2024-05-08T04:21:03.423 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T04:21:03.423 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289 --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} --verbose --name neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/orig.config.yaml 2024-05-08T04:21:03.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T04:21:03.572 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289 branch: wip-neesingh-testing1-20240507 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: neesingh@redhat.com first_in_suite: false job_id: '7697289' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing1-20240507 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-neesingh-testing1-20240507 sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 owner: scheduled_neesingh@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4374 sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 sleep_before_teardown: 0 suite: fs:volumes suite_branch: wip-neesingh-testing1-20240507 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnS/3cYF83fvr2BaYE8Va8ASSrF+m2VisCVVn/fgNEYJNj1uesTjrUBynghqpdH1q2GwXAQmUum+3EExQ6XLI0= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRm1XvoHIXIVd8PIF/BL0esi4XwWpYetJ7hmo0GEjmxAW4xn7mAK0Hn60JgRhKulQR+VLqYNGDfdRp3h35qmbI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_04:04:15 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T04:21:03.572 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa; will attempt to use it 2024-05-08T04:21:03.573 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks 2024-05-08T04:21:03.573 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T04:21:03.575 INFO:teuthology.task.internal:Checking packages... 2024-05-08T04:21:03.598 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20' 2024-05-08T04:21:03.599 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T04:21:03.599 INFO:teuthology.packaging:ref: None 2024-05-08T04:21:03.599 INFO:teuthology.packaging:tag: None 2024-05-08T04:21:03.599 INFO:teuthology.packaging:branch: wip-neesingh-testing1-20240507 2024-05-08T04:21:03.599 INFO:teuthology.packaging:sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-08T04:21:03.599 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing1-20240507 2024-05-08T04:21:03.752 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:21:03.753 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T04:21:03.759 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T04:21:03.760 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T04:21:03.805 INFO:teuthology.task.internal:Saving configuration 2024-05-08T04:21:03.822 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T04:21:03.838 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T04:21:03.865 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 04:15:03.613620', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMIbSqbA1KZxC5hKBtCTyUEj0ZO+nd6I61qUQH2DCPuT'} 2024-05-08T04:21:03.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 04:15:03.612223', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKtJBqNDsP9sriCRnATaPOLVSmu7PhcGd50PUuVYUnD9'} 2024-05-08T04:21:03.888 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T04:21:03.894 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T04:21:03.894 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T04:21:03.895 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T04:21:03.993 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f511cc524c0>, signals=[15]) 2024-05-08T04:21:03.993 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T04:21:03.999 INFO:teuthology.task.internal:Opening connections... 2024-05-08T04:21:04.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:21:04.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:21:04.083 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:21:04.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:21:04.169 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T04:21:04.176 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-05-08T04:21:04.180 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-05-08T04:21:04.181 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="22.04" 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=jammy 2024-05-08T04:21:04.230 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T04:21:04.231 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=jammy 2024-05-08T04:21:04.231 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-05-08T04:21:04.252 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-05-08T04:21:04.257 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-05-08T04:21:04.258 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-05-08T04:21:04.309 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-08T04:21:04.309 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Ubuntu" 2024-05-08T04:21:04.309 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="22.04" 2024-05-08T04:21:04.309 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-08T04:21:04.309 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_CODENAME=jammy 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:ID=ubuntu 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE=debian 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T04:21:04.310 INFO:teuthology.orchestra.run.smithi172.stdout:UBUNTU_CODENAME=jammy 2024-05-08T04:21:04.310 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-05-08T04:21:04.335 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T04:21:04.344 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T04:21:04.380 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T04:21:04.381 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-05-08T04:21:04.383 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-05-08T04:21:04.388 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T04:21:04.413 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T04:21:04.413 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-05-08T04:21:04.429 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-05-08T04:21:04.521 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T04:21:04.583 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T04:21:04.598 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-05-08T04:21:04.598 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-08T04:21:04.598 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-08T04:21:04.599 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T04:21:04.599 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T04:21:04.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:21:04.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-05-08T04:21:04.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:21:04.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-05-08T04:21:04.837 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:21:04.837 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T04:21:04.837 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-05-08T04:21:04.843 INFO:teuthology.orchestra.run.smithi164.stdout:5.15.0-56-generic 2024-05-08T04:21:04.843 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:21:04.843 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T04:21:04.843 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-05-08T04:21:04.844 DEBUG:teuthology.orchestra.run.smithi164:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-08T04:21:04.854 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:21:04.854 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T04:21:04.854 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-05-08T04:21:04.860 INFO:teuthology.orchestra.run.smithi172.stdout:5.15.0-56-generic 2024-05-08T04:21:04.861 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:21:04.861 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T04:21:04.861 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-05-08T04:21:04.861 DEBUG:teuthology.orchestra.run.smithi172:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-08T04:21:04.903 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-08T04:21:04.922 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: git config --global init.defaultBranch 2024-05-08T04:21:05.033 INFO:teuthology.orchestra.run.smithi164.stderr:hint: 2024-05-08T04:21:05.034 INFO:teuthology.orchestra.run.smithi164.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-08T04:21:05.034 INFO:teuthology.orchestra.run.smithi164.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-08T04:21:05.034 INFO:teuthology.orchestra.run.smithi164.stderr:hint: 2024-05-08T04:21:05.034 INFO:teuthology.orchestra.run.smithi164.stderr:hint: git branch -m 2024-05-08T04:21:05.034 INFO:teuthology.orchestra.run.smithi164.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-08T04:21:05.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-08T04:21:05.042 INFO:teuthology.orchestra.run.smithi172.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: git config --global init.defaultBranch 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: 2024-05-08T04:21:05.043 INFO:teuthology.orchestra.run.smithi172.stderr:hint: git branch -m 2024-05-08T04:21:05.044 INFO:teuthology.orchestra.run.smithi172.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-08T04:21:05.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-08T04:21:05.102 DEBUG:teuthology.orchestra.run.smithi164:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-08T04:21:05.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-08T04:21:05.161 DEBUG:teuthology.orchestra.run.smithi164:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-08T04:21:05.170 DEBUG:teuthology.orchestra.run.smithi172:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-08T04:22:01.111 INFO:teuthology.orchestra.run.smithi164.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-08T04:22:01.111 INFO:teuthology.orchestra.run.smithi164.stderr: * [new branch] main -> origin/main 2024-05-08T04:22:01.145 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190312 -> 20190312 2024-05-08T04:22:01.145 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190416 -> 20190416 2024-05-08T04:22:01.145 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190514 -> 20190514 2024-05-08T04:22:01.146 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190618 -> 20190618 2024-05-08T04:22:01.146 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190717 -> 20190717 2024-05-08T04:22:01.146 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190815 -> 20190815 2024-05-08T04:22:01.146 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20190923 -> 20190923 2024-05-08T04:22:01.146 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20191022 -> 20191022 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20191215 -> 20191215 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200122 -> 20200122 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200316 -> 20200316 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200421 -> 20200421 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200519 -> 20200519 2024-05-08T04:22:01.147 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200619 -> 20200619 2024-05-08T04:22:01.148 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200721 -> 20200721 2024-05-08T04:22:01.148 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200817 -> 20200817 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20200918 -> 20200918 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20201022 -> 20201022 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20201118 -> 20201118 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20201218 -> 20201218 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210208 -> 20210208 2024-05-08T04:22:01.149 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210315 -> 20210315 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210511 -> 20210511 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210716 -> 20210716 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210818 -> 20210818 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20210919 -> 20210919 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20211027 -> 20211027 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20211216 -> 20211216 2024-05-08T04:22:01.150 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220209 -> 20220209 2024-05-08T04:22:01.151 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220310 -> 20220310 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220411 -> 20220411 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220509 -> 20220509 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220610 -> 20220610 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220708 -> 20220708 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220815 -> 20220815 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20220913 -> 20220913 2024-05-08T04:22:01.152 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20221012 -> 20221012 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20221109 -> 20221109 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20221214 -> 20221214 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230117 -> 20230117 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230210 -> 20230210 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230310 -> 20230310 2024-05-08T04:22:01.153 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230404 -> 20230404 2024-05-08T04:22:01.154 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230515 -> 20230515 2024-05-08T04:22:01.154 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230625 -> 20230625 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230804 -> 20230804 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20230919 -> 20230919 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20231030 -> 20231030 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20231111 -> 20231111 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20231211 -> 20231211 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20240115 -> 20240115 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20240220 -> 20240220 2024-05-08T04:22:01.155 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20240312 -> 20240312 2024-05-08T04:22:01.156 INFO:teuthology.orchestra.run.smithi164.stderr: * [new tag] 20240410 -> 20240410 2024-05-08T04:22:02.138 INFO:teuthology.orchestra.run.smithi172.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-08T04:22:02.138 INFO:teuthology.orchestra.run.smithi172.stderr: * [new branch] main -> origin/main 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190312 -> 20190312 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190416 -> 20190416 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190514 -> 20190514 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190618 -> 20190618 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190717 -> 20190717 2024-05-08T04:22:02.172 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190815 -> 20190815 2024-05-08T04:22:02.173 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20190923 -> 20190923 2024-05-08T04:22:02.173 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20191022 -> 20191022 2024-05-08T04:22:02.173 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20191215 -> 20191215 2024-05-08T04:22:02.173 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200122 -> 20200122 2024-05-08T04:22:02.174 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200316 -> 20200316 2024-05-08T04:22:02.174 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200421 -> 20200421 2024-05-08T04:22:02.174 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200519 -> 20200519 2024-05-08T04:22:02.174 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200619 -> 20200619 2024-05-08T04:22:02.174 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200721 -> 20200721 2024-05-08T04:22:02.176 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200817 -> 20200817 2024-05-08T04:22:02.176 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20200918 -> 20200918 2024-05-08T04:22:02.176 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20201022 -> 20201022 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20201118 -> 20201118 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20201218 -> 20201218 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210208 -> 20210208 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210315 -> 20210315 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210511 -> 20210511 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210716 -> 20210716 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210818 -> 20210818 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20210919 -> 20210919 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20211027 -> 20211027 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20211216 -> 20211216 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220209 -> 20220209 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220310 -> 20220310 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220411 -> 20220411 2024-05-08T04:22:02.177 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220509 -> 20220509 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220610 -> 20220610 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220708 -> 20220708 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220815 -> 20220815 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20220913 -> 20220913 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20221012 -> 20221012 2024-05-08T04:22:02.178 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20221109 -> 20221109 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20221214 -> 20221214 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230117 -> 20230117 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230210 -> 20230210 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230310 -> 20230310 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230404 -> 20230404 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230515 -> 20230515 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230625 -> 20230625 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230804 -> 20230804 2024-05-08T04:22:02.180 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20230919 -> 20230919 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20231030 -> 20231030 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20231111 -> 20231111 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20231211 -> 20231211 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20240115 -> 20240115 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20240220 -> 20240220 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20240312 -> 20240312 2024-05-08T04:22:02.181 INFO:teuthology.orchestra.run.smithi172.stderr: * [new tag] 20240410 -> 20240410 2024-05-08T04:22:05.431 INFO:teuthology.orchestra.run.smithi164.stderr:Updating files: 45% (1560/3414) Updating files: 46% (1571/3414) Updating files: 47% (1605/3414) Updating files: 48% (1639/3414) Updating files: 49% (1673/3414) Updating files: 50% (1707/3414) Updating files: 51% (1742/3414) Updating files: 52% (1776/3414) Updating files: 53% (1810/3414) Updating files: 54% (1844/3414) Updating files: 55% (1878/3414) Updating files: 56% (1912/3414) Updating files: 57% (1946/3414) Updating files: 58% (1981/3414) Updating files: 59% (2015/3414) Updating files: 59% (2042/3414) Updating files: 60% (2049/3414) Updating files: 61% (2083/3414) Updating files: 62% (2117/3414) Updating files: 63% (2151/3414) Updating files: 64% (2185/3414) Updating files: 65% (2220/3414) Updating files: 66% (2254/3414) Updating files: 67% (2288/3414) Updating files: 68% (2322/3414) Updating files: 69% (2356/3414) Updating files: 70% (2390/3414) Updating files: 70% (2395/3414) Updating files: 71% (2424/3414) Updating files: 72% (2459/3414) Updating files: 73% (2493/3414) Updating files: 74% (2527/3414) Updating files: 75% (2561/3414) Updating files: 76% (2595/3414) Updating files: 77% (2629/3414) Updating files: 78% (2663/3414) Updating files: 79% (2698/3414) Updating files: 80% (2732/3414) Updating files: 81% (2766/3414) Updating files: 82% (2800/3414) Updating files: 83% (2834/3414) Updating files: 83% (2844/3414) Updating files: 84% (2868/3414) Updating files: 85% (2902/3414) Updating files: 86% (2937/3414) Updating files: 87% (2971/3414) Updating files: 88% (3005/3414) Updating files: 89% (3039/3414) Updating files: 90% (3073/3414) Updating files: 91% (3107/3414) Updating files: 92% (3141/3414) Updating files: 93% (3176/3414) Updating files: 94% (3210/3414) Updating files: 95% (3244/3414) Updating files: 96% (3278/3414) Updating files: 97% (3312/3414) Updating files: 98% (3346/3414) Updating files: 99% (3380/3414) Updating files: 100% (3414/3414) Updating files: 100% (3414/3414), done. 2024-05-08T04:22:05.436 INFO:teuthology.orchestra.run.smithi164.stdout:HEAD is now at b93493c0 Merge branch 'robot/pr-0-1714981108' into 'main' 2024-05-08T04:22:05.458 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T04:22:05.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:22:05.613 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:05.613 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-05-08T04:22:05.616 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi164.front.sepia.ceph.com to complete... 2024-05-08T04:22:06.481 INFO:teuthology.orchestra.run.smithi172.stderr:Updating files: 45% (1544/3414) Updating files: 46% (1571/3414) Updating files: 47% (1605/3414) Updating files: 48% (1639/3414) Updating files: 49% (1673/3414) Updating files: 50% (1707/3414) Updating files: 51% (1742/3414) Updating files: 52% (1776/3414) Updating files: 53% (1810/3414) Updating files: 54% (1844/3414) Updating files: 55% (1878/3414) Updating files: 56% (1912/3414) Updating files: 57% (1946/3414) Updating files: 58% (1981/3414) Updating files: 59% (2015/3414) Updating files: 59% (2036/3414) Updating files: 60% (2049/3414) Updating files: 61% (2083/3414) Updating files: 62% (2117/3414) Updating files: 63% (2151/3414) Updating files: 64% (2185/3414) Updating files: 65% (2220/3414) Updating files: 66% (2254/3414) Updating files: 67% (2288/3414) Updating files: 68% (2322/3414) Updating files: 69% (2356/3414) Updating files: 70% (2390/3414) Updating files: 70% (2395/3414) Updating files: 71% (2424/3414) Updating files: 72% (2459/3414) Updating files: 73% (2493/3414) Updating files: 74% (2527/3414) Updating files: 75% (2561/3414) Updating files: 76% (2595/3414) Updating files: 77% (2629/3414) Updating files: 78% (2663/3414) Updating files: 79% (2698/3414) Updating files: 80% (2732/3414) Updating files: 81% (2766/3414) Updating files: 82% (2800/3414) Updating files: 83% (2834/3414) Updating files: 83% (2839/3414) Updating files: 84% (2868/3414) Updating files: 85% (2902/3414) Updating files: 86% (2937/3414) Updating files: 87% (2971/3414) Updating files: 88% (3005/3414) Updating files: 89% (3039/3414) Updating files: 90% (3073/3414) Updating files: 91% (3107/3414) Updating files: 92% (3141/3414) Updating files: 93% (3176/3414) Updating files: 94% (3210/3414) Updating files: 95% (3244/3414) Updating files: 96% (3278/3414) Updating files: 97% (3312/3414) Updating files: 98% (3346/3414) Updating files: 99% (3380/3414) Updating files: 100% (3414/3414) Updating files: 100% (3414/3414), done. 2024-05-08T04:22:06.486 INFO:teuthology.orchestra.run.smithi172.stdout:HEAD is now at b93493c0 Merge branch 'robot/pr-0-1714981108' into 'main' 2024-05-08T04:22:06.508 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-05-08T04:22:06.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T04:22:06.657 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:06.657 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-05-08T04:22:06.660 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi172.front.sepia.ceph.com to complete... 2024-05-08T04:22:07.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08 04:22:07 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-05-08T04:22:07.431 INFO:teuthology.orchestra.run.smithi164.stderr:FINISHED --2024-05-08 04:22:07-- 2024-05-08T04:22:07.431 INFO:teuthology.orchestra.run.smithi164.stderr:Total wall clock time: 1.7s 2024-05-08T04:22:07.431 INFO:teuthology.orchestra.run.smithi164.stderr:Downloaded: 1 files, 43M in 1.6s (26.7 MB/s) 2024-05-08T04:22:07.435 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T04:22:07.435 DEBUG:teuthology.orchestra.run.smithi164:> sudo dpkg -i /tmp/linux-image.deb 2024-05-08T04:22:07.663 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-05-08T04:22:09.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08 04:22:09 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-05-08T04:22:09.306 INFO:teuthology.orchestra.run.smithi172.stderr:FINISHED --2024-05-08 04:22:09-- 2024-05-08T04:22:09.307 INFO:teuthology.orchestra.run.smithi172.stderr:Total wall clock time: 2.4s 2024-05-08T04:22:09.307 INFO:teuthology.orchestra.run.smithi172.stderr:Downloaded: 1 files, 43M in 1.9s (22.3 MB/s) 2024-05-08T04:22:09.311 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-05-08T04:22:09.311 DEBUG:teuthology.orchestra.run.smithi172:> sudo dpkg -i /tmp/linux-image.deb 2024-05-08T04:22:09.575 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-08T04:22:09.579 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-08T04:22:09.595 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-05-08T04:22:09.675 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-05-08T04:22:11.598 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-08T04:22:11.602 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-08T04:22:11.703 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-05-08T04:22:13.695 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-05-08T04:22:13.787 INFO:teuthology.orchestra.run.smithi164.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:15.485 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-05-08T04:22:15.586 INFO:teuthology.orchestra.run.smithi172.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:35.960 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2024-05-08T04:22:35.975 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T04:22:36.038 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-05-08T04:22:36.373 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:36.393 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:36.669 INFO:teuthology.orchestra.run.smithi172.stderr:Sourcing file `/etc/default/grub' 2024-05-08T04:22:36.680 INFO:teuthology.orchestra.run.smithi172.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T04:22:36.710 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-05-08T04:22:36.765 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-08T04:22:36.775 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-08T04:22:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-08T04:22:36.843 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-08T04:22:36.998 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-08T04:22:36.998 INFO:teuthology.orchestra.run.smithi164.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-08T04:22:36.998 INFO:teuthology.orchestra.run.smithi164.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-08T04:22:37.026 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.039 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-05-08T04:22:37.041 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.221 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-deb -c /tmp/linux-image.deb 2024-05-08T04:22:37.229 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-05-08T04:22:37.230 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-05-08T04:22:37.230 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.247 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.248 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-05-08T04:22:37.267 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T04:22:37.268 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T04:22:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T04:22:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T04:22:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T04:22:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T04:22:37.272 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-05-08T04:22:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-05-08T04:22:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-05-08T04:22:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T04:22:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-05-08T04:22:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-05-08T04:22:37.274 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-05-08T04:22:37.274 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T04:22:37.274 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T04:22:37.274 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T04:22:37.350 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-08T04:22:37.351 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-05-08T04:22:37.351 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-05-08T04:22:37.351 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T04:22:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T04:22:37.353 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T04:22:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T04:22:37.355 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T04:22:37.356 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T04:22:37.357 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T04:22:37.358 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T04:22:37.359 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T04:22:37.360 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-05-08T04:22:37.361 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T04:22:37.362 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T04:22:37.363 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-08T04:22:37.364 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-05-08T04:22:37.365 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T04:22:37.366 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T04:22:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-08T04:22:37.368 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-08T04:22:37.369 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-08T04:22:37.370 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-05-08T04:22:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-05-08T04:22:37.372 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T04:22:37.373 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T04:22:37.374 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T04:22:37.375 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-08T04:22:37.376 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T04:22:37.377 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T04:22:37.378 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T04:22:37.379 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T04:22:37.380 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T04:22:37.381 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T04:22:37.382 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T04:22:37.383 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T04:22:37.384 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T04:22:37.385 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T04:22:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-08T04:22:37.387 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T04:22:37.388 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-08T04:22:37.389 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-08T04:22:37.390 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-05-08T04:22:37.391 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T04:22:37.392 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-05-08T04:22:37.393 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-08T04:22:37.394 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T04:22:37.395 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T04:22:37.396 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-05-08T04:22:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-08T04:22:37.398 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T04:22:37.399 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T04:22:37.400 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T04:22:37.401 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-05-08T04:22:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-05-08T04:22:37.403 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T04:22:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-08T04:22:37.405 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T04:22:37.406 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T04:22:37.407 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T04:22:37.408 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T04:22:37.409 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T04:22:37.410 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-08T04:22:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T04:22:37.412 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-08T04:22:37.418 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-08T04:22:37.425 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T04:22:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-05-08T04:22:37.481 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-08T04:22:37.482 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T04:22:37.483 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-05-08T04:22:37.484 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T04:22:37.485 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T04:22:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T04:22:37.487 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-05-08T04:22:37.488 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T04:22:37.489 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T04:22:37.490 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T04:22:37.491 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-05-08T04:22:37.492 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T04:22:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T04:22:37.494 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-08T04:22:37.495 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-08T04:22:37.496 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T04:22:37.497 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T04:22:37.498 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-05-08T04:22:37.499 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-05-08T04:22:37.500 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-05-08T04:22:37.501 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-05-08T04:22:37.502 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T04:22:37.503 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-08T04:22:37.504 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-08T04:22:37.505 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-05-08T04:22:37.506 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T04:22:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T04:22:37.508 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T04:22:37.509 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-05-08T04:22:37.510 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T04:22:37.511 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T04:22:37.512 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-05-08T04:22:37.513 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T04:22:37.514 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T04:22:37.515 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-08T04:22:37.516 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-05-08T04:22:37.517 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-08T04:22:37.518 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-05-08T04:22:37.519 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-08T04:22:37.520 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-08T04:22:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-08T04:22:37.522 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-05-08T04:22:37.523 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T04:22:37.524 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T04:22:37.525 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T04:22:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T04:22:37.527 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T04:22:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-05-08T04:22:37.529 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T04:22:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T04:22:37.531 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-08T04:22:37.532 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-08T04:22:37.533 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T04:22:37.534 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T04:22:37.537 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T04:22:37.537 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-08T04:22:37.538 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-08T04:22:37.539 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T04:22:37.540 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T04:22:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T04:22:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-05-08T04:22:37.543 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-05-08T04:22:37.544 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T04:22:37.545 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-05-08T04:22:37.546 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T04:22:37.560 INFO:teuthology.orchestra.run.smithi172.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-08T04:22:37.560 INFO:teuthology.orchestra.run.smithi172.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-08T04:22:37.560 INFO:teuthology.orchestra.run.smithi172.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T04:22:37.591 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-05-08T04:22:37.592 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-05-08T04:22:37.593 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T04:22:37.594 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T04:22:37.595 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-05-08T04:22:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-05-08T04:22:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-05-08T04:22:37.598 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-05-08T04:22:37.599 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-05-08T04:22:37.600 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-05-08T04:22:37.601 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T04:22:37.602 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T04:22:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T04:22:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T04:22:37.605 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-05-08T04:22:37.606 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T04:22:37.607 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T04:22:37.608 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-08T04:22:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T04:22:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-05-08T04:22:37.611 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T04:22:37.612 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T04:22:37.613 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T04:22:37.614 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T04:22:37.615 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T04:22:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T04:22:37.617 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T04:22:37.618 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T04:22:37.619 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T04:22:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T04:22:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T04:22:37.622 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T04:22:37.623 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T04:22:37.624 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T04:22:37.625 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T04:22:37.626 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-05-08T04:22:37.627 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T04:22:37.628 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T04:22:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T04:22:37.630 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-05-08T04:22:37.631 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T04:22:37.632 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T04:22:37.633 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-05-08T04:22:37.634 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-05-08T04:22:37.639 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.639 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.640 DEBUG:teuthology.orchestra.run.smithi164:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-05-08T04:22:37.640 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-05-08T04:22:37.646 INFO:teuthology.orchestra.run.smithi164.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:37.646 INFO:teuthology.orchestra.run.smithi164.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:37.647 INFO:teuthology.orchestra.run.smithi164.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:37.647 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-08T04:22:37.647 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.647 DEBUG:teuthology.orchestra.run.smithi164:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-08T04:22:37.691 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi164.front.sepia.ceph.com to complete... 2024-05-08T04:22:37.766 DEBUG:teuthology.orchestra.run.smithi172:> dpkg-deb -c /tmp/linux-image.deb 2024-05-08T04:22:37.814 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-05-08T04:22:37.814 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-05-08T04:22:37.815 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.832 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.832 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-05-08T04:22:37.853 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:37.854 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-05-08T04:22:37.854 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-05-08T04:22:37.854 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-05-08T04:22:37.854 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-05-08T04:22:37.854 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-08T04:22:37.856 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T04:22:37.857 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T04:22:37.858 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T04:22:37.859 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T04:22:37.876 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-05-08T04:22:37.876 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-05-08T04:22:37.876 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-05-08T04:22:37.876 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T04:22:37.877 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T04:22:37.878 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T04:22:37.879 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-05-08T04:22:37.880 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-05-08T04:22:37.881 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T04:22:37.882 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T04:22:37.883 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T04:22:37.884 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T04:22:37.885 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T04:22:37.886 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T04:22:37.887 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-05-08T04:22:37.887 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T04:22:37.947 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T04:22:37.948 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-05-08T04:22:37.949 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-08T04:22:37.950 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T04:22:37.951 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-05-08T04:22:37.952 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T04:22:37.953 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-05-08T04:22:37.954 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T04:22:37.955 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-08T04:22:37.956 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-05-08T04:22:37.957 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T04:22:37.958 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T04:22:37.959 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T04:22:37.960 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T04:22:37.961 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T04:22:37.962 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-05-08T04:22:37.963 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T04:22:37.964 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T04:22:37.965 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T04:22:37.966 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T04:22:37.967 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T04:22:37.968 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T04:22:37.969 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T04:22:37.970 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T04:22:37.971 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T04:22:37.972 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-08T04:22:37.973 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-08T04:22:37.974 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-08T04:22:37.975 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-08T04:22:37.976 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T04:22:37.977 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-08T04:22:37.978 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-05-08T04:22:37.979 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-08T04:22:37.980 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-08T04:22:37.981 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T04:22:37.982 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T04:22:37.983 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-05-08T04:22:37.984 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T04:22:37.985 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T04:22:37.986 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T04:22:37.987 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-05-08T04:22:37.988 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-05-08T04:22:37.989 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-08T04:22:37.990 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-05-08T04:22:37.991 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-05-08T04:22:37.992 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-05-08T04:22:37.993 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-05-08T04:22:37.994 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T04:22:37.995 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T04:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T04:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T04:22:38.049 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-05-08T04:22:38.049 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-08T04:22:38.049 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T04:22:38.049 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-05-08T04:22:38.049 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T04:22:38.050 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-05-08T04:22:38.051 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T04:22:38.052 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T04:22:38.053 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-05-08T04:22:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T04:22:38.055 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T04:22:38.056 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T04:22:38.057 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-05-08T04:22:38.058 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T04:22:38.059 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T04:22:38.060 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T04:22:38.061 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T04:22:38.062 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T04:22:38.063 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-05-08T04:22:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T04:22:38.065 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-05-08T04:22:38.066 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T04:22:38.067 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T04:22:38.068 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-08T04:22:38.069 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-05-08T04:22:38.070 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-05-08T04:22:38.071 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-08T04:22:38.072 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T04:22:38.073 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-08T04:22:38.074 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T04:22:38.075 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T04:22:38.076 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T04:22:38.077 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-08T04:22:38.078 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T04:22:38.079 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T04:22:38.080 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-08T04:22:38.081 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-05-08T04:22:38.082 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T04:22:38.083 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T04:22:38.084 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T04:22:38.085 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-05-08T04:22:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-05-08T04:22:38.087 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T04:22:38.088 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T04:22:38.089 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-08T04:22:38.090 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-08T04:22:38.091 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-08T04:22:38.092 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-08T04:22:38.093 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-05-08T04:22:38.094 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-05-08T04:22:38.095 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-05-08T04:22:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T04:22:38.124 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T04:22:38.125 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T04:22:38.126 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T04:22:38.127 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T04:22:38.128 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T04:22:38.129 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T04:22:38.130 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-08T04:22:38.131 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T04:22:38.132 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T04:22:38.133 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-05-08T04:22:38.134 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-08T04:22:38.135 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T04:22:38.136 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T04:22:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T04:22:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-05-08T04:22:38.139 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-05-08T04:22:38.140 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-05-08T04:22:38.141 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-05-08T04:22:38.142 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T04:22:38.143 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T04:22:38.144 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T04:22:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T04:22:38.146 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T04:22:38.147 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T04:22:38.147 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T04:22:38.147 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T04:22:38.147 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T04:22:38.149 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T04:22:38.149 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T04:22:38.149 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T04:22:38.150 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-05-08T04:22:38.151 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T04:22:38.152 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-05-08T04:22:38.153 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T04:22:38.153 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-05-08T04:22:38.153 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T04:22:38.153 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-05-08T04:22:38.153 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T04:22:38.155 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2024-05-08T04:22:38.155 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T04:22:38.155 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T04:22:38.193 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-05-08T04:22:38.194 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-05-08T04:22:38.195 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T04:22:38.196 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T04:22:38.197 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T04:22:38.198 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T04:22:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T04:22:38.200 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T04:22:38.201 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T04:22:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T04:22:38.203 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T04:22:38.204 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T04:22:38.205 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T04:22:38.206 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T04:22:38.207 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T04:22:38.208 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T04:22:38.209 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T04:22:38.210 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T04:22:38.211 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T04:22:38.212 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T04:22:38.213 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T04:22:38.214 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T04:22:38.215 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T04:22:38.216 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T04:22:38.217 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T04:22:38.218 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-05-08T04:22:38.219 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T04:22:38.220 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T04:22:38.221 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T04:22:38.222 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T04:22:38.223 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-05-08T04:22:38.224 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T04:22:38.225 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-05-08T04:22:38.226 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-05-08T04:22:38.227 INFO:teuthology.orchestra.run.smithi172.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-05-08T04:22:38.227 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-05-08T04:22:38.227 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-05-08T04:22:38.231 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.231 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.231 DEBUG:teuthology.orchestra.run.smithi172:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-05-08T04:22:38.241 INFO:teuthology.orchestra.run.smithi172.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:38.241 INFO:teuthology.orchestra.run.smithi172.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:38.241 INFO:teuthology.orchestra.run.smithi172.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-08T04:22:38.242 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-08T04:22:38.242 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.242 DEBUG:teuthology.orchestra.run.smithi172:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-08T04:22:38.285 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi172.front.sepia.ceph.com to complete... 2024-05-08T04:22:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.389 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-08T04:22:38.691 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-08T04:22:38.694 INFO:teuthology.orchestra.run.smithi172.stderr:Sourcing file `/etc/default/grub' 2024-05-08T04:22:38.695 INFO:teuthology.orchestra.run.smithi172.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T04:22:38.699 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-05-08T04:22:38.732 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-08T04:22:38.737 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-08T04:22:38.866 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-08T04:22:38.866 INFO:teuthology.orchestra.run.smithi164.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-08T04:22:38.866 INFO:teuthology.orchestra.run.smithi164.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-08T04:22:38.884 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-05-08T04:22:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-05-08T04:22:39.127 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-08T04:22:39.132 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-08T04:22:39.173 INFO:teuthology.orchestra.run.smithi172.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-08T04:22:39.178 INFO:teuthology.orchestra.run.smithi172.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-08T04:22:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-08T04:22:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-08T04:22:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-08T04:22:39.343 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-05-08T04:23:10.034 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T04:23:10.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:23:10.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:23:10.434 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-08T04:23:10.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:23:10.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:23:28.543 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-08T04:23:28.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-05-08T04:23:37.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:23:37.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:23:37.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:23:37.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:23:40.610 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-08T04:23:40.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-05-08T04:23:52.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:23:52.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:23:52.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:23:52.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:24:11.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-08T04:24:26.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:24:26.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:24:26.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-05-08T04:24:44.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:24:44.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:24:44.308 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-05-08T04:24:44.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:24:44.994 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T04:24:44.994 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-05-08T04:24:45.045 INFO:teuthology.orchestra.run.smithi164.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T04:24:45.045 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T04:24:45.045 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T04:24:45.045 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T04:24:46.046 DEBUG:teuthology.parallel:result is None 2024-05-08T04:24:52.866 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T04:25:07.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:25:07.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:25:07.993 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-05-08T04:25:08.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:25:08.573 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T04:25:08.573 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-05-08T04:25:08.627 INFO:teuthology.orchestra.run.smithi172.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T04:25:08.627 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T04:25:08.627 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T04:25:08.627 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T04:25:09.628 DEBUG:teuthology.parallel:result is None 2024-05-08T04:25:09.628 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T04:25:09.641 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T04:25:09.642 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T04:25:09.645 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T04:25:09.654 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T04:25:09.683 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T04:25:09.733 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T04:25:09.733 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T04:25:09.736 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T04:25:09.770 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T04:25:09.783 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T04:25:09.783 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T04:25:09.799 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T04:25:09.910 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:25:09.912 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:25:09.929 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:25:09.931 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:25:09.933 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T04:25:09.940 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T04:25:09.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T04:25:09.943 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T04:25:09.965 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T04:25:09.996 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T04:25:09.996 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:25:10.012 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:25:10.021 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:25:10.069 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:25:10.125 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:25:10.125 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T04:25:10.202 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:25:10.211 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:25:10.263 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:25:10.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T04:25:10.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-05-08T04:25:10.342 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-05-08T04:25:10.489 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T04:25:10.497 INFO:teuthology.task.internal:Starting timer... 2024-05-08T04:25:10.497 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T04:25:10.552 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T04:25:10.601 DEBUG:teuthology.task.selinux:Excluding smithi164: OS 'ubuntu' does not support SELinux 2024-05-08T04:25:10.601 DEBUG:teuthology.task.selinux:Excluding smithi172: OS 'ubuntu' does not support SELinux 2024-05-08T04:25:10.602 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T04:25:10.602 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-08T04:25:10.602 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T04:25:10.602 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T04:25:10.652 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T04:25:10.658 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T04:25:10.781 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T04:25:10.797 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T04:25:10.798 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi164.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T04:33:55.254 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-05-08T04:33:55.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:33:55.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:33:55.342 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-05-08T04:33:55.461 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-08T04:33:55.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:33:55.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:33:55.549 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-05-08T04:33:55.673 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-08T04:33:55.673 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T04:33:55.688 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T04:33:55.689 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T04:33:55.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:33:55.692 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T04:33:55.692 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:33:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-08T04:33:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Command line: ntpd -gq 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: ---------------------------------------------------- 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: ntp-4 is maintained by Network Time Foundation, 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: corporation. Support and training for ntp-4 are 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: available at https://www.nwtime.org/support 2024-05-08T04:33:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: ---------------------------------------------------- 2024-05-08T04:33:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: proto: precision = 0.042 usec (-24) 2024-05-08T04:33:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: basedate set to 2022-02-04 2024-05-08T04:33:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: gps base set to 2022-02-06 (week 2196) 2024-05-08T04:33:55.730 INFO:teuthology.orchestra.run.smithi164.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T04:33:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T04:33:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T04:33:55.731 INFO:teuthology.orchestra.run.smithi164.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen normally on 3 enp3s0f1 172.21.15.164:123 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen normally on 4 lo [::1]:123 2024-05-08T04:33:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:73f3%5]:123 2024-05-08T04:33:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:33:55 ntpd[17246]: Listening on routing socket on fd #22 for interface updates 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Command line: ntpd -gq 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: ---------------------------------------------------- 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: ntp-4 is maintained by Network Time Foundation, 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: corporation. Support and training for ntp-4 are 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: available at https://www.nwtime.org/support 2024-05-08T04:33:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: ---------------------------------------------------- 2024-05-08T04:33:55.754 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: proto: precision = 0.042 usec (-24) 2024-05-08T04:33:55.754 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: basedate set to 2022-02-04 2024-05-08T04:33:55.754 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: gps base set to 2022-02-06 (week 2196) 2024-05-08T04:33:55.754 INFO:teuthology.orchestra.run.smithi172.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T04:33:55.755 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T04:33:55.755 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T04:33:55.755 INFO:teuthology.orchestra.run.smithi172.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen normally on 3 enp3s0f1 172.21.15.172:123 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen normally on 4 lo [::1]:123 2024-05-08T04:33:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f91%5]:123 2024-05-08T04:33:55.757 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:33:55 ntpd[17263]: Listening on routing socket on fd #22 for interface updates 2024-05-08T04:33:56.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:56 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:56.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:56 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:56.753 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:56.753 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:56.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:56.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:56.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:56.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:56 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:57.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:57 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:57.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:57.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:57 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:58.753 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:58.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:58.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:58.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:58.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:58.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:58 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:59.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:59.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:33:59 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:59.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:33:59.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:33:59 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:00.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:00.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:00.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:00 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:00.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:00 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:01.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:01 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:01.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:01.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:01 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:02.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:02.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:02.731 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:02 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:02.754 INFO:teuthology.orchestra.run.smithi172.stdout: 8 May 04:34:02 ntpd[17263]: ntpd: time slew -0.001669 s 2024-05-08T04:34:02.755 INFO:teuthology.orchestra.run.smithi172.stdout:ntpd: time slew -0.001669s 2024-05-08T04:34:02.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T04:34:02.755 INFO:teuthology.orchestra.run.smithi172.stderr: 8 May 04:34:02 ntpd[17263]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T04:34:02.844 INFO:teuthology.orchestra.run.smithi172.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T04:34:02.845 INFO:teuthology.orchestra.run.smithi172.stdout:============================================================================== 2024-05-08T04:34:02.845 INFO:teuthology.orchestra.run.smithi172.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:02.845 INFO:teuthology.orchestra.run.smithi172.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:02.845 INFO:teuthology.orchestra.run.smithi172.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:02.845 INFO:teuthology.orchestra.run.smithi172.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:03.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:03.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:03.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T04:34:03.729 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stdout: 8 May 04:34:03 ntpd[17246]: ntpd: time slew -0.000773 s 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stdout:ntpd: time slew -0.000773s 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T04:34:03.730 INFO:teuthology.orchestra.run.smithi164.stderr: 8 May 04:34:03 ntpd[17246]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-08T04:34:03.827 INFO:teuthology.run_tasks:Running task install... 2024-05-08T04:34:03.836 DEBUG:teuthology.task.install:project ceph 2024-05-08T04:34:03.836 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}} 2024-05-08T04:34:03.836 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'} 2024-05-08T04:34:03.836 INFO:teuthology.task.install:Using flavor: default 2024-05-08T04:34:03.842 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T04:34:03.842 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T04:34:03.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-key list | grep Ceph 2024-05-08T04:34:03.843 DEBUG:teuthology.orchestra.run.smithi172:> sudo apt-key list | grep Ceph 2024-05-08T04:34:03.948 INFO:teuthology.orchestra.run.smithi172.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-08T04:34:03.970 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-08T04:34:03.999 INFO:teuthology.orchestra.run.smithi172.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T04:34:03.999 INFO:teuthology.orchestra.run.smithi172.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T04:34:04.000 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T04:34:04.000 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T04:34:04.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-08T04:34:04.020 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T04:34:04.020 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T04:34:04.021 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T04:34:04.021 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T04:34:04.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-08T04:34:04.153 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default/ 2024-05-08T04:34:04.153 INFO:teuthology.task.install.deb:Package version is 19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:34:04.246 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default/ 2024-05-08T04:34:04.246 INFO:teuthology.task.install.deb:Package version is 19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:34:04.339 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:34:04.340 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T04:34:04.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo apt-get update 2024-05-08T04:34:04.466 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:34:04.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T04:34:04.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-05-08T04:34:04.549 INFO:teuthology.orchestra.run.smithi172.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-08T04:34:04.565 INFO:teuthology.orchestra.run.smithi172.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-08T04:34:04.567 INFO:teuthology.orchestra.run.smithi172.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-08T04:34:04.586 INFO:teuthology.orchestra.run.smithi172.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-08T04:34:04.592 INFO:teuthology.orchestra.run.smithi172.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy InRelease 2024-05-08T04:34:04.613 INFO:teuthology.orchestra.run.smithi172.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-08T04:34:04.635 INFO:teuthology.orchestra.run.smithi172.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-08T04:34:04.656 INFO:teuthology.orchestra.run.smithi172.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-05-08T04:34:04.674 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-08T04:34:04.688 INFO:teuthology.orchestra.run.smithi172.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-05-08T04:34:04.786 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-08T04:34:04.788 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy InRelease 2024-05-08T04:34:04.827 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-08T04:34:04.867 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-08T04:34:04.867 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-08T04:34:04.905 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-05-08T04:34:04.949 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-08T04:34:04.959 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-05-08T04:34:05.137 INFO:teuthology.orchestra.run.smithi172.stdout:Fetched 28.9 kB in 1s (52.0 kB/s) 2024-05-08T04:34:05.260 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 28.9 kB in 1s (52.1 kB/s) 2024-05-08T04:34:06.119 INFO:teuthology.orchestra.run.smithi172.stdout:Reading package lists... 2024-05-08T04:34:06.133 DEBUG:teuthology.orchestra.run.smithi172:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1629-g4f7075d1-1jammy cephadm=19.0.0-1629-g4f7075d1-1jammy ceph-mds=19.0.0-1629-g4f7075d1-1jammy ceph-mgr=19.0.0-1629-g4f7075d1-1jammy ceph-common=19.0.0-1629-g4f7075d1-1jammy ceph-fuse=19.0.0-1629-g4f7075d1-1jammy ceph-test=19.0.0-1629-g4f7075d1-1jammy ceph-volume=19.0.0-1629-g4f7075d1-1jammy radosgw=19.0.0-1629-g4f7075d1-1jammy python3-rados=19.0.0-1629-g4f7075d1-1jammy python3-rgw=19.0.0-1629-g4f7075d1-1jammy python3-cephfs=19.0.0-1629-g4f7075d1-1jammy python3-rbd=19.0.0-1629-g4f7075d1-1jammy libcephfs2=19.0.0-1629-g4f7075d1-1jammy libcephfs-dev=19.0.0-1629-g4f7075d1-1jammy librados2=19.0.0-1629-g4f7075d1-1jammy librbd1=19.0.0-1629-g4f7075d1-1jammy rbd-fuse=19.0.0-1629-g4f7075d1-1jammy python3-cephfs=19.0.0-1629-g4f7075d1-1jammy cephfs-shell=19.0.0-1629-g4f7075d1-1jammy cephfs-top=19.0.0-1629-g4f7075d1-1jammy cephfs-mirror=19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:34:06.207 INFO:teuthology.orchestra.run.smithi172.stdout:Reading package lists... 2024-05-08T04:34:06.249 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-05-08T04:34:06.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1629-g4f7075d1-1jammy cephadm=19.0.0-1629-g4f7075d1-1jammy ceph-mds=19.0.0-1629-g4f7075d1-1jammy ceph-mgr=19.0.0-1629-g4f7075d1-1jammy ceph-common=19.0.0-1629-g4f7075d1-1jammy ceph-fuse=19.0.0-1629-g4f7075d1-1jammy ceph-test=19.0.0-1629-g4f7075d1-1jammy ceph-volume=19.0.0-1629-g4f7075d1-1jammy radosgw=19.0.0-1629-g4f7075d1-1jammy python3-rados=19.0.0-1629-g4f7075d1-1jammy python3-rgw=19.0.0-1629-g4f7075d1-1jammy python3-cephfs=19.0.0-1629-g4f7075d1-1jammy python3-rbd=19.0.0-1629-g4f7075d1-1jammy libcephfs2=19.0.0-1629-g4f7075d1-1jammy libcephfs-dev=19.0.0-1629-g4f7075d1-1jammy librados2=19.0.0-1629-g4f7075d1-1jammy librbd1=19.0.0-1629-g4f7075d1-1jammy rbd-fuse=19.0.0-1629-g4f7075d1-1jammy python3-cephfs=19.0.0-1629-g4f7075d1-1jammy cephfs-shell=19.0.0-1629-g4f7075d1-1jammy cephfs-top=19.0.0-1629-g4f7075d1-1jammy cephfs-mirror=19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:34:06.337 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-05-08T04:34:06.382 INFO:teuthology.orchestra.run.smithi172.stdout:Building dependency tree... 2024-05-08T04:34:06.383 INFO:teuthology.orchestra.run.smithi172.stdout:Reading state information... 2024-05-08T04:34:06.513 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-05-08T04:34:06.514 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-05-08T04:34:06.555 INFO:teuthology.orchestra.run.smithi172.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T04:34:06.555 INFO:teuthology.orchestra.run.smithi172.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-08T04:34:06.555 INFO:teuthology.orchestra.run.smithi172.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-08T04:34:06.555 INFO:teuthology.orchestra.run.smithi172.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-08T04:34:06.556 INFO:teuthology.orchestra.run.smithi172.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-08T04:34:06.556 INFO:teuthology.orchestra.run.smithi172.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-08T04:34:06.556 INFO:teuthology.orchestra.run.smithi172.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T04:34:06.556 INFO:teuthology.orchestra.run.smithi172.stdout:The following additional packages will be installed: 2024-05-08T04:34:06.556 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T04:34:06.557 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-08T04:34:06.557 INFO:teuthology.orchestra.run.smithi172.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-08T04:34:06.557 INFO:teuthology.orchestra.run.smithi172.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-08T04:34:06.558 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-08T04:34:06.558 INFO:teuthology.orchestra.run.smithi172.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-08T04:34:06.558 INFO:teuthology.orchestra.run.smithi172.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-08T04:34:06.559 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout:Suggested packages: 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout:Recommended packages: 2024-05-08T04:34:06.560 INFO:teuthology.orchestra.run.smithi172.stdout: btrfs-tools 2024-05-08T04:34:06.688 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T04:34:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-08T04:34:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-08T04:34:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-08T04:34:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-08T04:34:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-08T04:34:06.690 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T04:34:06.690 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-05-08T04:34:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T04:34:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-08T04:34:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-08T04:34:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-08T04:34:06.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-05-08T04:34:06.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-05-08T04:34:06.694 INFO:teuthology.orchestra.run.smithi164.stdout: btrfs-tools 2024-05-08T04:34:06.710 INFO:teuthology.orchestra.run.smithi172.stdout:The following NEW packages will be installed: 2024-05-08T04:34:06.710 INFO:teuthology.orchestra.run.smithi172.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T04:34:06.710 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T04:34:06.710 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T04:34:06.710 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-08T04:34:06.712 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-08T04:34:06.713 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-08T04:34:06.714 INFO:teuthology.orchestra.run.smithi172.stdout: socat unzip xmlstarlet zip 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-08T04:34:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-08T04:34:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-08T04:34:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-08T04:34:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: socat unzip xmlstarlet zip 2024-05-08T04:34:06.862 INFO:teuthology.orchestra.run.smithi172.stdout:0 upgraded, 124 newly installed, 0 to remove and 306 not upgraded. 2024-05-08T04:34:06.862 INFO:teuthology.orchestra.run.smithi172.stdout:Need to get 176 MB of archives. 2024-05-08T04:34:06.862 INFO:teuthology.orchestra.run.smithi172.stdout:After this operation, 732 MB of additional disk space will be used. 2024-05-08T04:34:06.862 INFO:teuthology.orchestra.run.smithi172.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1629-g4f7075d1-1jammy [3,533 kB] 2024-05-08T04:34:06.907 INFO:teuthology.orchestra.run.smithi172.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-08T04:34:06.919 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 124 newly installed, 0 to remove and 306 not upgraded. 2024-05-08T04:34:06.919 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 176 MB of archives. 2024-05-08T04:34:06.919 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 732 MB of additional disk space will be used. 2024-05-08T04:34:06.919 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-08T04:34:06.941 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-08T04:34:06.952 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-08T04:34:06.990 INFO:teuthology.orchestra.run.smithi172.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-08T04:34:06.996 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1629-g4f7075d1-1jammy [3,533 kB] 2024-05-08T04:34:07.004 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-08T04:34:07.030 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-08T04:34:07.033 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-08T04:34:07.035 INFO:teuthology.orchestra.run.smithi172.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-08T04:34:07.047 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-08T04:34:07.099 INFO:teuthology.orchestra.run.smithi172.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1629-g4f7075d1-1jammy [3,217 kB] 2024-05-08T04:34:07.099 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-08T04:34:07.102 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-08T04:34:07.110 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-08T04:34:07.113 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-08T04:34:07.114 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-08T04:34:07.120 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-08T04:34:07.122 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-08T04:34:07.124 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-08T04:34:07.125 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-08T04:34:07.126 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-08T04:34:07.143 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-08T04:34:07.145 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-08T04:34:07.146 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-08T04:34:07.147 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-08T04:34:07.163 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-08T04:34:07.164 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-08T04:34:07.165 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-08T04:34:07.165 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-08T04:34:07.165 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-08T04:34:07.166 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-08T04:34:07.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-08T04:34:07.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-08T04:34:07.187 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-08T04:34:07.187 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-08T04:34:07.202 INFO:teuthology.orchestra.run.smithi172.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1629-g4f7075d1-1jammy [710 kB] 2024-05-08T04:34:07.204 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-08T04:34:07.205 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-08T04:34:07.206 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T04:34:07.206 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-08T04:34:07.207 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-08T04:34:07.208 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-08T04:34:07.225 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-08T04:34:07.225 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-08T04:34:07.228 INFO:teuthology.orchestra.run.smithi172.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1629-g4f7075d1-1jammy [356 kB] 2024-05-08T04:34:07.230 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-08T04:34:07.231 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-08T04:34:07.236 INFO:teuthology.orchestra.run.smithi172.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-08T04:34:07.240 INFO:teuthology.orchestra.run.smithi172.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1629-g4f7075d1-1jammy [32.8 kB] 2024-05-08T04:34:07.240 INFO:teuthology.orchestra.run.smithi172.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1629-g4f7075d1-1jammy [180 kB] 2024-05-08T04:34:07.245 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-08T04:34:07.246 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-08T04:34:07.247 INFO:teuthology.orchestra.run.smithi172.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1629-g4f7075d1-1jammy [64.8 kB] 2024-05-08T04:34:07.247 INFO:teuthology.orchestra.run.smithi164.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-08T04:34:07.249 INFO:teuthology.orchestra.run.smithi164.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-08T04:34:07.249 INFO:teuthology.orchestra.run.smithi164.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-08T04:34:07.251 INFO:teuthology.orchestra.run.smithi164.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-08T04:34:07.252 INFO:teuthology.orchestra.run.smithi172.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1629-g4f7075d1-1jammy [330 kB] 2024-05-08T04:34:07.263 INFO:teuthology.orchestra.run.smithi172.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1629-g4f7075d1-1jammy [6,575 kB] 2024-05-08T04:34:07.266 INFO:teuthology.orchestra.run.smithi164.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-08T04:34:07.268 INFO:teuthology.orchestra.run.smithi164.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-08T04:34:07.269 INFO:teuthology.orchestra.run.smithi164.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-08T04:34:07.270 INFO:teuthology.orchestra.run.smithi164.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-08T04:34:07.287 INFO:teuthology.orchestra.run.smithi164.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-08T04:34:07.330 INFO:teuthology.orchestra.run.smithi172.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-08T04:34:07.334 INFO:teuthology.orchestra.run.smithi164.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-08T04:34:07.336 INFO:teuthology.orchestra.run.smithi164.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-08T04:34:07.337 INFO:teuthology.orchestra.run.smithi164.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T04:34:07.337 INFO:teuthology.orchestra.run.smithi164.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-08T04:34:07.339 INFO:teuthology.orchestra.run.smithi164.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-08T04:34:07.343 INFO:teuthology.orchestra.run.smithi172.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-08T04:34:07.359 INFO:teuthology.orchestra.run.smithi164.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-08T04:34:07.362 INFO:teuthology.orchestra.run.smithi164.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-08T04:34:07.362 INFO:teuthology.orchestra.run.smithi164.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-08T04:34:07.380 INFO:teuthology.orchestra.run.smithi164.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-08T04:34:07.380 INFO:teuthology.orchestra.run.smithi164.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-08T04:34:07.395 INFO:teuthology.orchestra.run.smithi172.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-08T04:34:07.395 INFO:teuthology.orchestra.run.smithi164.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-08T04:34:07.396 INFO:teuthology.orchestra.run.smithi164.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-08T04:34:07.396 INFO:teuthology.orchestra.run.smithi164.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-08T04:34:07.397 INFO:teuthology.orchestra.run.smithi164.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-08T04:34:07.397 INFO:teuthology.orchestra.run.smithi164.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-08T04:34:07.398 INFO:teuthology.orchestra.run.smithi164.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-08T04:34:07.402 INFO:teuthology.orchestra.run.smithi164.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-08T04:34:07.416 INFO:teuthology.orchestra.run.smithi164.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-08T04:34:07.417 INFO:teuthology.orchestra.run.smithi164.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-08T04:34:07.418 INFO:teuthology.orchestra.run.smithi164.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-08T04:34:07.436 INFO:teuthology.orchestra.run.smithi164.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-08T04:34:07.439 INFO:teuthology.orchestra.run.smithi164.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-08T04:34:07.440 INFO:teuthology.orchestra.run.smithi164.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-08T04:34:07.441 INFO:teuthology.orchestra.run.smithi164.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-08T04:34:07.442 INFO:teuthology.orchestra.run.smithi164.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-08T04:34:07.447 INFO:teuthology.orchestra.run.smithi164.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-08T04:34:07.447 INFO:teuthology.orchestra.run.smithi164.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-08T04:34:07.457 INFO:teuthology.orchestra.run.smithi164.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-08T04:34:07.457 INFO:teuthology.orchestra.run.smithi164.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T04:34:07.458 INFO:teuthology.orchestra.run.smithi164.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-08T04:34:07.478 INFO:teuthology.orchestra.run.smithi164.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-08T04:34:07.479 INFO:teuthology.orchestra.run.smithi164.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-08T04:34:07.481 INFO:teuthology.orchestra.run.smithi164.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-08T04:34:07.488 INFO:teuthology.orchestra.run.smithi164.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-08T04:34:07.488 INFO:teuthology.orchestra.run.smithi164.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-08T04:34:07.489 INFO:teuthology.orchestra.run.smithi164.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-08T04:34:07.493 INFO:teuthology.orchestra.run.smithi164.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-08T04:34:07.497 INFO:teuthology.orchestra.run.smithi172.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1629-g4f7075d1-1jammy [112 kB] 2024-05-08T04:34:07.498 INFO:teuthology.orchestra.run.smithi164.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-08T04:34:07.504 INFO:teuthology.orchestra.run.smithi172.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1629-g4f7075d1-1jammy [466 kB] 2024-05-08T04:34:07.519 INFO:teuthology.orchestra.run.smithi172.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1629-g4f7075d1-1jammy [24.2 MB] 2024-05-08T04:34:07.569 INFO:teuthology.orchestra.run.smithi172.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-08T04:34:07.579 INFO:teuthology.orchestra.run.smithi172.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-08T04:34:07.606 INFO:teuthology.orchestra.run.smithi172.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-08T04:34:07.614 INFO:teuthology.orchestra.run.smithi172.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-08T04:34:07.779 INFO:teuthology.orchestra.run.smithi172.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-08T04:34:08.328 INFO:teuthology.orchestra.run.smithi164.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1629-g4f7075d1-1jammy [3,217 kB] 2024-05-08T04:34:08.359 INFO:teuthology.orchestra.run.smithi172.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-08T04:34:08.375 INFO:teuthology.orchestra.run.smithi172.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-08T04:34:08.391 INFO:teuthology.orchestra.run.smithi172.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-08T04:34:08.399 INFO:teuthology.orchestra.run.smithi172.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-08T04:34:08.399 INFO:teuthology.orchestra.run.smithi172.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-08T04:34:08.416 INFO:teuthology.orchestra.run.smithi172.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-08T04:34:08.442 INFO:teuthology.orchestra.run.smithi172.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1629-g4f7075d1-1jammy [5,076 kB] 2024-05-08T04:34:08.453 INFO:teuthology.orchestra.run.smithi172.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-08T04:34:08.467 INFO:teuthology.orchestra.run.smithi172.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-08T04:34:08.471 INFO:teuthology.orchestra.run.smithi172.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-08T04:34:08.471 INFO:teuthology.orchestra.run.smithi172.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-08T04:34:08.641 INFO:teuthology.orchestra.run.smithi172.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-08T04:34:08.644 INFO:teuthology.orchestra.run.smithi172.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1629-g4f7075d1-1jammy [243 kB] 2024-05-08T04:34:08.651 INFO:teuthology.orchestra.run.smithi172.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1629-g4f7075d1-1jammy [124 kB] 2024-05-08T04:34:08.655 INFO:teuthology.orchestra.run.smithi172.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1629-g4f7075d1-1jammy [1,521 kB] 2024-05-08T04:34:08.716 INFO:teuthology.orchestra.run.smithi172.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1629-g4f7075d1-1jammy [6,236 kB] 2024-05-08T04:34:08.723 INFO:teuthology.orchestra.run.smithi172.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-08T04:34:08.747 INFO:teuthology.orchestra.run.smithi172.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-08T04:34:08.805 INFO:teuthology.orchestra.run.smithi172.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-08T04:34:08.841 INFO:teuthology.orchestra.run.smithi164.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1629-g4f7075d1-1jammy [710 kB] 2024-05-08T04:34:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-08T04:34:08.873 INFO:teuthology.orchestra.run.smithi172.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-08T04:34:08.895 INFO:teuthology.orchestra.run.smithi172.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-08T04:34:08.948 INFO:teuthology.orchestra.run.smithi164.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1629-g4f7075d1-1jammy [356 kB] 2024-05-08T04:34:08.986 INFO:teuthology.orchestra.run.smithi172.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1629-g4f7075d1-1jammy [22.7 MB] 2024-05-08T04:34:09.003 INFO:teuthology.orchestra.run.smithi164.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1629-g4f7075d1-1jammy [32.8 kB] 2024-05-08T04:34:09.004 INFO:teuthology.orchestra.run.smithi164.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1629-g4f7075d1-1jammy [180 kB] 2024-05-08T04:34:09.010 INFO:teuthology.orchestra.run.smithi164.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1629-g4f7075d1-1jammy [64.8 kB] 2024-05-08T04:34:09.013 INFO:teuthology.orchestra.run.smithi164.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1629-g4f7075d1-1jammy [330 kB] 2024-05-08T04:34:09.067 INFO:teuthology.orchestra.run.smithi164.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1629-g4f7075d1-1jammy [6,575 kB] 2024-05-08T04:34:09.068 INFO:teuthology.orchestra.run.smithi172.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-08T04:34:09.084 INFO:teuthology.orchestra.run.smithi172.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-08T04:34:09.090 INFO:teuthology.orchestra.run.smithi172.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-08T04:34:09.096 INFO:teuthology.orchestra.run.smithi172.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-08T04:34:09.281 INFO:teuthology.orchestra.run.smithi172.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T04:34:09.363 INFO:teuthology.orchestra.run.smithi172.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-08T04:34:09.379 INFO:teuthology.orchestra.run.smithi172.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-08T04:34:09.564 INFO:teuthology.orchestra.run.smithi172.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-08T04:34:09.614 INFO:teuthology.orchestra.run.smithi172.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-08T04:34:09.627 INFO:teuthology.orchestra.run.smithi172.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-08T04:34:09.785 INFO:teuthology.orchestra.run.smithi172.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-08T04:34:09.785 INFO:teuthology.orchestra.run.smithi172.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-08T04:34:09.790 INFO:teuthology.orchestra.run.smithi172.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-08T04:34:09.795 INFO:teuthology.orchestra.run.smithi172.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-08T04:34:09.800 INFO:teuthology.orchestra.run.smithi164.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1629-g4f7075d1-1jammy [112 kB] 2024-05-08T04:34:09.801 INFO:teuthology.orchestra.run.smithi164.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1629-g4f7075d1-1jammy [466 kB] 2024-05-08T04:34:09.809 INFO:teuthology.orchestra.run.smithi172.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-08T04:34:09.816 INFO:teuthology.orchestra.run.smithi164.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1629-g4f7075d1-1jammy [24.2 MB] 2024-05-08T04:34:09.997 INFO:teuthology.orchestra.run.smithi172.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-08T04:34:10.107 INFO:teuthology.orchestra.run.smithi172.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1629-g4f7075d1-1jammy [14.1 kB] 2024-05-08T04:34:10.107 INFO:teuthology.orchestra.run.smithi172.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1629-g4f7075d1-1jammy [866 kB] 2024-05-08T04:34:10.148 INFO:teuthology.orchestra.run.smithi172.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1629-g4f7075d1-1jammy [2,191 kB] 2024-05-08T04:34:10.244 INFO:teuthology.orchestra.run.smithi172.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1629-g4f7075d1-1jammy [777 kB] 2024-05-08T04:34:10.260 INFO:teuthology.orchestra.run.smithi172.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-08T04:34:10.269 INFO:teuthology.orchestra.run.smithi172.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1629-g4f7075d1-1jammy [144 kB] 2024-05-08T04:34:10.274 INFO:teuthology.orchestra.run.smithi172.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1629-g4f7075d1-1jammy [1,921 kB] 2024-05-08T04:34:10.381 INFO:teuthology.orchestra.run.smithi172.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1629-g4f7075d1-1jammy [8,625 kB] 2024-05-08T04:34:10.391 INFO:teuthology.orchestra.run.smithi172.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-08T04:34:10.434 INFO:teuthology.orchestra.run.smithi172.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-08T04:34:10.524 INFO:teuthology.orchestra.run.smithi172.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-08T04:34:10.534 INFO:teuthology.orchestra.run.smithi172.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-08T04:34:10.546 INFO:teuthology.orchestra.run.smithi172.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-08T04:34:10.588 INFO:teuthology.orchestra.run.smithi172.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-08T04:34:10.848 INFO:teuthology.orchestra.run.smithi172.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1629-g4f7075d1-1jammy [14.2 kB] 2024-05-08T04:34:10.848 INFO:teuthology.orchestra.run.smithi172.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1629-g4f7075d1-1jammy [49.9 MB] 2024-05-08T04:34:10.850 INFO:teuthology.orchestra.run.smithi172.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-08T04:34:10.851 INFO:teuthology.orchestra.run.smithi172.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-08T04:34:10.853 INFO:teuthology.orchestra.run.smithi172.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T04:34:11.017 INFO:teuthology.orchestra.run.smithi172.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-08T04:34:11.362 INFO:teuthology.orchestra.run.smithi172.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-08T04:34:11.764 INFO:teuthology.orchestra.run.smithi172.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-08T04:34:11.773 INFO:teuthology.orchestra.run.smithi172.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-08T04:34:11.774 INFO:teuthology.orchestra.run.smithi172.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-08T04:34:11.844 INFO:teuthology.orchestra.run.smithi172.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-08T04:34:11.844 INFO:teuthology.orchestra.run.smithi172.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-08T04:34:11.845 INFO:teuthology.orchestra.run.smithi172.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-08T04:34:11.847 INFO:teuthology.orchestra.run.smithi172.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-08T04:34:11.847 INFO:teuthology.orchestra.run.smithi172.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-08T04:34:11.848 INFO:teuthology.orchestra.run.smithi172.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-08T04:34:11.908 INFO:teuthology.orchestra.run.smithi164.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1629-g4f7075d1-1jammy [5,076 kB] 2024-05-08T04:34:12.013 INFO:teuthology.orchestra.run.smithi172.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-08T04:34:12.226 INFO:teuthology.orchestra.run.smithi172.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-08T04:34:12.309 INFO:teuthology.orchestra.run.smithi164.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1629-g4f7075d1-1jammy [243 kB] 2024-05-08T04:34:12.356 INFO:teuthology.orchestra.run.smithi164.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1629-g4f7075d1-1jammy [124 kB] 2024-05-08T04:34:12.360 INFO:teuthology.orchestra.run.smithi164.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1629-g4f7075d1-1jammy [1,521 kB] 2024-05-08T04:34:12.492 INFO:teuthology.orchestra.run.smithi164.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1629-g4f7075d1-1jammy [6,236 kB] 2024-05-08T04:34:12.553 INFO:teuthology.orchestra.run.smithi172.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-08T04:34:12.602 INFO:teuthology.orchestra.run.smithi172.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-08T04:34:12.627 INFO:teuthology.orchestra.run.smithi172.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-08T04:34:12.635 INFO:teuthology.orchestra.run.smithi172.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-08T04:34:12.681 INFO:teuthology.orchestra.run.smithi172.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-08T04:34:12.706 INFO:teuthology.orchestra.run.smithi172.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-08T04:34:12.709 INFO:teuthology.orchestra.run.smithi172.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-08T04:34:12.715 INFO:teuthology.orchestra.run.smithi172.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-08T04:34:12.718 INFO:teuthology.orchestra.run.smithi172.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-08T04:34:12.914 INFO:teuthology.orchestra.run.smithi172.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-08T04:34:12.987 INFO:teuthology.orchestra.run.smithi164.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1629-g4f7075d1-1jammy [22.7 MB] 2024-05-08T04:34:13.173 INFO:teuthology.orchestra.run.smithi172.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-08T04:34:13.436 INFO:teuthology.orchestra.run.smithi172.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-08T04:34:13.438 INFO:teuthology.orchestra.run.smithi172.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T04:34:13.454 INFO:teuthology.orchestra.run.smithi172.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-08T04:34:13.459 INFO:teuthology.orchestra.run.smithi172.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-08T04:34:13.497 INFO:teuthology.orchestra.run.smithi172.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-08T04:34:13.511 INFO:teuthology.orchestra.run.smithi172.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-08T04:34:13.581 INFO:teuthology.orchestra.run.smithi172.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1629-g4f7075d1-1jammy [121 kB] 2024-05-08T04:34:13.582 INFO:teuthology.orchestra.run.smithi172.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1629-g4f7075d1-1jammy [228 kB] 2024-05-08T04:34:13.585 INFO:teuthology.orchestra.run.smithi172.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1629-g4f7075d1-1jammy [34.0 kB] 2024-05-08T04:34:13.585 INFO:teuthology.orchestra.run.smithi172.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1629-g4f7075d1-1jammy [23.4 kB] 2024-05-08T04:34:13.585 INFO:teuthology.orchestra.run.smithi172.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1629-g4f7075d1-1jammy [38.4 kB] 2024-05-08T04:34:13.586 INFO:teuthology.orchestra.run.smithi172.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1629-g4f7075d1-1jammy [13.0 MB] 2024-05-08T04:34:13.601 INFO:teuthology.orchestra.run.smithi172.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-08T04:34:13.603 INFO:teuthology.orchestra.run.smithi172.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-08T04:34:13.605 INFO:teuthology.orchestra.run.smithi172.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-08T04:34:13.786 INFO:teuthology.orchestra.run.smithi172.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-08T04:34:14.061 INFO:teuthology.orchestra.run.smithi172.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-08T04:34:14.301 INFO:teuthology.orchestra.run.smithi172.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1629-g4f7075d1-1jammy [91.5 kB] 2024-05-08T04:34:14.524 INFO:teuthology.orchestra.run.smithi164.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1629-g4f7075d1-1jammy [14.1 kB] 2024-05-08T04:34:14.524 INFO:teuthology.orchestra.run.smithi164.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1629-g4f7075d1-1jammy [866 kB] 2024-05-08T04:34:14.544 INFO:teuthology.orchestra.run.smithi164.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1629-g4f7075d1-1jammy [2,191 kB] 2024-05-08T04:34:14.617 INFO:teuthology.orchestra.run.smithi164.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1629-g4f7075d1-1jammy [777 kB] 2024-05-08T04:34:14.642 INFO:teuthology.orchestra.run.smithi164.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1629-g4f7075d1-1jammy [144 kB] 2024-05-08T04:34:14.646 INFO:teuthology.orchestra.run.smithi164.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1629-g4f7075d1-1jammy [1,921 kB] 2024-05-08T04:34:14.710 INFO:teuthology.orchestra.run.smithi164.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1629-g4f7075d1-1jammy [8,625 kB] 2024-05-08T04:34:14.995 INFO:teuthology.orchestra.run.smithi164.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1629-g4f7075d1-1jammy [14.2 kB] 2024-05-08T04:34:14.996 INFO:teuthology.orchestra.run.smithi164.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1629-g4f7075d1-1jammy [49.9 MB] 2024-05-08T04:34:15.145 INFO:teuthology.orchestra.run.smithi172.stdout:Fetched 176 MB in 8s (22.6 MB/s) 2024-05-08T04:34:15.311 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T04:34:15.378 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-08T04:34:15.381 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-08T04:34:15.435 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-08T04:34:15.763 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T04:34:15.776 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-08T04:34:15.821 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-08T04:34:16.149 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T04:34:16.163 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-08T04:34:16.207 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-08T04:34:16.536 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-08T04:34:16.549 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-08T04:34:16.636 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-08T04:34:16.645 INFO:teuthology.orchestra.run.smithi164.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1629-g4f7075d1-1jammy [121 kB] 2024-05-08T04:34:16.647 INFO:teuthology.orchestra.run.smithi164.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1629-g4f7075d1-1jammy [228 kB] 2024-05-08T04:34:16.649 INFO:teuthology.orchestra.run.smithi164.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1629-g4f7075d1-1jammy [34.0 kB] 2024-05-08T04:34:16.649 INFO:teuthology.orchestra.run.smithi164.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1629-g4f7075d1-1jammy [23.4 kB] 2024-05-08T04:34:16.650 INFO:teuthology.orchestra.run.smithi164.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1629-g4f7075d1-1jammy [38.4 kB] 2024-05-08T04:34:16.650 INFO:teuthology.orchestra.run.smithi164.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1629-g4f7075d1-1jammy [13.0 MB] 2024-05-08T04:34:16.997 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T04:34:17.012 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-08T04:34:17.055 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-08T04:34:17.074 INFO:teuthology.orchestra.run.smithi164.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1629-g4f7075d1-1jammy [91.5 kB] 2024-05-08T04:34:17.401 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T04:34:17.416 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-08T04:34:17.459 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-08T04:34:17.716 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 176 MB in 10s (17.3 MB/s) 2024-05-08T04:34:17.829 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T04:34:17.844 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:17.861 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T04:34:17.929 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:17.929 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-08T04:34:17.932 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-08T04:34:17.977 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-08T04:34:18.246 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T04:34:18.259 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-08T04:34:18.304 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-08T04:34:18.324 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T04:34:18.339 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:18.382 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:18.607 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T04:34:18.621 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-08T04:34:18.665 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-08T04:34:18.702 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T04:34:18.717 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:18.760 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:18.977 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-08T04:34:18.990 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-08T04:34:19.077 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-08T04:34:19.113 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-08T04:34:19.128 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-08T04:34:19.171 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-08T04:34:19.422 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T04:34:19.436 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-08T04:34:19.480 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-08T04:34:19.499 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package librados2. 2024-05-08T04:34:19.514 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../010-librados2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:19.558 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking librados2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:19.825 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T04:34:19.840 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-08T04:34:19.875 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-08T04:34:19.935 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package librbd1. 2024-05-08T04:34:19.950 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../011-librbd1_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:19.985 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking librbd1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:20.279 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T04:34:20.293 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:20.345 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:20.405 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T04:34:20.419 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:20.463 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libcephfs2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:20.732 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T04:34:20.732 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-rados. 2024-05-08T04:34:20.746 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:20.747 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:20.790 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:20.790 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-rados (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:21.085 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T04:34:21.099 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:21.135 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T04:34:21.150 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-08T04:34:21.160 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-ceph-argparse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:21.185 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:34:21.387 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T04:34:21.402 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:21.437 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-cephfs (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:21.504 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-08T04:34:21.519 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-08T04:34:21.563 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-08T04:34:21.732 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T04:34:21.747 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:21.790 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:21.891 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librados2. 2024-05-08T04:34:21.905 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../010-librados2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:21.949 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librados2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:22.144 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T04:34:22.159 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-08T04:34:22.202 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-08T04:34:22.385 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librbd1. 2024-05-08T04:34:22.400 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../011-librbd1_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:22.455 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T04:34:22.460 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librbd1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:22.469 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:22.513 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-rbd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:22.821 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T04:34:22.832 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T04:34:22.835 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:22.847 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-08T04:34:22.871 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:22.890 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-08T04:34:23.141 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rados. 2024-05-08T04:34:23.155 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:23.193 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-08T04:34:23.198 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rados (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:23.208 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-08T04:34:23.251 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-08T04:34:23.527 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T04:34:23.541 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:23.571 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-08T04:34:23.585 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-argparse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:23.586 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-08T04:34:23.629 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-08T04:34:23.854 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T04:34:23.869 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:23.912 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cephfs (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:23.924 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package lua5.1. 2024-05-08T04:34:23.939 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-08T04:34:23.974 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-08T04:34:24.207 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T04:34:24.221 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:24.265 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:24.277 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package lua-any. 2024-05-08T04:34:24.292 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-08T04:34:24.335 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-08T04:34:24.601 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T04:34:24.616 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-08T04:34:24.621 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package zip. 2024-05-08T04:34:24.637 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-08T04:34:24.668 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-08T04:34:24.679 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking zip (3.0-12build2) ... 2024-05-08T04:34:24.904 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T04:34:24.919 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:24.954 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rbd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:25.000 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package unzip. 2024-05-08T04:34:25.015 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-08T04:34:25.058 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-08T04:34:25.273 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T04:34:25.288 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-08T04:34:25.331 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-08T04:34:25.412 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package luarocks. 2024-05-08T04:34:25.427 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-08T04:34:25.470 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-08T04:34:25.609 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-08T04:34:25.624 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-08T04:34:25.667 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-08T04:34:25.838 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package librgw2. 2024-05-08T04:34:25.852 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../027-librgw2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:25.896 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking librgw2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:25.964 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-08T04:34:25.979 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-08T04:34:26.022 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-08T04:34:26.332 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T04:34:26.342 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua5.1. 2024-05-08T04:34:26.346 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:26.356 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-08T04:34:26.382 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-rgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:26.400 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-08T04:34:26.702 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-08T04:34:26.703 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-any. 2024-05-08T04:34:26.716 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-08T04:34:26.718 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-08T04:34:26.752 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-08T04:34:26.753 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-08T04:34:27.006 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package zip. 2024-05-08T04:34:27.020 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-08T04:34:27.029 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T04:34:27.044 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:27.064 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking zip (3.0-12build2) ... 2024-05-08T04:34:27.088 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libradosstriper1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:27.384 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package unzip. 2024-05-08T04:34:27.399 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-08T04:34:27.442 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-08T04:34:27.499 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-common. 2024-05-08T04:34:27.515 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:27.557 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:27.821 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package luarocks. 2024-05-08T04:34:27.836 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-08T04:34:27.871 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-08T04:34:28.261 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librgw2. 2024-05-08T04:34:28.275 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../027-librgw2_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:28.319 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librgw2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:28.571 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-base. 2024-05-08T04:34:28.587 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:28.630 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-base (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:28.989 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T04:34:29.004 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:29.074 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:29.142 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-08T04:34:29.156 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-08T04:34:29.200 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-08T04:34:29.350 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-08T04:34:29.365 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-08T04:34:29.400 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-08T04:34:29.512 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-cheroot. 2024-05-08T04:34:29.526 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-08T04:34:29.661 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T04:34:29.676 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:29.711 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libradosstriper1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:29.837 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-08T04:34:30.098 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-common. 2024-05-08T04:34:30.113 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:30.156 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:30.165 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-08T04:34:30.181 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-08T04:34:30.223 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-08T04:34:30.484 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-08T04:34:30.499 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-08T04:34:30.534 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-08T04:34:30.779 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-08T04:34:30.793 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-08T04:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-08T04:34:31.181 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-tempora. 2024-05-08T04:34:31.196 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-08T04:34:31.236 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-base. 2024-05-08T04:34:31.239 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-08T04:34:31.251 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:31.303 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-base (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:31.542 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-portend. 2024-05-08T04:34:31.557 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-08T04:34:31.601 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-08T04:34:31.854 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-08T04:34:31.868 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-08T04:34:31.903 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-08T04:34:31.907 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-08T04:34:31.921 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-08T04:34:31.965 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-08T04:34:32.215 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T04:34:32.226 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cheroot. 2024-05-08T04:34:32.230 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-08T04:34:32.240 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-08T04:34:32.265 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-08T04:34:32.276 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-08T04:34:32.554 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-08T04:34:32.568 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T04:34:32.569 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-08T04:34:32.583 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-08T04:34:32.604 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-08T04:34:32.626 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-08T04:34:32.890 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-08T04:34:32.905 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-08T04:34:32.921 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T04:34:32.937 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-08T04:34:32.948 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-08T04:34:32.980 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-08T04:34:33.276 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-08T04:34:33.291 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-08T04:34:33.316 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T04:34:33.331 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-08T04:34:33.334 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-08T04:34:33.374 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-08T04:34:33.637 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempora. 2024-05-08T04:34:33.652 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-mako. 2024-05-08T04:34:33.652 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-08T04:34:33.667 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-08T04:34:33.687 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-08T04:34:33.710 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-08T04:34:33.923 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-portend. 2024-05-08T04:34:33.939 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-08T04:34:33.980 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T04:34:33.981 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-08T04:34:33.995 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T04:34:34.030 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T04:34:34.284 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-08T04:34:34.299 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-08T04:34:34.342 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-08T04:34:34.349 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T04:34:34.365 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-08T04:34:34.407 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-08T04:34:34.670 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T04:34:34.686 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-webob. 2024-05-08T04:34:34.686 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-08T04:34:34.701 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-08T04:34:34.737 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-08T04:34:34.744 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-08T04:34:35.082 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T04:34:35.089 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T04:34:35.097 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-08T04:34:35.104 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-08T04:34:35.140 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-08T04:34:35.149 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-08T04:34:35.452 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T04:34:35.458 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T04:34:35.467 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-08T04:34:35.474 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-08T04:34:35.508 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-08T04:34:35.510 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-08T04:34:35.770 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-paste. 2024-05-08T04:34:35.785 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-08T04:34:35.813 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T04:34:35.828 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-08T04:34:35.828 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-08T04:34:35.888 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-08T04:34:36.131 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T04:34:36.141 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-mako. 2024-05-08T04:34:36.146 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-08T04:34:36.156 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-08T04:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-08T04:34:36.190 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-08T04:34:36.492 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T04:34:36.502 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T04:34:36.507 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-08T04:34:36.518 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T04:34:36.550 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-08T04:34:36.560 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T04:34:36.870 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T04:34:36.880 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T04:34:36.886 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-08T04:34:36.895 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-08T04:34:36.928 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-08T04:34:36.938 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-08T04:34:37.240 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T04:34:37.255 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-08T04:34:37.257 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webob. 2024-05-08T04:34:37.273 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-08T04:34:37.298 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-08T04:34:37.316 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-08T04:34:37.635 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T04:34:37.650 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-08T04:34:37.678 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-certifi. 2024-05-08T04:34:37.688 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-08T04:34:37.693 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-08T04:34:37.736 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-08T04:34:37.988 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T04:34:38.004 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-08T04:34:38.014 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-idna. 2024-05-08T04:34:38.030 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-08T04:34:38.038 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-08T04:34:38.064 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-08T04:34:38.341 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-paste. 2024-05-08T04:34:38.357 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-08T04:34:38.366 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-urllib3. 2024-05-08T04:34:38.382 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-08T04:34:38.399 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-08T04:34:38.450 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-08T04:34:38.719 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T04:34:38.733 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-08T04:34:38.769 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-08T04:34:39.047 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T04:34:39.064 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-08T04:34:39.105 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-08T04:34:39.408 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T04:34:39.423 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-08T04:34:39.466 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-08T04:34:39.470 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-requests. 2024-05-08T04:34:39.485 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-08T04:34:39.520 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-08T04:34:39.840 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T04:34:39.855 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-08T04:34:39.898 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-08T04:34:40.326 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T04:34:40.341 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:40.384 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:40.486 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T04:34:40.501 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-08T04:34:40.544 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-08T04:34:40.764 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T04:34:40.779 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:40.814 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:40.924 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-certifi. 2024-05-08T04:34:40.939 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-08T04:34:40.974 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-08T04:34:41.092 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T04:34:41.107 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-08T04:34:41.150 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-08T04:34:41.252 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-idna. 2024-05-08T04:34:41.267 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-08T04:34:41.302 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-08T04:34:41.461 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T04:34:41.477 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-08T04:34:41.520 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-08T04:34:41.638 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-urllib3. 2024-05-08T04:34:41.653 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-08T04:34:41.704 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-08T04:34:41.881 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T04:34:41.896 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:41.939 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:42.024 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests. 2024-05-08T04:34:42.039 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-08T04:34:42.082 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-08T04:34:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T04:34:42.299 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:42.334 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mon (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T04:34:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-08T04:34:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-08T04:34:42.738 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T04:34:42.753 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:42.796 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:42.896 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T04:34:42.910 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:42.979 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-osd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:43.184 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T04:34:43.200 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:43.234 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:43.512 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T04:34:43.528 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-08T04:34:43.570 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-08T04:34:43.707 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph. 2024-05-08T04:34:43.722 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../068-ceph_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:43.765 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:43.882 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T04:34:43.897 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-08T04:34:43.940 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-08T04:34:44.076 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T04:34:44.092 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:44.126 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:44.259 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T04:34:44.275 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:44.343 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:44.488 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T04:34:44.503 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:44.546 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mds (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:44.696 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T04:34:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:44.745 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mon (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:44.932 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package cephadm. 2024-05-08T04:34:44.948 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../071-cephadm_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:44.990 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:45.232 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T04:34:45.246 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:45.282 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-osd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:45.327 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T04:34:45.342 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-08T04:34:45.376 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-08T04:34:45.671 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-08T04:34:45.687 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-08T04:34:45.729 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-08T04:34:46.027 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph. 2024-05-08T04:34:46.041 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../068-ceph_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:46.085 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:46.363 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T04:34:46.377 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:46.412 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:46.717 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-babel. 2024-05-08T04:34:46.731 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-08T04:34:46.748 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T04:34:46.762 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:46.775 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-08T04:34:46.799 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mds (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:47.070 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T04:34:47.084 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-08T04:34:47.128 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-08T04:34:47.143 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephadm. 2024-05-08T04:34:47.157 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../071-cephadm_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:47.201 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:47.498 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T04:34:47.511 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:47.520 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T04:34:47.535 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-08T04:34:47.556 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:47.579 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-08T04:34:47.865 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-08T04:34:47.880 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-08T04:34:47.901 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T04:34:47.914 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T04:34:47.923 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-08T04:34:47.959 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T04:34:48.270 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-routes. 2024-05-08T04:34:48.284 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-08T04:34:48.329 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-08T04:34:48.690 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T04:34:48.704 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:48.749 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:48.953 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-babel. 2024-05-08T04:34:48.962 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-08T04:34:49.028 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-08T04:34:49.160 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-08T04:34:49.174 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-08T04:34:49.218 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-08T04:34:49.356 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T04:34:49.363 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-08T04:34:49.405 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-08T04:34:49.747 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T04:34:49.761 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-08T04:34:49.775 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T04:34:49.784 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:49.805 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-08T04:34:49.825 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:50.142 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-08T04:34:50.156 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-08T04:34:50.170 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T04:34:50.179 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T04:34:50.220 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T04:34:50.277 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-08T04:34:50.498 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-routes. 2024-05-08T04:34:50.509 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-08T04:34:50.547 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-08T04:34:50.605 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T04:34:50.619 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-08T04:34:50.663 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-08T04:34:50.809 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T04:34:50.819 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:50.850 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:51.287 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-08T04:34:51.296 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-08T04:34:51.337 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-08T04:34:51.348 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T04:34:51.360 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:51.397 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:51.807 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T04:34:51.819 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-08T04:34:51.857 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-08T04:34:52.218 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-08T04:34:52.228 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-08T04:34:52.370 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-08T04:34:52.409 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T04:34:52.426 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-08T04:34:52.467 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-08T04:34:52.673 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T04:34:52.683 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-08T04:34:52.712 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T04:34:52.723 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-08T04:34:52.726 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-08T04:34:52.786 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-08T04:34:53.040 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T04:34:53.055 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-08T04:34:53.098 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-08T04:34:53.382 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T04:34:53.392 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:53.393 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T04:34:53.407 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-08T04:34:53.432 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:53.451 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-08T04:34:53.772 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T04:34:53.786 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-08T04:34:53.863 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-08T04:34:54.183 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T04:34:54.198 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-08T04:34:54.241 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-08T04:34:54.529 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T04:34:54.542 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-08T04:34:54.568 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T04:34:54.577 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-08T04:34:54.579 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-08T04:34:54.618 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-08T04:34:54.906 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T04:34:54.921 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-08T04:34:54.946 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T04:34:54.960 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-08T04:34:54.964 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-08T04:34:55.029 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-08T04:34:55.366 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T04:34:55.378 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-08T04:34:55.416 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-08T04:34:55.620 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T04:34:55.634 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:55.678 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:55.727 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T04:34:55.739 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-08T04:34:55.777 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-08T04:34:56.039 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T04:34:56.055 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-08T04:34:56.098 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-08T04:34:56.131 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T04:34:56.142 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-08T04:34:56.189 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-08T04:34:56.434 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T04:34:56.449 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-08T04:34:56.492 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-08T04:34:56.609 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T04:34:56.621 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-08T04:34:56.667 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-08T04:34:56.695 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package jq. 2024-05-08T04:34:56.710 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-08T04:34:56.745 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-08T04:34:56.998 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package socat. 2024-05-08T04:34:57.013 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-08T04:34:57.045 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T04:34:57.056 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-08T04:34:57.057 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-08T04:34:57.103 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-08T04:34:57.392 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T04:34:57.407 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T04:34:57.408 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-08T04:34:57.421 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-08T04:34:57.450 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-08T04:34:57.456 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-08T04:34:57.805 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-test. 2024-05-08T04:34:57.819 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:34:57.863 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-test (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T04:34:58.132 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:58.170 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:34:58.473 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T04:34:58.484 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-08T04:34:58.523 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-08T04:34:58.826 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T04:34:58.836 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-08T04:34:58.876 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-08T04:34:59.136 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package jq. 2024-05-08T04:34:59.147 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-08T04:34:59.186 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-08T04:34:59.623 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package socat. 2024-05-08T04:34:59.637 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-08T04:34:59.681 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-08T04:34:59.742 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T04:34:59.758 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:34:59.892 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ceph-volume (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:00.084 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T04:35:00.097 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-08T04:35:00.134 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-08T04:35:00.263 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T04:35:00.277 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:00.321 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking cephfs-mirror (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:00.521 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-test. 2024-05-08T04:35:00.533 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:00.571 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-test (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:00.674 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T04:35:00.689 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-08T04:35:00.732 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-08T04:35:00.968 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T04:35:00.983 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:35:01.018 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking cephfs-shell (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:01.254 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T04:35:01.268 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:35:01.312 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking cephfs-top (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:01.565 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T04:35:01.578 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:01.640 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libcephfs-dev (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:01.959 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-08T04:35:01.972 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-08T04:35:02.017 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-08T04:35:02.339 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-08T04:35:02.352 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-08T04:35:02.388 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-08T04:35:02.484 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T04:35:02.497 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:35:02.542 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-volume (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:02.768 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T04:35:02.783 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-08T04:35:02.826 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-08T04:35:02.929 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T04:35:02.942 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:02.979 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-mirror (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:03.233 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package pkg-config. 2024-05-08T04:35:03.246 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-08T04:35:03.284 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T04:35:03.291 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-08T04:35:03.297 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-08T04:35:03.325 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-08T04:35:03.552 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T04:35:03.566 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-08T04:35:03.569 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T04:35:03.582 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:35:03.602 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-08T04:35:03.619 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-shell (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:03.864 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T04:35:03.876 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1629-g4f7075d1-1jammy_all.deb ... 2024-05-08T04:35:03.922 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-top (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:04.013 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-08T04:35:04.027 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-08T04:35:04.072 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-08T04:35:04.174 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T04:35:04.187 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:04.232 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs-dev (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:04.350 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T04:35:04.363 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T04:35:04.399 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T04:35:04.568 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-08T04:35:04.581 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-08T04:35:04.618 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-08T04:35:04.711 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T04:35:04.725 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-08T04:35:04.769 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-08T04:35:04.914 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-08T04:35:04.927 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-08T04:35:04.955 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-08T04:35:05.089 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T04:35:05.103 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-08T04:35:05.155 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-08T04:35:05.385 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T04:35:05.397 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-08T04:35:05.443 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-08T04:35:05.475 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-py. 2024-05-08T04:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-08T04:35:05.533 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-08T04:35:05.845 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pygments. 2024-05-08T04:35:05.859 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-08T04:35:05.911 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-08T04:35:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package pkg-config. 2024-05-08T04:35:05.961 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-08T04:35:06.006 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-08T04:35:06.309 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T04:35:06.322 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-08T04:35:06.367 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-08T04:35:06.374 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T04:35:06.386 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-08T04:35:06.440 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-08T04:35:06.743 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-toml. 2024-05-08T04:35:06.745 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-08T04:35:06.757 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-08T04:35:06.757 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-08T04:35:06.786 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-08T04:35:06.801 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-08T04:35:07.129 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T04:35:07.143 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-08T04:35:07.172 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T04:35:07.185 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T04:35:07.187 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-08T04:35:07.222 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T04:35:07.532 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T04:35:07.534 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T04:35:07.546 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-08T04:35:07.547 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-08T04:35:07.584 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-08T04:35:07.590 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-08T04:35:07.860 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T04:35:07.874 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-08T04:35:07.895 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T04:35:07.908 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-08T04:35:07.918 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-08T04:35:07.945 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-08T04:35:08.273 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-py. 2024-05-08T04:35:08.285 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-08T04:35:08.322 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-08T04:35:08.664 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package radosgw. 2024-05-08T04:35:08.676 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../122-radosgw_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:08.701 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pygments. 2024-05-08T04:35:08.713 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking radosgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:08.714 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-08T04:35:08.751 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-08T04:35:09.163 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T04:35:09.175 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-08T04:35:09.213 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-08T04:35:09.300 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T04:35:09.313 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:09.341 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking rbd-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:09.524 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-toml. 2024-05-08T04:35:09.536 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-08T04:35:09.573 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-08T04:35:09.634 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-08T04:35:09.888 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-08T04:35:09.909 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T04:35:09.921 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-08T04:35:09.959 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-08T04:35:10.014 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-08T04:35:10.271 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T04:35:10.277 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-08T04:35:10.283 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-08T04:35:10.321 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-08T04:35:10.590 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T04:35:10.602 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-08T04:35:10.639 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-08T04:35:10.679 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-08T04:35:11.344 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package radosgw. 2024-05-08T04:35:11.356 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../122-radosgw_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:11.356 INFO:teuthology.orchestra.run.smithi172.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-08T04:35:11.370 INFO:teuthology.orchestra.run.smithi172.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-08T04:35:11.393 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking radosgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:11.416 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:11.909 INFO:teuthology.orchestra.run.smithi172.stdout:Adding system user cephadm....done 2024-05-08T04:35:11.963 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T04:35:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1629-g4f7075d1-1jammy_amd64.deb ... 2024-05-08T04:35:12.004 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking rbd-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:12.063 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-08T04:35:12.271 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-08T04:35:12.348 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-08T04:35:12.543 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-08T04:35:12.649 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-08T04:35:12.669 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-08T04:35:12.775 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-08T04:35:12.931 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-08T04:35:13.035 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T04:35:13.302 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-08T04:35:13.323 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-08T04:35:13.412 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-08T04:35:13.747 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-08T04:35:13.922 INFO:teuthology.orchestra.run.smithi164.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-08T04:35:13.966 INFO:teuthology.orchestra.run.smithi164.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-08T04:35:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:14.124 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-08T04:35:14.383 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-08T04:35:14.668 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-08T04:35:14.782 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user cephadm....done 2024-05-08T04:35:14.944 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-08T04:35:14.952 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-08T04:35:15.204 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-08T04:35:15.236 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-08T04:35:15.472 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-08T04:35:15.496 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-08T04:35:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-08T04:35:15.748 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-08T04:35:15.890 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T04:35:16.016 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-ceph-argparse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:16.157 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-08T04:35:16.275 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-08T04:35:16.292 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-08T04:35:16.376 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-08T04:35:16.510 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-08T04:35:16.594 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-08T04:35:16.770 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-08T04:35:16.870 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-08T04:35:16.979 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-08T04:35:17.147 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-08T04:35:17.247 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-08T04:35:17.515 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-08T04:35:17.539 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-08T04:35:17.569 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-08T04:35:17.633 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-08T04:35:17.759 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-08T04:35:17.799 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-08T04:35:18.092 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-08T04:35:18.127 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-08T04:35:18.253 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-08T04:35:18.401 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-08T04:35:18.521 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-08T04:35:18.661 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-08T04:35:18.797 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-08T04:35:18.903 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-argparse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:19.115 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up zip (3.0-12build2) ... 2024-05-08T04:35:19.179 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-08T04:35:19.266 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-08T04:35:19.306 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-08T04:35:19.431 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-08T04:35:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-08T04:35:19.800 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-08T04:35:19.959 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-08T04:35:20.260 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-08T04:35:20.386 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-08T04:35:20.409 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-08T04:35:20.512 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-08T04:35:20.794 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-08T04:35:20.823 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-08T04:35:20.830 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-08T04:35:20.887 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-08T04:35:21.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-08T04:35:21.231 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-08T04:35:21.381 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-08T04:35:21.507 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-08T04:35:21.533 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-08T04:35:21.784 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-08T04:35:21.917 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-08T04:35:22.043 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-08T04:35:22.227 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-08T04:35:22.335 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up zip (3.0-12build2) ... 2024-05-08T04:35:22.461 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-08T04:35:22.545 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-08T04:35:22.888 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-08T04:35:23.155 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-08T04:35:23.165 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-08T04:35:23.290 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:23.422 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-08T04:35:23.548 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-08T04:35:23.583 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-idna (3.3-1) ... 2024-05-08T04:35:23.683 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-08T04:35:23.925 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-08T04:35:23.992 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-08T04:35:24.368 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-08T04:35:24.628 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-08T04:35:25.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-08T04:35:25.148 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-08T04:35:25.314 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-08T04:35:25.374 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:25.484 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-08T04:35:25.665 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-08T04:35:25.752 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-08T04:35:25.878 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-08T04:35:26.004 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-08T04:35:26.024 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-08T04:35:26.267 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-08T04:35:26.305 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-08T04:35:26.368 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:26.590 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-08T04:35:26.653 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-idna (3.3-1) ... 2024-05-08T04:35:26.841 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:26.966 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-08T04:35:27.012 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-08T04:35:27.285 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-08T04:35:27.544 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-08T04:35:27.812 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-08T04:35:28.122 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T04:35:28.235 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-08T04:35:28.389 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-08T04:35:28.478 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:28.612 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-08T04:35:28.673 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-08T04:35:28.799 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-08T04:35:28.880 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-08T04:35:29.005 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-08T04:35:29.092 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-08T04:35:29.131 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-08T04:35:29.235 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-08T04:35:29.416 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-08T04:35:29.527 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-08T04:35:29.701 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-08T04:35:29.820 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-08T04:35:29.944 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:30.069 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-08T04:35:30.088 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-08T04:35:30.198 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-08T04:35:30.307 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-08T04:35:30.396 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-08T04:35:30.554 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-08T04:35:30.633 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-08T04:35:30.759 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-08T04:35:30.813 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-08T04:35:31.081 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-08T04:35:31.169 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-08T04:35:31.350 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T04:35:31.437 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-08T04:35:31.571 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-08T04:35:31.633 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-08T04:35:31.831 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:31.892 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-08T04:35:31.932 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-08T04:35:32.002 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-08T04:35:32.269 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-08T04:35:32.358 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-08T04:35:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-08T04:35:32.667 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-08T04:35:32.721 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-08T04:35:32.952 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-08T04:35:33.039 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-08T04:35:33.163 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-08T04:35:33.307 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-08T04:35:33.424 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-08T04:35:33.433 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-08T04:35:33.542 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-08T04:35:33.614 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-08T04:35:33.691 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-08T04:35:33.852 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-08T04:35:33.967 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-08T04:35:33.978 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-08T04:35:34.352 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T04:35:34.380 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-08T04:35:34.647 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-08T04:35:34.678 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-08T04:35:34.773 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-08T04:35:35.031 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-08T04:35:35.041 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-08T04:35:35.132 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up librados2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:35.142 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-08T04:35:35.258 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:35.384 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-08T04:35:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-08T04:35:35.737 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libcephfs2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:35.744 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-08T04:35:35.863 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libradosstriper1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:35.973 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-08T04:35:36.020 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-08T04:35:36.223 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-08T04:35:36.249 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up librbd1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:36.375 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr-modules-core (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:36.501 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:36.516 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-08T04:35:36.746 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T04:35:36.746 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T04:35:36.767 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-08T04:35:37.018 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-08T04:35:37.404 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T04:35:37.410 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libcephfs-dev (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:37.536 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-rados (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:37.682 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up librgw2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:37.722 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-08T04:35:37.788 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up cephfs-top (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:37.898 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-08T04:35:38.074 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-08T04:35:38.192 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librados2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:38.293 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:38.394 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-08T04:35:38.722 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:38.847 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libradosstriper1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:38.974 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-08T04:35:39.160 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-rbd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.286 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up rbd-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.292 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librbd1 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.409 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-modules-core (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.412 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-rgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.535 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.537 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up python3-cephfs (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.664 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up cephfs-shell (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:39.743 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T04:35:39.743 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T04:35:39.765 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:40.326 INFO:teuthology.orchestra.run.smithi172.stdout:Adding group ceph....done 2024-05-08T04:35:40.370 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs-dev (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:40.488 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rados (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:40.597 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librgw2 (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:40.723 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-top (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:40.727 INFO:teuthology.orchestra.run.smithi172.stdout:Adding system user ceph....done 2024-05-08T04:35:40.841 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-08T04:35:40.854 INFO:teuthology.orchestra.run.smithi172.stdout:Setting system user ceph properties....done 2024-05-08T04:35:40.872 INFO:teuthology.orchestra.run.smithi172.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T04:35:41.015 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T04:35:41.442 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T04:35:42.094 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rbd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.124 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up cephfs-mirror (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up rbd-fuse (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.333 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T04:35:42.333 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T04:35:42.338 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cephfs (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.539 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-shell (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.657 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-common (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:42.975 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-test (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:43.143 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up radosgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:43.202 INFO:teuthology.orchestra.run.smithi164.stdout:Adding group ceph....done 2024-05-08T04:35:43.611 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user ceph....done 2024-05-08T04:35:43.701 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:35:43.701 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:35:43.780 INFO:teuthology.orchestra.run.smithi164.stdout:Setting system user ceph properties....done 2024-05-08T04:35:43.798 INFO:teuthology.orchestra.run.smithi164.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T04:35:43.940 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T04:35:44.335 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T04:35:44.378 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-base (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:44.723 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T04:35:45.075 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-mirror (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:45.283 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T04:35:45.284 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T04:35:45.380 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mds (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:45.611 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T04:35:45.611 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T04:35:45.884 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-test (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:46.010 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up radosgw (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:46.273 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:46.518 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T04:35:46.519 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T04:35:46.534 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:35:46.534 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:35:47.137 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-base (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:47.167 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-osd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:47.481 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T04:35:47.481 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T04:35:47.498 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T04:35:48.138 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mds (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:48.143 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:48.269 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:48.345 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T04:35:48.346 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T04:35:48.412 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mon (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:48.613 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T04:35:48.613 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T04:35:48.949 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:49.145 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T04:35:49.146 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T04:35:49.255 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr-cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:49.372 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:49.498 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-mgr-dashboard (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:49.658 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ceph-volume (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:49.741 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-osd (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:50.010 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-08T04:35:50.053 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T04:35:50.053 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T04:35:50.127 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-08T04:35:50.271 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-08T04:35:50.651 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:50.777 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:51.366 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mon (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:51.406 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-08T04:35:51.574 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T04:35:51.574 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T04:35:52.099 INFO:teuthology.orchestra.run.smithi172.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T04:35:52.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T04:35:52.125 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-cephadm (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:52.184 INFO:teuthology.orchestra.run.smithi172.stdout:Reading package lists... 2024-05-08T04:35:52.268 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:52.360 INFO:teuthology.orchestra.run.smithi172.stdout:Building dependency tree... 2024-05-08T04:35:52.361 INFO:teuthology.orchestra.run.smithi172.stdout:Reading state information... 2024-05-08T04:35:52.394 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-dashboard (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:52.538 INFO:teuthology.orchestra.run.smithi172.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-08T04:35:52.539 INFO:teuthology.orchestra.run.smithi172.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-08T04:35:52.539 INFO:teuthology.orchestra.run.smithi172.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-08T04:35:52.539 INFO:teuthology.orchestra.run.smithi172.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T04:35:52.539 INFO:teuthology.orchestra.run.smithi172.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-08T04:35:52.539 INFO:teuthology.orchestra.run.smithi172.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-08T04:35:52.540 INFO:teuthology.orchestra.run.smithi172.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-08T04:35:52.540 INFO:teuthology.orchestra.run.smithi172.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-08T04:35:52.540 INFO:teuthology.orchestra.run.smithi172.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-08T04:35:52.540 INFO:teuthology.orchestra.run.smithi172.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T04:35:52.542 INFO:teuthology.orchestra.run.smithi172.stdout:The following additional packages will be installed: 2024-05-08T04:35:52.542 INFO:teuthology.orchestra.run.smithi172.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-08T04:35:52.542 INFO:teuthology.orchestra.run.smithi172.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-08T04:35:52.542 INFO:teuthology.orchestra.run.smithi172.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout:Suggested packages: 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-08T04:35:52.543 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-08T04:35:52.561 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-volume (19.0.0-1629-g4f7075d1-1jammy) ... 2024-05-08T04:35:52.595 INFO:teuthology.orchestra.run.smithi172.stdout:The following NEW packages will be installed: 2024-05-08T04:35:52.596 INFO:teuthology.orchestra.run.smithi172.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-08T04:35:52.596 INFO:teuthology.orchestra.run.smithi172.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-08T04:35:52.596 INFO:teuthology.orchestra.run.smithi172.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-08T04:35:52.597 INFO:teuthology.orchestra.run.smithi172.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-08T04:35:52.597 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-08T04:35:52.597 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-08T04:35:52.597 INFO:teuthology.orchestra.run.smithi172.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T04:35:52.598 INFO:teuthology.orchestra.run.smithi172.stdout:The following packages will be upgraded: 2024-05-08T04:35:52.598 INFO:teuthology.orchestra.run.smithi172.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-08T04:35:52.810 INFO:teuthology.orchestra.run.smithi172.stdout:4 upgraded, 39 newly installed, 0 to remove and 302 not upgraded. 2024-05-08T04:35:52.811 INFO:teuthology.orchestra.run.smithi172.stdout:Need to get 55.7 MB of archives. 2024-05-08T04:35:52.811 INFO:teuthology.orchestra.run.smithi172.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-08T04:35:52.811 INFO:teuthology.orchestra.run.smithi172.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-08T04:35:52.922 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-08T04:35:53.023 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-08T04:35:53.166 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-08T04:35:53.465 INFO:teuthology.orchestra.run.smithi172.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-08T04:35:53.483 INFO:teuthology.orchestra.run.smithi172.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-08T04:35:53.609 INFO:teuthology.orchestra.run.smithi172.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-08T04:35:53.609 INFO:teuthology.orchestra.run.smithi172.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-08T04:35:53.611 INFO:teuthology.orchestra.run.smithi172.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-08T04:35:53.643 INFO:teuthology.orchestra.run.smithi172.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-08T04:35:53.646 INFO:teuthology.orchestra.run.smithi172.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-08T04:35:53.650 INFO:teuthology.orchestra.run.smithi172.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-08T04:35:53.654 INFO:teuthology.orchestra.run.smithi172.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-08T04:35:53.655 INFO:teuthology.orchestra.run.smithi172.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-08T04:35:53.822 INFO:teuthology.orchestra.run.smithi172.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-08T04:35:54.117 INFO:teuthology.orchestra.run.smithi172.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-08T04:35:54.127 INFO:teuthology.orchestra.run.smithi172.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-08T04:35:54.132 INFO:teuthology.orchestra.run.smithi172.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-08T04:35:54.221 INFO:teuthology.orchestra.run.smithi172.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-08T04:35:54.294 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-08T04:35:54.970 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T04:35:54.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T04:35:55.049 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-05-08T04:35:55.181 INFO:teuthology.orchestra.run.smithi172.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-08T04:35:55.203 INFO:teuthology.orchestra.run.smithi172.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-08T04:35:55.203 INFO:teuthology.orchestra.run.smithi172.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-08T04:35:55.206 INFO:teuthology.orchestra.run.smithi172.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-08T04:35:55.206 INFO:teuthology.orchestra.run.smithi172.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-08T04:35:55.211 INFO:teuthology.orchestra.run.smithi172.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-08T04:35:55.228 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-05-08T04:35:55.228 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-05-08T04:35:55.375 INFO:teuthology.orchestra.run.smithi172.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-08T04:35:55.400 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-08T04:35:55.401 INFO:teuthology.orchestra.run.smithi164.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-08T04:35:55.401 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T04:35:55.401 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-05-08T04:35:55.401 INFO:teuthology.orchestra.run.smithi164.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-08T04:35:55.401 INFO:teuthology.orchestra.run.smithi164.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-08T04:35:55.402 INFO:teuthology.orchestra.run.smithi164.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-08T04:35:55.402 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-08T04:35:55.402 INFO:teuthology.orchestra.run.smithi164.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T04:35:55.403 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-05-08T04:35:55.403 INFO:teuthology.orchestra.run.smithi164.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-08T04:35:55.404 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-08T04:35:55.404 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-08T04:35:55.457 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-05-08T04:35:55.458 INFO:teuthology.orchestra.run.smithi164.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-08T04:35:55.458 INFO:teuthology.orchestra.run.smithi164.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-08T04:35:55.459 INFO:teuthology.orchestra.run.smithi164.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-08T04:35:55.459 INFO:teuthology.orchestra.run.smithi164.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-08T04:35:55.459 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-08T04:35:55.459 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-08T04:35:55.459 INFO:teuthology.orchestra.run.smithi164.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T04:35:55.460 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-05-08T04:35:55.461 INFO:teuthology.orchestra.run.smithi164.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-08T04:35:55.523 INFO:teuthology.orchestra.run.smithi164.stdout:4 upgraded, 39 newly installed, 0 to remove and 302 not upgraded. 2024-05-08T04:35:55.523 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 55.7 MB of archives. 2024-05-08T04:35:55.523 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-08T04:35:55.523 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-08T04:35:55.681 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-08T04:35:55.685 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-08T04:35:55.718 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-08T04:35:55.719 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-08T04:35:55.719 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-08T04:35:55.768 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-08T04:35:55.768 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-08T04:35:55.770 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-08T04:35:55.771 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-08T04:35:55.771 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-08T04:35:55.785 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-08T04:35:55.786 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-08T04:35:55.787 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-08T04:35:55.787 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-08T04:35:55.790 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-08T04:35:55.999 INFO:teuthology.orchestra.run.smithi172.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-08T04:35:55.999 INFO:teuthology.orchestra.run.smithi172.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-08T04:35:56.028 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-08T04:35:56.030 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-08T04:35:56.030 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-08T04:35:56.033 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-08T04:35:56.034 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-08T04:35:56.039 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-08T04:35:56.040 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-08T04:35:56.048 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-08T04:35:56.049 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-08T04:35:56.070 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-08T04:35:56.070 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-08T04:35:56.081 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-08T04:35:56.103 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-08T04:35:56.107 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-08T04:35:56.108 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-08T04:35:56.109 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-08T04:35:56.109 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-08T04:35:56.121 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-08T04:35:56.123 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-08T04:35:56.181 INFO:teuthology.orchestra.run.smithi172.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-08T04:35:56.181 INFO:teuthology.orchestra.run.smithi172.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-08T04:35:56.219 INFO:teuthology.orchestra.run.smithi172.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-08T04:35:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-08T04:35:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-08T04:35:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-08T04:35:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-08T04:35:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-08T04:35:56.283 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-08T04:35:56.283 INFO:teuthology.orchestra.run.smithi172.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-08T04:35:56.288 INFO:teuthology.orchestra.run.smithi172.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-08T04:35:56.288 INFO:teuthology.orchestra.run.smithi172.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-08T04:35:56.290 INFO:teuthology.orchestra.run.smithi172.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-08T04:35:56.290 INFO:teuthology.orchestra.run.smithi172.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-08T04:35:56.295 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-08T04:35:56.296 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-08T04:35:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-08T04:35:56.673 INFO:teuthology.orchestra.run.smithi164.stdout:Preconfiguring packages ... 2024-05-08T04:35:56.819 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 55.7 MB in 1s (67.4 MB/s) 2024-05-08T04:35:56.937 INFO:teuthology.orchestra.run.smithi172.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-08T04:35:57.105 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-05-08T04:35:57.109 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:35:57.252 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:35:57.879 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-08T04:35:58.929 INFO:teuthology.orchestra.run.smithi164.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-08T04:35:59.197 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package flex. 2024-05-08T04:35:59.268 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-05-08T04:35:59.272 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-08T04:35:59.321 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-08T04:35:59.815 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-08T04:35:59.993 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-08T04:36:00.441 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:36:01.014 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-05-08T04:36:01.017 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:36:01.143 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:36:01.521 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:36:01.657 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:36:01.987 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-08T04:36:02.390 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package bison. 2024-05-08T04:36:02.461 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-05-08T04:36:02.464 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-08T04:36:02.510 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-08T04:36:02.910 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T04:36:02.924 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-08T04:36:02.968 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-08T04:36:03.296 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package dump. 2024-05-08T04:36:03.310 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-08T04:36:03.354 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking dump (0.4b47-3) ... 2024-05-08T04:36:03.608 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package indent. 2024-05-08T04:36:03.622 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:03.667 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-08T04:36:03.919 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T04:36:03.933 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-08T04:36:03.978 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-08T04:36:04.281 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T04:36:04.296 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-08T04:36:04.331 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-08T04:36:04.575 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T04:36:04.591 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-08T04:36:04.633 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-08T04:36:04.970 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-08T04:36:04.986 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-08T04:36:05.029 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:05.281 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-08T04:36:05.297 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-08T04:36:05.339 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:05.559 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjson-perl. 2024-05-08T04:36:05.574 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-08T04:36:05.608 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-08T04:36:05.912 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-08T04:36:05.927 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-08T04:36:05.970 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-08T04:36:07.314 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T04:36:07.330 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T04:36:07.373 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:07.625 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T04:36:07.640 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T04:36:07.675 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:07.944 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T04:36:07.960 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-08T04:36:08.002 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:08.322 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T04:36:08.338 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-08T04:36:08.380 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-08T04:36:08.708 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T04:36:08.724 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-08T04:36:08.758 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-08T04:36:09.094 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T04:36:09.110 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:09.152 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:09.505 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T04:36:09.521 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:09.563 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:09.866 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T04:36:09.881 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:09.924 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:10.144 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T04:36:10.160 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-08T04:36:10.193 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:36:10.689 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T04:36:10.704 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-08T04:36:10.756 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-08T04:36:11.083 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package modemmanager. 2024-05-08T04:36:11.099 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-08T04:36:11.167 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:11.790 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package network-manager. 2024-05-08T04:36:11.805 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-08T04:36:11.856 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-08T04:36:12.455 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ppp. 2024-05-08T04:36:12.470 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-08T04:36:12.826 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-08T04:36:13.135 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T04:36:13.149 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-08T04:36:13.202 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-08T04:36:13.454 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T04:36:13.469 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-08T04:36:13.512 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-08T04:36:13.823 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T04:36:13.838 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-08T04:36:13.882 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client-common (238) ... 2024-05-08T04:36:14.160 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-08T04:36:14.175 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:14.210 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:14.790 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T04:36:14.804 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-08T04:36:14.865 INFO:teuthology.orchestra.run.smithi164.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T04:36:14.888 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-common (238) ... 2024-05-08T04:36:15.355 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-14. 2024-05-08T04:36:15.369 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:15.416 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:16.566 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql. 2024-05-08T04:36:16.581 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-08T04:36:16.624 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql (14+238) ... 2024-05-08T04:36:16.868 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T04:36:16.883 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-08T04:36:16.918 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client (14+238) ... 2024-05-08T04:36:17.095 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T04:36:17.110 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-08T04:36:17.154 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-08T04:36:17.473 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package tcl8.6. 2024-05-08T04:36:17.489 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-08T04:36:17.531 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-08T04:36:17.850 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package tcl. 2024-05-08T04:36:17.866 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-08T04:36:17.913 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-08T04:36:18.232 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T04:36:18.247 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-08T04:36:18.278 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-08T04:36:18.624 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T04:36:18.640 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-08T04:36:18.673 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-08T04:36:19.035 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T04:36:19.052 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-08T04:36:19.101 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-08T04:36:19.412 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client-common (238) ... 2024-05-08T04:36:19.591 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-08T04:36:19.717 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-08T04:36:19.843 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:19.969 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-08T04:36:20.095 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-08T04:36:20.221 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:21.169 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T04:36:21.267 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-08T04:36:21.646 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T04:36:21.647 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T04:36:22.346 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-08T04:36:22.471 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-08T04:36:22.589 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:36:22.690 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-08T04:36:22.791 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-08T04:36:23.484 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:23.609 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-08T04:36:23.736 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-08T04:36:24.357 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-08T04:36:24.458 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up dump (0.4b47-3) ... 2024-05-08T04:36:24.578 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T04:36:24.635 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:24.769 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-08T04:36:24.895 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-08T04:36:25.021 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-08T04:36:25.147 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-08T04:36:25.237 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-08T04:36:25.299 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-08T04:36:25.399 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-08T04:36:25.559 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-08T04:36:25.685 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:25.845 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-08T04:36:26.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client (14+238) ... 2024-05-08T04:36:26.138 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-08T04:36:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout:which means they are currently configured by ifupdown: 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout:- enp3s0f1 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T04:36:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:26.807 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T04:36:27.190 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T04:36:27.597 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T04:36:28.282 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:28.409 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:28.902 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-08T04:36:29.036 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-common (238) ... 2024-05-08T04:36:29.982 INFO:teuthology.orchestra.run.smithi164.stdout:Adding user postgres to group ssl-cert 2024-05-08T04:36:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:30.231 INFO:teuthology.orchestra.run.smithi164.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T04:36:30.484 INFO:teuthology.orchestra.run.smithi164.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T04:36:30.484 INFO:teuthology.orchestra.run.smithi164.stdout:Removing obsolete dictionary files: 2024-05-08T04:36:31.179 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T04:36:31.762 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:32.101 INFO:teuthology.orchestra.run.smithi164.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-08T04:36:32.122 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:This user must also own the server process. 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T04:36:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:The default text search configuration will be set to "english". 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout:Data page checksums are disabled. 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout:creating subdirectories ... ok 2024-05-08T04:36:32.180 INFO:teuthology.orchestra.run.smithi164.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T04:36:32.204 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default max_connections ... 100 2024-05-08T04:36:32.242 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default shared_buffers ... 128MB 2024-05-08T04:36:32.252 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default time zone ... Etc/UTC 2024-05-08T04:36:32.253 INFO:teuthology.orchestra.run.smithi164.stdout:creating configuration files ... ok 2024-05-08T04:36:32.418 INFO:teuthology.orchestra.run.smithi164.stdout:running bootstrap script ... ok 2024-05-08T04:36:33.035 INFO:teuthology.orchestra.run.smithi164.stdout:performing post-bootstrap initialization ... ok 2024-05-08T04:36:33.542 INFO:teuthology.orchestra.run.smithi164.stdout:syncing data to disk ... ok 2024-05-08T04:36:34.034 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T04:36:35.714 INFO:teuthology.orchestra.run.smithi172.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-08T04:36:35.717 INFO:teuthology.orchestra.run.smithi172.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-08T04:36:35.721 INFO:teuthology.orchestra.run.smithi172.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-08T04:36:35.724 INFO:teuthology.orchestra.run.smithi172.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-08T04:36:35.810 INFO:teuthology.orchestra.run.smithi172.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-08T04:36:35.822 INFO:teuthology.orchestra.run.smithi172.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-08T04:36:37.077 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:37.186 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-08T04:36:37.312 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql (14+238) ... 2024-05-08T04:36:37.438 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:37.564 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:37.825 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T04:36:37.825 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T04:36:38.483 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-08T04:36:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T04:36:38.880 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-08T04:36:39.014 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-08T04:36:39.785 INFO:teuthology.orchestra.run.smithi172.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-08T04:36:39.863 INFO:teuthology.orchestra.run.smithi172.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-08T04:36:40.341 INFO:teuthology.orchestra.run.smithi172.stdout:Preconfiguring packages ... 2024-05-08T04:36:40.417 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-08T04:36:40.511 INFO:teuthology.orchestra.run.smithi172.stdout:Fetched 55.7 MB in 47s (1,177 kB/s) 2024-05-08T04:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-08T04:36:40.798 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-05-08T04:36:40.801 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:36:40.952 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:36:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T04:36:41.076 DEBUG:teuthology.parallel:result is None 2024-05-08T04:36:41.537 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-08T04:36:42.645 INFO:teuthology.orchestra.run.smithi172.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-08T04:36:42.934 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package flex. 2024-05-08T04:36:43.005 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-05-08T04:36:43.008 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-08T04:36:43.049 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-08T04:36:43.543 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-08T04:36:43.721 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-08T04:36:44.134 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:36:44.665 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-05-08T04:36:44.669 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:36:44.778 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:36:45.173 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-08T04:36:45.309 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-08T04:36:45.622 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-08T04:36:45.993 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package bison. 2024-05-08T04:36:46.062 INFO:teuthology.orchestra.run.smithi172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-05-08T04:36:46.066 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-08T04:36:46.111 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-08T04:36:46.537 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T04:36:46.551 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-08T04:36:46.595 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-08T04:36:46.907 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package dump. 2024-05-08T04:36:46.921 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-08T04:36:46.958 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking dump (0.4b47-3) ... 2024-05-08T04:36:47.202 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package indent. 2024-05-08T04:36:47.216 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:47.260 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-08T04:36:47.546 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T04:36:47.560 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-08T04:36:47.604 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-08T04:36:47.916 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T04:36:47.931 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-08T04:36:47.966 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-08T04:36:48.194 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T04:36:48.208 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-08T04:36:48.252 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-08T04:36:48.897 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-08T04:36:48.912 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-08T04:36:48.955 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:49.217 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-08T04:36:49.231 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-08T04:36:49.275 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-08T04:36:49.519 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libjson-perl. 2024-05-08T04:36:49.533 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-08T04:36:49.577 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-08T04:36:49.888 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-08T04:36:49.902 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-08T04:36:49.947 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-08T04:36:51.308 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T04:36:51.324 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T04:36:51.366 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:51.619 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T04:36:51.635 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T04:36:51.677 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:36:51.979 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T04:36:51.995 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-08T04:36:52.029 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:52.383 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T04:36:52.398 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-08T04:36:52.441 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-08T04:36:52.752 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T04:36:52.768 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-08T04:36:52.810 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-08T04:36:53.146 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T04:36:53.162 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:53.196 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:53.516 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T04:36:53.531 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:53.574 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:53.860 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T04:36:53.875 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:53.918 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:36:54.179 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T04:36:54.195 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-08T04:36:54.229 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:36:54.732 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T04:36:54.748 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-08T04:36:54.790 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-08T04:36:55.135 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package modemmanager. 2024-05-08T04:36:55.150 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-08T04:36:55.218 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:36:55.833 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package network-manager. 2024-05-08T04:36:55.848 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-08T04:36:55.899 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-08T04:36:56.873 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package ppp. 2024-05-08T04:36:56.889 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-08T04:36:57.262 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-08T04:36:57.586 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T04:36:57.601 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-08T04:36:57.636 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-08T04:36:58.039 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T04:36:58.053 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-08T04:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-08T04:36:58.584 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T04:36:58.598 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-08T04:36:58.658 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-client-common (238) ... 2024-05-08T04:36:59.104 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-08T04:36:59.119 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:36:59.179 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:36:59.858 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T04:36:59.873 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-08T04:36:59.935 INFO:teuthology.orchestra.run.smithi172.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T04:36:59.965 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-common (238) ... 2024-05-08T04:37:00.423 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-14. 2024-05-08T04:37:00.438 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-08T04:37:00.484 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:37:01.618 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql. 2024-05-08T04:37:01.633 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-08T04:37:01.667 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql (14+238) ... 2024-05-08T04:37:01.919 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T04:37:01.934 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-08T04:37:01.978 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-client (14+238) ... 2024-05-08T04:37:02.222 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T04:37:02.236 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-08T04:37:02.280 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-08T04:37:02.607 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package tcl8.6. 2024-05-08T04:37:02.622 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-08T04:37:02.665 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-08T04:37:02.968 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package tcl. 2024-05-08T04:37:02.983 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-08T04:37:03.031 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-08T04:37:03.338 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T04:37:03.353 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-08T04:37:03.395 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-08T04:37:03.733 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T04:37:03.749 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-08T04:37:03.792 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-08T04:37:04.178 INFO:teuthology.orchestra.run.smithi172.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T04:37:04.193 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-08T04:37:04.244 INFO:teuthology.orchestra.run.smithi172.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-08T04:37:04.521 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-client-common (238) ... 2024-05-08T04:37:04.742 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-08T04:37:04.868 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-08T04:37:05.011 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:37:05.111 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-08T04:37:05.221 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-08T04:37:05.322 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:37:06.266 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T04:37:06.377 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-08T04:37:06.704 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T04:37:06.705 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T04:37:07.421 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-08T04:37:07.531 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-08T04:37:07.656 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-08T04:37:07.791 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-08T04:37:07.917 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-08T04:37:08.034 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-08T04:37:08.135 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-08T04:37:08.270 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-08T04:37:08.908 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-08T04:37:09.034 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up dump (0.4b47-3) ... 2024-05-08T04:37:09.130 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T04:37:09.194 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:37:09.295 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-08T04:37:09.421 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-08T04:37:09.547 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-08T04:37:09.673 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-08T04:37:09.762 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-08T04:37:09.858 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-08T04:37:09.984 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-08T04:37:10.160 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-08T04:37:10.261 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:37:10.362 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-08T04:37:10.538 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-client (14+238) ... 2024-05-08T04:37:10.664 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-08T04:37:11.230 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:11.230 INFO:teuthology.orchestra.run.smithi172.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T04:37:11.230 INFO:teuthology.orchestra.run.smithi172.stdout:which means they are currently configured by ifupdown: 2024-05-08T04:37:11.231 INFO:teuthology.orchestra.run.smithi172.stdout:- enp3s0f1 2024-05-08T04:37:11.231 INFO:teuthology.orchestra.run.smithi172.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T04:37:11.231 INFO:teuthology.orchestra.run.smithi172.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T04:37:11.231 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:11.363 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T04:37:11.766 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T04:37:12.203 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T04:37:13.042 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-08T04:37:13.184 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-08T04:37:13.310 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-08T04:37:13.436 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-common (238) ... 2024-05-08T04:37:14.364 INFO:teuthology.orchestra.run.smithi172.stdout:Adding user postgres to group ssl-cert 2024-05-08T04:37:14.707 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:14.707 INFO:teuthology.orchestra.run.smithi172.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T04:37:14.952 INFO:teuthology.orchestra.run.smithi172.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T04:37:14.952 INFO:teuthology.orchestra.run.smithi172.stdout:Removing obsolete dictionary files: 2024-05-08T04:37:15.669 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T04:37:16.104 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-08T04:37:16.445 INFO:teuthology.orchestra.run.smithi172.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-08T04:37:16.461 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout:This user must also own the server process. 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T04:37:16.513 INFO:teuthology.orchestra.run.smithi172.stdout:The default text search configuration will be set to "english". 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout:Data page checksums are disabled. 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout:creating subdirectories ... ok 2024-05-08T04:37:16.514 INFO:teuthology.orchestra.run.smithi172.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T04:37:16.538 INFO:teuthology.orchestra.run.smithi172.stdout:selecting default max_connections ... 100 2024-05-08T04:37:16.571 INFO:teuthology.orchestra.run.smithi172.stdout:selecting default shared_buffers ... 128MB 2024-05-08T04:37:16.586 INFO:teuthology.orchestra.run.smithi172.stdout:selecting default time zone ... Etc/UTC 2024-05-08T04:37:16.586 INFO:teuthology.orchestra.run.smithi172.stdout:creating configuration files ... ok 2024-05-08T04:37:16.751 INFO:teuthology.orchestra.run.smithi172.stdout:running bootstrap script ... ok 2024-05-08T04:37:17.360 INFO:teuthology.orchestra.run.smithi172.stdout:performing post-bootstrap initialization ... ok 2024-05-08T04:37:17.898 INFO:teuthology.orchestra.run.smithi172.stdout:syncing data to disk ... ok 2024-05-08T04:37:18.436 INFO:teuthology.orchestra.run.smithi172.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T04:37:21.528 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:37:21.653 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-08T04:37:21.779 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up postgresql (14+238) ... 2024-05-08T04:37:21.905 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-08T04:37:22.040 INFO:teuthology.orchestra.run.smithi172.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-08T04:37:22.293 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T04:37:22.293 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T04:37:23.025 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-08T04:37:23.118 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T04:37:23.495 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-08T04:37:23.638 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-08T04:37:25.390 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-08T04:37:25.493 INFO:teuthology.orchestra.run.smithi172.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-08T04:37:26.067 INFO:teuthology.orchestra.run.smithi172.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T04:37:26.074 DEBUG:teuthology.parallel:result is None 2024-05-08T04:37:26.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-08T04:37:26.223 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-query -W -f '${Version}' ceph 2024-05-08T04:37:26.247 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:37:26.248 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:37:26.248 INFO:teuthology.task.install:The correct ceph version 19.0.0-1629-g4f7075d1-1jammy is installed. 2024-05-08T04:37:26.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-08T04:37:26.476 DEBUG:teuthology.orchestra.run.smithi172:> dpkg-query -W -f '${Version}' ceph 2024-05-08T04:37:26.499 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:37:26.499 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1629-g4f7075d1-1jammy 2024-05-08T04:37:26.499 INFO:teuthology.task.install:The correct ceph version 19.0.0-1629-g4f7075d1-1jammy is installed. 2024-05-08T04:37:26.501 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T04:37:26.501 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:26.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T04:37:26.523 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:26.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T04:37:26.560 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T04:37:26.560 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:26.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T04:37:26.587 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T04:37:26.655 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:26.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T04:37:26.674 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T04:37:26.735 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T04:37:26.735 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:26.735 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T04:37:26.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T04:37:26.819 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:26.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T04:37:26.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T04:37:26.899 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T04:37:26.899 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:26.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T04:37:26.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T04:37:26.982 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:26.982 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T04:37:27.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T04:37:27.063 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T04:37:27.147 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T04:37:27.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-05-08T04:37:27.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2024-05-08T04:37:27.170 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T04:37:27.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T04:37:27.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T04:37:27.238 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T04:37:27.238 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T04:37:27.285 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T04:37:27.311 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T04:37:27.311 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T04:37:27.311 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T04:37:27.311 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:27.311 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T04:37:27.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T04:37:27.450 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:27.451 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T04:37:27.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T04:37:27.556 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T04:37:27.556 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T04:37:27.556 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289', 'branch': 'wip-neesingh-testing1-20240507', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7697289', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing1-20240507'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-neesingh-testing1-20240507', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4374, 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'wip-neesingh-testing1-20240507', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnS/3cYF83fvr2BaYE8Va8ASSrF+m2VisCVVn/fgNEYJNj1uesTjrUBynghqpdH1q2GwXAQmUum+3EExQ6XLI0=', 'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRm1XvoHIXIVd8PIF/BL0esi4XwWpYetJ7hmo0GEjmxAW4xn7mAK0Hn60JgRhKulQR+VLqYNGDfdRp3h35qmbI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_04:04:15', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T04:37:27.557 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T04:37:27.560 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T04:37:27.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T04:37:27.609 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T04:37:27.668 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:27.668 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T04:37:27.682 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T04:37:27.683 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 894 Links: 1 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-08 04:29:19.700674302 +0000 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-08 04:29:19.696674339 +0000 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-08 04:29:19.696674339 +0000 2024-05-08T04:37:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-08 04:29:19.696674339 +0000 2024-05-08T04:37:27.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T04:37:27.801 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-08T04:37:27.801 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-08T04:37:27.801 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000398149 s, 1.3 MB/s 2024-05-08T04:37:27.803 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T04:37:27.861 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-08 04:29:20.580666442 +0000 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-08 04:29:20.304668907 +0000 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-08 04:29:20.304668907 +0000 2024-05-08T04:37:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-08 04:29:20.304668907 +0000 2024-05-08T04:37:27.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T04:37:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-08T04:37:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-08T04:37:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000372538 s, 1.4 MB/s 2024-05-08T04:37:27.979 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T04:37:28.033 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-08 04:29:21.144661405 +0000 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-08 04:29:20.864663906 +0000 2024-05-08T04:37:28.083 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-08 04:29:20.864663906 +0000 2024-05-08T04:37:28.084 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-08 04:29:20.864663906 +0000 2024-05-08T04:37:28.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T04:37:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-08T04:37:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-08T04:37:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000380214 s, 1.3 MB/s 2024-05-08T04:37:28.151 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T04:37:28.209 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 946 Links: 1 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-08 04:29:21.728656189 +0000 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-08 04:29:21.448658689 +0000 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-08 04:29:21.448658689 +0000 2024-05-08T04:37:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-08 04:29:21.448658689 +0000 2024-05-08T04:37:28.259 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T04:37:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-08T04:37:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-08T04:37:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000673177 s, 761 kB/s 2024-05-08T04:37:28.323 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T04:37:28.377 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:28.377 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:28.377 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T04:37:28.386 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T04:37:28.386 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-05-08T04:37:28.437 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 895 Links: 1 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-08 04:29:27.144898292 +0000 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-08 04:29:26.872891610 +0000 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-08 04:29:26.872891610 +0000 2024-05-08T04:37:28.438 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-08 04:29:26.872891610 +0000 2024-05-08T04:37:28.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T04:37:28.499 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-08T04:37:28.499 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-08T04:37:28.499 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000405403 s, 1.3 MB/s 2024-05-08T04:37:28.501 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T04:37:28.555 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-05-08T04:37:28.605 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T04:37:28.605 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.605 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 910 Links: 1 2024-05-08T04:37:28.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.606 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-08 04:29:27.680911205 +0000 2024-05-08T04:37:28.606 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-08 04:29:27.412904748 +0000 2024-05-08T04:37:28.606 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-08 04:29:27.412904748 +0000 2024-05-08T04:37:28.606 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-08 04:29:27.412904748 +0000 2024-05-08T04:37:28.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T04:37:28.667 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-08T04:37:28.667 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-08T04:37:28.667 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000382773 s, 1.3 MB/s 2024-05-08T04:37:28.669 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T04:37:28.723 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-05-08T04:37:28.773 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T04:37:28.773 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.773 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-05-08T04:37:28.773 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.774 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-08 04:29:28.280925658 +0000 2024-05-08T04:37:28.774 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-08 04:29:27.988918624 +0000 2024-05-08T04:37:28.774 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-08 04:29:27.988918624 +0000 2024-05-08T04:37:28.774 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-08 04:29:27.988918624 +0000 2024-05-08T04:37:28.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T04:37:28.840 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-08T04:37:28.840 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-08T04:37:28.840 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00040706 s, 1.3 MB/s 2024-05-08T04:37:28.842 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T04:37:28.895 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-05-08T04:37:28.945 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 946 Links: 1 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-08 04:29:28.884940210 +0000 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-08 04:29:28.596933272 +0000 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-08 04:29:28.596933272 +0000 2024-05-08T04:37:28.946 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-08 04:29:28.596933272 +0000 2024-05-08T04:37:28.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T04:37:29.011 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-08T04:37:29.011 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-08T04:37:29.012 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000401814 s, 1.3 MB/s 2024-05-08T04:37:29.014 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T04:37:29.067 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:29.067 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T04:37:29.067 INFO:tasks.ceph:Generating config... 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] pid file = 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T04:37:29.069 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T04:37:29.070 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-08T04:37:29.070 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T04:37:29.071 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T04:37:29.071 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T04:37:29.071 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T04:37:29.072 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T04:37:29.072 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T04:37:29.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T04:37:29.116 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T04:37:29.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T04:37:29.226 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:37:29.246 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.164'), ('mon.b', '172.21.15.172'), ('mon.c', '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]')] 2024-05-08T04:37:29.247 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.164,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T04:37:29.248 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:29.248 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T04:37:29.302 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.164 --add b 172.21.15.172 --addv c '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: generated fsid 6150cf8f-12a4-433b-b39d-bc24e302fd2a 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:setting min_mon_release = quincy 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:epoch 0 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:fsid 6150cf8f-12a4-433b-b39d-bc24e302fd2a 2024-05-08T04:37:29.392 INFO:teuthology.orchestra.run.smithi164.stdout:last_changed 2024-05-08T04:37:29.393638+0000 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:created 2024-05-08T04:37:29.393638+0000 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:min_mon_release 17 (quincy) 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:election_strategy: 1 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:2: [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] mon.c 2024-05-08T04:37:29.393 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T04:37:29.396 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T04:37:29.404 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6150cf8f-12a4-433b-b39d-bc24e302fd2a... 2024-05-08T04:37:29.405 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T04:37:29.449 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T04:37:29.487 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-05-08T04:37:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-05-08T04:37:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T04:37:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-05-08T04:37:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.164,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on skipped message = False 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 6150cf8f-12a4-433b-b39d-bc24e302fd2a 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-05-08T04:37:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-05-08T04:37:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: bdev async discard = True 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: bdev enable discard = True 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore allocator = bitmap 2024-05-08T04:37:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-05-08T04:37:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug finisher = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-05-08T04:37:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = "" 2024-05-08T04:37:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2024-05-08T04:37:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-05-08T04:37:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-05-08T04:37:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T04:37:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.164,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-05-08T04:37:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 6150cf8f-12a4-433b-b39d-bc24e302fd2a 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-05-08T04:37:29.502 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-05-08T04:37:29.503 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bdev async discard = True 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bdev enable discard = True 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore allocator = bitmap 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 1/20 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 1/20 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 4/10 2024-05-08T04:37:29.504 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-05-08T04:37:29.505 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: debug finisher = 20 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-05-08T04:37:29.506 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-05-08T04:37:29.507 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-05-08T04:37:29.508 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2024-05-08T04:37:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-05-08T04:37:29.510 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-05-08T04:37:29.510 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-05-08T04:37:29.511 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T04:37:29.511 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T04:37:29.611 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T04:37:29.611 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:29.611 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T04:37:29.620 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:29.620 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T04:37:29.670 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:37:29.670 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:29.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T04:37:29.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:37:29.744 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:37:29.745 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:29.798 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:37:29.798 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:29.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T04:37:29.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:37:29.832 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:37:29.832 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:29.881 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T04:37:29.881 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T04:37:29.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T04:37:29.934 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T04:37:29.940 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T04:37:29.996 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T04:37:30.024 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T04:37:30.024 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T04:37:30.078 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T04:37:30.083 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T04:37:30.102 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T04:37:30.196 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T04:37:30.200 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T04:37:30.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T04:37:30.275 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T04:37:30.280 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T04:37:30.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T04:37:30.395 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T04:37:30.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T04:37:30.418 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T04:37:30.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T04:37:30.460 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T04:37:30.477 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T04:37:30.559 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T04:37:30.575 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T04:37:30.618 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T04:37:30.634 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T04:37:30.717 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T04:37:30.733 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T04:37:30.733 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T04:37:30.734 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T04:37:30.753 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:30.753 INFO:tasks.ceph:role: osd.0 2024-05-08T04:37:30.753 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:37:30.753 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T04:37:30.820 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:30.821 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:31.229 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-05-08T04:37:31.231 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-05-08T04:37:31.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T04:37:31.368 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T04:37:31.384 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:31.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:31.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T04:37:31.445 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:31.445 INFO:tasks.ceph:role: osd.1 2024-05-08T04:37:31.445 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:37:31.445 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:31.517 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:31.878 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-05-08T04:37:31.880 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-05-08T04:37:31.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T04:37:31.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T04:37:32.013 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:32.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:32.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T04:37:32.077 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:32.077 INFO:tasks.ceph:role: osd.2 2024-05-08T04:37:32.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:37:32.077 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T04:37:32.148 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:32.148 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:32.148 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:32.148 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:32.148 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:32.149 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:32.149 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:32.149 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:32.149 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:32.149 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:32.538 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-05-08T04:37:32.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-05-08T04:37:32.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T04:37:32.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T04:37:32.659 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:32.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:32.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T04:37:32.725 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:32.725 INFO:tasks.ceph:role: osd.3 2024-05-08T04:37:32.725 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:37:32.725 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:32.797 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:32.798 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:32.798 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:33.183 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-05-08T04:37:33.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-05-08T04:37:33.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T04:37:33.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T04:37:33.329 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:33.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:33.330 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:33.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.449+0000 7f16635197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T04:37:33.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.453+0000 7f16635197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T04:37:33.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.453+0000 7f16635197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:37:33.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.453+0000 7f16635197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:37:33.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.453+0000 7f16635197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:37:33.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:33.465+0000 7f16635197c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T04:37:36.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T04:37:36.566 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:36.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T04:37:36.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T04:37:36.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:37:36.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:37:36.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:37:36.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:36.652+0000 7f6b09e727c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T04:37:39.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T04:37:39.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.788+0000 7f802862b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T04:37:39.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.788+0000 7f802862b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T04:37:39.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.788+0000 7f802862b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:37:39.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.788+0000 7f802862b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:37:39.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.788+0000 7f802862b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:37:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:39.792+0000 7f802862b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T04:37:42.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T04:37:42.862 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:42.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T04:37:42.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T04:37:42.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:37:42.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:37:42.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:37:42.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:37:42.952+0000 7fe6a92cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T04:37:46.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T04:37:46.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T04:37:46.052 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:46.053 INFO:tasks.ceph:role: osd.4 2024-05-08T04:37:46.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:37:46.053 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:46.122 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:46.123 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:46.523 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-05-08T04:37:46.526 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-05-08T04:37:46.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T04:37:46.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T04:37:46.640 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:46.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:46.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T04:37:46.705 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:46.705 INFO:tasks.ceph:role: osd.5 2024-05-08T04:37:46.705 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:37:46.705 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:46.776 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:46.777 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:46.777 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:46.777 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:46.777 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:47.193 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-05-08T04:37:47.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-05-08T04:37:47.196 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T04:37:47.336 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T04:37:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:47.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:47.354 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T04:37:47.418 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:47.418 INFO:tasks.ceph:role: osd.6 2024-05-08T04:37:47.418 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:37:47.418 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:47.489 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:47.904 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-05-08T04:37:47.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-05-08T04:37:47.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T04:37:48.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T04:37:48.054 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:48.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:48.055 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T04:37:48.121 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:37:48.121 INFO:tasks.ceph:role: osd.7 2024-05-08T04:37:48.121 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:37:48.122 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T04:37:48.192 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:37:48.193 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:37:48.558 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-05-08T04:37:48.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-05-08T04:37:48.562 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T04:37:48.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T04:37:48.705 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T04:37:48.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:37:48.707 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.828+0000 7f77229c67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T04:37:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.832+0000 7f77229c67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T04:37:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.832+0000 7f77229c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:37:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.832+0000 7f77229c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:37:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.832+0000 7f77229c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:37:48.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:48.836+0000 7f77229c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T04:37:51.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T04:37:51.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:52.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T04:37:52.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T04:37:52.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:37:52.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:37:52.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:37:52.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:52.027+0000 7fcb9a25e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T04:37:55.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T04:37:55.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:55.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.207+0000 7ffa70cc17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T04:37:55.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.207+0000 7ffa70cc17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T04:37:55.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.207+0000 7ffa70cc17c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:37:55.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.207+0000 7ffa70cc17c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:37:55.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.207+0000 7ffa70cc17c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:37:55.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:55.211+0000 7ffa70cc17c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T04:37:57.558 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:37:57.562 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:37:58.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T04:37:58.275 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:37:58.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.363+0000 7f6b0a1d37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T04:37:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.363+0000 7f6b0a1d37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T04:37:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.363+0000 7f6b0a1d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:37:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.363+0000 7f6b0a1d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:37:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.363+0000 7f6b0a1d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:37:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:37:58.367+0000 7f6b0a1d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T04:38:01.415 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T04:38:01.436 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T04:38:01.436 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T04:38:01.458 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T04:38:01.519 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T04:38:01.583 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T04:38:01.647 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T04:38:01.711 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T04:38:01.775 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:01.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T04:38:01.834 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:01.835 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T04:38:01.855 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:01.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T04:38:01.917 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:01.917 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T04:38:01.981 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:01.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T04:38:02.041 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:02.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T04:38:02.109 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:02.109 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T04:38:02.177 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:02.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T04:38:02.241 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:02.241 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T04:38:02.250 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:02.251 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T04:38:02.299 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:02.299 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T04:38:02.308 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:02.308 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T04:38:02.357 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T04:38:02.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T04:38:02.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ== 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A== 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ== 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ== 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA== 2024-05-08T04:38:02.405 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA== 2024-05-08T04:38:02.406 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA== 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw== 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout:[client.2] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg== 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg== 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout:[client.3] 2024-05-08T04:38:02.407 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ== 2024-05-08T04:38:02.419 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2024-05-08T04:38:02.419 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ== 2024-05-08T04:38:02.419 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-05-08T04:38:02.419 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A== 2024-05-08T04:38:02.419 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A== 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-05-08T04:38:02.420 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg== 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2024-05-08T04:38:02.421 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ== 2024-05-08T04:38:02.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:38:02.454 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:38:02.523 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.589 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.760 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.812 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.816 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:02.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:38:02.872 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:38:02.926 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.929 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:02.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T04:38:03.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.048 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.101 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:38:03.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.273 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.390 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.445 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.448 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:38:03.505 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T04:38:03.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T04:38:03.524 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:38:03.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T04:38:03.651 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T04:38:03.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:38:03.785 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T04:38:03.805 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T04:38:03.864 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:38:03.974 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T04:38:03.994 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:38:03.997 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:38:04.044 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T04:38:04.044 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T04:38:04.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T04:38:04.047 INFO:tasks.ceph.mon.a:Started 2024-05-08T04:38:04.048 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T04:38:04.048 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T04:38:04.087 INFO:tasks.ceph.mon.b:Started 2024-05-08T04:38:04.088 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T04:38:04.088 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T04:38:04.090 INFO:tasks.ceph.mon.c:Started 2024-05-08T04:38:04.091 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T04:38:04.091 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T04:38:04.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T04:38:04.094 INFO:tasks.ceph.mgr.y:Started 2024-05-08T04:38:04.094 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T04:38:04.094 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T04:38:04.097 INFO:tasks.ceph.mgr.x:Started 2024-05-08T04:38:04.097 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T04:38:04.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T04:38:04.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 Processor -- start 2024-05-08T04:38:04.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 -- start start 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc54806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5480c6600 con 0x7fc54815ba20 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54815fbe0 con 0x7fc54806e390 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc54e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480c6040 con 0x7fc54815ce10 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc546a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc54806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38616/0 (socket says 172.21.15.164:38616) 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 -- 172.21.15.164:0/2173657337 learned_addr learned my addr 172.21.15.164:0/2173657337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc546a00640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 msgr2=0x7fc54806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T04:38:04.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 -- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 msgr2=0x7fc5480ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 -- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 msgr2=0x7fc5480ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc546a00640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 msgr2=0x7fc54806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 --2- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc54806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc54806de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:38:04.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.236+0000 7fc547e00640 1 --2- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:38:04.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.392+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089884671 0 0) 0x7fc5480c6600 con 0x7fc54815ba20 2024-05-08T04:38:04.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.392+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c003530 con 0x7fc54815ba20 2024-05-08T04:38:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499988734 0 0) 0x7fc52c003530 con 0x7fc54815ba20 2024-05-08T04:38:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 msgr2=0x7fc54806de50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:38:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc54806de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 msgr2=0x7fc5480ae6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 --2- 172.21.15.164:0/2173657337 >> v2:172.21.15.164:3300/0 conn(0x7fc54806e390 0x7fc5480ae6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480c6320 con 0x7fc54815ba20 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc538001680 con 0x7fc54815ba20 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 >> v1:172.21.15.172:6789/0 conn(0x7fc54815ba20 legacy=0x7fc548160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c0039c0 0x7fc52c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c004810 0x7fc52c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc52c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54815fbe0 con 0x7fc52c0039c0 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480c6040 con 0x7fc52c004810 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c00bb60 con 0x7fc54815ce10 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c004810 0x7fc52c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc547e00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc52c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc547400640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c0039c0 0x7fc52c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 msgr2=0x7fc52c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54815ce10 0x7fc52c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c0039c0 msgr2=0x7fc52c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c0039c0 0x7fc52c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.396+0000 7fc546a00640 1 -- 172.21.15.164:0/2173657337 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c003530 con 0x7fc52c004810 2024-05-08T04:38:04.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.400+0000 7fc546a00640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c004810 0x7fc52c004c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53400b9e0 tx=0x7fc53401c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a7455a2ed8180c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:04.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53400b5e0 con 0x7fc52c004810 2024-05-08T04:38:04.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340af640 con 0x7fc52c004810 2024-05-08T04:38:04.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc546000640 1 -- 172.21.15.164:0/2173657337 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340afa30 con 0x7fc52c004810 2024-05-08T04:38:04.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 -- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c004810 msgr2=0x7fc52c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:04.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 --2- 172.21.15.164:0/2173657337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c004810 0x7fc52c004c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53400b9e0 tx=0x7fc53401c640 comp rx=0 tx=0).stop 2024-05-08T04:38:04.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc547e00640 1 -- 172.21.15.164:0/2173657337 reap_dead start 2024-05-08T04:38:04.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 -- 172.21.15.164:0/2173657337 shutdown_connections 2024-05-08T04:38:04.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 -- 172.21.15.164:0/2173657337 >> 172.21.15.164:0/2173657337 conn(0x7fc5480cf770 msgr2=0x7fc5480cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:04.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 -- 172.21.15.164:0/2173657337 shutdown_connections 2024-05-08T04:38:04.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.404+0000 7fc54e000640 1 -- 172.21.15.164:0/2173657337 wait complete. 2024-05-08T04:38:04.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 Processor -- start 2024-05-08T04:38:04.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- start start 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480b0d50 0x7fc548152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480ad890 0x7fc5480add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480af030 con 0x7fc52c004810 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480ce090 con 0x7fc5480b0d50 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc548166b60 con 0x7fc5480ad890 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38656/0 (socket says 172.21.15.164:38656) 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 -- 172.21.15.164:0/2477818988 learned_addr learned my addr 172.21.15.164:0/2477818988 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc546a00640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480b0d50 0x7fc548152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547e00640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480ad890 0x7fc5480add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 -- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480ad890 msgr2=0x7fc5480add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480ad890 0x7fc5480add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 -- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480b0d50 msgr2=0x7fc548152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480b0d50 0x7fc548152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 -- 172.21.15.164:0/2477818988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480c9f60 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc547400640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53800f860 tx=0x7fc53800fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5380052e0 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538018070 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53800d910 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5480ca240 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5480ca800 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc53800dab0 con 0x7fc52c004810 2024-05-08T04:38:04.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.408+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc53801d460 con 0x7fc52c004810 2024-05-08T04:38:04.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.412+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548072d30 con 0x7fc52c004810 2024-05-08T04:38:04.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.424+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc538005480 con 0x7fc52c004810 2024-05-08T04:38:04.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.483+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc54804f010 con 0x7fc52c004810 2024-05-08T04:38:04.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc53f400640 1 -- 172.21.15.164:0/2477818988 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc538003600 con 0x7fc52c004810 2024-05-08T04:38:04.488 INFO:teuthology.orchestra.run.smithi164.stderr:adjusted tunables profile to default 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 msgr2=0x7fc5480cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc53800f860 tx=0x7fc53800fd20 comp rx=0 tx=0).stop 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 shutdown_connections 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480ad890 0x7fc5480add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480b0d50 0x7fc548152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 --2- 172.21.15.164:0/2477818988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52c004810 0x7fc5480cb810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 >> 172.21.15.164:0/2477818988 conn(0x7fc5480cf770 msgr2=0x7fc548165850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 shutdown_connections 2024-05-08T04:38:04.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:04.487+0000 7fc54e000640 1 -- 172.21.15.164:0/2477818988 wait complete. 2024-05-08T04:38:04.513 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T04:38:04.513 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T04:38:04.513 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:04.514 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T04:38:04.533 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:04.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T04:38:04.595 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:04.595 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T04:38:04.608 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:04.603+0000 7f428ee0a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T04:38:04.628 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:04.627+0000 7ffadaa0a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T04:38:04.646 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:04.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T04:38:04.710 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:04.710 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T04:38:04.730 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:04.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T04:38:04.792 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:04.793 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T04:38:04.856 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:04.856 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T04:38:04.916 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f 0 2024-05-08T04:38:05.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.087+0000 7f3004000640 1 Processor -- start 2024-05-08T04:38:05.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.087+0000 7f3004000640 1 -- start start 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3004000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2ffc0ae2c0 0x7f2ffc072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3004000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc06e370 con 0x7f2ffc072cb0 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3004000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc06e650 con 0x7f2ffc15bd20 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3004000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc071f70 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3002000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2ffc072cb0 0x7f2ffc0acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.172:42554/0 (socket says 172.21.15.172:42554) 2024-05-08T04:38:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3002000640 1 -- 172.21.15.172:0/2403035066 learned_addr learned my addr 172.21.15.172:0/2403035066 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3000c00640 1 --2- 172.21.15.172:0/2403035066 >> v2:172.21.15.164:3300/0 conn(0x7f2ffc0ae2c0 0x7f2ffc072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512494426 0 0) 0x7f2ffc06e370 con 0x7f2ffc072cb0 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe4003530 con 0x7f2ffc072cb0 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456897937 0 0) 0x7f2ffc06e650 con 0x7f2ffc15bd20 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ffc06e370 con 0x7f2ffc15bd20 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3000c00640 1 -- 172.21.15.172:0/2403035066 >> v1:172.21.15.172:6789/0 conn(0x7f2ffc15bd20 legacy=0x7f2ffc06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3000c00640 1 -- 172.21.15.172:0/2403035066 >> v1:172.21.15.164:6789/0 conn(0x7f2ffc072cb0 legacy=0x7f2ffc0acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.091+0000 7f3000c00640 1 -- 172.21.15.172:0/2403035066 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc072250 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3000c00640 1 --2- 172.21.15.172:0/2403035066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc072770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097c0 tx=0x7f2fe801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19267f7bf88b906 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80b5070 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80a8990 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f2ffbe00640 1 -- 172.21.15.172:0/2403035066 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80a8d60 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 -- 172.21.15.172:0/2403035066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 msgr2=0x7f2ffc072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 --2- 172.21.15.172:0/2403035066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc072770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097c0 tx=0x7f2fe801ec90 comp rx=0 tx=0).stop 2024-05-08T04:38:05.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 -- 172.21.15.172:0/2403035066 shutdown_connections 2024-05-08T04:38:05.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 --2- 172.21.15.172:0/2403035066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc072770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 -- 172.21.15.172:0/2403035066 >> 172.21.15.172:0/2403035066 conn(0x7f2ffc0b1bc0 msgr2=0x7f2ffc0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 -- 172.21.15.172:0/2403035066 shutdown_connections 2024-05-08T04:38:05.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.095+0000 7f3004000640 1 -- 172.21.15.172:0/2403035066 wait complete. 2024-05-08T04:38:05.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 Processor -- start 2024-05-08T04:38:05.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- start start 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 0x7f2ffc14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0ad730 con 0x7f2ffc0ae2c0 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0ad5b0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc15c100 con 0x7f2ffc072cb0 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3001600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 0x7f2ffc14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3001600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39650/0 (socket says 172.21.15.172:39650) 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3001600640 1 -- 172.21.15.172:0/4202224147 learned_addr learned my addr 172.21.15.172:0/4202224147 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3000c00640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 -- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 msgr2=0x7f2ffc09cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 -- 172.21.15.172:0/4202224147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 msgr2=0x7f2ffc1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 -- 172.21.15.172:0/4202224147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc09d4c0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3001600640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3000c00640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3002000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 0x7f2ffc14a6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400ca70 tx=0x7f2ff400cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4017070 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff400abd0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4012400 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc15c8e0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc15cea0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2ff401a070 con 0x7f2ffc15bd20 2024-05-08T04:38:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.099+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2ff401cb30 con 0x7f2ffc15bd20 2024-05-08T04:38:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.103+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc09e220 con 0x7f2ffc15bd20 2024-05-08T04:38:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.103+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2ff4021070 con 0x7f2ffc15bd20 2024-05-08T04:38:05.132 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:05.127+0000 7f428ee0a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T04:38:05.151 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:05.147+0000 7ffadaa0a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T04:38:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.171+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f", "id": 0} v 0) v1 -- 0x7f2ffc04f010 con 0x7f2ffc15bd20 2024-05-08T04:38:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f2ff9600640 1 -- 172.21.15.172:0/4202224147 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2ffc09d4c0 con 0x7f2ffc15bd20 2024-05-08T04:38:05.180 INFO:teuthology.orchestra.run.smithi172.stdout:0 2024-05-08T04:38:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 msgr2=0x7f2ffc14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 0x7f2ffc14a6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400ca70 tx=0x7f2ff400cf30 comp rx=0 tx=0).stop 2024-05-08T04:38:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 shutdown_connections 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ffc15bd20 0x7f2ffc14a6d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ffc0ae2c0 0x7f2ffc1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 --2- 172.21.15.172:0/4202224147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ffc072cb0 0x7f2ffc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 >> 172.21.15.172:0/4202224147 conn(0x7f2ffc0b1bc0 msgr2=0x7f2ffc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 shutdown_connections 2024-05-08T04:38:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.175+0000 7f3004000640 1 -- 172.21.15.172:0/4202224147 wait complete. 2024-05-08T04:38:05.199 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 342a2bc7-f643-4df7-baf3-b04d228500ad 1 2024-05-08T04:38:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.327+0000 7f7960600640 1 Processor -- start 2024-05-08T04:38:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.327+0000 7f7960600640 1 -- start start 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f7960600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f7960600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f795806d220 0x7f795806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f7960600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580ce760 con 0x7f795806d680 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f7960600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79580cea00 con 0x7f795806d220 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f7960600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580cece0 con 0x7f795815bd20 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f795806d220 0x7f795806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39658/0 (socket says 172.21.15.172:39658) 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795dc00640 1 -- 172.21.15.172:0/422174077 learned_addr learned my addr 172.21.15.172:0/422174077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795d200640 1 -- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 msgr2=0x7f7958071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795d200640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795d200640 1 -- 172.21.15.172:0/422174077 >> v1:172.21.15.164:6789/0 conn(0x7f795806d680 legacy=0x7f79580cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795d200640 1 -- 172.21.15.172:0/422174077 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580cefc0 con 0x7f795806d220 2024-05-08T04:38:05.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.331+0000 7f795dc00640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:05.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f795d200640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795806d220 0x7f795806ac90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f79440097c0 tx=0x7f794401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67919914716d6fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7957e00640 1 -- 172.21.15.172:0/422174077 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79440aa5f0 con 0x7f795806d220 2024-05-08T04:38:05.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7957e00640 1 -- 172.21.15.172:0/422174077 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79440af070 con 0x7f795806d220 2024-05-08T04:38:05.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7957e00640 1 -- 172.21.15.172:0/422174077 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79440aa9b0 con 0x7f795806d220 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 -- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795806d220 msgr2=0x7f795806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795806d220 0x7f795806ac90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f79440097c0 tx=0x7f794401c470 comp rx=0 tx=0).stop 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 -- 172.21.15.172:0/422174077 shutdown_connections 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795806d220 0x7f795806ac90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 --2- 172.21.15.172:0/422174077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795815bd20 0x7f7958071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 -- 172.21.15.172:0/422174077 >> 172.21.15.172:0/422174077 conn(0x7f79580b25b0 msgr2=0x7f79580b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 -- 172.21.15.172:0/422174077 shutdown_connections 2024-05-08T04:38:05.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.335+0000 7f7960600640 1 -- 172.21.15.172:0/422174077 wait complete. 2024-05-08T04:38:05.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 Processor -- start 2024-05-08T04:38:05.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- start start 2024-05-08T04:38:05.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 0x7f7958139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 0x7f795815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795815bd20 0x7f795815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b0140 con 0x7f795806d220 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795806b5e0 con 0x7f795815bd20 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b02c0 con 0x7f795806d680 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 0x7f795815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795815bd20 0x7f795815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 0x7f795815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39674/0 (socket says 172.21.15.172:39674) 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795d200640 1 -- 172.21.15.172:0/4118707316 learned_addr learned my addr 172.21.15.172:0/4118707316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 0x7f7958139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 -- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 msgr2=0x7f795815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 0x7f795815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 -- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795815bd20 msgr2=0x7f795815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795815bd20 0x7f795815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 -- 172.21.15.172:0/4118707316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795815c430 con 0x7f795806d220 2024-05-08T04:38:05.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f795dc00640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 0x7f7958139fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f794c00e8b0 tx=0x7f794c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c004030 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c0041d0 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79581518d0 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00c8e0 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795815c710 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f794c003620 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.339+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f794c013b60 con 0x7f795806d220 2024-05-08T04:38:05.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.343+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958156b70 con 0x7f795806d220 2024-05-08T04:38:05.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.343+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f794c017020 con 0x7f795806d220 2024-05-08T04:38:05.365 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:05.359+0000 7f428ee0a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T04:38:05.383 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:05.379+0000 7ffadaa0a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T04:38:05.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.415+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "342a2bc7-f643-4df7-baf3-b04d228500ad", "id": 1} v 0) v1 -- 0x7f795804f010 con 0x7f795806d220 2024-05-08T04:38:05.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.415+0000 7f7955600640 1 -- 172.21.15.172:0/4118707316 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "342a2bc7-f643-4df7-baf3-b04d228500ad", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f794c0093d0 con 0x7f795806d220 2024-05-08T04:38:05.422 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 msgr2=0x7f7958139fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 0x7f7958139fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f794c00e8b0 tx=0x7f794c00ed70 comp rx=0 tx=0).stop 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 shutdown_connections 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f795806d680 0x7f795815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f795815bd20 0x7f795815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 --2- 172.21.15.172:0/4118707316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f795806d220 0x7f7958139fa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 >> 172.21.15.172:0/4118707316 conn(0x7f79580b25b0 msgr2=0x7f79580ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 shutdown_connections 2024-05-08T04:38:05.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.419+0000 7f7960600640 1 -- 172.21.15.172:0/4118707316 wait complete. 2024-05-08T04:38:05.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 454494bb-a031-44bb-806e-9922df32773d 2 2024-05-08T04:38:05.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.571+0000 7f0c4bc00640 1 Processor -- start 2024-05-08T04:38:05.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.571+0000 7f0c4bc00640 1 -- start start 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.571+0000 7f0c4bc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0c4415f2b0 0x7f0c440d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c4bc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c44070280 con 0x7f0c440b3fe0 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c4bc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c44070560 con 0x7f0c440cd560 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c4bc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4415f690 con 0x7f0c4415f2b0 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49c00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0c440cd560 0x7f0c4415f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56102/0 (socket says 172.21.15.172:56102) 2024-05-08T04:38:05.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49c00640 1 -- 172.21.15.172:0/2213778290 learned_addr learned my addr 172.21.15.172:0/2213778290 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49200640 1 --2- 172.21.15.172:0/2213778290 >> v2:172.21.15.164:3300/0 conn(0x7f0c4415f2b0 0x7f0c440d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606179723 0 0) 0x7f0c44070560 con 0x7f0c440cd560 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c28003530 con 0x7f0c440cd560 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49200640 1 -- 172.21.15.172:0/2213778290 >> v1:172.21.15.172:6789/0 conn(0x7f0c440cd560 legacy=0x7f0c4415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49200640 1 -- 172.21.15.172:0/2213778290 >> v1:172.21.15.164:6789/0 conn(0x7f0c440b3fe0 legacy=0x7f0c440cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478947321 0 0) 0x7f0c44070280 con 0x7f0c440b3fe0 2024-05-08T04:38:05.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49200640 1 -- 172.21.15.172:0/2213778290 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4406f7e0 con 0x7f0c4415f2b0 2024-05-08T04:38:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.575+0000 7f0c49200640 1 --2- 172.21.15.172:0/2213778290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440d0410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097c0 tx=0x7f0c3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb034d429fb8840 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340b1070 con 0x7f0c4415f2b0 2024-05-08T04:38:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c340ac5f0 con 0x7f0c4415f2b0 2024-05-08T04:38:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c43e00640 1 -- 172.21.15.172:0/2213778290 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340ac9c0 con 0x7f0c4415f2b0 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 -- 172.21.15.172:0/2213778290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 msgr2=0x7f0c440d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/2213778290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440d0410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097c0 tx=0x7f0c3401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 -- 172.21.15.172:0/2213778290 shutdown_connections 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/2213778290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440d0410 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 -- 172.21.15.172:0/2213778290 >> 172.21.15.172:0/2213778290 conn(0x7f0c440b34f0 msgr2=0x7f0c440b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 -- 172.21.15.172:0/2213778290 shutdown_connections 2024-05-08T04:38:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 -- 172.21.15.172:0/2213778290 wait complete. 2024-05-08T04:38:05.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.579+0000 7f0c4bc00640 1 Processor -- start 2024-05-08T04:38:05.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- start start 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c440cd560 0x7f0c441505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44169f90 con 0x7f0c4415f2b0 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44169e10 con 0x7f0c440cd560 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4416a110 con 0x7f0c440b3fe0 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c440cd560 0x7f0c441505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39686/0 (socket says 172.21.15.172:39686) 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 -- 172.21.15.172:0/1085220552 learned_addr learned my addr 172.21.15.172:0/1085220552 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49c00640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 -- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c440cd560 msgr2=0x7f0c441505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c440cd560 0x7f0c441505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 -- 172.21.15.172:0/1085220552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 msgr2=0x7f0c440c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 -- 172.21.15.172:0/1085220552 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440c6820 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c49200640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097c0 tx=0x7f0c3401ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340bc020 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c340b2040 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440c6b00 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340b1050 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c440c70c0 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0c340bf070 con 0x7f0c440b3fe0 2024-05-08T04:38:05.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0c34003620 con 0x7f0c440b3fe0 2024-05-08T04:38:05.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.583+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4409fae0 con 0x7f0c440b3fe0 2024-05-08T04:38:05.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.607+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0c340c5020 con 0x7f0c440b3fe0 2024-05-08T04:38:05.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.675+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "454494bb-a031-44bb-806e-9922df32773d", "id": 2} v 0) v1 -- 0x7f0c4404f010 con 0x7f0c440b3fe0 2024-05-08T04:38:05.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c41600640 1 -- 172.21.15.172:0/1085220552 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "454494bb-a031-44bb-806e-9922df32773d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c340a9070 con 0x7f0c440b3fe0 2024-05-08T04:38:05.684 INFO:teuthology.orchestra.run.smithi172.stdout:2 2024-05-08T04:38:05.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 msgr2=0x7f0c4414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097c0 tx=0x7f0c3401ace0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 shutdown_connections 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c4415f2b0 0x7f0c440c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c440cd560 0x7f0c441505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 --2- 172.21.15.172:0/1085220552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c440b3fe0 0x7f0c4414ffa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 >> 172.21.15.172:0/1085220552 conn(0x7f0c440b34f0 msgr2=0x7f0c44157370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 shutdown_connections 2024-05-08T04:38:05.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.679+0000 7f0c4bc00640 1 -- 172.21.15.172:0/1085220552 wait complete. 2024-05-08T04:38:05.705 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new d4386cf2-89cd-4873-8d2f-d1831eea8277 3 2024-05-08T04:38:05.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 Processor -- start 2024-05-08T04:38:05.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 -- start start 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 0x7f79680b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79680b5820 0x7f796815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f796815c5b0 0x7f79680c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.835+0000 7f7970600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680b5cf0 con 0x7f796815c5b0 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f7970600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680b5fd0 con 0x7f79680b5820 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f7970600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680b62b0 con 0x7f796815bc90 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79680b5820 0x7f796815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79680b5820 0x7f796815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37032/0 (socket says 172.21.15.172:37032) 2024-05-08T04:38:05.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 -- 172.21.15.172:0/442159376 learned_addr learned my addr 172.21.15.172:0/442159376 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796dc00640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 0x7f79680b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796e600640 1 --2- 172.21.15.172:0/442159376 >> v2:172.21.15.164:3300/0 conn(0x7f796815c5b0 0x7f79680c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 -- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 msgr2=0x7f79680b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:05.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 0x7f79680b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 -- 172.21.15.172:0/442159376 >> v2:172.21.15.164:3300/0 conn(0x7f796815c5b0 msgr2=0x7f79680c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:05.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 --2- 172.21.15.172:0/442159376 >> v2:172.21.15.164:3300/0 conn(0x7f796815c5b0 0x7f79680c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796d200640 1 -- 172.21.15.172:0/442159376 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680b6590 con 0x7f79680b5820 2024-05-08T04:38:05.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.839+0000 7f796dc00640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 0x7f79680b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:05.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f796d200640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796815c070 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79680719b0 tx=0x7f795401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa2c3944f24ebe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7967e00640 1 -- 172.21.15.172:0/442159376 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540b1070 con 0x7f79680b5820 2024-05-08T04:38:05.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7967e00640 1 -- 172.21.15.172:0/442159376 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79540ac620 con 0x7f79680b5820 2024-05-08T04:38:05.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7967e00640 1 -- 172.21.15.172:0/442159376 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540ac9f0 con 0x7f79680b5820 2024-05-08T04:38:05.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 -- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 msgr2=0x7f796815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796815c070 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79680719b0 tx=0x7f795401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 -- 172.21.15.172:0/442159376 shutdown_connections 2024-05-08T04:38:05.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 --2- 172.21.15.172:0/442159376 >> v2:172.21.15.164:3300/0 conn(0x7f796815c5b0 0x7f79680c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796815c070 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 --2- 172.21.15.172:0/442159376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815bc90 0x7f79680b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 -- 172.21.15.172:0/442159376 >> 172.21.15.172:0/442159376 conn(0x7f79680b35d0 msgr2=0x7f79680b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 -- 172.21.15.172:0/442159376 shutdown_connections 2024-05-08T04:38:05.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.843+0000 7f7970600640 1 -- 172.21.15.172:0/442159376 wait complete. 2024-05-08T04:38:05.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 Processor -- start 2024-05-08T04:38:05.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- start start 2024-05-08T04:38:05.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f796815bc90 0x7f796815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815c5b0 0x7f796815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680b7930 con 0x7f796815bc90 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680b77b0 con 0x7f79680b5820 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680c79f0 con 0x7f796815c5b0 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815c5b0 0x7f796815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37040/0 (socket says 172.21.15.172:37040) 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 -- 172.21.15.172:0/2690419479 learned_addr learned my addr 172.21.15.172:0/2690419479 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796d200640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f796815bc90 0x7f796815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 -- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815c5b0 msgr2=0x7f796815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815c5b0 0x7f796815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 -- 172.21.15.172:0/2690419479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f796815bc90 msgr2=0x7f796815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f796815bc90 0x7f796815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 -- 172.21.15.172:0/2690419479 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680cef50 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f796dc00640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f795c00d800 tx=0x7f795c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c018070 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c004030 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c004400 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79680ce340 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79680ce900 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f795c0096a0 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f795c003c60 con 0x7f79680b5820 2024-05-08T04:38:05.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.847+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796809fbc0 con 0x7f79680b5820 2024-05-08T04:38:05.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.851+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f795c003620 con 0x7f79680b5820 2024-05-08T04:38:05.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.919+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4386cf2-89cd-4873-8d2f-d1831eea8277", "id": 3} v 0) v1 -- 0x7f796804f010 con 0x7f79680b5820 2024-05-08T04:38:05.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7965600640 1 -- 172.21.15.172:0/2690419479 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4386cf2-89cd-4873-8d2f-d1831eea8277", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f795c009990 con 0x7f79680b5820 2024-05-08T04:38:05.928 INFO:teuthology.orchestra.run.smithi172.stdout:3 2024-05-08T04:38:05.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 msgr2=0x7f796814e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:05.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f795c00d800 tx=0x7f795c00dcc0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 shutdown_connections 2024-05-08T04:38:05.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796815c5b0 0x7f796815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f796815bc90 0x7f796815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 --2- 172.21.15.172:0/2690419479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79680b5820 0x7f796814e0d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:05.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 >> 172.21.15.172:0/2690419479 conn(0x7f79680b35d0 msgr2=0x7f7968151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:05.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 shutdown_connections 2024-05-08T04:38:05.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:05.923+0000 7f7970600640 1 -- 172.21.15.172:0/2690419479 wait complete. 2024-05-08T04:38:05.948 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new d500c818-d468-4b1e-ac4c-274147bc4bab 4 2024-05-08T04:38:05.983 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:05.979+0000 7ffadaa0a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T04:38:06.010 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.007+0000 7f428ee0a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T04:38:06.045 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.043+0000 7ffadaa0a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T04:38:06.074 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.071+0000 7f428ee0a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T04:38:06.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 Processor -- start 2024-05-08T04:38:06.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 -- start start 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff2c0b4180 0x7eff2c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7eff2c0cd3e0 0x7eff2c0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0cf340 con 0x7eff2c15bea0 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0d02f0 con 0x7eff2c0cd3e0 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff31800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c15c280 con 0x7eff2c0b4180 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff2a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff2c0b4180 0x7eff2c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.079+0000 7eff2aa00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7eff2c15bea0 0x7eff2c0cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56106/0 (socket says 172.21.15.172:56106) 2024-05-08T04:38:06.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2aa00640 1 -- 172.21.15.172:0/2556124081 learned_addr learned my addr 172.21.15.172:0/2556124081 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-05-08T04:38:06.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2b400640 1 --2- 172.21.15.172:0/2556124081 >> v2:172.21.15.164:3300/0 conn(0x7eff2c0cd3e0 0x7eff2c0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2a000640 1 -- 172.21.15.172:0/2556124081 >> v2:172.21.15.164:3300/0 conn(0x7eff2c0cd3e0 msgr2=0x7eff2c0c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:06.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2a000640 1 --2- 172.21.15.172:0/2556124081 >> v2:172.21.15.164:3300/0 conn(0x7eff2c0cd3e0 0x7eff2c0c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2a000640 1 -- 172.21.15.172:0/2556124081 >> v1:172.21.15.172:6789/0 conn(0x7eff2c15bea0 legacy=0x7eff2c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2a000640 1 -- 172.21.15.172:0/2556124081 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0d0540 con 0x7eff2c0b4180 2024-05-08T04:38:06.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff2a000640 1 --2- 172.21.15.172:0/2556124081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c0ccea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff2c071c60 tx=0x7eff1801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b992795b9873b27d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff29600640 1 -- 172.21.15.172:0/2556124081 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180ab650 con 0x7eff2c0b4180 2024-05-08T04:38:06.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff29600640 1 -- 172.21.15.172:0/2556124081 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180b0070 con 0x7eff2c0b4180 2024-05-08T04:38:06.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.083+0000 7eff29600640 1 -- 172.21.15.172:0/2556124081 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180aba10 con 0x7eff2c0b4180 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- 172.21.15.172:0/2556124081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 msgr2=0x7eff2c0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- 172.21.15.172:0/2556124081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c0ccea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff2c071c60 tx=0x7eff1801c450 comp rx=0 tx=0).stop 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- 172.21.15.172:0/2556124081 shutdown_connections 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- 172.21.15.172:0/2556124081 >> v2:172.21.15.164:3300/0 conn(0x7eff2c0cd3e0 0x7eff2c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- 172.21.15.172:0/2556124081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c0ccea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- 172.21.15.172:0/2556124081 >> 172.21.15.172:0/2556124081 conn(0x7eff2c0b35d0 msgr2=0x7eff2c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- 172.21.15.172:0/2556124081 shutdown_connections 2024-05-08T04:38:06.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- 172.21.15.172:0/2556124081 wait complete. 2024-05-08T04:38:06.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 Processor -- start 2024-05-08T04:38:06.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- start start 2024-05-08T04:38:06.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff2c0cd3e0 0x7eff2c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0c6b20 con 0x7eff2c0cd3e0 2024-05-08T04:38:06.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0c58b0 con 0x7eff2c0b4180 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff31800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0c6ca0 con 0x7eff2c15bea0 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39700/0 (socket says 172.21.15.172:39700) 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 -- 172.21.15.172:0/988719475 learned_addr learned my addr 172.21.15.172:0/988719475 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2a000640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff2c0cd3e0 0x7eff2c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 -- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 msgr2=0x7eff2c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 -- 172.21.15.172:0/988719475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff2c0cd3e0 msgr2=0x7eff2c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff2c0cd3e0 0x7eff2c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.087+0000 7eff2b400640 1 -- 172.21.15.172:0/988719475 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c1532c0 con 0x7eff2c15bea0 2024-05-08T04:38:06.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff2b400640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff2400efc0 tx=0x7eff2400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24018020 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24014480 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c1535a0 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24014870 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c153b60 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7eff2401b020 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7eff24014da0 con 0x7eff2c15bea0 2024-05-08T04:38:06.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c06bb70 con 0x7eff2c15bea0 2024-05-08T04:38:06.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.091+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7eff24022030 con 0x7eff2c15bea0 2024-05-08T04:38:06.105 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.103+0000 7ffadaa0a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T04:38:06.137 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.131+0000 7f428ee0a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T04:38:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.159+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d500c818-d468-4b1e-ac4c-274147bc4bab", "id": 4} v 0) v1 -- 0x7eff2c09fc30 con 0x7eff2c15bea0 2024-05-08T04:38:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.163+0000 7eff22a00640 1 -- 172.21.15.172:0/988719475 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d500c818-d468-4b1e-ac4c-274147bc4bab", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7eff24010040 con 0x7eff2c15bea0 2024-05-08T04:38:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:4 2024-05-08T04:38:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 msgr2=0x7eff2c15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff2400efc0 tx=0x7eff2400c490 comp rx=0 tx=0).stop 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 shutdown_connections 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff2c15bea0 0x7eff2c15d470 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff2c0cd3e0 0x7eff2c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 --2- 172.21.15.172:0/988719475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff2c0b4180 0x7eff2c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 >> 172.21.15.172:0/988719475 conn(0x7eff2c0b35d0 msgr2=0x7eff2c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 shutdown_connections 2024-05-08T04:38:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.167+0000 7eff31800640 1 -- 172.21.15.172:0/988719475 wait complete. 2024-05-08T04:38:06.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new e28992ff-444b-42c9-b859-87d309757426 5 2024-05-08T04:38:06.195 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.191+0000 7ffadaa0a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T04:38:06.229 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.223+0000 7f428ee0a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T04:38:06.255 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.251+0000 7ffadaa0a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T04:38:06.292 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.287+0000 7f428ee0a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T04:38:06.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 Processor -- start 2024-05-08T04:38:06.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 -- start start 2024-05-08T04:38:06.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafe40d0c20 0x7fafe40ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe40cfe60 con 0x7fafe40cdb10 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe40d0790 con 0x7fafe40d0c20 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40d0140 con 0x7fafe40ce760 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafdbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafe40d0c20 0x7fafe40ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe1e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fafe40cdb10 0x7fafe406efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56114/0 (socket says 172.21.15.172:56114) 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe1e00640 1 -- 172.21.15.172:0/174517435 learned_addr learned my addr 172.21.15.172:0/174517435 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-05-08T04:38:06.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafe2800640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafdbe00640 1 -- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 msgr2=0x7fafe40cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:06.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafdbe00640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe40cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafdbe00640 1 -- 172.21.15.172:0/174517435 >> v1:172.21.15.172:6789/0 conn(0x7fafe40cdb10 legacy=0x7fafe406efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.319+0000 7fafdbe00640 1 -- 172.21.15.172:0/174517435 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe40d0420 con 0x7fafe40d0c20 2024-05-08T04:38:06.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafdbe00640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe40ce220 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097c0 tx=0x7fafcc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4e706ae83493c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe1400640 1 -- 172.21.15.172:0/174517435 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0b0070 con 0x7fafe40d0c20 2024-05-08T04:38:06.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe1400640 1 -- 172.21.15.172:0/174517435 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0ab600 con 0x7fafe40d0c20 2024-05-08T04:38:06.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe1400640 1 -- 172.21.15.172:0/174517435 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0ab9d0 con 0x7fafe40d0c20 2024-05-08T04:38:06.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 -- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 msgr2=0x7fafe40ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe40ce220 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097c0 tx=0x7fafcc01c410 comp rx=0 tx=0).stop 2024-05-08T04:38:06.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 -- 172.21.15.172:0/174517435 shutdown_connections 2024-05-08T04:38:06.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe40cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 --2- 172.21.15.172:0/174517435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe40ce220 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 -- 172.21.15.172:0/174517435 >> 172.21.15.172:0/174517435 conn(0x7fafe40b34f0 msgr2=0x7fafe40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 -- 172.21.15.172:0/174517435 shutdown_connections 2024-05-08T04:38:06.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.323+0000 7fafe9200640 1 -- 172.21.15.172:0/174517435 wait complete. 2024-05-08T04:38:06.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 Processor -- start 2024-05-08T04:38:06.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 -- start start 2024-05-08T04:38:06.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 0x7fafe415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe414d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe416b050 con 0x7fafe40cdb10 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe416aed0 con 0x7fafe40d0c20 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe416b1d0 con 0x7fafe40ce760 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39720/0 (socket says 172.21.15.172:39720) 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe414d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 -- 172.21.15.172:0/4083177895 learned_addr learned my addr 172.21.15.172:0/4083177895 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe1e00640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 0x7fafe415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 -- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 msgr2=0x7fafe414d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe414d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 -- 172.21.15.172:0/4083177895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 msgr2=0x7fafe415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 0x7fafe415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 -- 172.21.15.172:0/4083177895 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe40c9060 con 0x7fafe40ce760 2024-05-08T04:38:06.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe1e00640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 0x7fafe415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:06.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe2800640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe414d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafdbe00640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01a040 tx=0x7fafcc0191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0ba070 con 0x7fafe40ce760 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0b5460 con 0x7fafe40ce760 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0b0040 con 0x7fafe40ce760 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.327+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe40b4a00 con 0x7fafe40ce760 2024-05-08T04:38:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.331+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe40b4cc0 con 0x7fafe40ce760 2024-05-08T04:38:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.331+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fafcc0bd070 con 0x7fafe40ce760 2024-05-08T04:38:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.331+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fafcc0b5850 con 0x7fafe40ce760 2024-05-08T04:38:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.331+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe409fae0 con 0x7fafe40ce760 2024-05-08T04:38:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.331+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fafcc016090 con 0x7fafe40ce760 2024-05-08T04:38:06.343 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.339+0000 7ffadaa0a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T04:38:06.384 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.379+0000 7f428ee0a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T04:38:06.404 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.403+0000 7ffadaa0a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T04:38:06.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.399+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e28992ff-444b-42c9-b859-87d309757426", "id": 5} v 0) v1 -- 0x7fafe404f010 con 0x7fafe40ce760 2024-05-08T04:38:06.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.403+0000 7fafda000640 1 -- 172.21.15.172:0/4083177895 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e28992ff-444b-42c9-b859-87d309757426", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fafcc0b5a50 con 0x7fafe40ce760 2024-05-08T04:38:06.409 INFO:teuthology.orchestra.run.smithi172.stdout:5 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 msgr2=0x7fafe414d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01a040 tx=0x7fafcc0191b0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 shutdown_connections 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafe40ce760 0x7fafe414d090 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafe40d0c20 0x7fafe414d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 --2- 172.21.15.172:0/4083177895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafe40cdb10 0x7fafe415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 >> 172.21.15.172:0/4083177895 conn(0x7fafe40b34f0 msgr2=0x7fafe4157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 shutdown_connections 2024-05-08T04:38:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.407+0000 7fafe9200640 1 -- 172.21.15.172:0/4083177895 wait complete. 2024-05-08T04:38:06.436 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new eda60f02-6743-495e-b97a-0fb6ff0d9bc6 6 2024-05-08T04:38:06.459 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.455+0000 7f428ee0a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T04:38:06.571 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.567+0000 7ffadaa0a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T04:38:06.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 Processor -- start 2024-05-08T04:38:06.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 -- start start 2024-05-08T04:38:06.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56c00d00c0 0x7f56c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 0x7f56c00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c00d0630 con 0x7f56c0071460 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c00b58f0 con 0x7f56c00cd8d0 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.587+0000 7f56c4e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c00b3970 con 0x7f56c00d00c0 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56c00d00c0 0x7f56c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56c00d00c0 0x7f56c00b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37082/0 (socket says 172.21.15.172:37082) 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 -- 172.21.15.172:0/1706270210 learned_addr learned my addr 172.21.15.172:0/1706270210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56be800640 1 --2- 172.21.15.172:0/1706270210 >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 0x7f56c00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 -- 172.21.15.172:0/1706270210 >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 msgr2=0x7f56c00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:06.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 --2- 172.21.15.172:0/1706270210 >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 0x7f56c00cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 -- 172.21.15.172:0/1706270210 >> v1:172.21.15.172:6789/0 conn(0x7f56c0071460 legacy=0x7f56c0071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 -- 172.21.15.172:0/1706270210 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00b3c50 con 0x7f56c00d00c0 2024-05-08T04:38:06.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56be800640 1 --2- 172.21.15.172:0/1706270210 >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 0x7f56c00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:06.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56b5400640 1 --2- 172.21.15.172:0/1706270210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00d00c0 0x7f56c00b4f80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a800bb10 tx=0x7f56a801eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6fab1db0d99a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56bd400640 1 -- 172.21.15.172:0/1706270210 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80b4070 con 0x7f56c00d00c0 2024-05-08T04:38:06.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56bd400640 1 -- 172.21.15.172:0/1706270210 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80a7b20 con 0x7f56c00d00c0 2024-05-08T04:38:06.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.591+0000 7f56bd400640 1 -- 172.21.15.172:0/1706270210 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80af440 con 0x7f56c00d00c0 2024-05-08T04:38:06.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- 172.21.15.172:0/1706270210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00d00c0 msgr2=0x7f56c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- 172.21.15.172:0/1706270210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00d00c0 0x7f56c00b4f80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a800bb10 tx=0x7f56a801eb60 comp rx=0 tx=0).stop 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- 172.21.15.172:0/1706270210 shutdown_connections 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- 172.21.15.172:0/1706270210 >> v2:172.21.15.164:3300/0 conn(0x7f56c00cd8d0 0x7f56c00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- 172.21.15.172:0/1706270210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00d00c0 0x7f56c00b4f80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- 172.21.15.172:0/1706270210 >> 172.21.15.172:0/1706270210 conn(0x7f56c00b2d00 msgr2=0x7f56c00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- 172.21.15.172:0/1706270210 shutdown_connections 2024-05-08T04:38:06.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- 172.21.15.172:0/1706270210 wait complete. 2024-05-08T04:38:06.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 Processor -- start 2024-05-08T04:38:06.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- start start 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 0x7f56c01439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c00d00c0 0x7f56c0150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00b48b0 con 0x7f56c00d00c0 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00b4730 con 0x7f56c00cd8d0 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56c4e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c006ac90 con 0x7f56c0071460 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37088/0 (socket says 172.21.15.172:37088) 2024-05-08T04:38:06.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56bde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 0x7f56c01439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 -- 172.21.15.172:0/955108757 learned_addr learned my addr 172.21.15.172:0/955108757 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56be800640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c00d00c0 0x7f56c0150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 -- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 msgr2=0x7f56c01439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 0x7f56c01439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 -- 172.21.15.172:0/955108757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c00d00c0 msgr2=0x7f56c0150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c00d00c0 0x7f56c0150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56b5400640 1 -- 172.21.15.172:0/955108757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0150c60 con 0x7f56c00cd8d0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.595+0000 7f56bde00640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 0x7f56c01439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b5400640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b2c0 tx=0x7f56a801b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80b4040 con 0x7f56c00cd8d0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80a7920 con 0x7f56c00cd8d0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c0150f40 con 0x7f56c00cd8d0 2024-05-08T04:38:06.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80ba6a0 con 0x7f56c00cd8d0 2024-05-08T04:38:06.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0151500 con 0x7f56c00cd8d0 2024-05-08T04:38:06.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f56a80c1020 con 0x7f56c00cd8d0 2024-05-08T04:38:06.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f56a80c3dc0 con 0x7f56c00cd8d0 2024-05-08T04:38:06.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0072e80 con 0x7f56c00cd8d0 2024-05-08T04:38:06.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.599+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f56a80c8020 con 0x7f56c00cd8d0 2024-05-08T04:38:06.632 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.627+0000 7f428ee0a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T04:38:06.642 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:06.639+0000 7ffadaa0a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T04:38:06.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.667+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eda60f02-6743-495e-b97a-0fb6ff0d9bc6", "id": 6} v 0) v1 -- 0x7f56c00cddf0 con 0x7f56c00cd8d0 2024-05-08T04:38:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56b6a00640 1 -- 172.21.15.172:0/955108757 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eda60f02-6743-495e-b97a-0fb6ff0d9bc6", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f56a80187e0 con 0x7f56c00cd8d0 2024-05-08T04:38:06.677 INFO:teuthology.orchestra.run.smithi172.stdout:6 2024-05-08T04:38:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 msgr2=0x7f56c014fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b2c0 tx=0x7f56a801b900 comp rx=0 tx=0).stop 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 shutdown_connections 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c00d00c0 0x7f56c0150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c00cd8d0 0x7f56c014fc10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 --2- 172.21.15.172:0/955108757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c0071460 0x7f56c01439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 >> 172.21.15.172:0/955108757 conn(0x7f56c00b2d00 msgr2=0x7f56c00ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 shutdown_connections 2024-05-08T04:38:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.671+0000 7f56c4e00640 1 -- 172.21.15.172:0/955108757 wait complete. 2024-05-08T04:38:06.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 67d26eaa-8ce3-4724-8384-b144879d1272 7 2024-05-08T04:38:06.706 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:06.703+0000 7f428ee0a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T04:38:06.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 Processor -- start 2024-05-08T04:38:06.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 -- start start 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f925806ac90 0x7f925806b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580727c0 con 0x7f92580723e0 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925815e790 con 0x7f925815e270 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925806b620 con 0x7f925806ac90 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9257400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f925815e270 0x7f92580cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56134/0 (socket says 172.21.15.172:56134) 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9257400640 1 -- 172.21.15.172:0/2131407877 learned_addr learned my addr 172.21.15.172:0/2131407877 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-05-08T04:38:06.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f925cc00640 1 --2- 172.21.15.172:0/2131407877 >> v2:172.21.15.164:3300/0 conn(0x7f925806ac90 0x7f925806b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334117274 0 0) 0x7f925815e790 con 0x7f925815e270 2024-05-08T04:38:06.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923c003530 con 0x7f925815e270 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228343075 0 0) 0x7f92580727c0 con 0x7f92580723e0 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.827+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925815e790 con 0x7f92580723e0 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.831+0000 7f925cc00640 1 -- 172.21.15.172:0/2131407877 >> v1:172.21.15.172:6789/0 conn(0x7f925815e270 legacy=0x7f92580cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.831+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333632398 0 0) 0x7f923c003530 con 0x7f925815e270 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.831+0000 7f925cc00640 1 -- 172.21.15.172:0/2131407877 >> v1:172.21.15.164:6789/0 conn(0x7f92580723e0 legacy=0x7f92580714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.831+0000 7f925cc00640 1 -- 172.21.15.172:0/2131407877 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925815ea70 con 0x7f925806ac90 2024-05-08T04:38:06.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925cc00640 1 --2- 172.21.15.172:0/2131407877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925806ac90 0x7f925806b0e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d90 tx=0x7f925001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f91f8d3328e2ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244001680 con 0x7f925806ac90 2024-05-08T04:38:06.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92500a6500 con 0x7f925806ac90 2024-05-08T04:38:06.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f9256a00640 1 -- 172.21.15.172:0/2131407877 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500a68d0 con 0x7f925806ac90 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- 172.21.15.172:0/2131407877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925806ac90 msgr2=0x7f925806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 --2- 172.21.15.172:0/2131407877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925806ac90 0x7f925806b0e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d90 tx=0x7f925001c8b0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- 172.21.15.172:0/2131407877 shutdown_connections 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 --2- 172.21.15.172:0/2131407877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925806ac90 0x7f925806b0e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- 172.21.15.172:0/2131407877 >> 172.21.15.172:0/2131407877 conn(0x7f925806bda0 msgr2=0x7f925806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- 172.21.15.172:0/2131407877 shutdown_connections 2024-05-08T04:38:06.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- 172.21.15.172:0/2131407877 wait complete. 2024-05-08T04:38:06.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 Processor -- start 2024-05-08T04:38:06.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.835+0000 7f925ec00640 1 -- start start 2024-05-08T04:38:06.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925806ac90 0x7f92581f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925815e270 0x7f925815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581626e0 con 0x7f925815e270 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580ce0d0 con 0x7f925806ac90 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92581f8090 con 0x7f92580723e0 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925806ac90 0x7f92581f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:39740/0 (socket says 172.21.15.172:39740) 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 -- 172.21.15.172:0/2362563156 learned_addr learned my addr 172.21.15.172:0/2362563156 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925cc00640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925815e270 0x7f925815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 -- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925806ac90 msgr2=0x7f92581f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925806ac90 0x7f92581f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 -- 172.21.15.172:0/2362563156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925815e270 msgr2=0x7f925815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925815e270 0x7f925815ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 -- 172.21.15.172:0/2362563156 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92581f7ad0 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f9257400640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f924400c500 tx=0x7f924400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244018070 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9244007560 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244004a30 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92581f7db0 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92580ae950 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f924401b070 con 0x7f92580723e0 2024-05-08T04:38:06.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9244004bd0 con 0x7f92580723e0 2024-05-08T04:38:06.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.839+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9258072f00 con 0x7f92580723e0 2024-05-08T04:38:06.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.843+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9244022020 con 0x7f92580723e0 2024-05-08T04:38:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.907+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "67d26eaa-8ce3-4724-8384-b144879d1272", "id": 7} v 0) v1 -- 0x7f925804f010 con 0x7f92580723e0 2024-05-08T04:38:06.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.911+0000 7f924fe00640 1 -- 172.21.15.172:0/2362563156 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "67d26eaa-8ce3-4724-8384-b144879d1272", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f924401d440 con 0x7f92580723e0 2024-05-08T04:38:06.917 INFO:teuthology.orchestra.run.smithi172.stdout:7 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.911+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 msgr2=0x7f92581f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.911+0000 7f925ec00640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f924400c500 tx=0x7f924400c9c0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 shutdown_connections 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92580723e0 0x7f92581f7110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f925815e270 0x7f925815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 --2- 172.21.15.172:0/2362563156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925806ac90 0x7f92581f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 >> 172.21.15.172:0/2362563156 conn(0x7f925806bda0 msgr2=0x7f92580cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 shutdown_connections 2024-05-08T04:38:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-08T04:38:06.915+0000 7f925ec00640 1 -- 172.21.15.172:0/2362563156 wait complete. 2024-05-08T04:38:06.935 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T04:38:06.935 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T04:38:06.938 INFO:tasks.ceph.osd.0:Started 2024-05-08T04:38:06.939 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T04:38:06.939 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T04:38:06.944 INFO:tasks.ceph.osd.1:Started 2024-05-08T04:38:06.944 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T04:38:06.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T04:38:06.947 INFO:tasks.ceph.osd.2:Started 2024-05-08T04:38:06.947 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T04:38:06.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T04:38:06.950 INFO:tasks.ceph.osd.3:Started 2024-05-08T04:38:06.950 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T04:38:06.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T04:38:06.984 INFO:tasks.ceph.osd.4:Started 2024-05-08T04:38:06.984 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T04:38:06.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T04:38:06.987 INFO:tasks.ceph.osd.5:Started 2024-05-08T04:38:06.987 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T04:38:06.987 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T04:38:06.990 INFO:tasks.ceph.osd.6:Started 2024-05-08T04:38:06.990 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T04:38:06.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T04:38:06.993 INFO:tasks.ceph.osd.7:Started 2024-05-08T04:38:06.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:07.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 Processor -- start 2024-05-08T04:38:07.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 -- start start 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb5a40b26c0 0x7fb5a40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb5a40cfbd0 0x7fb5a40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a40d0060 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a4072640 con 0x7fb5a40b26c0 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a406c770 con 0x7fb5a415ab60 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb5a40cfbd0 0x7fb5a40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb5a40cfbd0 0x7fb5a40cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38700/0 (socket says 172.21.15.164:38700) 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 -- 172.21.15.164:0/1874404112 learned_addr learned my addr 172.21.15.164:0/1874404112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a2a00640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb59a000640 1 --2- 172.21.15.164:0/1874404112 >> v2:172.21.15.172:3300/0 conn(0x7fb5a40b26c0 0x7fb5a40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 -- 172.21.15.164:0/1874404112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 msgr2=0x7fb5a4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 -- 172.21.15.164:0/1874404112 >> v2:172.21.15.172:3300/0 conn(0x7fb5a40b26c0 msgr2=0x7fb5a40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 --2- 172.21.15.164:0/1874404112 >> v2:172.21.15.172:3300/0 conn(0x7fb5a40b26c0 0x7fb5a40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.171+0000 7fb5a3400640 1 -- 172.21.15.164:0/1874404112 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a406ca50 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a3400640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a40cd910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4067430 tx=0x7fb59c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9787f9c262578b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a2000640 1 -- 172.21.15.164:0/1874404112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0b0070 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a2000640 1 -- 172.21.15.164:0/1874404112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c0ab600 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a2000640 1 -- 172.21.15.164:0/1874404112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0ab9d0 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 -- 172.21.15.164:0/1874404112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40cfbd0 msgr2=0x7fb5a40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a40cd910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4067430 tx=0x7fb59c01c520 comp rx=0 tx=0).stop 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 -- 172.21.15.164:0/1874404112 shutdown_connections 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a40cd910 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 --2- 172.21.15.164:0/1874404112 >> v2:172.21.15.172:3300/0 conn(0x7fb5a40b26c0 0x7fb5a40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 --2- 172.21.15.164:0/1874404112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 -- 172.21.15.164:0/1874404112 >> 172.21.15.164:0/1874404112 conn(0x7fb5a406ac90 msgr2=0x7fb5a406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 -- 172.21.15.164:0/1874404112 shutdown_connections 2024-05-08T04:38:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.175+0000 7fb5a9800640 1 -- 172.21.15.164:0/1874404112 wait complete. 2024-05-08T04:38:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 Processor -- start 2024-05-08T04:38:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- start start 2024-05-08T04:38:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a4140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a406c550 con 0x7fb5a40b26c0 2024-05-08T04:38:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a406c3d0 con 0x7fb5a40cfbd0 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40aed30 con 0x7fb5a415ab60 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38714/0 (socket says 172.21.15.164:38714) 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 -- 172.21.15.164:0/2514718925 learned_addr learned my addr 172.21.15.164:0/2514718925 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59a000640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a4140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a3400640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 -- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 msgr2=0x7fb5a4153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 -- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 msgr2=0x7fb5a4140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a4140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 -- 172.21.15.164:0/2514718925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a414c590 con 0x7fb5a40b26c0 2024-05-08T04:38:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59a000640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a4140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a2a00640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb59400efc0 tx=0x7fb59400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594018020 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb594014480 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a40afec0 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594014850 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a40b0480 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.179+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb59401b020 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.183+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb59401de00 con 0x7fb5a40b26c0 2024-05-08T04:38:07.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.183+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a4069d60 con 0x7fb5a40b26c0 2024-05-08T04:38:07.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.183+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb594022030 con 0x7fb5a40b26c0 2024-05-08T04:38:07.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.239+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5a41549a0 con 0x7fb5a40b26c0 2024-05-08T04:38:07.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.239+0000 7fb59b400640 1 -- 172.21.15.164:0/2514718925 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fb594010040 con 0x7fb5a40b26c0 2024-05-08T04:38:07.243 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:07.243 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":10,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:06.913892+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:07.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 msgr2=0x7fb5a41526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:07.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb59400efc0 tx=0x7fb59400c490 comp rx=0 tx=0).stop 2024-05-08T04:38:07.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 shutdown_connections 2024-05-08T04:38:07.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5a415ab60 0x7fb5a4153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5a40cfbd0 0x7fb5a4140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 --2- 172.21.15.164:0/2514718925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a40b26c0 0x7fb5a41526d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:07.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 >> 172.21.15.164:0/2514718925 conn(0x7fb5a406ac90 msgr2=0x7fb5a406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:07.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 shutdown_connections 2024-05-08T04:38:07.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:07.243+0000 7fb5a9800640 1 -- 172.21.15.164:0/2514718925 wait complete. 2024-05-08T04:38:07.262 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T04:38:07.262 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T04:38:07.331 INFO:tasks.ceph.mgr.y.smithi164.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-08T04:38:07.331 INFO:tasks.ceph.mgr.y.smithi164.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-08T04:38:07.331 INFO:tasks.ceph.mgr.y.smithi164.stderr: from numpy import show_config as show_numpy_config 2024-05-08T04:38:07.359 INFO:tasks.ceph.mgr.x.smithi172.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-08T04:38:07.359 INFO:tasks.ceph.mgr.x.smithi172.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-08T04:38:07.359 INFO:tasks.ceph.mgr.x.smithi172.stderr: from numpy import show_config as show_numpy_config 2024-05-08T04:38:07.379 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:07.375+0000 7ffadaa0a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T04:38:07.422 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:07.419+0000 7f428ee0a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T04:38:07.527 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:07.523+0000 7ffadaa0a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:38:07.568 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:07.563+0000 7f428ee0a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:38:07.620 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-05-08T04:38:07.619+0000 7faf01a127c0 -1 Falling back to public interface 2024-05-08T04:38:07.645 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-05-08T04:38:07.643+0000 7f8fb96697c0 -1 Falling back to public interface 2024-05-08T04:38:07.646 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-05-08T04:38:07.643+0000 7fefb7f057c0 -1 Falling back to public interface 2024-05-08T04:38:07.656 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-05-08T04:38:07.655+0000 7f02c743b7c0 -1 Falling back to public interface 2024-05-08T04:38:07.683 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-05-08T04:38:07.679+0000 7f7aba9057c0 -1 Falling back to public interface 2024-05-08T04:38:07.690 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-05-08T04:38:07.687+0000 7f872f46b7c0 -1 Falling back to public interface 2024-05-08T04:38:07.694 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-05-08T04:38:07.691+0000 7f0eb2a8a7c0 -1 Falling back to public interface 2024-05-08T04:38:07.702 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-05-08T04:38:07.699+0000 7f91f26c17c0 -1 Falling back to public interface 2024-05-08T04:38:07.716 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:07.715+0000 7ffadaa0a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T04:38:07.743 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:07.739+0000 7f428ee0a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T04:38:07.788 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:07.787+0000 7ffadaa0a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T04:38:07.826 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:07.823+0000 7f428ee0a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T04:38:07.937 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:07.935+0000 7ffadaa0a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:38:07.973 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:07.967+0000 7f428ee0a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:38:08.066 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:08.194 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.191+0000 7fd254000640 1 Processor -- start 2024-05-08T04:38:08.194 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.191+0000 7fd254000640 1 -- start start 2024-05-08T04:38:08.196 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd254000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd24c0d3620 0x7fd24c0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:08.196 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd254000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:08.196 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd254000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0d2580 con 0x7fd24c15dce0 2024-05-08T04:38:08.196 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd254000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0d2320 con 0x7fd24c0d3620 2024-05-08T04:38:08.196 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd254000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b7c40 con 0x7fd24c15e0c0 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd252000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd252000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:60688/0 (socket says 172.21.15.164:60688) 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd252000640 1 -- 172.21.15.164:0/443123022 learned_addr learned my addr 172.21.15.164:0/443123022 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd250c00640 1 --2- 172.21.15.164:0/443123022 >> v2:172.21.15.172:3300/0 conn(0x7fd24c0d3620 0x7fd24c0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd250c00640 1 -- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 msgr2=0x7fd24c0b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd250c00640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd250c00640 1 -- 172.21.15.164:0/443123022 >> v1:172.21.15.172:6789/0 conn(0x7fd24c15dce0 legacy=0x7fd24c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd250c00640 1 -- 172.21.15.164:0/443123022 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0d2860 con 0x7fd24c0d3620 2024-05-08T04:38:08.197 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.195+0000 7fd252000640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:08.199 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd250c00640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c0d3620 0x7fd24c0b5580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097f0 tx=0x7fd23801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a371c450765aa59d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:08.200 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd24be00640 1 -- 172.21.15.164:0/443123022 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380a7070 con 0x7fd24c0d3620 2024-05-08T04:38:08.200 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd24be00640 1 -- 172.21.15.164:0/443123022 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2380b0070 con 0x7fd24c0d3620 2024-05-08T04:38:08.200 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd24be00640 1 -- 172.21.15.164:0/443123022 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380ab890 con 0x7fd24c0d3620 2024-05-08T04:38:08.200 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c0d3620 msgr2=0x7fd24c0b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c0d3620 0x7fd24c0b5580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097f0 tx=0x7fd23801c470 comp rx=0 tx=0).stop 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- 172.21.15.164:0/443123022 shutdown_connections 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15e0c0 0x7fd24c0b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- 172.21.15.164:0/443123022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c0d3620 0x7fd24c0b5580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- 172.21.15.164:0/443123022 >> 172.21.15.164:0/443123022 conn(0x7fd24c0718b0 msgr2=0x7fd24c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- 172.21.15.164:0/443123022 shutdown_connections 2024-05-08T04:38:08.201 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- 172.21.15.164:0/443123022 wait complete. 2024-05-08T04:38:08.202 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 Processor -- start 2024-05-08T04:38:08.202 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- start start 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15dce0 0x7fd24c0d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c145140 0x7fd24c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b75c0 con 0x7fd24c15e0c0 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b7440 con 0x7fd24c145140 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd254000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b7740 con 0x7fd24c15dce0 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd250c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd250c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38718/0 (socket says 172.21.15.164:38718) 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd250c00640 1 -- 172.21.15.164:0/3156300064 learned_addr learned my addr 172.21.15.164:0/3156300064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.199+0000 7fd251600640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15dce0 0x7fd24c0d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:08.204 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd252000640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c145140 0x7fd24c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 -- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15dce0 msgr2=0x7fd24c0d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15dce0 0x7fd24c0d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 -- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c145140 msgr2=0x7fd24c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c145140 0x7fd24c0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 -- 172.21.15.164:0/3156300064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0cabd0 con 0x7fd24c15e0c0 2024-05-08T04:38:08.205 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd250c00640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd23801a0a0 tx=0x7fd23801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380bf610 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2380bf7d0 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0caeb0 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380bfbc0 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0cb470 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd2380bd020 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd2380b4a60 con 0x7fd24c15e0c0 2024-05-08T04:38:08.206 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.203+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c068610 con 0x7fd24c15e0c0 2024-05-08T04:38:08.207 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.207+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd2380c6030 con 0x7fd24c15e0c0 2024-05-08T04:38:08.265 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd24c168800 con 0x7fd24c15e0c0 2024-05-08T04:38:08.265 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd249600640 1 -- 172.21.15.164:0/3156300064 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fd238016090 con 0x7fd24c15e0c0 2024-05-08T04:38:08.266 INFO:teuthology.misc.health.smithi164.stdout: 2024-05-08T04:38:08.266 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":10,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:06.913892+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 msgr2=0x7fd24c0d3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd23801a0a0 tx=0x7fd23801acb0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 shutdown_connections 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd24c145140 0x7fd24c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd24c15e0c0 0x7fd24c0d3fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 --2- 172.21.15.164:0/3156300064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd24c15dce0 0x7fd24c0d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 >> 172.21.15.164:0/3156300064 conn(0x7fd24c0718b0 msgr2=0x7fd24c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 shutdown_connections 2024-05-08T04:38:08.267 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:08.263+0000 7fd254000640 1 -- 172.21.15.164:0/3156300064 wait complete. 2024-05-08T04:38:08.283 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T04:38:08.641 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:08.639+0000 7ffadaa0a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T04:38:08.654 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:08.651+0000 7f428ee0a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T04:38:08.728 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:08.727+0000 7ffadaa0a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T04:38:08.739 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:08.735+0000 7f428ee0a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T04:38:08.930 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:08.927+0000 7ffadaa0a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T04:38:08.939 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:08.935+0000 7f428ee0a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T04:38:09.015 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:09.011+0000 7ffadaa0a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T04:38:09.023 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:09.019+0000 7f428ee0a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T04:38:09.345 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:09.343+0000 7ffadaa0a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T04:38:09.348 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:09.343+0000 7f428ee0a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T04:38:09.416 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:09.415+0000 7ffadaa0a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T04:38:09.418 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-05-08T04:38:09.415+0000 7f428ee0a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T04:38:10.438 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:10.435+0000 7ffa80a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T04:38:10.438 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:38:10.435+0000 7ffa80a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T04:38:11.570 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-05-08T04:38:11.567+0000 7faf01a127c0 -1 osd.0 0 log_to_monitors true 2024-05-08T04:38:11.592 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-05-08T04:38:11.587+0000 7f872f46b7c0 -1 osd.7 0 log_to_monitors true 2024-05-08T04:38:11.662 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-05-08T04:38:11.659+0000 7f8fb96697c0 -1 osd.1 0 log_to_monitors true 2024-05-08T04:38:11.669 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-05-08T04:38:11.667+0000 7fefb7f057c0 -1 osd.3 0 log_to_monitors true 2024-05-08T04:38:11.680 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-05-08T04:38:11.675+0000 7f0eb2a8a7c0 -1 osd.5 0 log_to_monitors true 2024-05-08T04:38:11.719 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-05-08T04:38:11.715+0000 7f02c743b7c0 -1 osd.2 0 log_to_monitors true 2024-05-08T04:38:11.891 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-05-08T04:38:11.887+0000 7f7aba9057c0 -1 osd.4 0 log_to_monitors true 2024-05-08T04:38:11.973 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-05-08T04:38:11.967+0000 7f91f26c17c0 -1 osd.6 0 log_to_monitors true 2024-05-08T04:38:15.089 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:15.288 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 Processor -- start 2024-05-08T04:38:15.289 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 -- start start 2024-05-08T04:38:15.290 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e3470 0x7f05d40e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05d40e4140 0x7f05d4052a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d40e4660 con 0x7f05d4052fa0 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d40e4920 con 0x7f05d40e4140 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05e0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d40391c0 con 0x7f05d40e3470 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05dbe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f05d4052fa0 0x7f05d40399c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:43580/0 (socket says 172.21.15.164:43580) 2024-05-08T04:38:15.291 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05dbe00640 1 -- 172.21.15.164:0/3089908270 learned_addr learned my addr 172.21.15.164:0/3089908270 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05daa00640 1 --2- 172.21.15.164:0/3089908270 >> v2:172.21.15.172:3300/0 conn(0x7f05d40e4140 0x7f05d4052a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.287+0000 7f05db400640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e3470 0x7f05d40e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05da000640 1 -- 172.21.15.164:0/3089908270 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783558532 0 0) 0x7f05d40e4660 con 0x7f05d4052fa0 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05da000640 1 -- 172.21.15.164:0/3089908270 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c4003530 con 0x7f05d4052fa0 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05daa00640 1 -- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e3470 msgr2=0x7f05d40e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05daa00640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e3470 0x7f05d40e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05daa00640 1 -- 172.21.15.164:0/3089908270 >> v1:172.21.15.172:6789/0 conn(0x7f05d4052fa0 legacy=0x7f05d40399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.292 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.291+0000 7f05daa00640 1 -- 172.21.15.164:0/3089908270 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40394a0 con 0x7f05d40e4140 2024-05-08T04:38:15.296 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05daa00640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d40e4140 0x7f05d4052a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097c0 tx=0x7f05c801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe564f4dafa5941 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:15.296 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05da000640 1 -- 172.21.15.164:0/3089908270 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80b0070 con 0x7f05d40e4140 2024-05-08T04:38:15.296 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05da000640 1 -- 172.21.15.164:0/3089908270 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8004230 con 0x7f05d40e4140 2024-05-08T04:38:15.296 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05da000640 1 -- 172.21.15.164:0/3089908270 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8003620 con 0x7f05d40e4140 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 -- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d40e4140 msgr2=0x7f05d4052a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d40e4140 0x7f05d4052a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097c0 tx=0x7f05c801c480 comp rx=0 tx=0).stop 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 -- 172.21.15.164:0/3089908270 shutdown_connections 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d40e4140 0x7f05d4052a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 --2- 172.21.15.164:0/3089908270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e3470 0x7f05d40e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.298 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 -- 172.21.15.164:0/3089908270 >> 172.21.15.164:0/3089908270 conn(0x7f05d400a410 msgr2=0x7f05d400a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:15.299 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 -- 172.21.15.164:0/3089908270 shutdown_connections 2024-05-08T04:38:15.299 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.295+0000 7f05e0e00640 1 -- 172.21.15.164:0/3089908270 wait complete. 2024-05-08T04:38:15.301 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.299+0000 7f05e0e00640 1 Processor -- start 2024-05-08T04:38:15.303 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.299+0000 7f05e0e00640 1 -- start start 2024-05-08T04:38:15.303 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.299+0000 7f05e0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 0x7f05d40d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.303 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.299+0000 7f05e0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 0x7f05d40cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.299+0000 7f05daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 0x7f05d40cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 0x7f05d40cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55770/0 (socket says 172.21.15.164:55770) 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 0x7f05d40d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05daa00640 1 -- 172.21.15.164:0/4045915959 learned_addr learned my addr 172.21.15.164:0/4045915959 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e4140 0x7f05d41848f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d403a250 con 0x7f05d40e3470 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d403a0d0 con 0x7f05d4052fa0 2024-05-08T04:38:15.304 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4006b80 con 0x7f05d40e4140 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 -- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e4140 msgr2=0x7f05d41848f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e4140 0x7f05d41848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 -- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 msgr2=0x7f05d40cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 0x7f05d40cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40d0f00 con 0x7f05d4052fa0 2024-05-08T04:38:15.305 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05db400640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 0x7f05d40d16a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e8a0 tx=0x7f05d000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:15.306 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0017070 con 0x7f05d4052fa0 2024-05-08T04:38:15.306 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000c940 con 0x7f05d4052fa0 2024-05-08T04:38:15.306 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d40d11e0 con 0x7f05d4052fa0 2024-05-08T04:38:15.306 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d40d3670 con 0x7f05d4052fa0 2024-05-08T04:38:15.306 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000ccf0 con 0x7f05d4052fa0 2024-05-08T04:38:15.307 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.303+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d4008e30 con 0x7f05d4052fa0 2024-05-08T04:38:15.312 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.307+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f05d001a070 con 0x7f05d4052fa0 2024-05-08T04:38:15.312 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.307+0000 7f05cf400640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 0x7f05bc069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:15.312 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.307+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f05d0084800 con 0x7f05d4052fa0 2024-05-08T04:38:15.312 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.311+0000 7f05daa00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 0x7f05bc069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:15.314 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.311+0000 7f05daa00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 0x7f05bc069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097c0 tx=0x7f05c80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:15.319 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.315+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05d00560a0 con 0x7f05d4052fa0 2024-05-08T04:38:15.635 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-05-08T04:38:15.631+0000 7f7ab6c00640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T04:38:15.636 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-05-08T04:38:15.635+0000 7f02c1800640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T04:38:15.663 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-05-08T04:38:15.659+0000 7f7aab800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.665 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.663+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05d40057f0 con 0x7f05d4052fa0 2024-05-08T04:38:15.669 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-05-08T04:38:15.667+0000 7f02b8200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.670 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-05-08T04:38:15.667+0000 7f8729a00640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T04:38:15.670 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.667+0000 7f05cf400640 1 -- 172.21.15.164:0/4045915959 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f05d0059f00 con 0x7f05d4052fa0 2024-05-08T04:38:15.671 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-05-08T04:38:15.667+0000 7f91ecc00640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T04:38:15.673 INFO:teuthology.misc.health.smithi164.stdout: 2024-05-08T04:38:15.673 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":12,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:13.436373+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 msgr2=0x7f05bc069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 0x7f05bc069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097c0 tx=0x7f05c80a6000 comp rx=0 tx=0).stop 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 msgr2=0x7f05d40d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 0x7f05d40d16a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e8a0 tx=0x7f05d000ed60 comp rx=0 tx=0).stop 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 shutdown_connections 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066ed0 0x7f05bc069390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05d40e4140 0x7f05d41848f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.678 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05d40e3470 0x7f05d40cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.679 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 --2- 172.21.15.164:0/4045915959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05d4052fa0 0x7f05d40d16a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:15.679 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 >> 172.21.15.164:0/4045915959 conn(0x7f05d400a410 msgr2=0x7f05d404fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:15.679 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 shutdown_connections 2024-05-08T04:38:15.679 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:15.675+0000 7f05e0e00640 1 -- 172.21.15.164:0/4045915959 wait complete. 2024-05-08T04:38:15.682 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-05-08T04:38:15.679+0000 7f8fb3c00640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T04:38:15.696 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-05-08T04:38:15.695+0000 7faefbe00640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T04:38:15.709 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T04:38:15.715 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-05-08T04:38:15.711+0000 7f8720400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.715 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-05-08T04:38:15.711+0000 7f91e3600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.725 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-05-08T04:38:15.723+0000 7f8faa600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.731 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-05-08T04:38:15.727+0000 7f0ead000640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T04:38:15.739 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-05-08T04:38:15.735+0000 7faef2800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.775 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-05-08T04:38:15.771+0000 7f0ea3a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:15.845 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-05-08T04:38:15.843+0000 7fefb2400640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T04:38:15.869 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-05-08T04:38:15.867+0000 7fefa8e00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:38:22.516 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:22.649 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 Processor -- start 2024-05-08T04:38:22.649 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 -- start start 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae980c8530 0x7fae980cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae980cf220 con 0x7fae9815e290 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9815ece0 con 0x7fae980d2000 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9f400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae980ce850 con 0x7fae980c8530 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae8fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fae9815e290 0x7fae980d18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44876/0 (socket says 172.21.15.164:44876) 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae8fe00640 1 -- 172.21.15.164:0/3220203870 learned_addr learned my addr 172.21.15.164:0/3220203870 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:22.651 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae9ca00640 1 --2- 172.21.15.164:0/3220203870 >> v2:172.21.15.172:3300/0 conn(0x7fae980c8530 0x7fae980cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838758807 0 0) 0x7fae980cf220 con 0x7fae9815e290 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c003530 con 0x7fae9815e290 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226421986 0 0) 0x7fae9815ece0 con 0x7fae980d2000 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.647+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae980cf220 con 0x7fae980d2000 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9ca00640 1 -- 172.21.15.164:0/3220203870 >> v1:172.21.15.172:6789/0 conn(0x7fae980d2000 legacy=0x7fae9815b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9ca00640 1 -- 172.21.15.164:0/3220203870 >> v1:172.21.15.164:6789/0 conn(0x7fae9815e290 legacy=0x7fae980d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.652 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9ca00640 1 -- 172.21.15.164:0/3220203870 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9815efc0 con 0x7fae980c8530 2024-05-08T04:38:22.655 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9ca00640 1 --2- 172.21.15.164:0/3220203870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae980c8530 0x7fae980cece0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae88009820 tx=0x7fae8801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1796a805e987fd95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:22.655 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880b1070 con 0x7fae980c8530 2024-05-08T04:38:22.655 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880ac5f0 con 0x7fae980c8530 2024-05-08T04:38:22.655 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae97e00640 1 -- 172.21.15.164:0/3220203870 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ac9a0 con 0x7fae980c8530 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9f400640 1 -- 172.21.15.164:0/3220203870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae980c8530 msgr2=0x7fae980cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9f400640 1 --2- 172.21.15.164:0/3220203870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae980c8530 0x7fae980cece0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae88009820 tx=0x7fae8801c480 comp rx=0 tx=0).stop 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9f400640 1 -- 172.21.15.164:0/3220203870 shutdown_connections 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.651+0000 7fae9f400640 1 --2- 172.21.15.164:0/3220203870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae980c8530 0x7fae980cece0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- 172.21.15.164:0/3220203870 >> 172.21.15.164:0/3220203870 conn(0x7fae9809aa60 msgr2=0x7fae9809ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- 172.21.15.164:0/3220203870 shutdown_connections 2024-05-08T04:38:22.656 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- 172.21.15.164:0/3220203870 wait complete. 2024-05-08T04:38:22.657 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 Processor -- start 2024-05-08T04:38:22.657 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- start start 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae980d2000 0x7fae980b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 0x7fae980b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c74e0 con 0x7fae980c8530 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9806b900 con 0x7fae9815e290 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c7660 con 0x7fae980d2000 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:22.659 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48060/0 (socket says 172.21.15.164:48060) 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 -- 172.21.15.164:0/3511904003 learned_addr learned my addr 172.21.15.164:0/3511904003 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae8fe00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae980d2000 0x7fae980b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9d400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 0x7fae980b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 -- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae980d2000 msgr2=0x7fae980b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae980d2000 0x7fae980b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 -- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 msgr2=0x7fae980b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 0x7fae980b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 -- 172.21.15.164:0/3511904003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9815b540 con 0x7fae980c8530 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9d400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 0x7fae980b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:22.660 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae9ca00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae8801a0b0 tx=0x7fae88019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.655+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ba020 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880ac9a0 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae980b47d0 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880b1040 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae980b4d70 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae98068610 con 0x7fae980c8530 2024-05-08T04:38:22.661 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae880bd020 con 0x7fae980c8530 2024-05-08T04:38:22.662 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae95600640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 0x7fae74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:22.662 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae8811e8f0 con 0x7fae980c8530 2024-05-08T04:38:22.662 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae8fe00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 0x7fae74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:22.663 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.659+0000 7fae8fe00640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 0x7fae74069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae9815db70 tx=0x7fae80007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:22.667 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.663+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae880c4030 con 0x7fae980c8530 2024-05-08T04:38:22.823 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.819+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae9816aa10 con 0x7fae980c8530 2024-05-08T04:38:22.824 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae95600640 1 -- 172.21.15.164:0/3511904003 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7fae880a9070 con 0x7fae980c8530 2024-05-08T04:38:22.824 INFO:teuthology.misc.health.smithi164.stdout: 2024-05-08T04:38:22.824 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":16,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:19.677806+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:22.826 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 msgr2=0x7fae74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.826 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 0x7fae74069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae9815db70 tx=0x7fae80007d80 comp rx=0 tx=0).stop 2024-05-08T04:38:22.826 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 msgr2=0x7fae9815b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:22.826 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae8801a0b0 tx=0x7fae88019220 comp rx=0 tx=0).stop 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 shutdown_connections 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae74066f90 0x7fae74069450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae980d2000 0x7fae980b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9815e290 0x7fae980b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 --2- 172.21.15.164:0/3511904003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae980c8530 0x7fae9815b000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 >> 172.21.15.164:0/3511904003 conn(0x7fae9809aa60 msgr2=0x7fae9806f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 shutdown_connections 2024-05-08T04:38:22.827 INFO:teuthology.misc.health.smithi164.stderr:2024-05-08T04:38:22.823+0000 7fae9f400640 1 -- 172.21.15.164:0/3511904003 wait complete. 2024-05-08T04:38:22.843 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T04:38:22.843 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T04:38:22.843 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:38:22.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:38:22.847 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:38:22.847 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:38:22.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:38:22.849 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:38:22.850 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:38:22.850 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:38:22.854 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:38:22.854 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:38:22.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:38:22.857 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:38:22.857 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T04:38:22.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:22.940 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:38:22.942 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:38:22.965 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:38:22.965 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:38:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 Processor -- start 2024-05-08T04:38:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 -- start start 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940b26c0 0x7ff8940cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff8940cfbd0 0x7ff8940cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89406c770 con 0x7ff89415ab60 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940d0060 con 0x7ff8940cfbd0 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff899e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89406ca50 con 0x7ff8940b26c0 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff892a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff89415ab60 0x7ff894072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44904/0 (socket says 172.21.15.164:44904) 2024-05-08T04:38:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff892a00640 1 -- 172.21.15.164:0/1333537043 learned_addr learned my addr 172.21.15.164:0/1333537043 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff893400640 1 --2- 172.21.15.164:0/1333537043 >> v2:172.21.15.172:3300/0 conn(0x7ff8940cfbd0 0x7ff8940cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff88a000640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940b26c0 0x7ff8940cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff892000640 1 -- 172.21.15.164:0/1333537043 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075966224 0 0) 0x7ff89406c770 con 0x7ff89415ab60 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff892000640 1 -- 172.21.15.164:0/1333537043 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff878003530 con 0x7ff89415ab60 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff893400640 1 -- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940b26c0 msgr2=0x7ff8940cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:22.999+0000 7ff893400640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940b26c0 0x7ff8940cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff893400640 1 -- 172.21.15.164:0/1333537043 >> v1:172.21.15.164:6789/0 conn(0x7ff89415ab60 legacy=0x7ff894072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff893400640 1 -- 172.21.15.164:0/1333537043 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89406cd30 con 0x7ff8940cfbd0 2024-05-08T04:38:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff893400640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8940cfbd0 0x7ff8940cd910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d89306e7d725474 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff892000640 1 -- 172.21.15.164:0/1333537043 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0b2070 con 0x7ff8940cfbd0 2024-05-08T04:38:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff892000640 1 -- 172.21.15.164:0/1333537043 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c0a5830 con 0x7ff8940cfbd0 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff892000640 1 -- 172.21.15.164:0/1333537043 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0a5c00 con 0x7ff8940cfbd0 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 -- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8940cfbd0 msgr2=0x7ff8940cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8940cfbd0 0x7ff8940cd910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009d90 tx=0x7ff88c01c970 comp rx=0 tx=0).stop 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 -- 172.21.15.164:0/1333537043 shutdown_connections 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8940cfbd0 0x7ff8940cd910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 --2- 172.21.15.164:0/1333537043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940b26c0 0x7ff8940cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 -- 172.21.15.164:0/1333537043 >> 172.21.15.164:0/1333537043 conn(0x7ff89406ac90 msgr2=0x7ff89406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 -- 172.21.15.164:0/1333537043 shutdown_connections 2024-05-08T04:38:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.003+0000 7ff899e00640 1 -- 172.21.15.164:0/1333537043 wait complete. 2024-05-08T04:38:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 Processor -- start 2024-05-08T04:38:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- start start 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940cfbd0 0x7ff8940b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff89415ab60 0x7ff8940ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940aeeb0 con 0x7ff8940b26c0 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940aed30 con 0x7ff89415ab60 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff894167970 con 0x7ff8940cfbd0 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48096/0 (socket says 172.21.15.164:48096) 2024-05-08T04:38:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 -- 172.21.15.164:0/3652755969 learned_addr learned my addr 172.21.15.164:0/3652755969 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff88a000640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940cfbd0 0x7ff8940b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff893400640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff89415ab60 0x7ff8940ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 -- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940cfbd0 msgr2=0x7ff8940b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940cfbd0 0x7ff8940b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 -- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff89415ab60 msgr2=0x7ff8940ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff89415ab60 0x7ff8940ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 -- 172.21.15.164:0/3652755969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894158d10 con 0x7ff8940b26c0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff892a00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c3c0 tx=0x7ff88400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884017070 con 0x7ff8940b26c0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff884005cf0 con 0x7ff8940b26c0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840072f0 con 0x7ff8940b26c0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894158ff0 con 0x7ff8940b26c0 2024-05-08T04:38:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.007+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8941595b0 con 0x7ff8940b26c0 2024-05-08T04:38:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff894069d60 con 0x7ff8940b26c0 2024-05-08T04:38:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff88401a070 con 0x7ff8940b26c0 2024-05-08T04:38:23.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff88be00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff884084120 con 0x7ff8940b26c0 2024-05-08T04:38:23.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff88a000640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.011+0000 7ff88a000640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8941f9ff0 tx=0x7ff87c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.015+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff884054d00 con 0x7ff8940b26c0 2024-05-08T04:38:23.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.167+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff89415e760 con 0x7ff8940b26c0 2024-05-08T04:38:23.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff88be00640 1 -- 172.21.15.164:0/3652755969 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7ff884058da0 con 0x7ff8940b26c0 2024-05-08T04:38:23.172 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:23.172 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":16,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:19.677806+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 msgr2=0x7ff870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8941f9ff0 tx=0x7ff87c009190 comp rx=0 tx=0).stop 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 msgr2=0x7ff89414dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c3c0 tx=0x7ff88400c880 comp rx=0 tx=0).stop 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 shutdown_connections 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8940cfbd0 0x7ff8940b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff89415ab60 0x7ff8940ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 --2- 172.21.15.164:0/3652755969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8940b26c0 0x7ff89414dcc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 >> 172.21.15.164:0/3652755969 conn(0x7ff89406ac90 msgr2=0x7ff89415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 shutdown_connections 2024-05-08T04:38:23.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.171+0000 7ff899e00640 1 -- 172.21.15.164:0/3652755969 wait complete. 2024-05-08T04:38:23.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:23.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:23.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 Processor -- start 2024-05-08T04:38:23.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 -- start start 2024-05-08T04:38:23.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f888815ba20 0x7f8888160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f888806de50 0x7f888806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88880c6600 con 0x7f88880ae6e0 2024-05-08T04:38:23.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888815fbe0 con 0x7f888806de50 2024-05-08T04:38:23.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8890200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88880c6040 con 0x7f888815ba20 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f888806de50 0x7f888806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888e200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f88880ae6e0 0x7f88880cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44908/0 (socket says 172.21.15.164:44908) 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888e200640 1 -- 172.21.15.164:0/3583927428 learned_addr learned my addr 172.21.15.164:0/3583927428 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888d800640 1 --2- 172.21.15.164:0/3583927428 >> v2:172.21.15.172:3300/0 conn(0x7f888815ba20 0x7f8888160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888ce00640 1 -- 172.21.15.164:0/3583927428 >> v2:172.21.15.172:3300/0 conn(0x7f888815ba20 msgr2=0x7f8888160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888ce00640 1 --2- 172.21.15.164:0/3583927428 >> v2:172.21.15.172:3300/0 conn(0x7f888815ba20 0x7f8888160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888ce00640 1 -- 172.21.15.164:0/3583927428 >> v1:172.21.15.164:6789/0 conn(0x7f88880ae6e0 legacy=0x7f88880cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f888ce00640 1 -- 172.21.15.164:0/3583927428 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88880c6320 con 0x7f888806de50 2024-05-08T04:38:23.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.331+0000 7f8887e00640 1 -- 172.21.15.164:0/3583927428 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104050271 0 0) 0x7f88880c6600 con 0x7f88880ae6e0 2024-05-08T04:38:23.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f888ce00640 1 --2- 172.21.15.164:0/3583927428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f888806de50 0x7f888806e2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88880cc0e0 tx=0x7f887401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee24062345c7916 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8887e00640 1 -- 172.21.15.164:0/3583927428 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740ab5f0 con 0x7f888806de50 2024-05-08T04:38:23.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8887e00640 1 -- 172.21.15.164:0/3583927428 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740b0070 con 0x7f888806de50 2024-05-08T04:38:23.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8887e00640 1 -- 172.21.15.164:0/3583927428 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740ab9b0 con 0x7f888806de50 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- 172.21.15.164:0/3583927428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f888806de50 msgr2=0x7f888806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 --2- 172.21.15.164:0/3583927428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f888806de50 0x7f888806e2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88880cc0e0 tx=0x7f887401c450 comp rx=0 tx=0).stop 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- 172.21.15.164:0/3583927428 shutdown_connections 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 --2- 172.21.15.164:0/3583927428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f888806de50 0x7f888806e2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 --2- 172.21.15.164:0/3583927428 >> v2:172.21.15.172:3300/0 conn(0x7f888815ba20 0x7f8888160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- 172.21.15.164:0/3583927428 >> 172.21.15.164:0/3583927428 conn(0x7f88880cf770 msgr2=0x7f88880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- 172.21.15.164:0/3583927428 shutdown_connections 2024-05-08T04:38:23.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- 172.21.15.164:0/3583927428 wait complete. 2024-05-08T04:38:23.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 Processor -- start 2024-05-08T04:38:23.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.335+0000 7f8890200640 1 -- start start 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f888806de50 0x7f888814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 0x7f888814f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880aec60 con 0x7f88880ae6e0 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880aeae0 con 0x7f888806de50 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88880aede0 con 0x7f888815ba20 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48118/0 (socket says 172.21.15.164:48118) 2024-05-08T04:38:23.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 -- 172.21.15.164:0/1513240657 learned_addr learned my addr 172.21.15.164:0/1513240657 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888d800640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f888806de50 0x7f888814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888e200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 0x7f888814f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 -- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 msgr2=0x7f888814f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 0x7f888814f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 -- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f888806de50 msgr2=0x7f888814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f888806de50 0x7f888814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 -- 172.21.15.164:0/1513240657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888814fd30 con 0x7f88880ae6e0 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888e200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 0x7f888814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:23.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f888ce00640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88740ad040 tx=0x7f887401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740bf410 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740bf5d0 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740bf9c0 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888814ffd0 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88880cade0 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8888072d30 con 0x7f88880ae6e0 2024-05-08T04:38:23.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.339+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88740bd020 con 0x7f88880ae6e0 2024-05-08T04:38:23.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.343+0000 7f8885600640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 0x7f8868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.343+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f887411eff0 con 0x7f88880ae6e0 2024-05-08T04:38:23.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.343+0000 7f888d800640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 0x7f8868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.343+0000 7f888d800640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 0x7f8868069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f887c009ac0 tx=0x7f887c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.347+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8874016070 con 0x7f88880ae6e0 2024-05-08T04:38:23.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8888147d60 con 0x7f88880ae6e0 2024-05-08T04:38:23.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8885600640 1 -- 172.21.15.164:0/1513240657 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88740efc80 con 0x7f88880ae6e0 2024-05-08T04:38:23.505 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 msgr2=0x7f8868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 0x7f8868069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f887c009ac0 tx=0x7f887c009190 comp rx=0 tx=0).stop 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 msgr2=0x7f88880cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88740ad040 tx=0x7f887401ac90 comp rx=0 tx=0).stop 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 shutdown_connections 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8868066f60 0x7f8868069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f888815ba20 0x7f888814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88880ae6e0 0x7f88880cb8c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 --2- 172.21.15.164:0/1513240657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f888806de50 0x7f888814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 >> 172.21.15.164:0/1513240657 conn(0x7f88880cf770 msgr2=0x7f888815d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 shutdown_connections 2024-05-08T04:38:23.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.503+0000 7f8890200640 1 -- 172.21.15.164:0/1513240657 wait complete. 2024-05-08T04:38:23.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:23.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 Processor -- start 2024-05-08T04:38:23.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 -- start start 2024-05-08T04:38:23.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53d015ab60 0x7f53d0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d0072640 con 0x7f53d00b26c0 2024-05-08T04:38:23.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d006c770 con 0x7f53d00cfda0 2024-05-08T04:38:23.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d7c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d006ca50 con 0x7f53d015ab60 2024-05-08T04:38:23.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f53d00b26c0 0x7f53d00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44916/0 (socket says 172.21.15.164:44916) 2024-05-08T04:38:23.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 -- 172.21.15.164:0/2992269389 learned_addr learned my addr 172.21.15.164:0/2992269389 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221933296 0 0) 0x7f53d0072640 con 0x7f53d00b26c0 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d5200640 1 --2- 172.21.15.164:0/2992269389 >> v2:172.21.15.172:3300/0 conn(0x7f53d015ab60 0x7f53d0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b8003530 con 0x7f53d00b26c0 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421197034 0 0) 0x7f53b8003530 con 0x7f53d00b26c0 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 >> v2:172.21.15.172:3300/0 conn(0x7f53d015ab60 msgr2=0x7f53d0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:23.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 --2- 172.21.15.164:0/2992269389 >> v2:172.21.15.172:3300/0 conn(0x7f53d015ab60 0x7f53d0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 >> v1:172.21.15.172:6789/0 conn(0x7f53d00cfda0 legacy=0x7f53d00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d006cd30 con 0x7f53d00b26c0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f53bc001680 con 0x7f53d00b26c0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 >> v1:172.21.15.164:6789/0 conn(0x7f53d00b26c0 legacy=0x7f53d00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b80039c0 0x7f53b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 0x7f53b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b80051d0 0x7f53b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d006ca50 con 0x7f53b8004810 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b800c070 con 0x7f53b80051d0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b800c270 con 0x7f53b80039c0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 0x7f53b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d5200640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b80039c0 0x7f53b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d5c00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b80051d0 0x7f53b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 -- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b80039c0 msgr2=0x7f53b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:23.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b80039c0 0x7f53b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 -- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b80051d0 msgr2=0x7f53b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:23.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b80051d0 0x7f53b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53cfe00640 1 -- 172.21.15.164:0/2992269389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8003530 con 0x7f53b8004810 2024-05-08T04:38:23.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d5c00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b80051d0 0x7f53b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:23.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.659+0000 7f53d5c00640 1 -- 172.21.15.164:0/2992269389 reap_dead start 2024-05-08T04:38:23.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53cfe00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 0x7f53b8004c90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c1c0 tx=0x7f53bc01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3070c455b53e16b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007d00 con 0x7f53b8004810 2024-05-08T04:38:23.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc005b00 con 0x7f53b8004810 2024-05-08T04:38:23.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53cf400640 1 -- 172.21.15.164:0/2992269389 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007070 con 0x7f53b8004810 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 -- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 msgr2=0x7f53b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 0x7f53b8004c90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c1c0 tx=0x7f53bc01f3d0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 -- 172.21.15.164:0/2992269389 shutdown_connections 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 --2- 172.21.15.164:0/2992269389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b8004810 0x7f53b8004c90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 -- 172.21.15.164:0/2992269389 >> 172.21.15.164:0/2992269389 conn(0x7f53d006ac90 msgr2=0x7f53d006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 -- 172.21.15.164:0/2992269389 shutdown_connections 2024-05-08T04:38:23.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.663+0000 7f53d7c00640 1 -- 172.21.15.164:0/2992269389 wait complete. 2024-05-08T04:38:23.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 Processor -- start 2024-05-08T04:38:23.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- start start 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b8004810 0x7f53d01f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53d01e7a60 0x7f53d0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d006c3d0 con 0x7f53d0155e60 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d006d2f0 con 0x7f53b8004810 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d006c550 con 0x7f53d01e7a60 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48128/0 (socket says 172.21.15.164:48128) 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 -- 172.21.15.164:0/767156895 learned_addr learned my addr 172.21.15.164:0/767156895 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d5200640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b8004810 0x7f53d01f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d5c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53d01e7a60 0x7f53d0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 -- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53d01e7a60 msgr2=0x7f53d0159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53d01e7a60 0x7f53d0159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 -- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b8004810 msgr2=0x7f53d01f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b8004810 0x7f53d01f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 -- 172.21.15.164:0/767156895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d015a2e0 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53cfe00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c1c0 tx=0x7f53b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc001c40 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc000ee0 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007d00 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d01e4d50 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.667+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d01e5310 con 0x7f53d0155e60 2024-05-08T04:38:23.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d0069d60 con 0x7f53d0155e60 2024-05-08T04:38:23.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53bc001480 con 0x7f53d0155e60 2024-05-08T04:38:23.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53ccc00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 0x7f53b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:23.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53bc121ef0 con 0x7f53d0155e60 2024-05-08T04:38:23.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53d5200640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 0x7f53b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:23.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.671+0000 7f53d5200640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 0x7f53b0069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53d006b740 tx=0x7f53c4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:23.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.675+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53bc0190e0 con 0x7f53d0155e60 2024-05-08T04:38:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d01e7e40 con 0x7f53d0155e60 2024-05-08T04:38:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53ccc00640 1 -- 172.21.15.164:0/767156895 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f53bc0f3240 con 0x7f53d0155e60 2024-05-08T04:38:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:23.865 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-05-08T04:38:23.866 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 2 2024-05-08T04:38:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 msgr2=0x7f53b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 0x7f53b0069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53d006b740 tx=0x7f53c4009190 comp rx=0 tx=0).stop 2024-05-08T04:38:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 msgr2=0x7f53d01e7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c1c0 tx=0x7f53b8005840 comp rx=0 tx=0).stop 2024-05-08T04:38:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 shutdown_connections 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53b0066f60 0x7f53b0069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53d01e7a60 0x7f53d0159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d0155e60 0x7f53d01e7520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 --2- 172.21.15.164:0/767156895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b8004810 0x7f53d01f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 >> 172.21.15.164:0/767156895 conn(0x7f53d006ac90 msgr2=0x7f53d015ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 shutdown_connections 2024-05-08T04:38:23.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:23.863+0000 7f53d7c00640 1 -- 172.21.15.164:0/767156895 wait complete. 2024-05-08T04:38:25.886 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:26.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 Processor -- start 2024-05-08T04:38:26.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 -- start start 2024-05-08T04:38:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07681625f0 0x7f07680b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076806cc70 con 0x7f076815dd70 2024-05-08T04:38:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076806cf50 con 0x7f07681625f0 2024-05-08T04:38:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680b1570 con 0x7f076806c7b0 2024-05-08T04:38:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f076d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07681625f0 0x7f07680b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:52386/0 (socket says 172.21.15.164:52386) 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 -- 172.21.15.164:0/138247133 learned_addr learned my addr 172.21.15.164:0/138247133 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 -- 172.21.15.164:0/138247133 >> v2:172.21.15.172:3300/0 conn(0x7f07681625f0 msgr2=0x7f07680b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 --2- 172.21.15.164:0/138247133 >> v2:172.21.15.172:3300/0 conn(0x7f07681625f0 0x7f07680b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 -- 172.21.15.164:0/138247133 >> v1:172.21.15.172:6789/0 conn(0x7f076815dd70 legacy=0x7f07681619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f075fe00640 1 -- 172.21.15.164:0/138247133 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680b1850 con 0x7f076806c7b0 2024-05-08T04:38:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.031+0000 7f0767e00640 1 -- 172.21.15.164:0/138247133 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356572819 0 0) 0x7f076806cc70 con 0x7f076815dd70 2024-05-08T04:38:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f075fe00640 1 --2- 172.21.15.164:0/138247133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0750009870 tx=0x7f075001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ba5a124ff63935 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f0767e00640 1 -- 172.21.15.164:0/138247133 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07500b5070 con 0x7f076806c7b0 2024-05-08T04:38:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f0767e00640 1 -- 172.21.15.164:0/138247133 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07500a8a50 con 0x7f076806c7b0 2024-05-08T04:38:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f0767e00640 1 -- 172.21.15.164:0/138247133 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07500a8e20 con 0x7f076806c7b0 2024-05-08T04:38:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 -- 172.21.15.164:0/138247133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 msgr2=0x7f07681620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 --2- 172.21.15.164:0/138247133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0750009870 tx=0x7f075001ed50 comp rx=0 tx=0).stop 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 -- 172.21.15.164:0/138247133 shutdown_connections 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 --2- 172.21.15.164:0/138247133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076806c7b0 0x7f07681620b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 --2- 172.21.15.164:0/138247133 >> v2:172.21.15.172:3300/0 conn(0x7f07681625f0 0x7f07680b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 -- 172.21.15.164:0/138247133 >> 172.21.15.164:0/138247133 conn(0x7f07680b34f0 msgr2=0x7f07680b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.035+0000 7f076f400640 1 -- 172.21.15.164:0/138247133 shutdown_connections 2024-05-08T04:38:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- 172.21.15.164:0/138247133 wait complete. 2024-05-08T04:38:26.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 Processor -- start 2024-05-08T04:38:26.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- start start 2024-05-08T04:38:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f076806c7b0 0x7f0768161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076815dd70 0x7f076814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076815e2d0 con 0x7f07681625f0 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076815e150 con 0x7f076806c7b0 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680b22b0 con 0x7f076815dd70 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48130/0 (socket says 172.21.15.164:48130) 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 -- 172.21.15.164:0/1396155189 learned_addr learned my addr 172.21.15.164:0/1396155189 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f075fe00640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076815dd70 0x7f076814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076ca00640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f076806c7b0 0x7f0768161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 -- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076815dd70 msgr2=0x7f076814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076815dd70 0x7f076814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 -- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f076806c7b0 msgr2=0x7f0768161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f076806c7b0 0x7f0768161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 -- 172.21.15.164:0/1396155189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076814caf0 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076d400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0760009cb0 tx=0x7f076000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760018020 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760014480 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0768160190 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.039+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760014870 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0768160750 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768069d60 con 0x7f07681625f0 2024-05-08T04:38:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f076001b020 con 0x7f07681625f0 2024-05-08T04:38:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f0765600640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 0x7f0744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07600858c0 con 0x7f07681625f0 2024-05-08T04:38:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f076ca00640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 0x7f0744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.043+0000 7f076ca00640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 0x7f0744069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0758000f70 tx=0x7f0758009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.047+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0760022030 con 0x7f07681625f0 2024-05-08T04:38:26.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.231+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07680b4010 con 0x7f07681625f0 2024-05-08T04:38:26.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f0765600640 1 -- 172.21.15.164:0/1396155189 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f0760010040 con 0x7f07681625f0 2024-05-08T04:38:26.236 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:26.236 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-05-08T04:38:26.236 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 3 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 msgr2=0x7f0744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 0x7f0744069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0758000f70 tx=0x7f0758009040 comp rx=0 tx=0).stop 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 msgr2=0x7f07681491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0760009cb0 tx=0x7f076000c490 comp rx=0 tx=0).stop 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 shutdown_connections 2024-05-08T04:38:26.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0744066f60 0x7f0744069420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f076815dd70 0x7f076814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07681625f0 0x7f07681491e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 --2- 172.21.15.164:0/1396155189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f076806c7b0 0x7f0768161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 >> 172.21.15.164:0/1396155189 conn(0x7f07680b34f0 msgr2=0x7f07680b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 shutdown_connections 2024-05-08T04:38:26.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.235+0000 7f076f400640 1 -- 172.21.15.164:0/1396155189 wait complete. 2024-05-08T04:38:26.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:26.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 Processor -- start 2024-05-08T04:38:26.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 -- start start 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc49015dd20 0x7fc4900afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc4900ae2a0 0x7fc49006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc4900b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49006b600 con 0x7fc49015dd20 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900ae730 con 0x7fc4900ae2a0 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc495200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49015e100 con 0x7fc49006b1d0 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc49015dd20 0x7fc4900afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc49015dd20 0x7fc4900afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48142/0 (socket says 172.21.15.164:48142) 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 -- 172.21.15.164:0/3344096131 learned_addr learned my addr 172.21.15.164:0/3344096131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:26.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48e800640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc4900b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48d400640 1 --2- 172.21.15.164:0/3344096131 >> v2:172.21.15.172:3300/0 conn(0x7fc4900ae2a0 0x7fc49006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 -- 172.21.15.164:0/3344096131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 msgr2=0x7fc4900b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc4900b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 -- 172.21.15.164:0/3344096131 >> v2:172.21.15.172:3300/0 conn(0x7fc4900ae2a0 msgr2=0x7fc49006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 --2- 172.21.15.164:0/3344096131 >> v2:172.21.15.172:3300/0 conn(0x7fc4900ae2a0 0x7fc49006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.391+0000 7fc48de00640 1 -- 172.21.15.164:0/3344096131 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900b5a30 con 0x7fc49015dd20 2024-05-08T04:38:26.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc48de00640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc49015dd20 0x7fc4900afb80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc49015e2b0 tx=0x7fc48001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4b15c5cfdc0eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc48ca00640 1 -- 172.21.15.164:0/3344096131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800b0070 con 0x7fc49015dd20 2024-05-08T04:38:26.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc48ca00640 1 -- 172.21.15.164:0/3344096131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800ab600 con 0x7fc49015dd20 2024-05-08T04:38:26.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc48ca00640 1 -- 172.21.15.164:0/3344096131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800ab9d0 con 0x7fc49015dd20 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 -- 172.21.15.164:0/3344096131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc49015dd20 msgr2=0x7fc4900afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc49015dd20 0x7fc4900afb80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc49015e2b0 tx=0x7fc48001c260 comp rx=0 tx=0).stop 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 -- 172.21.15.164:0/3344096131 shutdown_connections 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc4900b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 --2- 172.21.15.164:0/3344096131 >> v2:172.21.15.172:3300/0 conn(0x7fc4900ae2a0 0x7fc49006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 --2- 172.21.15.164:0/3344096131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc49015dd20 0x7fc4900afb80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 -- 172.21.15.164:0/3344096131 >> 172.21.15.164:0/3344096131 conn(0x7fc4900b24d0 msgr2=0x7fc4900b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 -- 172.21.15.164:0/3344096131 shutdown_connections 2024-05-08T04:38:26.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 -- 172.21.15.164:0/3344096131 wait complete. 2024-05-08T04:38:26.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.395+0000 7fc495200640 1 Processor -- start 2024-05-08T04:38:26.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- start start 2024-05-08T04:38:26.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc490152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 0x7fc490152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901621a0 con 0x7fc4900ae2a0 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490161ec0 con 0x7fc49015dd20 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901626c0 con 0x7fc49006b1d0 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48146/0 (socket says 172.21.15.164:48146) 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 -- 172.21.15.164:0/2012073063 learned_addr learned my addr 172.21.15.164:0/2012073063 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48de00640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc490152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48e800640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 0x7fc490152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 -- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 msgr2=0x7fc490152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc490152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 -- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 msgr2=0x7fc490152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 0x7fc490152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 -- 172.21.15.164:0/2012073063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490148910 con 0x7fc4900ae2a0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48e800640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 0x7fc490152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc48d400640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d800 tx=0x7fc47c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00b6c0 con 0x7fc4900ae2a0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c019070 con 0x7fc4900ae2a0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00ba60 con 0x7fc4900ae2a0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490148bf0 con 0x7fc4900ae2a0 2024-05-08T04:38:26.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4901491b0 con 0x7fc4900ae2a0 2024-05-08T04:38:26.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.399+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc490069d60 con 0x7fc4900ae2a0 2024-05-08T04:38:26.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.403+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc47c00bc00 con 0x7fc4900ae2a0 2024-05-08T04:38:26.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.403+0000 7fc486000640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 0x7fc470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.403+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc47c085100 con 0x7fc4900ae2a0 2024-05-08T04:38:26.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.403+0000 7fc48de00640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 0x7fc470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.403+0000 7fc48de00640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 0x7fc470069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ad000 tx=0x7fc480008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.407+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc47c056450 con 0x7fc4900ae2a0 2024-05-08T04:38:26.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.559+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc49006bb40 con 0x7fc4900ae2a0 2024-05-08T04:38:26.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.559+0000 7fc486000640 1 -- 172.21.15.164:0/2012073063 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12944 (secure 0 0 0) 0x7fc47c055d50 con 0x7fc4900ae2a0 2024-05-08T04:38:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:26.564 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":16,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:19.677806+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:26.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 msgr2=0x7fc470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 0x7fc470069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ad000 tx=0x7fc480008040 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 msgr2=0x7fc490152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d800 tx=0x7fc47c00dcc0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 shutdown_connections 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc470066f60 0x7fc470069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc49015dd20 0x7fc490152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4900ae2a0 0x7fc490152ab0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 --2- 172.21.15.164:0/2012073063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc49006b1d0 0x7fc490152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 >> 172.21.15.164:0/2012073063 conn(0x7fc4900b24d0 msgr2=0x7fc4900b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 shutdown_connections 2024-05-08T04:38:26.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.563+0000 7fc495200640 1 -- 172.21.15.164:0/2012073063 wait complete. 2024-05-08T04:38:26.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:26.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:26.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.719+0000 7f596c400640 1 Processor -- start 2024-05-08T04:38:26.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 -- start start 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f596415ce10 0x7f596406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f596406e390 0x7f59640ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640c6600 con 0x7f596415ba20 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596415fbe0 con 0x7f596406e390 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596c400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59640c6040 con 0x7f596415ce10 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596a400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f596406e390 0x7f59640ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f5969a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f596415ba20 0x7f5964160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44920/0 (socket says 172.21.15.164:44920) 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f5969a00640 1 -- 172.21.15.164:0/3562476120 learned_addr learned my addr 172.21.15.164:0/3562476120 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:26.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f5969000640 1 --2- 172.21.15.164:0/3562476120 >> v2:172.21.15.172:3300/0 conn(0x7f596415ce10 0x7f596406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596a400640 1 -- 172.21.15.164:0/3562476120 >> v2:172.21.15.172:3300/0 conn(0x7f596415ce10 msgr2=0x7f596406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:26.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596a400640 1 --2- 172.21.15.164:0/3562476120 >> v2:172.21.15.172:3300/0 conn(0x7f596415ce10 0x7f596406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596a400640 1 -- 172.21.15.164:0/3562476120 >> v1:172.21.15.164:6789/0 conn(0x7f596415ba20 legacy=0x7f5964160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f596a400640 1 -- 172.21.15.164:0/3562476120 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640c6320 con 0x7f596406e390 2024-05-08T04:38:26.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.723+0000 7f5963e00640 1 -- 172.21.15.164:0/3562476120 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218115000 0 0) 0x7f59640c6600 con 0x7f596415ba20 2024-05-08T04:38:26.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596a400640 1 --2- 172.21.15.164:0/3562476120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640ae6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59640b5980 tx=0x7f595c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c463f8106cd93e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f5963e00640 1 -- 172.21.15.164:0/3562476120 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0b2070 con 0x7f596406e390 2024-05-08T04:38:26.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f5963e00640 1 -- 172.21.15.164:0/3562476120 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c0a57f0 con 0x7f596406e390 2024-05-08T04:38:26.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f5963e00640 1 -- 172.21.15.164:0/3562476120 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0a5bc0 con 0x7f596406e390 2024-05-08T04:38:26.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- 172.21.15.164:0/3562476120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 msgr2=0x7f59640ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 --2- 172.21.15.164:0/3562476120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640ae6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59640b5980 tx=0x7f595c01c930 comp rx=0 tx=0).stop 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- 172.21.15.164:0/3562476120 shutdown_connections 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 --2- 172.21.15.164:0/3562476120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640ae6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 --2- 172.21.15.164:0/3562476120 >> v2:172.21.15.172:3300/0 conn(0x7f596415ce10 0x7f596406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- 172.21.15.164:0/3562476120 >> 172.21.15.164:0/3562476120 conn(0x7f59640cf770 msgr2=0x7f59640cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- 172.21.15.164:0/3562476120 shutdown_connections 2024-05-08T04:38:26.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- 172.21.15.164:0/3562476120 wait complete. 2024-05-08T04:38:26.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 Processor -- start 2024-05-08T04:38:26.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.727+0000 7f596c400640 1 -- start start 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f596415ba20 0x7f596414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596415ce10 0x7f59640b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964168b70 con 0x7f596406e390 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640ce090 con 0x7f596415ba20 2024-05-08T04:38:26.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5964168cf0 con 0x7f596415ce10 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48162/0 (socket says 172.21.15.164:48162) 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 -- 172.21.15.164:0/805363715 learned_addr learned my addr 172.21.15.164:0/805363715 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969000640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f596415ba20 0x7f596414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596a400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596415ce10 0x7f59640b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 -- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596415ce10 msgr2=0x7f59640b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596415ce10 0x7f59640b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 -- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f596415ba20 msgr2=0x7f596414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f596415ba20 0x7f596414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 -- 172.21.15.164:0/805363715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640704c0 con 0x7f596406e390 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5969a00640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59580011f0 tx=0x7f595800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958018070 con 0x7f596406e390 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59580074e0 con 0x7f596406e390 2024-05-08T04:38:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640707a0 con 0x7f596406e390 2024-05-08T04:38:26.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958004a10 con 0x7f596406e390 2024-05-08T04:38:26.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5964070d60 con 0x7f596406e390 2024-05-08T04:38:26.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5964072d30 con 0x7f596406e390 2024-05-08T04:38:26.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.731+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f595801b070 con 0x7f596406e390 2024-05-08T04:38:26.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.735+0000 7f5961600640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 0x7f5944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:26.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.735+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5958084060 con 0x7f596406e390 2024-05-08T04:38:26.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.735+0000 7f5969000640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 0x7f5944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:26.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.735+0000 7f5969000640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 0x7f5944069450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f59640ae6a0 tx=0x7f5950008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:26.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.739+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59580553b0 con 0x7f596406e390 2024-05-08T04:38:26.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5964166940 con 0x7f596406e390 2024-05-08T04:38:26.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f5961600640 1 -- 172.21.15.164:0/805363715 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5958054cb0 con 0x7f596406e390 2024-05-08T04:38:26.896 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 msgr2=0x7f5944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 0x7f5944069450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f59640ae6a0 tx=0x7f5950008040 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 msgr2=0x7f59640c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59580011f0 tx=0x7f595800c9b0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 shutdown_connections 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5944066f90 0x7f5944069450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596415ce10 0x7f59640b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f596415ba20 0x7f596414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 --2- 172.21.15.164:0/805363715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596406e390 0x7f59640c91e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 >> 172.21.15.164:0/805363715 conn(0x7f59640cf770 msgr2=0x7f596415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 shutdown_connections 2024-05-08T04:38:26.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:26.895+0000 7f596c400640 1 -- 172.21.15.164:0/805363715 wait complete. 2024-05-08T04:38:26.917 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:38:27.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 Processor -- start 2024-05-08T04:38:27.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 -- start start 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f491815dd70 0x7f49181619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 0x7f4918161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491806cc70 con 0x7f4918162420 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491806cf50 con 0x7f491815dd70 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491fc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180b1570 con 0x7f491806c7b0 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f491815dd70 0x7f49181619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f491815dd70 0x7f49181619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48174/0 (socket says 172.21.15.164:48174) 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 -- 172.21.15.164:0/3086189776 learned_addr learned my addr 172.21.15.164:0/3086189776 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f4917e00640 1 --2- 172.21.15.164:0/3086189776 >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 0x7f4918161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 -- 172.21.15.164:0/3086189776 >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 msgr2=0x7f4918161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 --2- 172.21.15.164:0/3086189776 >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 0x7f4918161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 -- 172.21.15.164:0/3086189776 >> v1:172.21.15.172:6789/0 conn(0x7f4918162420 legacy=0x7f49180cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f491d200640 1 -- 172.21.15.164:0/3086189776 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180b1850 con 0x7f491815dd70 2024-05-08T04:38:27.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.051+0000 7f4917e00640 1 --2- 172.21.15.164:0/3086189776 >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 0x7f4918161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:27.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491d200640 1 --2- 172.21.15.164:0/3086189776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491815dd70 0x7f49181619a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4918070fa0 tx=0x7f490c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40af6a05f5f6bf09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f4917400640 1 -- 172.21.15.164:0/3086189776 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b5070 con 0x7f491815dd70 2024-05-08T04:38:27.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f4917400640 1 -- 172.21.15.164:0/3086189776 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0a8a10 con 0x7f491815dd70 2024-05-08T04:38:27.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f4917400640 1 -- 172.21.15.164:0/3086189776 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0a8de0 con 0x7f491815dd70 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 -- 172.21.15.164:0/3086189776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491815dd70 msgr2=0x7f49181619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 --2- 172.21.15.164:0/3086189776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491815dd70 0x7f49181619a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4918070fa0 tx=0x7f490c01ed10 comp rx=0 tx=0).stop 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 -- 172.21.15.164:0/3086189776 shutdown_connections 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 --2- 172.21.15.164:0/3086189776 >> v2:172.21.15.172:3300/0 conn(0x7f491806c7b0 0x7f4918161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 --2- 172.21.15.164:0/3086189776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491815dd70 0x7f49181619a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 -- 172.21.15.164:0/3086189776 >> 172.21.15.164:0/3086189776 conn(0x7f49180b34f0 msgr2=0x7f49180b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 -- 172.21.15.164:0/3086189776 shutdown_connections 2024-05-08T04:38:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.055+0000 7f491fc00640 1 -- 172.21.15.164:0/3086189776 wait complete. 2024-05-08T04:38:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 Processor -- start 2024-05-08T04:38:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- start start 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f491815dd70 0x7f4918158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918162420 0x7f4918160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491815e150 con 0x7f491806c7b0 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180b4230 con 0x7f4918162420 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491815e2d0 con 0x7f491815dd70 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48188/0 (socket says 172.21.15.164:48188) 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 -- 172.21.15.164:0/3360652034 learned_addr learned my addr 172.21.15.164:0/3360652034 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f4917e00640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f491815dd70 0x7f4918158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491dc00640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918162420 0x7f4918160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 -- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f491815dd70 msgr2=0x7f4918158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f491815dd70 0x7f4918158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 -- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918162420 msgr2=0x7f4918160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918162420 0x7f4918160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 -- 172.21.15.164:0/3360652034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49181f4ba0 con 0x7f491806c7b0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491d200640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f490c01b920 tx=0x7f490c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b5070 con 0x7f491806c7b0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0b0b40 con 0x7f491806c7b0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0bb410 con 0x7f491806c7b0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- 172.21.15.164:0/3360652034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491815f4c0 con 0x7f491806c7b0 2024-05-08T04:38:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f491fc00640 1 -- 172.21.15.164:0/3360652034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491815fa80 con 0x7f491806c7b0 2024-05-08T04:38:27.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.059+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4918069d60 con 0x7f491806c7b0 2024-05-08T04:38:27.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.063+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f490c0c2060 con 0x7f491806c7b0 2024-05-08T04:38:27.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.063+0000 7f4914c00640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 0x7f48f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.063+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f490c121f10 con 0x7f491806c7b0 2024-05-08T04:38:27.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.063+0000 7f4917e00640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 0x7f48f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.063+0000 7f4917e00640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 0x7f48f8069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49181e9dc0 tx=0x7f4904008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.067+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f490c018870 con 0x7f491806c7b0 2024-05-08T04:38:27.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49180b4010 con 0x7f491806c7b0 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f4914c00640 1 -- 172.21.15.164:0/3360652034 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f490c0f31d0 con 0x7f491806c7b0 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:38:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:38:27.226 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 msgr2=0x7f48f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 0x7f48f8069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49181e9dc0 tx=0x7f4904008040 comp rx=0 tx=0).stop 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 msgr2=0x7f4918160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f490c01b920 tx=0x7f490c01bab0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 shutdown_connections 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f48f8066f60 0x7f48f8069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f491815dd70 0x7f4918158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918162420 0x7f4918160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 --2- 172.21.15.164:0/3360652034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f491806c7b0 0x7f4918160280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 >> 172.21.15.164:0/3360652034 conn(0x7f49180b34f0 msgr2=0x7f491806e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 shutdown_connections 2024-05-08T04:38:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.223+0000 7f490a000640 1 -- 172.21.15.164:0/3360652034 wait complete. 2024-05-08T04:38:27.245 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:38:27.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:38:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 Processor -- start 2024-05-08T04:38:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 -- start start 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0c240cc0f0 0x7f0c240af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0c240b1c00 0x7f0c24071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2415b020 con 0x7f0c240b2060 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c240cf460 con 0x7f0c240b1c00 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2ae00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2415b300 con 0x7f0c240cc0f0 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2a400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0c240b2060 0x7f0c2415e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44936/0 (socket says 172.21.15.164:44936) 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c2a400640 1 -- 172.21.15.164:0/3372494844 learned_addr learned my addr 172.21.15.164:0/3372494844 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29000640 1 --2- 172.21.15.164:0/3372494844 >> v2:172.21.15.164:3300/0 conn(0x7f0c240b1c00 0x7f0c24071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29a00640 1 --2- 172.21.15.164:0/3372494844 >> v2:172.21.15.172:3300/0 conn(0x7f0c240cc0f0 0x7f0c240af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29000640 1 -- 172.21.15.164:0/3372494844 >> v2:172.21.15.172:3300/0 conn(0x7f0c240cc0f0 msgr2=0x7f0c240af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29000640 1 --2- 172.21.15.164:0/3372494844 >> v2:172.21.15.172:3300/0 conn(0x7f0c240cc0f0 0x7f0c240af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29000640 1 -- 172.21.15.164:0/3372494844 >> v1:172.21.15.164:6789/0 conn(0x7f0c240b2060 legacy=0x7f0c2415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.383+0000 7f0c29000640 1 -- 172.21.15.164:0/3372494844 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2415b5e0 con 0x7f0c240b1c00 2024-05-08T04:38:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c29000640 1 --2- 172.21.15.164:0/3372494844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240b1c00 0x7f0c24071170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c2406e540 tx=0x7f0c1001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd505f30d9f4668b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c23e00640 1 -- 172.21.15.164:0/3372494844 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100ab650 con 0x7f0c240b1c00 2024-05-08T04:38:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c23e00640 1 -- 172.21.15.164:0/3372494844 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c100b0070 con 0x7f0c240b1c00 2024-05-08T04:38:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c23e00640 1 -- 172.21.15.164:0/3372494844 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100aba10 con 0x7f0c240b1c00 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 -- 172.21.15.164:0/3372494844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240b1c00 msgr2=0x7f0c24071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/3372494844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240b1c00 0x7f0c24071170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c2406e540 tx=0x7f0c1001c430 comp rx=0 tx=0).stop 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 -- 172.21.15.164:0/3372494844 shutdown_connections 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/3372494844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240b1c00 0x7f0c24071170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/3372494844 >> v2:172.21.15.172:3300/0 conn(0x7f0c240cc0f0 0x7f0c240af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 -- 172.21.15.164:0/3372494844 >> 172.21.15.164:0/3372494844 conn(0x7f0c2406ac90 msgr2=0x7f0c2406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 -- 172.21.15.164:0/3372494844 shutdown_connections 2024-05-08T04:38:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 -- 172.21.15.164:0/3372494844 wait complete. 2024-05-08T04:38:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.387+0000 7f0c2ae00640 1 Processor -- start 2024-05-08T04:38:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- start start 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c240b1c00 0x7f0c24168190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c240b2060 0x7f0c241578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2415bba0 con 0x7f0c240cc0f0 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24071ac0 con 0x7f0c240b2060 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2415ac60 con 0x7f0c240b1c00 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48218/0 (socket says 172.21.15.164:48218) 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 -- 172.21.15.164:0/1863218000 learned_addr learned my addr 172.21.15.164:0/1863218000 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c29000640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c240b2060 0x7f0c241578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c29a00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c240b1c00 0x7f0c24168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 -- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c240b1c00 msgr2=0x7f0c24168190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c240b1c00 0x7f0c24168190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 -- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c240b2060 msgr2=0x7f0c241578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c240b2060 0x7f0c241578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 -- 172.21.15.164:0/1863218000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24158950 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2a400640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00caa0 tx=0x7f0c1c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c017070 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00ac00 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c24156df0 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c012620 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c241573b0 con 0x7f0c240cc0f0 2024-05-08T04:38:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.391+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c24156bf0 con 0x7f0c240cc0f0 2024-05-08T04:38:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.395+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c1c01a070 con 0x7f0c240cc0f0 2024-05-08T04:38:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.395+0000 7f0c21600640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.395+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c1c084a20 con 0x7f0c240cc0f0 2024-05-08T04:38:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.395+0000 7f0c29a00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.395+0000 7f0c29a00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 0x7f0c04069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c241fa080 tx=0x7f0c18009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.399+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c1c055690 con 0x7f0c240cc0f0 2024-05-08T04:38:27.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.555+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c2404ef30 con 0x7f0c240cc0f0 2024-05-08T04:38:27.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:38:27.635 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:38:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.703+0000 7f0c21600640 1 -- 172.21.15.164:0/1863218000 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c1c059730 con 0x7f0c240cc0f0 2024-05-08T04:38:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 msgr2=0x7f0c04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 0x7f0c04069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c241fa080 tx=0x7f0c18009190 comp rx=0 tx=0).stop 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 msgr2=0x7f0c24158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00caa0 tx=0x7f0c1c00cf60 comp rx=0 tx=0).stop 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 shutdown_connections 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c240cc0f0 0x7f0c24158120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c240b2060 0x7f0c241578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 --2- 172.21.15.164:0/1863218000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c240b1c00 0x7f0c24168190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 >> 172.21.15.164:0/1863218000 conn(0x7f0c2406ac90 msgr2=0x7f0c24159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 shutdown_connections 2024-05-08T04:38:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.707+0000 7f0c2ae00640 1 -- 172.21.15.164:0/1863218000 wait complete. 2024-05-08T04:38:27.734 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:38:27.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 Processor -- start 2024-05-08T04:38:27.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 -- start start 2024-05-08T04:38:27.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a6806cce0 0x7f6a680ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a680cecc0 0x7f6a680ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6815e870 con 0x7f6a6815ab10 2024-05-08T04:38:27.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6806bb80 con 0x7f6a680cecc0 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a6d200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a680cf0f0 con 0x7f6a6806cce0 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a66800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a680cecc0 0x7f6a680ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a65e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6a6815ab10 0x7f6a680ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44946/0 (socket says 172.21.15.164:44946) 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a65e00640 1 -- 172.21.15.164:0/2386585685 learned_addr learned my addr 172.21.15.164:0/2386585685 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a65400640 1 --2- 172.21.15.164:0/2386585685 >> v2:172.21.15.172:3300/0 conn(0x7f6a6806cce0 0x7f6a680ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a64a00640 1 -- 172.21.15.164:0/2386585685 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242257622 0 0) 0x7f6a6815e870 con 0x7f6a6815ab10 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a64a00640 1 -- 172.21.15.164:0/2386585685 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a4c003530 con 0x7f6a6815ab10 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a66800640 1 -- 172.21.15.164:0/2386585685 >> v2:172.21.15.172:3300/0 conn(0x7f6a6806cce0 msgr2=0x7f6a680ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a66800640 1 --2- 172.21.15.164:0/2386585685 >> v2:172.21.15.172:3300/0 conn(0x7f6a6806cce0 0x7f6a680ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a66800640 1 -- 172.21.15.164:0/2386585685 >> v1:172.21.15.164:6789/0 conn(0x7f6a6815ab10 legacy=0x7f6a680ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.919+0000 7f6a66800640 1 -- 172.21.15.164:0/2386585685 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6815eb50 con 0x7f6a680cecc0 2024-05-08T04:38:27.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a66800640 1 --2- 172.21.15.164:0/2386585685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a680cecc0 0x7f6a680ca6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a680cf3c0 tx=0x7f6a6001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eca8b204bbfa332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a64a00640 1 -- 172.21.15.164:0/2386585685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600b2070 con 0x7f6a680cecc0 2024-05-08T04:38:27.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a64a00640 1 -- 172.21.15.164:0/2386585685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a600a5840 con 0x7f6a680cecc0 2024-05-08T04:38:27.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a64a00640 1 -- 172.21.15.164:0/2386585685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600a5bf0 con 0x7f6a680cecc0 2024-05-08T04:38:27.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 -- 172.21.15.164:0/2386585685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a680cecc0 msgr2=0x7f6a680ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2386585685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a680cecc0 0x7f6a680ca6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a680cf3c0 tx=0x7f6a6001c870 comp rx=0 tx=0).stop 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 -- 172.21.15.164:0/2386585685 shutdown_connections 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2386585685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a680cecc0 0x7f6a680ca6e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2386585685 >> v2:172.21.15.172:3300/0 conn(0x7f6a6806cce0 0x7f6a680ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 -- 172.21.15.164:0/2386585685 >> 172.21.15.164:0/2386585685 conn(0x7f6a680cf770 msgr2=0x7f6a680cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 -- 172.21.15.164:0/2386585685 shutdown_connections 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.923+0000 7f6a6d200640 1 -- 172.21.15.164:0/2386585685 wait complete. 2024-05-08T04:38:27.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 Processor -- start 2024-05-08T04:38:27.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- start start 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a680cecc0 0x7f6a680c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a6815ab10 0x7f6a680b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680aedf0 con 0x7f6a6806cce0 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6815aef0 con 0x7f6a6815ab10 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680aef70 con 0x7f6a680cecc0 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48238/0 (socket says 172.21.15.164:48238) 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 -- 172.21.15.164:0/2563372018 learned_addr learned my addr 172.21.15.164:0/2563372018 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:27.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a66800640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a6815ab10 0x7f6a680b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65400640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a680cecc0 0x7f6a680c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 -- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a680cecc0 msgr2=0x7f6a680c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a680cecc0 0x7f6a680c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 -- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a6815ab10 msgr2=0x7f6a680b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a6815ab10 0x7f6a680b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 -- 172.21.15.164:0/2563372018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680c82d0 con 0x7f6a6806cce0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a65e00640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800c450 tx=0x7f6a5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58017070 con 0x7f6a6806cce0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a58005cf0 con 0x7f6a6806cce0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a680c85b0 con 0x7f6a6806cce0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58007290 con 0x7f6a6806cce0 2024-05-08T04:38:27.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a680c8b70 con 0x7f6a6806cce0 2024-05-08T04:38:27.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.927+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68072d30 con 0x7f6a6806cce0 2024-05-08T04:38:27.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.931+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a5801a070 con 0x7f6a6806cce0 2024-05-08T04:38:27.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.931+0000 7f6a5e000640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 0x7f6a48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:27.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.931+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6a58084450 con 0x7f6a6806cce0 2024-05-08T04:38:27.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.931+0000 7f6a65400640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 0x7f6a48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:27.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.931+0000 7f6a65400640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 0x7f6a48069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a681fa630 tx=0x7f6a54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:27.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:27.935+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a58021080 con 0x7f6a6806cce0 2024-05-08T04:38:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.087+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6a68157e70 con 0x7f6a6806cce0 2024-05-08T04:38:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.711+0000 7f6a5e000640 1 -- 172.21.15.164:0/2563372018 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6a58055060 con 0x7f6a6806cce0 2024-05-08T04:38:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 msgr2=0x7f6a48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 0x7f6a48069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a681fa630 tx=0x7f6a54008040 comp rx=0 tx=0).stop 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 msgr2=0x7f6a6814d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800c450 tx=0x7f6a5800c910 comp rx=0 tx=0).stop 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 shutdown_connections 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a48066f60 0x7f6a48069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a680cecc0 0x7f6a680c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a6815ab10 0x7f6a680b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 --2- 172.21.15.164:0/2563372018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a6806cce0 0x7f6a6814d2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 >> 172.21.15.164:0/2563372018 conn(0x7f6a680cf770 msgr2=0x7f6a680cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 shutdown_connections 2024-05-08T04:38:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.715+0000 7f6a6d200640 1 -- 172.21.15.164:0/2563372018 wait complete. 2024-05-08T04:38:28.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:38:28.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 Processor -- start 2024-05-08T04:38:28.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 -- start start 2024-05-08T04:38:28.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbed00adfa0 0x7fbed00b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbed00b4dc0 0x7fbed006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed015e300 con 0x7fbed00af6d0 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed015e570 con 0x7fbed00b4dc0 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed8600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed015e850 con 0x7fbed00adfa0 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed6600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbed00b4dc0 0x7fbed006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed5c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbed00af6d0 0x7fbed00ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44960/0 (socket says 172.21.15.164:44960) 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.895+0000 7fbed5c00640 1 -- 172.21.15.164:0/343649515 learned_addr learned my addr 172.21.15.164:0/343649515 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed5200640 1 --2- 172.21.15.164:0/343649515 >> v2:172.21.15.172:3300/0 conn(0x7fbed00adfa0 0x7fbed00b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed6600640 1 -- 172.21.15.164:0/343649515 >> v2:172.21.15.172:3300/0 conn(0x7fbed00adfa0 msgr2=0x7fbed00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:28.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed6600640 1 --2- 172.21.15.164:0/343649515 >> v2:172.21.15.172:3300/0 conn(0x7fbed00adfa0 0x7fbed00b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed6600640 1 -- 172.21.15.164:0/343649515 >> v1:172.21.15.164:6789/0 conn(0x7fbed00af6d0 legacy=0x7fbed00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbecfe00640 1 -- 172.21.15.164:0/343649515 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373372152 0 0) 0x7fbed015e300 con 0x7fbed00af6d0 2024-05-08T04:38:28.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed6600640 1 -- 172.21.15.164:0/343649515 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00b5250 con 0x7fbed00b4dc0 2024-05-08T04:38:28.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbed6600640 1 --2- 172.21.15.164:0/343649515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed006ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbed0070b80 tx=0x7fbec801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cfffe57e132356 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:28.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbecfe00640 1 -- 172.21.15.164:0/343649515 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80b2070 con 0x7fbed00b4dc0 2024-05-08T04:38:28.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbecfe00640 1 -- 172.21.15.164:0/343649515 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec80a5870 con 0x7fbed00b4dc0 2024-05-08T04:38:28.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.899+0000 7fbecfe00640 1 -- 172.21.15.164:0/343649515 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80a5c40 con 0x7fbed00b4dc0 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- 172.21.15.164:0/343649515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 msgr2=0x7fbed006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- 172.21.15.164:0/343649515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed006ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbed0070b80 tx=0x7fbec801c8a0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- 172.21.15.164:0/343649515 shutdown_connections 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- 172.21.15.164:0/343649515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed006ac90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- 172.21.15.164:0/343649515 >> v2:172.21.15.172:3300/0 conn(0x7fbed00adfa0 0x7fbed00b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- 172.21.15.164:0/343649515 >> 172.21.15.164:0/343649515 conn(0x7fbed006cca0 msgr2=0x7fbed006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:28.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- 172.21.15.164:0/343649515 shutdown_connections 2024-05-08T04:38:28.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- 172.21.15.164:0/343649515 wait complete. 2024-05-08T04:38:28.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 Processor -- start 2024-05-08T04:38:28.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- start start 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed00adfa0 0x7fbed0151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 0x7fbed014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed016a950 con 0x7fbed00b4dc0 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed006bbf0 con 0x7fbed00af6d0 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed8600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed016aad0 con 0x7fbed00adfa0 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48252/0 (socket says 172.21.15.164:48252) 2024-05-08T04:38:28.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 -- 172.21.15.164:0/895306192 learned_addr learned my addr 172.21.15.164:0/895306192 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed5200640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 0x7fbed014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed5c00640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed00adfa0 0x7fbed0151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 -- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed00adfa0 msgr2=0x7fbed0151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed00adfa0 0x7fbed0151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 -- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 msgr2=0x7fbed014c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 0x7fbed014c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 -- 172.21.15.164:0/895306192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0151cf0 con 0x7fbed00b4dc0 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed5200640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 0x7fbed014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.903+0000 7fbed6600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b0040 tx=0x7fbec8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80b2050 con 0x7fbed00b4dc0 2024-05-08T04:38:28.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec80b3040 con 0x7fbed00b4dc0 2024-05-08T04:38:28.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80ad650 con 0x7fbed00b4dc0 2024-05-08T04:38:28.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed0151fd0 con 0x7fbed00b4dc0 2024-05-08T04:38:28.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed0152590 con 0x7fbed00b4dc0 2024-05-08T04:38:28.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed004f010 con 0x7fbed00b4dc0 2024-05-08T04:38:28.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbec80c0020 con 0x7fbed00b4dc0 2024-05-08T04:38:28.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 0x7fbeb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:28.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbec811f8e0 con 0x7fbed00b4dc0 2024-05-08T04:38:28.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.907+0000 7fbed5c00640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 0x7fbeb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:28.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.911+0000 7fbed5c00640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 0x7fbeb0069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000f70 tx=0x7fbec4007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:28.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:28.911+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbec8016520 con 0x7fbed00b4dc0 2024-05-08T04:38:29.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.099+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbed0067230 con 0x7fbed00b4dc0 2024-05-08T04:38:29.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.727+0000 7fbecd600640 1 -- 172.21.15.164:0/895306192 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbec80f0950 con 0x7fbed00b4dc0 2024-05-08T04:38:29.728 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T04:38:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.731+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 msgr2=0x7fbeb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.731+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 0x7fbeb0069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000f70 tx=0x7fbec4007b60 comp rx=0 tx=0).stop 2024-05-08T04:38:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.731+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 msgr2=0x7fbed0070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.731+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b0040 tx=0x7fbec8019640 comp rx=0 tx=0).stop 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 shutdown_connections 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbeb0066f60 0x7fbeb0069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00b4dc0 0x7fbed0070bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed00af6d0 0x7fbed014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 --2- 172.21.15.164:0/895306192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed00adfa0 0x7fbed0151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 >> 172.21.15.164:0/895306192 conn(0x7fbed006cca0 msgr2=0x7fbed006bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:29.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 shutdown_connections 2024-05-08T04:38:29.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.735+0000 7fbed8600640 1 -- 172.21.15.164:0/895306192 wait complete. 2024-05-08T04:38:29.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:29.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.899+0000 7f03c1400640 1 Processor -- start 2024-05-08T04:38:29.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c1400640 1 -- start start 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03bc070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c1400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc06dd90 con 0x7f03bc0718a0 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c1400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc06d7d0 con 0x7f03bc066d80 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c1400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc06dab0 con 0x7f03bc0669a0 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c0a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f03bc0718a0 0x7f03bc071d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57886/0 (socket says 172.21.15.164:57886) 2024-05-08T04:38:29.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c0a00640 1 -- 172.21.15.164:0/2099406394 learned_addr learned my addr 172.21.15.164:0/2099406394 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:29.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bbe00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03bc070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693978639 0 0) 0x7f03b40014a0 con 0x7f03bc0718a0 2024-05-08T04:38:29.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a0005740 con 0x7f03bc0718a0 2024-05-08T04:38:29.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010706860 0 0) 0x7f03bc06d7d0 con 0x7f03bc066d80 2024-05-08T04:38:29.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b40014a0 con 0x7f03bc066d80 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657752813 0 0) 0x7f03a0005740 con 0x7f03bc0718a0 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 msgr2=0x7f03bc070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03bc070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 >> v1:172.21.15.172:6789/0 conn(0x7f03bc066d80 legacy=0x7f03bc071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0b2f60 con 0x7f03bc0718a0 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f03b4001bd0 con 0x7f03bc0718a0 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 >> v1:172.21.15.164:6789/0 conn(0x7f03bc0718a0 legacy=0x7f03bc071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03a0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03a0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0006f60 0x7f03a00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc06dab0 con 0x7f03a0006340 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000dda0 con 0x7f03a0006f60 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000dfa0 con 0x7f03bc0669a0 2024-05-08T04:38:29.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03a0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bbe00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03a0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c0a00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0006f60 0x7f03a00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 -- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 msgr2=0x7f03a0005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0669a0 0x7f03a0005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 -- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0006f60 msgr2=0x7f03a00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0006f60 0x7f03a00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03bb400640 1 -- 172.21.15.164:0/2099406394 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0005740 con 0x7f03a0006340 2024-05-08T04:38:29.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.903+0000 7f03c0a00640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0006f60 0x7f03a00073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:29.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03bb400640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03a0006a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03a80018d0 tx=0x7f03a801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28dab421b9a1203 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:29.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8001520 con 0x7f03a0006340 2024-05-08T04:38:29.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a80050d0 con 0x7f03a0006340 2024-05-08T04:38:29.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03baa00640 1 -- 172.21.15.164:0/2099406394 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8003600 con 0x7f03a0006340 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 -- 172.21.15.164:0/2099406394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 msgr2=0x7f03a0006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 --2- 172.21.15.164:0/2099406394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03a0006a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03a80018d0 tx=0x7f03a801c580 comp rx=0 tx=0).stop 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c0a00640 1 -- 172.21.15.164:0/2099406394 reap_dead start 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 -- 172.21.15.164:0/2099406394 shutdown_connections 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 -- 172.21.15.164:0/2099406394 >> 172.21.15.164:0/2099406394 conn(0x7f03bc0d0410 msgr2=0x7f03bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 -- 172.21.15.164:0/2099406394 shutdown_connections 2024-05-08T04:38:29.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.907+0000 7f03c1400640 1 -- 172.21.15.164:0/2099406394 wait complete. 2024-05-08T04:38:29.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 Processor -- start 2024-05-08T04:38:29.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- start start 2024-05-08T04:38:29.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 0x7f03bc1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 0x7f03bc0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0b40f0 con 0x7f03a0006340 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0b3f70 con 0x7f03bc0cc8a0 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0b4270 con 0x7f03bc0cf370 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41054/0 (socket says 172.21.15.164:41054) 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 -- 172.21.15.164:0/3026130238 learned_addr learned my addr 172.21.15.164:0/3026130238 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c0a00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 0x7f03bc0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bb400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 0x7f03bc1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 -- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 msgr2=0x7f03bc1f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 0x7f03bc1f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 -- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 msgr2=0x7f03bc0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:29.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 0x7f03bc0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 -- 172.21.15.164:0/3026130238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc1f23e0 con 0x7f03a0006340 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c0a00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 0x7f03bc0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bb400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 0x7f03bc1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03bbe00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0119d0 tx=0x7f03ac011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00fa70 con 0x7f03a0006340 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac00fc10 con 0x7f03a0006340 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc1f69f0 con 0x7f03a0006340 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac014440 con 0x7f03a0006340 2024-05-08T04:38:29.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc1f6430 con 0x7f03a0006340 2024-05-08T04:38:29.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.911+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc0cf750 con 0x7f03a0006340 2024-05-08T04:38:29.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.915+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03ac012040 con 0x7f03a0006340 2024-05-08T04:38:29.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.915+0000 7f03b3e00640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 0x7f039c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:29.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.915+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ac0854f0 con 0x7f03a0006340 2024-05-08T04:38:29.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.915+0000 7f03bb400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 0x7f039c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:29.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.915+0000 7f03bb400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 0x7f039c069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03a80ac000 tx=0x7f03a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:29.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:29.919+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03ac019030 con 0x7f03a0006340 2024-05-08T04:38:30.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03bc04ef30 con 0x7f03a0006340 2024-05-08T04:38:30.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03b3e00640 1 -- 172.21.15.164:0/3026130238 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16678 (secure 0 0 0) 0x7f03ac0565b0 con 0x7f03a0006340 2024-05-08T04:38:30.078 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:30.078 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":19,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:29.101349+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:30.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 msgr2=0x7f039c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 0x7f039c069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03a80ac000 tx=0x7f03a8009040 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 msgr2=0x7f03bc1f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0119d0 tx=0x7f03ac011e90 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 shutdown_connections 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f039c066f60 0x7f039c069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0cf370 0x7f03bc1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03bc0cc8a0 0x7f03bc0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 --2- 172.21.15.164:0/3026130238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0006340 0x7f03bc1f5ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 >> 172.21.15.164:0/3026130238 conn(0x7f03bc0d0410 msgr2=0x7f03bc159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 shutdown_connections 2024-05-08T04:38:30.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.075+0000 7f03c1400640 1 -- 172.21.15.164:0/3026130238 wait complete. 2024-05-08T04:38:30.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:30.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 Processor -- start 2024-05-08T04:38:30.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 -- start start 2024-05-08T04:38:30.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15ba70 0x7fdf1c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0cfda0 0x7fdf1c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c0cd9f0 con 0x7fdf1c15be50 2024-05-08T04:38:30.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c15fe70 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.231+0000 7fdf24400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0cdcd0 con 0x7fdf1c15ba70 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf21000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdf1c15be50 0x7fdf1c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57890/0 (socket says 172.21.15.164:57890) 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf21000640 1 -- 172.21.15.164:0/1458766459 learned_addr learned my addr 172.21.15.164:0/1458766459 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 --2- 172.21.15.164:0/1458766459 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0cfda0 0x7fdf1c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf21a00640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15ba70 0x7fdf1c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf1be00640 1 -- 172.21.15.164:0/1458766459 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677970060 0 0) 0x7fdf1c0cd9f0 con 0x7fdf1c15be50 2024-05-08T04:38:30.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf1be00640 1 -- 172.21.15.164:0/1458766459 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf04003530 con 0x7fdf1c15be50 2024-05-08T04:38:30.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 -- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15ba70 msgr2=0x7fdf1c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:30.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15ba70 0x7fdf1c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 -- 172.21.15.164:0/1458766459 >> v1:172.21.15.164:6789/0 conn(0x7fdf1c15be50 legacy=0x7fdf1c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 -- 172.21.15.164:0/1458766459 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c15f820 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf22400640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c0b31a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009d90 tx=0x7fdf1401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be8dea9bb43c69c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.235+0000 7fdf1be00640 1 -- 172.21.15.164:0/1458766459 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140b2070 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf1be00640 1 -- 172.21.15.164:0/1458766459 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf140a5910 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf1be00640 1 -- 172.21.15.164:0/1458766459 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140a5ce0 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 msgr2=0x7fdf1c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c0b31a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009d90 tx=0x7fdf1401c940 comp rx=0 tx=0).stop 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- 172.21.15.164:0/1458766459 shutdown_connections 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c0b31a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- 172.21.15.164:0/1458766459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15ba70 0x7fdf1c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- 172.21.15.164:0/1458766459 >> 172.21.15.164:0/1458766459 conn(0x7fdf1c06ac90 msgr2=0x7fdf1c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:30.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- 172.21.15.164:0/1458766459 shutdown_connections 2024-05-08T04:38:30.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- 172.21.15.164:0/1458766459 wait complete. 2024-05-08T04:38:30.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 Processor -- start 2024-05-08T04:38:30.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- start start 2024-05-08T04:38:30.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15be50 0x7fdf1c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c168d40 con 0x7fdf1c15ba70 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c168bc0 con 0x7fdf1c0cfda0 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf24400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c168ec0 con 0x7fdf1c15be50 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf21000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf21000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41058/0 (socket says 172.21.15.164:41058) 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf21000640 1 -- 172.21.15.164:0/3659203710 learned_addr learned my addr 172.21.15.164:0/3659203710 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf21a00640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.239+0000 7fdf22400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15be50 0x7fdf1c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 -- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15be50 msgr2=0x7fdf1c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15be50 0x7fdf1c15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 -- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 msgr2=0x7fdf1c14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 -- 172.21.15.164:0/3659203710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c06db30 con 0x7fdf1c15ba70 2024-05-08T04:38:30.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf21000640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c390 tx=0x7fdf0800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08017070 con 0x7fdf1c15ba70 2024-05-08T04:38:30.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf08005d10 con 0x7fdf1c15ba70 2024-05-08T04:38:30.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08007310 con 0x7fdf1c15ba70 2024-05-08T04:38:30.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c155ee0 con 0x7fdf1c15ba70 2024-05-08T04:38:30.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c15d820 con 0x7fdf1c15ba70 2024-05-08T04:38:30.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c069d60 con 0x7fdf1c15ba70 2024-05-08T04:38:30.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf0801a070 con 0x7fdf1c15ba70 2024-05-08T04:38:30.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.243+0000 7fdf19600640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 0x7fdefc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.247+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf08084440 con 0x7fdf1c15ba70 2024-05-08T04:38:30.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.247+0000 7fdf21a00640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 0x7fdefc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.247+0000 7fdf21a00640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 0x7fdefc069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009b60 tx=0x7fdf10009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.251+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf1c155ee0 con 0x7fdf1c15ba70 2024-05-08T04:38:30.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.403+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf1c06b110 con 0x7fdf1c15ba70 2024-05-08T04:38:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.403+0000 7fdf19600640 1 -- 172.21.15.164:0/3659203710 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16678 (secure 0 0 0) 0x7fdf08054e20 con 0x7fdf1c15ba70 2024-05-08T04:38:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:30.408 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":19,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:29.101349+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 msgr2=0x7fdefc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 0x7fdefc069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009b60 tx=0x7fdf10009190 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 msgr2=0x7fdf1c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c390 tx=0x7fdf0800c850 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 shutdown_connections 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdefc066f60 0x7fdefc069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c15be50 0x7fdf1c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c15ba70 0x7fdf1c15ccb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 --2- 172.21.15.164:0/3659203710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0cfda0 0x7fdf1c14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 >> 172.21.15.164:0/3659203710 conn(0x7fdf1c06ac90 msgr2=0x7fdf1c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 shutdown_connections 2024-05-08T04:38:30.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.407+0000 7fdf24400640 1 -- 172.21.15.164:0/3659203710 wait complete. 2024-05-08T04:38:30.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:38:30.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.563+0000 7fda2a600640 1 Processor -- start 2024-05-08T04:38:30.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 -- start start 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda240d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda240d0a90 0x7fda240b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda240af9a0 con 0x7fda240b5ab0 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda240af750 con 0x7fda240d0a90 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda2a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda240afc80 con 0x7fda240aee90 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fda240b5ab0 0x7fda2415e900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57904/0 (socket says 172.21.15.164:57904) 2024-05-08T04:38:30.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23e00640 1 -- 172.21.15.164:0/1979631538 learned_addr learned my addr 172.21.15.164:0/1979631538 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:30.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda1aa00640 1 --2- 172.21.15.164:0/1979631538 >> v2:172.21.15.172:3300/0 conn(0x7fda240d0a90 0x7fda240b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda240d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066906055 0 0) 0x7fda240af9a0 con 0x7fda240b5ab0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda08003530 con 0x7fda240b5ab0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780312153 0 0) 0x7fda08003530 con 0x7fda240b5ab0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 msgr2=0x7fda240d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda240d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 >> v2:172.21.15.172:3300/0 conn(0x7fda240d0a90 msgr2=0x7fda240b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 --2- 172.21.15.164:0/1979631538 >> v2:172.21.15.172:3300/0 conn(0x7fda240d0a90 0x7fda240b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240aff60 con 0x7fda240b5ab0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fda1c001a00 con 0x7fda240b5ab0 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 >> v1:172.21.15.164:6789/0 conn(0x7fda240b5ab0 legacy=0x7fda2415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda080039c0 0x7fda08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda08004810 0x7fda08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240af750 con 0x7fda080039c0 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240afc80 con 0x7fda08004810 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0800b9d0 con 0x7fda240aee90 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda080039c0 0x7fda08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda1aa00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda08004810 0x7fda08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23e00640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 -- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 msgr2=0x7fda080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda240aee90 0x7fda080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 -- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda08004810 msgr2=0x7fda08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda08004810 0x7fda08004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.567+0000 7fda23400640 1 -- 172.21.15.164:0/1979631538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08003530 con 0x7fda080039c0 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda23400640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda080039c0 0x7fda08003de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda14009ae0 tx=0x7fda1401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6584016464e04347 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140b2070 con 0x7fda080039c0 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda140a5530 con 0x7fda080039c0 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda22a00640 1 -- 172.21.15.164:0/1979631538 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140096b0 con 0x7fda080039c0 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 -- 172.21.15.164:0/1979631538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda080039c0 msgr2=0x7fda08003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 --2- 172.21.15.164:0/1979631538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda080039c0 0x7fda08003de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda14009ae0 tx=0x7fda1401c9e0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda23e00640 1 -- 172.21.15.164:0/1979631538 reap_dead start 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 -- 172.21.15.164:0/1979631538 shutdown_connections 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 -- 172.21.15.164:0/1979631538 >> 172.21.15.164:0/1979631538 conn(0x7fda24067050 msgr2=0x7fda2406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 -- 172.21.15.164:0/1979631538 shutdown_connections 2024-05-08T04:38:30.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.571+0000 7fda2a600640 1 -- 172.21.15.164:0/1979631538 wait complete. 2024-05-08T04:38:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 Processor -- start 2024-05-08T04:38:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- start start 2024-05-08T04:38:30.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 0x7fda241494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 0x7fda2415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24072270 con 0x7fda241ea2d0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240720f0 con 0x7fda2415a390 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2406b4a0 con 0x7fda080039c0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41072/0 (socket says 172.21.15.164:41072) 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 -- 172.21.15.164:0/3884157778 learned_addr learned my addr 172.21.15.164:0/3884157778 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda23e00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 0x7fda2415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda23400640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 0x7fda241494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 -- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 msgr2=0x7fda241494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 0x7fda241494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 -- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 msgr2=0x7fda2415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 0x7fda2415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 -- 172.21.15.164:0/3884157778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2415f610 con 0x7fda241ea2d0 2024-05-08T04:38:30.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda23400640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 0x7fda241494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda23e00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 0x7fda2415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1aa00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0119b0 tx=0x7fda0c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c019070 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c004240 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2415f8b0 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00f880 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2415fe70 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.575+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda24072e80 con 0x7fda241ea2d0 2024-05-08T04:38:30.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.579+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda0c01c070 con 0x7fda241ea2d0 2024-05-08T04:38:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.579+0000 7fda1be00640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 0x7fda00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.579+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda0c085460 con 0x7fda241ea2d0 2024-05-08T04:38:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.579+0000 7fda23400640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 0x7fda00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.579+0000 7fda23400640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 0x7fda00069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda140af000 tx=0x7fda140046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:30.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.583+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda0c0563d0 con 0x7fda241ea2d0 2024-05-08T04:38:30.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:30.775+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fda2404f010 con 0x7fda241ea2d0 2024-05-08T04:38:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda1be00640 1 -- 172.21.15.164:0/3884157778 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fda0c05c0d0 con 0x7fda241ea2d0 2024-05-08T04:38:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 msgr2=0x7fda00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 0x7fda00069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda140af000 tx=0x7fda140046d0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 msgr2=0x7fda241ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0119b0 tx=0x7fda0c011e70 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 shutdown_connections 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda00066f90 0x7fda00069450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2415a390 0x7fda2415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda241ea2d0 0x7fda241ea730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 --2- 172.21.15.164:0/3884157778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda080039c0 0x7fda241494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 >> 172.21.15.164:0/3884157778 conn(0x7fda24067050 msgr2=0x7fda240ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 shutdown_connections 2024-05-08T04:38:31.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.743+0000 7fda2a600640 1 -- 172.21.15.164:0/3884157778 wait complete. 2024-05-08T04:38:31.768 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:38:31.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:38:31.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.903+0000 7f5746e00640 1 Processor -- start 2024-05-08T04:38:31.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.903+0000 7f5746e00640 1 -- start start 2024-05-08T04:38:31.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 0x7f574006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f574006fe10 0x7f57400cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57400cce80 0x7f57400cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5740070270 con 0x7f57400cce80 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57400cc4b0 con 0x7f574006fe10 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400cfdc0 con 0x7f574015efd0 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57400cce80 0x7f57400cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57400cce80 0x7f57400cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41086/0 (socket says 172.21.15.164:41086) 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 -- 172.21.15.164:0/385793047 learned_addr learned my addr 172.21.15.164:0/385793047 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5745a00640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 0x7f574006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5745000640 1 --2- 172.21.15.164:0/385793047 >> v2:172.21.15.172:3300/0 conn(0x7f574006fe10 0x7f57400cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 -- 172.21.15.164:0/385793047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 msgr2=0x7f574006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 0x7f574006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 -- 172.21.15.164:0/385793047 >> v2:172.21.15.172:3300/0 conn(0x7f574006fe10 msgr2=0x7f57400cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:31.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 --2- 172.21.15.164:0/385793047 >> v2:172.21.15.172:3300/0 conn(0x7f574006fe10 0x7f57400cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 -- 172.21.15.164:0/385793047 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400d00a0 con 0x7f57400cce80 2024-05-08T04:38:31.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5745a00640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 0x7f574006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:31.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.907+0000 7f5746400640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57400cce80 0x7f57400cd310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f57400d28e0 tx=0x7f573801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea98001998067af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:31.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f573fe00640 1 -- 172.21.15.164:0/385793047 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380b4070 con 0x7f57400cce80 2024-05-08T04:38:31.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f573fe00640 1 -- 172.21.15.164:0/385793047 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57380af400 con 0x7f57400cce80 2024-05-08T04:38:31.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f573fe00640 1 -- 172.21.15.164:0/385793047 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380af7d0 con 0x7f57400cce80 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- 172.21.15.164:0/385793047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57400cce80 msgr2=0x7f57400cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57400cce80 0x7f57400cd310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f57400d28e0 tx=0x7f573801f1e0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- 172.21.15.164:0/385793047 shutdown_connections 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57400cce80 0x7f57400cd310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- 172.21.15.164:0/385793047 >> v2:172.21.15.172:3300/0 conn(0x7f574006fe10 0x7f57400cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- 172.21.15.164:0/385793047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f574015efd0 0x7f574006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- 172.21.15.164:0/385793047 >> 172.21.15.164:0/385793047 conn(0x7f57400b3370 msgr2=0x7f57400b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- 172.21.15.164:0/385793047 shutdown_connections 2024-05-08T04:38:31.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- 172.21.15.164:0/385793047 wait complete. 2024-05-08T04:38:31.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 Processor -- start 2024-05-08T04:38:31.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- start start 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57400cce80 0x7f57400c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f574015efd0 0x7f57400c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400d0660 con 0x7f574006fe10 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574015f3b0 con 0x7f574015efd0 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400cdf70 con 0x7f57400cce80 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5745a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5745a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41102/0 (socket says 172.21.15.164:41102) 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5745a00640 1 -- 172.21.15.164:0/49360730 learned_addr learned my addr 172.21.15.164:0/49360730 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5745000640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57400cce80 0x7f57400c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.911+0000 7f5746400640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f574015efd0 0x7f57400c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 -- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57400cce80 msgr2=0x7f57400c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57400cce80 0x7f57400c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 -- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f574015efd0 msgr2=0x7f57400c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f574015efd0 0x7f57400c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 -- 172.21.15.164:0/49360730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740156410 con 0x7f574006fe10 2024-05-08T04:38:31.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745a00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f573400d830 tx=0x7f573400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400b6c0 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734019070 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400baa0 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57400c4530 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574015b3a0 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574004ef30 con 0x7f574006fe10 2024-05-08T04:38:31.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f573401e460 con 0x7f574006fe10 2024-05-08T04:38:31.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:31.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5734085610 con 0x7f574006fe10 2024-05-08T04:38:31.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.915+0000 7f5745000640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:31.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.919+0000 7f5745000640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 0x7f5720069450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574009f970 tx=0x7f572c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:31.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:31.919+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5734055ea0 con 0x7f574006fe10 2024-05-08T04:38:32.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.107+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5740067720 con 0x7f574006fe10 2024-05-08T04:38:32.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.747+0000 7f573d600640 1 -- 172.21.15.164:0/49360730 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5734059f40 con 0x7f574006fe10 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 msgr2=0x7f5720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 0x7f5720069450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574009f970 tx=0x7f572c008040 comp rx=0 tx=0).stop 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 msgr2=0x7f5740155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f573400d830 tx=0x7f573400dcf0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 shutdown_connections 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57400cce80 0x7f57400c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f574015efd0 0x7f57400c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 --2- 172.21.15.164:0/49360730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f574006fe10 0x7f5740155ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 >> 172.21.15.164:0/49360730 conn(0x7f57400b3370 msgr2=0x7f57400ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:32.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 shutdown_connections 2024-05-08T04:38:32.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.751+0000 7f5746e00640 1 -- 172.21.15.164:0/49360730 wait complete. 2024-05-08T04:38:32.772 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:38:32.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:32.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.903+0000 7fda51c00640 1 Processor -- start 2024-05-08T04:38:32.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.903+0000 7fda51c00640 1 -- start start 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda51c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fda4c0b26c0 0x7fda4c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda51c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c06c770 con 0x7fda4c0cfbd0 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda51c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c06ca50 con 0x7fda4c15ab60 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda51c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0d0060 con 0x7fda4c0b26c0 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda4b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fda4c0cfbd0 0x7fda4c0cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57920/0 (socket says 172.21.15.164:57920) 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda42000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fda4c0b26c0 0x7fda4c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda4b400640 1 -- 172.21.15.164:0/1147693359 learned_addr learned my addr 172.21.15.164:0/1147693359 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:32.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda42000640 1 -- 172.21.15.164:0/1147693359 >> v1:172.21.15.172:6789/0 conn(0x7fda4c15ab60 legacy=0x7fda4c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:32.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda42000640 1 -- 172.21.15.164:0/1147693359 >> v1:172.21.15.164:6789/0 conn(0x7fda4c0cfbd0 legacy=0x7fda4c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.907+0000 7fda42000640 1 -- 172.21.15.164:0/1147693359 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c06cd30 con 0x7fda4c0b26c0 2024-05-08T04:38:32.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda42000640 1 --2- 172.21.15.164:0/1147693359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c0b26c0 0x7fda4c0cf690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda340097c0 tx=0x7fda3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49734868b961b9a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:32.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda4a000640 1 -- 172.21.15.164:0/1147693359 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda340b0070 con 0x7fda4c0b26c0 2024-05-08T04:38:32.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda4a000640 1 -- 172.21.15.164:0/1147693359 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda340ab600 con 0x7fda4c0b26c0 2024-05-08T04:38:32.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda4a000640 1 -- 172.21.15.164:0/1147693359 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda340ab9d0 con 0x7fda4c0b26c0 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- 172.21.15.164:0/1147693359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c0b26c0 msgr2=0x7fda4c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 --2- 172.21.15.164:0/1147693359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c0b26c0 0x7fda4c0cf690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda340097c0 tx=0x7fda3401c460 comp rx=0 tx=0).stop 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- 172.21.15.164:0/1147693359 shutdown_connections 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 --2- 172.21.15.164:0/1147693359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c0b26c0 0x7fda4c0cf690 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- 172.21.15.164:0/1147693359 >> 172.21.15.164:0/1147693359 conn(0x7fda4c06ac90 msgr2=0x7fda4c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- 172.21.15.164:0/1147693359 shutdown_connections 2024-05-08T04:38:32.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- 172.21.15.164:0/1147693359 wait complete. 2024-05-08T04:38:32.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 Processor -- start 2024-05-08T04:38:32.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.911+0000 7fda51c00640 1 -- start start 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda4c0b26c0 0x7fda4c14a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4c0cfbd0 0x7fda4c14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0aeeb0 con 0x7fda4c15ab60 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0aed30 con 0x7fda4c0cfbd0 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c168980 con 0x7fda4c0b26c0 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41120/0 (socket says 172.21.15.164:41120) 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 -- 172.21.15.164:0/1615426786 learned_addr learned my addr 172.21.15.164:0/1615426786 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:32.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda42000640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4c0cfbd0 0x7fda4c14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4aa00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda4c0b26c0 0x7fda4c14a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 -- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda4c0b26c0 msgr2=0x7fda4c14a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda4c0b26c0 0x7fda4c14a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 -- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4c0cfbd0 msgr2=0x7fda4c14bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4c0cfbd0 0x7fda4c14bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 -- 172.21.15.164:0/1615426786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c14c480 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda4b400640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c430 tx=0x7fda4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44017070 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44005cf0 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c14c760 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440072d0 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c14c190 con 0x7fda4c15ab60 2024-05-08T04:38:32.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c069d60 con 0x7fda4c15ab60 2024-05-08T04:38:32.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.915+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda4401a070 con 0x7fda4c15ab60 2024-05-08T04:38:32.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.919+0000 7fda43400640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 0x7fda28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:32.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.919+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda44084330 con 0x7fda4c15ab60 2024-05-08T04:38:32.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.919+0000 7fda4aa00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 0x7fda28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:32.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.919+0000 7fda4aa00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 0x7fda28069450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda4c1fa2b0 tx=0x7fda3c0094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:32.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:32.923+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda44054cf0 con 0x7fda4c15ab60 2024-05-08T04:38:33.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda4c06b070 con 0x7fda4c15ab60 2024-05-08T04:38:33.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda43400640 1 -- 172.21.15.164:0/1615426786 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7fda44058d90 con 0x7fda4c15ab60 2024-05-08T04:38:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:33.110 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:38:33.110 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:38:33.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 msgr2=0x7fda28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 0x7fda28069450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda4c1fa2b0 tx=0x7fda3c0094d0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 msgr2=0x7fda4c1497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c430 tx=0x7fda4400c8f0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 shutdown_connections 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda28066f90 0x7fda28069450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda4c15ab60 0x7fda4c1497a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4c0cfbd0 0x7fda4c14bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 --2- 172.21.15.164:0/1615426786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda4c0b26c0 0x7fda4c14a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 >> 172.21.15.164:0/1615426786 conn(0x7fda4c06ac90 msgr2=0x7fda4c15eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 shutdown_connections 2024-05-08T04:38:33.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.107+0000 7fda51c00640 1 -- 172.21.15.164:0/1615426786 wait complete. 2024-05-08T04:38:33.131 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:33.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 Processor -- start 2024-05-08T04:38:33.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 -- start start 2024-05-08T04:38:33.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa07c163540 0x7fa07c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 0x7fa07c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0ceac0 con 0x7fa07c0aedb0 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0ce860 con 0x7fa07c163540 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa083600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0cc1e0 con 0x7fa07c0cf6c0 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa07c163540 0x7fa07c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa07c163540 0x7fa07c0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41132/0 (socket says 172.21.15.164:41132) 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 -- 172.21.15.164:0/737445805 learned_addr learned my addr 172.21.15.164:0/737445805 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa081600640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 0x7fa07c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 -- 172.21.15.164:0/737445805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 msgr2=0x7fa07c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 0x7fa07c0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.263+0000 7fa073e00640 1 -- 172.21.15.164:0/737445805 >> v1:172.21.15.172:6789/0 conn(0x7fa07c0aedb0 legacy=0x7fa07c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa073e00640 1 -- 172.21.15.164:0/737445805 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0cc4c0 con 0x7fa07c163540 2024-05-08T04:38:33.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa081600640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 0x7fa07c0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:33.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa073e00640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c0cccc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa064009870 tx=0x7fa06401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f79e9c3dbf58cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa07be00640 1 -- 172.21.15.164:0/737445805 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640b1070 con 0x7fa07c163540 2024-05-08T04:38:33.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa07be00640 1 -- 172.21.15.164:0/737445805 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0640a8050 con 0x7fa07c163540 2024-05-08T04:38:33.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa07be00640 1 -- 172.21.15.164:0/737445805 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640ac860 con 0x7fa07c163540 2024-05-08T04:38:33.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 -- 172.21.15.164:0/737445805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 msgr2=0x7fa07c0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c0cccc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa064009870 tx=0x7fa06401c4d0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 -- 172.21.15.164:0/737445805 shutdown_connections 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0cf6c0 0x7fa07c0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 --2- 172.21.15.164:0/737445805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c0cccc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 -- 172.21.15.164:0/737445805 >> 172.21.15.164:0/737445805 conn(0x7fa07c06ac90 msgr2=0x7fa07c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 -- 172.21.15.164:0/737445805 shutdown_connections 2024-05-08T04:38:33.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.267+0000 7fa083600640 1 -- 172.21.15.164:0/737445805 wait complete. 2024-05-08T04:38:33.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 Processor -- start 2024-05-08T04:38:33.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- start start 2024-05-08T04:38:33.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 0x7fa07c164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 0x7fa07c15f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c06dbc0 con 0x7fa07c163540 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c06da40 con 0x7fa07c0cf6c0 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c06dd40 con 0x7fa07c0aedb0 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41148/0 (socket says 172.21.15.164:41148) 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 -- 172.21.15.164:0/3911823311 learned_addr learned my addr 172.21.15.164:0/3911823311 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 -- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 msgr2=0x7fa07c164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa073e00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 0x7fa07c15f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa080c00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 0x7fa07c164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 0x7fa07c164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 -- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 msgr2=0x7fa07c15f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 0x7fa07c15f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 -- 172.21.15.164:0/3911823311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c15e840 con 0x7fa07c163540 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa073e00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 0x7fa07c15f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa080c00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 0x7fa07c164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa081600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa07400cef0 tx=0x7fa07400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07401a070 con 0x7fa07c163540 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07400d860 con 0x7fa07c163540 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400dc50 con 0x7fa07c163540 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c15eb20 con 0x7fa07c163540 2024-05-08T04:38:33.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.271+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c15f0e0 con 0x7fa07c163540 2024-05-08T04:38:33.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c069d60 con 0x7fa07c163540 2024-05-08T04:38:33.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa07401d070 con 0x7fa07c163540 2024-05-08T04:38:33.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa079600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 0x7fa058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa074085980 con 0x7fa07c163540 2024-05-08T04:38:33.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa080c00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 0x7fa058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.275+0000 7fa080c00640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 0x7fa058069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa06c001650 tx=0x7fa06c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.279+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0740569b0 con 0x7fa07c163540 2024-05-08T04:38:33.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.435+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa07c162300 con 0x7fa07c163540 2024-05-08T04:38:33.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.435+0000 7fa079600640 1 -- 172.21.15.164:0/3911823311 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fa0740562b0 con 0x7fa07c163540 2024-05-08T04:38:33.439 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:33.439 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 msgr2=0x7fa058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 0x7fa058069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa06c001650 tx=0x7fa06c009040 comp rx=0 tx=0).stop 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 msgr2=0x7fa07c15e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa07400cef0 tx=0x7fa07400a710 comp rx=0 tx=0).stop 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 shutdown_connections 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa058066f60 0x7fa058069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa07c163540 0x7fa07c15e010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa07c0cf6c0 0x7fa07c15f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 --2- 172.21.15.164:0/3911823311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa07c0aedb0 0x7fa07c164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 >> 172.21.15.164:0/3911823311 conn(0x7fa07c06ac90 msgr2=0x7fa07c0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 shutdown_connections 2024-05-08T04:38:33.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.439+0000 7fa083600640 1 -- 172.21.15.164:0/3911823311 wait complete. 2024-05-08T04:38:33.460 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:38:33.461 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:38:33.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 Processor -- start 2024-05-08T04:38:33.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 -- start start 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efcfc0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0d15c0 con 0x7efcfc15dac0 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0d18a0 con 0x7efcfc0d0780 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0d2ac0 con 0x7efcfc0d2660 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7efcfc15dac0 0x7efcfc0cf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57936/0 (socket says 172.21.15.164:57936) 2024-05-08T04:38:33.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9e00640 1 -- 172.21.15.164:0/2312613363 learned_addr learned my addr 172.21.15.164:0/2312613363 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:33.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efcfc0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173141255 0 0) 0x7efcfc0d15c0 con 0x7efcfc15dac0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce0003530 con 0x7efcfc15dac0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095573065 0 0) 0x7efce0003530 con 0x7efcfc15dac0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 msgr2=0x7efcfc0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efcfc0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 >> v1:172.21.15.172:6789/0 conn(0x7efcfc0d0780 legacy=0x7efcfc06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0d1b80 con 0x7efcfc15dac0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7efcec001680 con 0x7efcfc15dac0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 >> v1:172.21.15.164:6789/0 conn(0x7efcfc15dac0 legacy=0x7efcfc0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00039f0 0x7efce0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce00043a0 0x7efce0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efce00050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0d2ac0 con 0x7efce00043a0 2024-05-08T04:38:33.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce000b9d0 con 0x7efce00039f0 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce000bbd0 con 0x7efcfc0d2660 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce00043a0 0x7efce0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9e00640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00039f0 0x7efce0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfa800640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efce00050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 -- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 msgr2=0x7efce00050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efce00050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 -- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00039f0 msgr2=0x7efce0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00039f0 0x7efce0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcf9400640 1 -- 172.21.15.164:0/2312613363 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0003530 con 0x7efce00043a0 2024-05-08T04:38:33.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.595+0000 7efcfa800640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0d2660 0x7efce00050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcf9400640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce00043a0 0x7efce0004870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efce8009ae0 tx=0x7efce801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8db177a2c5b95c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce80b2070 con 0x7efce00043a0 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce80a5660 con 0x7efce00043a0 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcf8a00640 1 -- 172.21.15.164:0/2312613363 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce80096b0 con 0x7efce00043a0 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 -- 172.21.15.164:0/2312613363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce00043a0 msgr2=0x7efce0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 --2- 172.21.15.164:0/2312613363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce00043a0 0x7efce0004870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efce8009ae0 tx=0x7efce801ca70 comp rx=0 tx=0).stop 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfa800640 1 -- 172.21.15.164:0/2312613363 reap_dead start 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 -- 172.21.15.164:0/2312613363 shutdown_connections 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 -- 172.21.15.164:0/2312613363 >> 172.21.15.164:0/2312613363 conn(0x7efcfc09cce0 msgr2=0x7efcfc0d3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 -- 172.21.15.164:0/2312613363 shutdown_connections 2024-05-08T04:38:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.599+0000 7efcfb200640 1 -- 172.21.15.164:0/2312613363 wait complete. 2024-05-08T04:38:33.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 Processor -- start 2024-05-08T04:38:33.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- start start 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 0x7efcfc15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b7a70 0x7efcfc0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0d42f0 con 0x7efcfc0cd130 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0d2140 con 0x7efce00043a0 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc15dea0 con 0x7efcfc0b7a70 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41168/0 (socket says 172.21.15.164:41168) 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 -- 172.21.15.164:0/2089323518 learned_addr learned my addr 172.21.15.164:0/2089323518 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf9400640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b7a70 0x7efcfc0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf9e00640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 0x7efcfc15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 -- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b7a70 msgr2=0x7efcfc0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b7a70 0x7efcfc0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 -- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 msgr2=0x7efcfc15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 0x7efcfc15bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 -- 172.21.15.164:0/2089323518 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc15ee90 con 0x7efcfc0cd130 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf9e00640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 0x7efcfc15bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfa800640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcfc08cb00 tx=0x7efcf4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4003d90 con 0x7efcfc0cd130 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4019070 con 0x7efcfc0cd130 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0b6f90 con 0x7efcfc0cd130 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400ea90 con 0x7efcfc0cd130 2024-05-08T04:38:33.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc0b7ef0 con 0x7efcfc0cd130 2024-05-08T04:38:33.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.603+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc0cd5b0 con 0x7efcfc0cd130 2024-05-08T04:38:33.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.607+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcf4013070 con 0x7efcfc0cd130 2024-05-08T04:38:33.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.607+0000 7efcf2000640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 0x7efcdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.607+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcf4087e60 con 0x7efcfc0cd130 2024-05-08T04:38:33.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.607+0000 7efcf9e00640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 0x7efcdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.607+0000 7efcf9e00640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 0x7efcdc069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c10 tx=0x7efcec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.611+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcf4058e60 con 0x7efcfc0cd130 2024-05-08T04:38:33.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.763+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcfc15f170 con 0x7efcdc066f60 2024-05-08T04:38:33.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.771+0000 7efcf2000640 1 -- 172.21.15.164:0/2089323518 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7efcfc15f170 con 0x7efcdc066f60 2024-05-08T04:38:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:33.778 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 msgr2=0x7efcdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 0x7efcdc069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c10 tx=0x7efcec009040 comp rx=0 tx=0).stop 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 msgr2=0x7efcfc15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcfc08cb00 tx=0x7efcf4010f40 comp rx=0 tx=0).stop 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 shutdown_connections 2024-05-08T04:38:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efcdc066f60 0x7efcdc069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b7a70 0x7efcfc0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcfc0cd130 0x7efcfc15e1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 --2- 172.21.15.164:0/2089323518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efce00043a0 0x7efcfc15bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 >> 172.21.15.164:0/2089323518 conn(0x7efcfc09cce0 msgr2=0x7efcfc06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 shutdown_connections 2024-05-08T04:38:33.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.775+0000 7efcfb200640 1 -- 172.21.15.164:0/2089323518 wait complete. 2024-05-08T04:38:33.796 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T04:38:33.431834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.722938"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201972+0000","last_change":"2024-05-08T04:38:29.759413+0000","last_active":"2024-05-08T04:38:31.201972+0000","last_peered":"2024-05-08T04:38:31.201972+0000","last_clean":"2024-05-08T04:38:31.201972+0000","last_became_active":"2024-05-08T04:38:29.756280+0000","last_became_peered":"2024-05-08T04:38:29.756280+0000","last_unstale":"2024-05-08T04:38:31.201972+0000","last_undegraded":"2024-05-08T04:38:31.201972+0000","last_fullsized":"2024-05-08T04:38:31.201972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739754+0000","last_change":"2024-05-08T04:38:29.729346+0000","last_active":"2024-05-08T04:38:30.739754+0000","last_peered":"2024-05-08T04:38:30.739754+0000","last_clean":"2024-05-08T04:38:30.739754+0000","last_became_active":"2024-05-08T04:38:29.728984+0000","last_became_peered":"2024-05-08T04:38:29.728984+0000","last_unstale":"2024-05-08T04:38:30.739754+0000","last_undegraded":"2024-05-08T04:38:30.739754+0000","last_fullsized":"2024-05-08T04:38:30.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200859+0000","last_change":"2024-05-08T04:38:29.735729+0000","last_active":"2024-05-08T04:38:31.200859+0000","last_peered":"2024-05-08T04:38:31.200859+0000","last_clean":"2024-05-08T04:38:31.200859+0000","last_became_active":"2024-05-08T04:38:29.735510+0000","last_became_peered":"2024-05-08T04:38:29.735510+0000","last_unstale":"2024-05-08T04:38:31.200859+0000","last_undegraded":"2024-05-08T04:38:31.200859+0000","last_fullsized":"2024-05-08T04:38:31.200859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741119+0000","last_change":"2024-05-08T04:38:29.746582+0000","last_active":"2024-05-08T04:38:30.741119+0000","last_peered":"2024-05-08T04:38:30.741119+0000","last_clean":"2024-05-08T04:38:30.741119+0000","last_became_active":"2024-05-08T04:38:29.746344+0000","last_became_peered":"2024-05-08T04:38:29.746344+0000","last_unstale":"2024-05-08T04:38:30.741119+0000","last_undegraded":"2024-05-08T04:38:30.741119+0000","last_fullsized":"2024-05-08T04:38:30.741119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201549+0000","last_change":"2024-05-08T04:38:29.751201+0000","last_active":"2024-05-08T04:38:31.201549+0000","last_peered":"2024-05-08T04:38:31.201549+0000","last_clean":"2024-05-08T04:38:31.201549+0000","last_became_active":"2024-05-08T04:38:29.743374+0000","last_became_peered":"2024-05-08T04:38:29.743374+0000","last_unstale":"2024-05-08T04:38:31.201549+0000","last_undegraded":"2024-05-08T04:38:31.201549+0000","last_fullsized":"2024-05-08T04:38:31.201549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740625+0000","last_change":"2024-05-08T04:38:29.743465+0000","last_active":"2024-05-08T04:38:30.740625+0000","last_peered":"2024-05-08T04:38:30.740625+0000","last_clean":"2024-05-08T04:38:30.740625+0000","last_became_active":"2024-05-08T04:38:29.742816+0000","last_became_peered":"2024-05-08T04:38:29.742816+0000","last_unstale":"2024-05-08T04:38:30.740625+0000","last_undegraded":"2024-05-08T04:38:30.740625+0000","last_fullsized":"2024-05-08T04:38:30.740625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201320+0000","last_change":"2024-05-08T04:38:29.753051+0000","last_active":"2024-05-08T04:38:31.201320+0000","last_peered":"2024-05-08T04:38:31.201320+0000","last_clean":"2024-05-08T04:38:31.201320+0000","last_became_active":"2024-05-08T04:38:29.743659+0000","last_became_peered":"2024-05-08T04:38:29.743659+0000","last_unstale":"2024-05-08T04:38:31.201320+0000","last_undegraded":"2024-05-08T04:38:31.201320+0000","last_fullsized":"2024-05-08T04:38:31.201320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.736688+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.736468+0000","last_became_peered":"2024-05-08T04:38:29.736468+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201990+0000","last_change":"2024-05-08T04:38:29.767705+0000","last_active":"2024-05-08T04:38:31.201990+0000","last_peered":"2024-05-08T04:38:31.201990+0000","last_clean":"2024-05-08T04:38:31.201990+0000","last_became_active":"2024-05-08T04:38:29.767379+0000","last_became_peered":"2024-05-08T04:38:29.767379+0000","last_unstale":"2024-05-08T04:38:31.201990+0000","last_undegraded":"2024-05-08T04:38:31.201990+0000","last_fullsized":"2024-05-08T04:38:31.201990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:59.130461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746455+0000","last_change":"2024-05-08T04:38:29.752398+0000","last_active":"2024-05-08T04:38:30.746455+0000","last_peered":"2024-05-08T04:38:30.746455+0000","last_clean":"2024-05-08T04:38:30.746455+0000","last_became_active":"2024-05-08T04:38:29.751505+0000","last_became_peered":"2024-05-08T04:38:29.751505+0000","last_unstale":"2024-05-08T04:38:30.746455+0000","last_undegraded":"2024-05-08T04:38:30.746455+0000","last_fullsized":"2024-05-08T04:38:30.746455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746387+0000","last_change":"2024-05-08T04:38:29.762618+0000","last_active":"2024-05-08T04:38:30.746387+0000","last_peered":"2024-05-08T04:38:30.746387+0000","last_clean":"2024-05-08T04:38:30.746387+0000","last_became_active":"2024-05-08T04:38:29.762321+0000","last_became_peered":"2024-05-08T04:38:29.762321+0000","last_unstale":"2024-05-08T04:38:30.746387+0000","last_undegraded":"2024-05-08T04:38:30.746387+0000","last_fullsized":"2024-05-08T04:38:30.746387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740525+0000","last_change":"2024-05-08T04:38:29.756698+0000","last_active":"2024-05-08T04:38:30.740525+0000","last_peered":"2024-05-08T04:38:30.740525+0000","last_clean":"2024-05-08T04:38:30.740525+0000","last_became_active":"2024-05-08T04:38:29.756073+0000","last_became_peered":"2024-05-08T04:38:29.756073+0000","last_unstale":"2024-05-08T04:38:30.740525+0000","last_undegraded":"2024-05-08T04:38:30.740525+0000","last_fullsized":"2024-05-08T04:38:30.740525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877713+0000","last_change":"2024-05-08T04:38:29.751753+0000","last_active":"2024-05-08T04:38:30.877713+0000","last_peered":"2024-05-08T04:38:30.877713+0000","last_clean":"2024-05-08T04:38:30.877713+0000","last_became_active":"2024-05-08T04:38:29.750686+0000","last_became_peered":"2024-05-08T04:38:29.750686+0000","last_unstale":"2024-05-08T04:38:30.877713+0000","last_undegraded":"2024-05-08T04:38:30.877713+0000","last_fullsized":"2024-05-08T04:38:30.877713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.730987+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.730744+0000","last_became_peered":"2024-05-08T04:38:29.730744+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:55:21.772046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739297+0000","last_change":"2024-05-08T04:38:29.739983+0000","last_active":"2024-05-08T04:38:30.739297+0000","last_peered":"2024-05-08T04:38:30.739297+0000","last_clean":"2024-05-08T04:38:30.739297+0000","last_became_active":"2024-05-08T04:38:29.738511+0000","last_became_peered":"2024-05-08T04:38:29.738511+0000","last_unstale":"2024-05-08T04:38:30.739297+0000","last_undegraded":"2024-05-08T04:38:30.739297+0000","last_fullsized":"2024-05-08T04:38:30.739297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740242+0000","last_change":"2024-05-08T04:38:29.747325+0000","last_active":"2024-05-08T04:38:30.740242+0000","last_peered":"2024-05-08T04:38:30.740242+0000","last_clean":"2024-05-08T04:38:30.740242+0000","last_became_active":"2024-05-08T04:38:29.744848+0000","last_became_peered":"2024-05-08T04:38:29.744848+0000","last_unstale":"2024-05-08T04:38:30.740242+0000","last_undegraded":"2024-05-08T04:38:30.740242+0000","last_fullsized":"2024-05-08T04:38:30.740242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741529+0000","last_change":"2024-05-08T04:38:29.748850+0000","last_active":"2024-05-08T04:38:30.741529+0000","last_peered":"2024-05-08T04:38:30.741529+0000","last_clean":"2024-05-08T04:38:30.741529+0000","last_became_active":"2024-05-08T04:38:29.748560+0000","last_became_peered":"2024-05-08T04:38:29.748560+0000","last_unstale":"2024-05-08T04:38:30.741529+0000","last_undegraded":"2024-05-08T04:38:30.741529+0000","last_fullsized":"2024-05-08T04:38:30.741529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200471+0000","last_change":"2024-05-08T04:38:29.756416+0000","last_active":"2024-05-08T04:38:31.200471+0000","last_peered":"2024-05-08T04:38:31.200471+0000","last_clean":"2024-05-08T04:38:31.200471+0000","last_became_active":"2024-05-08T04:38:29.756194+0000","last_became_peered":"2024-05-08T04:38:29.756194+0000","last_unstale":"2024-05-08T04:38:31.200471+0000","last_undegraded":"2024-05-08T04:38:31.200471+0000","last_fullsized":"2024-05-08T04:38:31.200471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201900+0000","last_change":"2024-05-08T04:38:29.752005+0000","last_active":"2024-05-08T04:38:31.201900+0000","last_peered":"2024-05-08T04:38:31.201900+0000","last_clean":"2024-05-08T04:38:31.201900+0000","last_became_active":"2024-05-08T04:38:29.743651+0000","last_became_peered":"2024-05-08T04:38:29.743651+0000","last_unstale":"2024-05-08T04:38:31.201900+0000","last_undegraded":"2024-05-08T04:38:31.201900+0000","last_fullsized":"2024-05-08T04:38:31.201900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746095+0000","last_change":"2024-05-08T04:38:29.757006+0000","last_active":"2024-05-08T04:38:30.746095+0000","last_peered":"2024-05-08T04:38:30.746095+0000","last_clean":"2024-05-08T04:38:30.746095+0000","last_became_active":"2024-05-08T04:38:29.756756+0000","last_became_peered":"2024-05-08T04:38:29.756756+0000","last_unstale":"2024-05-08T04:38:30.746095+0000","last_undegraded":"2024-05-08T04:38:30.746095+0000","last_fullsized":"2024-05-08T04:38:30.746095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200787+0000","last_change":"2024-05-08T04:38:29.735804+0000","last_active":"2024-05-08T04:38:31.200787+0000","last_peered":"2024-05-08T04:38:31.200787+0000","last_clean":"2024-05-08T04:38:31.200787+0000","last_became_active":"2024-05-08T04:38:29.735527+0000","last_became_peered":"2024-05-08T04:38:29.735527+0000","last_unstale":"2024-05-08T04:38:31.200787+0000","last_undegraded":"2024-05-08T04:38:31.200787+0000","last_fullsized":"2024-05-08T04:38:31.200787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:18.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740806+0000","last_change":"2024-05-08T04:38:29.762810+0000","last_active":"2024-05-08T04:38:30.740806+0000","last_peered":"2024-05-08T04:38:30.740806+0000","last_clean":"2024-05-08T04:38:30.740806+0000","last_became_active":"2024-05-08T04:38:29.762532+0000","last_became_peered":"2024-05-08T04:38:29.762532+0000","last_unstale":"2024-05-08T04:38:30.740806+0000","last_undegraded":"2024-05-08T04:38:30.740806+0000","last_fullsized":"2024-05-08T04:38:30.740806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.756285+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.756024+0000","last_became_peered":"2024-05-08T04:38:29.756024+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740734+0000","last_change":"2024-05-08T04:38:29.747323+0000","last_active":"2024-05-08T04:38:30.740734+0000","last_peered":"2024-05-08T04:38:30.740734+0000","last_clean":"2024-05-08T04:38:30.740734+0000","last_became_active":"2024-05-08T04:38:29.747077+0000","last_became_peered":"2024-05-08T04:38:29.747077+0000","last_unstale":"2024-05-08T04:38:30.740734+0000","last_undegraded":"2024-05-08T04:38:30.740734+0000","last_fullsized":"2024-05-08T04:38:30.740734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878039+0000","last_change":"2024-05-08T04:38:29.748090+0000","last_active":"2024-05-08T04:38:30.878039+0000","last_peered":"2024-05-08T04:38:30.878039+0000","last_clean":"2024-05-08T04:38:30.878039+0000","last_became_active":"2024-05-08T04:38:29.747823+0000","last_became_peered":"2024-05-08T04:38:29.747823+0000","last_unstale":"2024-05-08T04:38:30.878039+0000","last_undegraded":"2024-05-08T04:38:30.878039+0000","last_fullsized":"2024-05-08T04:38:30.878039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746543+0000","last_change":"2024-05-08T04:38:29.766531+0000","last_active":"2024-05-08T04:38:30.746543+0000","last_peered":"2024-05-08T04:38:30.746543+0000","last_clean":"2024-05-08T04:38:30.746543+0000","last_became_active":"2024-05-08T04:38:29.765752+0000","last_became_peered":"2024-05-08T04:38:29.765752+0000","last_unstale":"2024-05-08T04:38:30.746543+0000","last_undegraded":"2024-05-08T04:38:30.746543+0000","last_fullsized":"2024-05-08T04:38:30.746543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:29.741993+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:29.740916+0000","last_became_peered":"2024-05-08T04:38:29.740916+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877964+0000","last_change":"2024-05-08T04:38:29.747721+0000","last_active":"2024-05-08T04:38:30.877964+0000","last_peered":"2024-05-08T04:38:30.877964+0000","last_clean":"2024-05-08T04:38:30.877964+0000","last_became_active":"2024-05-08T04:38:29.747443+0000","last_became_peered":"2024-05-08T04:38:29.747443+0000","last_unstale":"2024-05-08T04:38:30.877964+0000","last_undegraded":"2024-05-08T04:38:30.877964+0000","last_fullsized":"2024-05-08T04:38:30.877964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200927+0000","last_change":"2024-05-08T04:38:29.731332+0000","last_active":"2024-05-08T04:38:31.200927+0000","last_peered":"2024-05-08T04:38:31.200927+0000","last_clean":"2024-05-08T04:38:31.200927+0000","last_became_active":"2024-05-08T04:38:29.731075+0000","last_became_peered":"2024-05-08T04:38:29.731075+0000","last_unstale":"2024-05-08T04:38:31.200927+0000","last_undegraded":"2024-05-08T04:38:31.200927+0000","last_fullsized":"2024-05-08T04:38:31.200927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740872+0000","last_change":"2024-05-08T04:38:29.743680+0000","last_active":"2024-05-08T04:38:30.740872+0000","last_peered":"2024-05-08T04:38:30.740872+0000","last_clean":"2024-05-08T04:38:30.740872+0000","last_became_active":"2024-05-08T04:38:29.743208+0000","last_became_peered":"2024-05-08T04:38:29.743208+0000","last_unstale":"2024-05-08T04:38:30.740872+0000","last_undegraded":"2024-05-08T04:38:30.740872+0000","last_fullsized":"2024-05-08T04:38:30.740872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200255+0000","last_change":"2024-05-08T04:38:29.738213+0000","last_active":"2024-05-08T04:38:31.200255+0000","last_peered":"2024-05-08T04:38:31.200255+0000","last_clean":"2024-05-08T04:38:31.200255+0000","last_became_active":"2024-05-08T04:38:29.737935+0000","last_became_peered":"2024-05-08T04:38:29.737935+0000","last_unstale":"2024-05-08T04:38:31.200255+0000","last_undegraded":"2024-05-08T04:38:31.200255+0000","last_fullsized":"2024-05-08T04:38:31.200255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746039+0000","last_change":"2024-05-08T04:38:29.750290+0000","last_active":"2024-05-08T04:38:30.746039+0000","last_peered":"2024-05-08T04:38:30.746039+0000","last_clean":"2024-05-08T04:38:30.746039+0000","last_became_active":"2024-05-08T04:38:29.749840+0000","last_became_peered":"2024-05-08T04:38:29.749840+0000","last_unstale":"2024-05-08T04:38:30.746039+0000","last_undegraded":"2024-05-08T04:38:30.746039+0000","last_fullsized":"2024-05-08T04:38:30.746039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740040+0000","last_change":"2024-05-08T04:38:29.734132+0000","last_active":"2024-05-08T04:38:30.740040+0000","last_peered":"2024-05-08T04:38:30.740040+0000","last_clean":"2024-05-08T04:38:30.740040+0000","last_became_active":"2024-05-08T04:38:29.733869+0000","last_became_peered":"2024-05-08T04:38:29.733869+0000","last_unstale":"2024-05-08T04:38:30.740040+0000","last_undegraded":"2024-05-08T04:38:30.740040+0000","last_fullsized":"2024-05-08T04:38:30.740040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:28.424596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877812+0000","last_change":"2024-05-08T04:38:28.741244+0000","last_active":"2024-05-08T04:38:30.877812+0000","last_peered":"2024-05-08T04:38:30.877812+0000","last_clean":"2024-05-08T04:38:30.877812+0000","last_became_active":"2024-05-08T04:38:28.740897+0000","last_became_peered":"2024-05-08T04:38:28.740897+0000","last_unstale":"2024-05-08T04:38:30.877812+0000","last_undegraded":"2024-05-08T04:38:30.877812+0000","last_fullsized":"2024-05-08T04:38:30.877812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200940+0000","last_change":"2024-05-08T04:38:29.731992+0000","last_active":"2024-05-08T04:38:31.200940+0000","last_peered":"2024-05-08T04:38:31.200940+0000","last_clean":"2024-05-08T04:38:31.200940+0000","last_became_active":"2024-05-08T04:38:29.731683+0000","last_became_peered":"2024-05-08T04:38:29.731683+0000","last_unstale":"2024-05-08T04:38:31.200940+0000","last_undegraded":"2024-05-08T04:38:31.200940+0000","last_fullsized":"2024-05-08T04:38:31.200940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746000+0000","last_change":"2024-05-08T04:38:28.738380+0000","last_active":"2024-05-08T04:38:30.746000+0000","last_peered":"2024-05-08T04:38:30.746000+0000","last_clean":"2024-05-08T04:38:30.746000+0000","last_became_active":"2024-05-08T04:38:28.738109+0000","last_became_peered":"2024-05-08T04:38:28.738109+0000","last_unstale":"2024-05-08T04:38:30.746000+0000","last_undegraded":"2024-05-08T04:38:30.746000+0000","last_fullsized":"2024-05-08T04:38:30.746000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877587+0000","last_change":"2024-05-08T04:38:28.743973+0000","last_active":"2024-05-08T04:38:30.877587+0000","last_peered":"2024-05-08T04:38:30.877587+0000","last_clean":"2024-05-08T04:38:30.877587+0000","last_became_active":"2024-05-08T04:38:28.743755+0000","last_became_peered":"2024-05-08T04:38:28.743755+0000","last_unstale":"2024-05-08T04:38:30.877587+0000","last_undegraded":"2024-05-08T04:38:30.877587+0000","last_fullsized":"2024-05-08T04:38:30.877587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877578+0000","last_change":"2024-05-08T04:38:29.737294+0000","last_active":"2024-05-08T04:38:30.877578+0000","last_peered":"2024-05-08T04:38:30.877578+0000","last_clean":"2024-05-08T04:38:30.877578+0000","last_became_active":"2024-05-08T04:38:29.737004+0000","last_became_peered":"2024-05-08T04:38:29.737004+0000","last_unstale":"2024-05-08T04:38:30.877578+0000","last_undegraded":"2024-05-08T04:38:30.877578+0000","last_fullsized":"2024-05-08T04:38:30.877578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740741+0000","last_change":"2024-05-08T04:38:28.737564+0000","last_active":"2024-05-08T04:38:30.740741+0000","last_peered":"2024-05-08T04:38:30.740741+0000","last_clean":"2024-05-08T04:38:30.740741+0000","last_became_active":"2024-05-08T04:38:28.737280+0000","last_became_peered":"2024-05-08T04:38:28.737280+0000","last_unstale":"2024-05-08T04:38:30.740741+0000","last_undegraded":"2024-05-08T04:38:30.740741+0000","last_fullsized":"2024-05-08T04:38:30.740741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739295+0000","last_change":"2024-05-08T04:38:29.739772+0000","last_active":"2024-05-08T04:38:30.739295+0000","last_peered":"2024-05-08T04:38:30.739295+0000","last_clean":"2024-05-08T04:38:30.739295+0000","last_became_active":"2024-05-08T04:38:29.738258+0000","last_became_peered":"2024-05-08T04:38:29.738258+0000","last_unstale":"2024-05-08T04:38:30.739295+0000","last_undegraded":"2024-05-08T04:38:30.739295+0000","last_fullsized":"2024-05-08T04:38:30.739295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746413+0000","last_change":"2024-05-08T04:38:28.742877+0000","last_active":"2024-05-08T04:38:30.746413+0000","last_peered":"2024-05-08T04:38:30.746413+0000","last_clean":"2024-05-08T04:38:30.746413+0000","last_became_active":"2024-05-08T04:38:28.742495+0000","last_became_peered":"2024-05-08T04:38:28.742495+0000","last_unstale":"2024-05-08T04:38:30.746413+0000","last_undegraded":"2024-05-08T04:38:30.746413+0000","last_fullsized":"2024-05-08T04:38:30.746413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741690+0000","last_change":"2024-05-08T04:38:29.759053+0000","last_active":"2024-05-08T04:38:30.741690+0000","last_peered":"2024-05-08T04:38:30.741690+0000","last_clean":"2024-05-08T04:38:30.741690+0000","last_became_active":"2024-05-08T04:38:29.758357+0000","last_became_peered":"2024-05-08T04:38:29.758357+0000","last_unstale":"2024-05-08T04:38:30.741690+0000","last_undegraded":"2024-05-08T04:38:30.741690+0000","last_fullsized":"2024-05-08T04:38:30.741690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741139+0000","last_change":"2024-05-08T04:38:28.748874+0000","last_active":"2024-05-08T04:38:30.741139+0000","last_peered":"2024-05-08T04:38:30.741139+0000","last_clean":"2024-05-08T04:38:30.741139+0000","last_became_active":"2024-05-08T04:38:28.748071+0000","last_became_peered":"2024-05-08T04:38:28.748071+0000","last_unstale":"2024-05-08T04:38:30.741139+0000","last_undegraded":"2024-05-08T04:38:30.741139+0000","last_fullsized":"2024-05-08T04:38:30.741139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201063+0000","last_change":"2024-05-08T04:38:29.731689+0000","last_active":"2024-05-08T04:38:31.201063+0000","last_peered":"2024-05-08T04:38:31.201063+0000","last_clean":"2024-05-08T04:38:31.201063+0000","last_became_active":"2024-05-08T04:38:29.731433+0000","last_became_peered":"2024-05-08T04:38:29.731433+0000","last_unstale":"2024-05-08T04:38:31.201063+0000","last_undegraded":"2024-05-08T04:38:31.201063+0000","last_fullsized":"2024-05-08T04:38:31.201063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:28.752051+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:28.751792+0000","last_became_peered":"2024-05-08T04:38:28.751792+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740885+0000","last_change":"2024-05-08T04:38:29.756998+0000","last_active":"2024-05-08T04:38:30.740885+0000","last_peered":"2024-05-08T04:38:30.740885+0000","last_clean":"2024-05-08T04:38:30.740885+0000","last_became_active":"2024-05-08T04:38:29.756395+0000","last_became_peered":"2024-05-08T04:38:29.756395+0000","last_unstale":"2024-05-08T04:38:30.740885+0000","last_undegraded":"2024-05-08T04:38:30.740885+0000","last_fullsized":"2024-05-08T04:38:30.740885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746251+0000","last_change":"2024-05-08T04:38:28.744776+0000","last_active":"2024-05-08T04:38:30.746251+0000","last_peered":"2024-05-08T04:38:30.746251+0000","last_clean":"2024-05-08T04:38:30.746251+0000","last_became_active":"2024-05-08T04:38:28.744410+0000","last_became_peered":"2024-05-08T04:38:28.744410+0000","last_unstale":"2024-05-08T04:38:30.746251+0000","last_undegraded":"2024-05-08T04:38:30.746251+0000","last_fullsized":"2024-05-08T04:38:30.746251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201683+0000","last_change":"2024-05-08T04:38:29.755661+0000","last_active":"2024-05-08T04:38:31.201683+0000","last_peered":"2024-05-08T04:38:31.201683+0000","last_clean":"2024-05-08T04:38:31.201683+0000","last_became_active":"2024-05-08T04:38:29.754401+0000","last_became_peered":"2024-05-08T04:38:29.754401+0000","last_unstale":"2024-05-08T04:38:31.201683+0000","last_undegraded":"2024-05-08T04:38:31.201683+0000","last_fullsized":"2024-05-08T04:38:31.201683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201037+0000","last_change":"2024-05-08T04:38:28.742800+0000","last_active":"2024-05-08T04:38:31.201037+0000","last_peered":"2024-05-08T04:38:31.201037+0000","last_clean":"2024-05-08T04:38:31.201037+0000","last_became_active":"2024-05-08T04:38:28.742491+0000","last_became_peered":"2024-05-08T04:38:28.742491+0000","last_unstale":"2024-05-08T04:38:31.201037+0000","last_undegraded":"2024-05-08T04:38:31.201037+0000","last_fullsized":"2024-05-08T04:38:31.201037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:53.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200760+0000","last_change":"2024-05-08T04:38:29.740544+0000","last_active":"2024-05-08T04:38:31.200760+0000","last_peered":"2024-05-08T04:38:31.200760+0000","last_clean":"2024-05-08T04:38:31.200760+0000","last_became_active":"2024-05-08T04:38:29.740262+0000","last_became_peered":"2024-05-08T04:38:29.740262+0000","last_unstale":"2024-05-08T04:38:31.200760+0000","last_undegraded":"2024-05-08T04:38:31.200760+0000","last_fullsized":"2024-05-08T04:38:31.200760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:15.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740918+0000","last_change":"2024-05-08T04:38:28.751437+0000","last_active":"2024-05-08T04:38:30.740918+0000","last_peered":"2024-05-08T04:38:30.740918+0000","last_clean":"2024-05-08T04:38:30.740918+0000","last_became_active":"2024-05-08T04:38:28.751039+0000","last_became_peered":"2024-05-08T04:38:28.751039+0000","last_unstale":"2024-05-08T04:38:30.740918+0000","last_undegraded":"2024-05-08T04:38:30.740918+0000","last_fullsized":"2024-05-08T04:38:30.740918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201608+0000","last_change":"2024-05-08T04:38:29.756359+0000","last_active":"2024-05-08T04:38:31.201608+0000","last_peered":"2024-05-08T04:38:31.201608+0000","last_clean":"2024-05-08T04:38:31.201608+0000","last_became_active":"2024-05-08T04:38:29.756087+0000","last_became_peered":"2024-05-08T04:38:29.756087+0000","last_unstale":"2024-05-08T04:38:31.201608+0000","last_undegraded":"2024-05-08T04:38:31.201608+0000","last_fullsized":"2024-05-08T04:38:31.201608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.745976+0000","last_change":"2024-05-08T04:38:28.739635+0000","last_active":"2024-05-08T04:38:30.745976+0000","last_peered":"2024-05-08T04:38:30.745976+0000","last_clean":"2024-05-08T04:38:30.745976+0000","last_became_active":"2024-05-08T04:38:28.739255+0000","last_became_peered":"2024-05-08T04:38:28.739255+0000","last_unstale":"2024-05-08T04:38:30.745976+0000","last_undegraded":"2024-05-08T04:38:30.745976+0000","last_fullsized":"2024-05-08T04:38:30.745976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746057+0000","last_change":"2024-05-08T04:38:29.750751+0000","last_active":"2024-05-08T04:38:30.746057+0000","last_peered":"2024-05-08T04:38:30.746057+0000","last_clean":"2024-05-08T04:38:30.746057+0000","last_became_active":"2024-05-08T04:38:29.749978+0000","last_became_peered":"2024-05-08T04:38:29.749978+0000","last_unstale":"2024-05-08T04:38:30.746057+0000","last_undegraded":"2024-05-08T04:38:30.746057+0000","last_fullsized":"2024-05-08T04:38:30.746057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200130+0000","last_change":"2024-05-08T04:38:28.738083+0000","last_active":"2024-05-08T04:38:31.200130+0000","last_peered":"2024-05-08T04:38:31.200130+0000","last_clean":"2024-05-08T04:38:31.200130+0000","last_became_active":"2024-05-08T04:38:28.736994+0000","last_became_peered":"2024-05-08T04:38:28.736994+0000","last_unstale":"2024-05-08T04:38:31.200130+0000","last_undegraded":"2024-05-08T04:38:31.200130+0000","last_fullsized":"2024-05-08T04:38:31.200130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201379+0000","last_change":"2024-05-08T04:38:29.758699+0000","last_active":"2024-05-08T04:38:31.201379+0000","last_peered":"2024-05-08T04:38:31.201379+0000","last_clean":"2024-05-08T04:38:31.201379+0000","last_became_active":"2024-05-08T04:38:29.758439+0000","last_became_peered":"2024-05-08T04:38:29.758439+0000","last_unstale":"2024-05-08T04:38:31.201379+0000","last_undegraded":"2024-05-08T04:38:31.201379+0000","last_fullsized":"2024-05-08T04:38:31.201379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741696+0000","last_change":"2024-05-08T04:38:28.738978+0000","last_active":"2024-05-08T04:38:30.741696+0000","last_peered":"2024-05-08T04:38:30.741696+0000","last_clean":"2024-05-08T04:38:30.741696+0000","last_became_active":"2024-05-08T04:38:28.738688+0000","last_became_peered":"2024-05-08T04:38:28.738688+0000","last_unstale":"2024-05-08T04:38:30.741696+0000","last_undegraded":"2024-05-08T04:38:30.741696+0000","last_fullsized":"2024-05-08T04:38:30.741696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201862+0000","last_change":"2024-05-08T04:38:29.757772+0000","last_active":"2024-05-08T04:38:31.201862+0000","last_peered":"2024-05-08T04:38:31.201862+0000","last_clean":"2024-05-08T04:38:31.201862+0000","last_became_active":"2024-05-08T04:38:29.756730+0000","last_became_peered":"2024-05-08T04:38:29.756730+0000","last_unstale":"2024-05-08T04:38:31.201862+0000","last_undegraded":"2024-05-08T04:38:31.201862+0000","last_fullsized":"2024-05-08T04:38:31.201862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200418+0000","last_change":"2024-05-08T04:38:28.734548+0000","last_active":"2024-05-08T04:38:31.200418+0000","last_peered":"2024-05-08T04:38:31.200418+0000","last_clean":"2024-05-08T04:38:31.200418+0000","last_became_active":"2024-05-08T04:38:28.734179+0000","last_became_peered":"2024-05-08T04:38:28.734179+0000","last_unstale":"2024-05-08T04:38:31.200418+0000","last_undegraded":"2024-05-08T04:38:31.200418+0000","last_fullsized":"2024-05-08T04:38:31.200418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.202085+0000","last_change":"2024-05-08T04:38:29.758477+0000","last_active":"2024-05-08T04:38:31.202085+0000","last_peered":"2024-05-08T04:38:31.202085+0000","last_clean":"2024-05-08T04:38:31.202085+0000","last_became_active":"2024-05-08T04:38:29.756388+0000","last_became_peered":"2024-05-08T04:38:29.756388+0000","last_unstale":"2024-05-08T04:38:31.202085+0000","last_undegraded":"2024-05-08T04:38:31.202085+0000","last_fullsized":"2024-05-08T04:38:31.202085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:06.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739628+0000","last_change":"2024-05-08T04:38:28.739369+0000","last_active":"2024-05-08T04:38:30.739628+0000","last_peered":"2024-05-08T04:38:30.739628+0000","last_clean":"2024-05-08T04:38:30.739628+0000","last_became_active":"2024-05-08T04:38:28.739079+0000","last_became_peered":"2024-05-08T04:38:28.739079+0000","last_unstale":"2024-05-08T04:38:30.739628+0000","last_undegraded":"2024-05-08T04:38:30.739628+0000","last_fullsized":"2024-05-08T04:38:30.739628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741244+0000","last_change":"2024-05-08T04:38:29.748638+0000","last_active":"2024-05-08T04:38:30.741244+0000","last_peered":"2024-05-08T04:38:30.741244+0000","last_clean":"2024-05-08T04:38:30.741244+0000","last_became_active":"2024-05-08T04:38:29.748277+0000","last_became_peered":"2024-05-08T04:38:29.748277+0000","last_unstale":"2024-05-08T04:38:30.741244+0000","last_undegraded":"2024-05-08T04:38:30.741244+0000","last_fullsized":"2024-05-08T04:38:30.741244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200420+0000","last_change":"2024-05-08T04:38:28.737536+0000","last_active":"2024-05-08T04:38:31.200420+0000","last_peered":"2024-05-08T04:38:31.200420+0000","last_clean":"2024-05-08T04:38:31.200420+0000","last_became_active":"2024-05-08T04:38:28.737250+0000","last_became_peered":"2024-05-08T04:38:28.737250+0000","last_unstale":"2024-05-08T04:38:31.200420+0000","last_undegraded":"2024-05-08T04:38:31.200420+0000","last_fullsized":"2024-05-08T04:38:31.200420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201674+0000","last_change":"2024-05-08T04:38:29.762160+0000","last_active":"2024-05-08T04:38:31.201674+0000","last_peered":"2024-05-08T04:38:31.201674+0000","last_clean":"2024-05-08T04:38:31.201674+0000","last_became_active":"2024-05-08T04:38:29.761754+0000","last_became_peered":"2024-05-08T04:38:29.761754+0000","last_unstale":"2024-05-08T04:38:31.201674+0000","last_undegraded":"2024-05-08T04:38:31.201674+0000","last_fullsized":"2024-05-08T04:38:31.201674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201275+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:38:31.201275+0000","last_peered":"2024-05-08T04:38:31.201275+0000","last_clean":"2024-05-08T04:38:31.201275+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:38:31.201275+0000","last_undegraded":"2024-05-08T04:38:31.201275+0000","last_fullsized":"2024-05-08T04:38:31.201275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741367+0000","last_change":"2024-05-08T04:38:28.750045+0000","last_active":"2024-05-08T04:38:30.741367+0000","last_peered":"2024-05-08T04:38:30.741367+0000","last_clean":"2024-05-08T04:38:30.741367+0000","last_became_active":"2024-05-08T04:38:28.749775+0000","last_became_peered":"2024-05-08T04:38:28.749775+0000","last_unstale":"2024-05-08T04:38:30.741367+0000","last_undegraded":"2024-05-08T04:38:30.741367+0000","last_fullsized":"2024-05-08T04:38:30.741367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877692+0000","last_change":"2024-05-08T04:38:29.733287+0000","last_active":"2024-05-08T04:38:30.877692+0000","last_peered":"2024-05-08T04:38:30.877692+0000","last_clean":"2024-05-08T04:38:30.877692+0000","last_became_active":"2024-05-08T04:38:29.733010+0000","last_became_peered":"2024-05-08T04:38:29.733010+0000","last_unstale":"2024-05-08T04:38:30.877692+0000","last_undegraded":"2024-05-08T04:38:30.877692+0000","last_fullsized":"2024-05-08T04:38:30.877692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201257+0000","last_change":"2024-05-08T04:38:28.750803+0000","last_active":"2024-05-08T04:38:31.201257+0000","last_peered":"2024-05-08T04:38:31.201257+0000","last_clean":"2024-05-08T04:38:31.201257+0000","last_became_active":"2024-05-08T04:38:28.750389+0000","last_became_peered":"2024-05-08T04:38:28.750389+0000","last_unstale":"2024-05-08T04:38:31.201257+0000","last_undegraded":"2024-05-08T04:38:31.201257+0000","last_fullsized":"2024-05-08T04:38:31.201257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.734926+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.733112+0000","last_became_peered":"2024-05-08T04:38:29.733112+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200293+0000","last_change":"2024-05-08T04:38:28.733116+0000","last_active":"2024-05-08T04:38:31.200293+0000","last_peered":"2024-05-08T04:38:31.200293+0000","last_clean":"2024-05-08T04:38:31.200293+0000","last_became_active":"2024-05-08T04:38:28.732033+0000","last_became_peered":"2024-05-08T04:38:28.732033+0000","last_unstale":"2024-05-08T04:38:31.200293+0000","last_undegraded":"2024-05-08T04:38:31.200293+0000","last_fullsized":"2024-05-08T04:38:31.200293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739374+0000","last_change":"2024-05-08T04:38:29.740783+0000","last_active":"2024-05-08T04:38:30.739374+0000","last_peered":"2024-05-08T04:38:30.739374+0000","last_clean":"2024-05-08T04:38:30.739374+0000","last_became_active":"2024-05-08T04:38:29.738992+0000","last_became_peered":"2024-05-08T04:38:29.738992+0000","last_unstale":"2024-05-08T04:38:30.739374+0000","last_undegraded":"2024-05-08T04:38:30.739374+0000","last_fullsized":"2024-05-08T04:38:30.739374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740989+0000","last_change":"2024-05-08T04:38:28.744976+0000","last_active":"2024-05-08T04:38:30.740989+0000","last_peered":"2024-05-08T04:38:30.740989+0000","last_clean":"2024-05-08T04:38:30.740989+0000","last_became_active":"2024-05-08T04:38:28.744674+0000","last_became_peered":"2024-05-08T04:38:28.744674+0000","last_unstale":"2024-05-08T04:38:30.740989+0000","last_undegraded":"2024-05-08T04:38:30.740989+0000","last_fullsized":"2024-05-08T04:38:30.740989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740393+0000","last_change":"2024-05-08T04:38:29.743310+0000","last_active":"2024-05-08T04:38:30.740393+0000","last_peered":"2024-05-08T04:38:30.740393+0000","last_clean":"2024-05-08T04:38:30.740393+0000","last_became_active":"2024-05-08T04:38:29.741857+0000","last_became_peered":"2024-05-08T04:38:29.741857+0000","last_unstale":"2024-05-08T04:38:30.740393+0000","last_undegraded":"2024-05-08T04:38:30.740393+0000","last_fullsized":"2024-05-08T04:38:30.740393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:09.896832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739510+0000","last_change":"2024-05-08T04:38:28.735199+0000","last_active":"2024-05-08T04:38:30.739510+0000","last_peered":"2024-05-08T04:38:30.739510+0000","last_clean":"2024-05-08T04:38:30.739510+0000","last_became_active":"2024-05-08T04:38:28.734839+0000","last_became_peered":"2024-05-08T04:38:28.734839+0000","last_unstale":"2024-05-08T04:38:30.739510+0000","last_undegraded":"2024-05-08T04:38:30.739510+0000","last_fullsized":"2024-05-08T04:38:30.739510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741496+0000","last_change":"2024-05-08T04:38:29.743096+0000","last_active":"2024-05-08T04:38:30.741496+0000","last_peered":"2024-05-08T04:38:30.741496+0000","last_clean":"2024-05-08T04:38:30.741496+0000","last_became_active":"2024-05-08T04:38:29.742795+0000","last_became_peered":"2024-05-08T04:38:29.742795+0000","last_unstale":"2024-05-08T04:38:30.741496+0000","last_undegraded":"2024-05-08T04:38:30.741496+0000","last_fullsized":"2024-05-08T04:38:30.741496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201931+0000","last_change":"2024-05-08T04:38:28.749845+0000","last_active":"2024-05-08T04:38:31.201931+0000","last_peered":"2024-05-08T04:38:31.201931+0000","last_clean":"2024-05-08T04:38:31.201931+0000","last_became_active":"2024-05-08T04:38:28.749522+0000","last_became_peered":"2024-05-08T04:38:28.749522+0000","last_unstale":"2024-05-08T04:38:31.201931+0000","last_undegraded":"2024-05-08T04:38:31.201931+0000","last_fullsized":"2024-05-08T04:38:31.201931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:28.101827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739371+0000","last_change":"2024-05-08T04:38:29.733856+0000","last_active":"2024-05-08T04:38:30.739371+0000","last_peered":"2024-05-08T04:38:30.739371+0000","last_clean":"2024-05-08T04:38:30.739371+0000","last_became_active":"2024-05-08T04:38:29.733589+0000","last_became_peered":"2024-05-08T04:38:29.733589+0000","last_unstale":"2024-05-08T04:38:30.739371+0000","last_undegraded":"2024-05-08T04:38:30.739371+0000","last_fullsized":"2024-05-08T04:38:30.739371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.744254+0000","last_change":"2024-05-08T04:38:28.743259+0000","last_active":"2024-05-08T04:38:30.744254+0000","last_peered":"2024-05-08T04:38:30.744254+0000","last_clean":"2024-05-08T04:38:30.744254+0000","last_became_active":"2024-05-08T04:38:28.743022+0000","last_became_peered":"2024-05-08T04:38:28.743022+0000","last_unstale":"2024-05-08T04:38:30.744254+0000","last_undegraded":"2024-05-08T04:38:30.744254+0000","last_fullsized":"2024-05-08T04:38:30.744254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878061+0000","last_change":"2024-05-08T04:38:29.742718+0000","last_active":"2024-05-08T04:38:30.878061+0000","last_peered":"2024-05-08T04:38:30.878061+0000","last_clean":"2024-05-08T04:38:30.878061+0000","last_became_active":"2024-05-08T04:38:29.742135+0000","last_became_peered":"2024-05-08T04:38:29.742135+0000","last_unstale":"2024-05-08T04:38:30.878061+0000","last_undegraded":"2024-05-08T04:38:30.878061+0000","last_fullsized":"2024-05-08T04:38:30.878061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746492+0000","last_change":"2024-05-08T04:38:28.747934+0000","last_active":"2024-05-08T04:38:30.746492+0000","last_peered":"2024-05-08T04:38:30.746492+0000","last_clean":"2024-05-08T04:38:30.746492+0000","last_became_active":"2024-05-08T04:38:28.747524+0000","last_became_peered":"2024-05-08T04:38:28.747524+0000","last_unstale":"2024-05-08T04:38:30.746492+0000","last_undegraded":"2024-05-08T04:38:30.746492+0000","last_fullsized":"2024-05-08T04:38:30.746492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200901+0000","last_change":"2024-05-08T04:38:29.729062+0000","last_active":"2024-05-08T04:38:31.200901+0000","last_peered":"2024-05-08T04:38:31.200901+0000","last_clean":"2024-05-08T04:38:31.200901+0000","last_became_active":"2024-05-08T04:38:29.728777+0000","last_became_peered":"2024-05-08T04:38:29.728777+0000","last_unstale":"2024-05-08T04:38:31.200901+0000","last_undegraded":"2024-05-08T04:38:31.200901+0000","last_fullsized":"2024-05-08T04:38:31.200901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201185+0000","last_change":"2024-05-08T04:38:28.749529+0000","last_active":"2024-05-08T04:38:31.201185+0000","last_peered":"2024-05-08T04:38:31.201185+0000","last_clean":"2024-05-08T04:38:31.201185+0000","last_became_active":"2024-05-08T04:38:28.749167+0000","last_became_peered":"2024-05-08T04:38:28.749167+0000","last_unstale":"2024-05-08T04:38:31.201185+0000","last_undegraded":"2024-05-08T04:38:31.201185+0000","last_fullsized":"2024-05-08T04:38:31.201185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740076+0000","last_change":"2024-05-08T04:38:29.749670+0000","last_active":"2024-05-08T04:38:30.740076+0000","last_peered":"2024-05-08T04:38:30.740076+0000","last_clean":"2024-05-08T04:38:30.740076+0000","last_became_active":"2024-05-08T04:38:29.741613+0000","last_became_peered":"2024-05-08T04:38:29.741613+0000","last_unstale":"2024-05-08T04:38:30.740076+0000","last_undegraded":"2024-05-08T04:38:30.740076+0000","last_fullsized":"2024-05-08T04:38:30.740076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:10.004004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739385+0000","last_change":"2024-05-08T04:38:28.747163+0000","last_active":"2024-05-08T04:38:30.739385+0000","last_peered":"2024-05-08T04:38:30.739385+0000","last_clean":"2024-05-08T04:38:30.739385+0000","last_became_active":"2024-05-08T04:38:28.746783+0000","last_became_peered":"2024-05-08T04:38:28.746783+0000","last_unstale":"2024-05-08T04:38:30.739385+0000","last_undegraded":"2024-05-08T04:38:30.739385+0000","last_fullsized":"2024-05-08T04:38:30.739385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877757+0000","last_change":"2024-05-08T04:38:29.737910+0000","last_active":"2024-05-08T04:38:30.877757+0000","last_peered":"2024-05-08T04:38:30.877757+0000","last_clean":"2024-05-08T04:38:30.877757+0000","last_became_active":"2024-05-08T04:38:29.736005+0000","last_became_peered":"2024-05-08T04:38:29.736005+0000","last_unstale":"2024-05-08T04:38:30.877757+0000","last_undegraded":"2024-05-08T04:38:30.877757+0000","last_fullsized":"2024-05-08T04:38:30.877757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739337+0000","last_change":"2024-05-08T04:38:28.740345+0000","last_active":"2024-05-08T04:38:30.739337+0000","last_peered":"2024-05-08T04:38:30.739337+0000","last_clean":"2024-05-08T04:38:30.739337+0000","last_became_active":"2024-05-08T04:38:28.740070+0000","last_became_peered":"2024-05-08T04:38:28.740070+0000","last_unstale":"2024-05-08T04:38:30.739337+0000","last_undegraded":"2024-05-08T04:38:30.739337+0000","last_fullsized":"2024-05-08T04:38:30.739337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877804+0000","last_change":"2024-05-08T04:38:29.737168+0000","last_active":"2024-05-08T04:38:30.877804+0000","last_peered":"2024-05-08T04:38:30.877804+0000","last_clean":"2024-05-08T04:38:30.877804+0000","last_became_active":"2024-05-08T04:38:29.736816+0000","last_became_peered":"2024-05-08T04:38:29.736816+0000","last_unstale":"2024-05-08T04:38:30.877804+0000","last_undegraded":"2024-05-08T04:38:30.877804+0000","last_fullsized":"2024-05-08T04:38:30.877804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201887+0000","last_change":"2024-05-08T04:38:28.754098+0000","last_active":"2024-05-08T04:38:31.201887+0000","last_peered":"2024-05-08T04:38:31.201887+0000","last_clean":"2024-05-08T04:38:31.201887+0000","last_became_active":"2024-05-08T04:38:28.753787+0000","last_became_peered":"2024-05-08T04:38:28.753787+0000","last_unstale":"2024-05-08T04:38:31.201887+0000","last_undegraded":"2024-05-08T04:38:31.201887+0000","last_fullsized":"2024-05-08T04:38:31.201887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:56.969185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746089+0000","last_change":"2024-05-08T04:38:29.747896+0000","last_active":"2024-05-08T04:38:30.746089+0000","last_peered":"2024-05-08T04:38:30.746089+0000","last_clean":"2024-05-08T04:38:30.746089+0000","last_became_active":"2024-05-08T04:38:29.747661+0000","last_became_peered":"2024-05-08T04:38:29.747661+0000","last_unstale":"2024-05-08T04:38:30.746089+0000","last_undegraded":"2024-05-08T04:38:30.746089+0000","last_fullsized":"2024-05-08T04:38:30.746089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741307+0000","last_change":"2024-05-08T04:38:29.742488+0000","last_active":"2024-05-08T04:38:30.741307+0000","last_peered":"2024-05-08T04:38:30.741307+0000","last_clean":"2024-05-08T04:38:30.741307+0000","last_became_active":"2024-05-08T04:38:29.738959+0000","last_became_peered":"2024-05-08T04:38:29.738959+0000","last_unstale":"2024-05-08T04:38:30.741307+0000","last_undegraded":"2024-05-08T04:38:30.741307+0000","last_fullsized":"2024-05-08T04:38:30.741307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201514+0000","last_change":"2024-05-08T04:38:28.758434+0000","last_active":"2024-05-08T04:38:31.201514+0000","last_peered":"2024-05-08T04:38:31.201514+0000","last_clean":"2024-05-08T04:38:31.201514+0000","last_became_active":"2024-05-08T04:38:28.758108+0000","last_became_peered":"2024-05-08T04:38:28.758108+0000","last_unstale":"2024-05-08T04:38:31.201514+0000","last_undegraded":"2024-05-08T04:38:31.201514+0000","last_fullsized":"2024-05-08T04:38:31.201514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741246+0000","last_change":"2024-05-08T04:38:29.740724+0000","last_active":"2024-05-08T04:38:30.741246+0000","last_peered":"2024-05-08T04:38:30.741246+0000","last_clean":"2024-05-08T04:38:30.741246+0000","last_became_active":"2024-05-08T04:38:29.739520+0000","last_became_peered":"2024-05-08T04:38:29.739520+0000","last_unstale":"2024-05-08T04:38:30.741246+0000","last_undegraded":"2024-05-08T04:38:30.741246+0000","last_fullsized":"2024-05-08T04:38:30.741246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201827+0000","last_change":"2024-05-08T04:38:28.753194+0000","last_active":"2024-05-08T04:38:31.201827+0000","last_peered":"2024-05-08T04:38:31.201827+0000","last_clean":"2024-05-08T04:38:31.201827+0000","last_became_active":"2024-05-08T04:38:28.752937+0000","last_became_peered":"2024-05-08T04:38:28.752937+0000","last_unstale":"2024-05-08T04:38:31.201827+0000","last_undegraded":"2024-05-08T04:38:31.201827+0000","last_fullsized":"2024-05-08T04:38:31.201827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201009+0000","last_change":"2024-05-08T04:38:29.738092+0000","last_active":"2024-05-08T04:38:31.201009+0000","last_peered":"2024-05-08T04:38:31.201009+0000","last_clean":"2024-05-08T04:38:31.201009+0000","last_became_active":"2024-05-08T04:38:29.737806+0000","last_became_peered":"2024-05-08T04:38:29.737806+0000","last_unstale":"2024-05-08T04:38:31.201009+0000","last_undegraded":"2024-05-08T04:38:31.201009+0000","last_fullsized":"2024-05-08T04:38:31.201009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878056+0000","last_change":"2024-05-08T04:38:28.732592+0000","last_active":"2024-05-08T04:38:30.878056+0000","last_peered":"2024-05-08T04:38:30.878056+0000","last_clean":"2024-05-08T04:38:30.878056+0000","last_became_active":"2024-05-08T04:38:28.732270+0000","last_became_peered":"2024-05-08T04:38:28.732270+0000","last_unstale":"2024-05-08T04:38:30.878056+0000","last_undegraded":"2024-05-08T04:38:30.878056+0000","last_fullsized":"2024-05-08T04:38:30.878056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878079+0000","last_change":"2024-05-08T04:38:29.751389+0000","last_active":"2024-05-08T04:38:30.878079+0000","last_peered":"2024-05-08T04:38:30.878079+0000","last_clean":"2024-05-08T04:38:30.878079+0000","last_became_active":"2024-05-08T04:38:29.751025+0000","last_became_peered":"2024-05-08T04:38:29.751025+0000","last_unstale":"2024-05-08T04:38:30.878079+0000","last_undegraded":"2024-05-08T04:38:30.878079+0000","last_fullsized":"2024-05-08T04:38:30.878079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201069+0000","last_change":"2024-05-08T04:38:28.745873+0000","last_active":"2024-05-08T04:38:31.201069+0000","last_peered":"2024-05-08T04:38:31.201069+0000","last_clean":"2024-05-08T04:38:31.201069+0000","last_became_active":"2024-05-08T04:38:28.745607+0000","last_became_peered":"2024-05-08T04:38:28.745607+0000","last_unstale":"2024-05-08T04:38:31.201069+0000","last_undegraded":"2024-05-08T04:38:31.201069+0000","last_fullsized":"2024-05-08T04:38:31.201069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:38:33.801 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:38:33.969 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-05-08T04:38:33.966+0000 7f1867800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:38:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 Processor -- start 2024-05-08T04:38:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 -- start start 2024-05-08T04:38:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6da00cfda0 0x7f6da0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00cd9f0 con 0x7f6da015ba70 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da015fe70 con 0x7f6da00cfda0 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6da4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cdcd0 con 0x7f6da015be50 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6d9de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6da015ba70 0x7f6da00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57948/0 (socket says 172.21.15.164:57948) 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.975+0000 7f6d9de00640 1 -- 172.21.15.164:0/3213064417 learned_addr learned my addr 172.21.15.164:0/3213064417 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 --2- 172.21.15.164:0/3213064417 >> v2:172.21.15.164:3300/0 conn(0x7f6da00cfda0 0x7f6da0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9d400640 1 -- 172.21.15.164:0/3213064417 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103926277 0 0) 0x7f6da00cd9f0 con 0x7f6da015ba70 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9d400640 1 -- 172.21.15.164:0/3213064417 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d84003530 con 0x7f6da015ba70 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9e800640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 -- 172.21.15.164:0/3213064417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 msgr2=0x7f6da00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 -- 172.21.15.164:0/3213064417 >> v1:172.21.15.164:6789/0 conn(0x7f6da015ba70 legacy=0x7f6da00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 -- 172.21.15.164:0/3213064417 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da015f820 con 0x7f6da00cfda0 2024-05-08T04:38:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9e800640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:33.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d95400640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da0160300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d880097c0 tx=0x7f6d8801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2411c0f3831838d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9d400640 1 -- 172.21.15.164:0/3213064417 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880b2070 con 0x7f6da00cfda0 2024-05-08T04:38:33.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9d400640 1 -- 172.21.15.164:0/3213064417 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d880ad680 con 0x7f6da00cfda0 2024-05-08T04:38:33.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.979+0000 7f6d9d400640 1 -- 172.21.15.164:0/3213064417 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880ada30 con 0x7f6da00cfda0 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- 172.21.15.164:0/3213064417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 msgr2=0x7f6da0160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da0160300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d880097c0 tx=0x7f6d8801c470 comp rx=0 tx=0).stop 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- 172.21.15.164:0/3213064417 shutdown_connections 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3213064417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da0160300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- 172.21.15.164:0/3213064417 >> 172.21.15.164:0/3213064417 conn(0x7f6da006ac90 msgr2=0x7f6da006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:33.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- 172.21.15.164:0/3213064417 shutdown_connections 2024-05-08T04:38:33.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- 172.21.15.164:0/3213064417 wait complete. 2024-05-08T04:38:33.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 Processor -- start 2024-05-08T04:38:33.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- start start 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6da015ba70 0x7f6da0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da01548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da0168d40 con 0x7f6da00cfda0 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da0168bc0 con 0x7f6da015ba70 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6da4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da0168ec0 con 0x7f6da015be50 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41182/0 (socket says 172.21.15.164:41182) 2024-05-08T04:38:33.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 -- 172.21.15.164:0/3713048733 learned_addr learned my addr 172.21.15.164:0/3713048733 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d95400640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6da015ba70 0x7f6da0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9e800640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da01548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 -- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 msgr2=0x7f6da01548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da01548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 -- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6da015ba70 msgr2=0x7f6da0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6da015ba70 0x7f6da0071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 -- 172.21.15.164:0/3713048733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00b0ed0 con 0x7f6da00cfda0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.983+0000 7f6d9de00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c480 tx=0x7f6d9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90017070 con 0x7f6da00cfda0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90005cf0 con 0x7f6da00cfda0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da01502f0 con 0x7f6da00cfda0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90007290 con 0x7f6da00cfda0 2024-05-08T04:38:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da01508b0 con 0x7f6da00cfda0 2024-05-08T04:38:33.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da0069d60 con 0x7f6da00cfda0 2024-05-08T04:38:33.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d9001a070 con 0x7f6da00cfda0 2024-05-08T04:38:33.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 0x7f6d7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:33.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d90084390 con 0x7f6da00cfda0 2024-05-08T04:38:33.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d95400640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 0x7f6d7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:33.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.987+0000 7f6d95400640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 0x7f6d7c069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d880af000 tx=0x7f6d880a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:33.991+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d90054d40 con 0x7f6da00cfda0 2024-05-08T04:38:34.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.143+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6da006e210 con 0x7f6d7c066f90 2024-05-08T04:38:34.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6d96a00640 1 -- 172.21.15.164:0/3713048733 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f6da006e210 con 0x7f6d7c066f90 2024-05-08T04:38:34.154 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:34.156 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 msgr2=0x7f6d7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 0x7f6d7c069450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d880af000 tx=0x7f6d880a7000 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 msgr2=0x7f6da014a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c480 tx=0x7f6d9000c940 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 shutdown_connections 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6d7c066f90 0x7f6d7c069450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6da015be50 0x7f6da01548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6da015ba70 0x7f6da0071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.151+0000 7f6da4a00640 1 --2- 172.21.15.164:0/3713048733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6da00cfda0 0x7f6da014a1f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.155+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 >> 172.21.15.164:0/3713048733 conn(0x7f6da006ac90 msgr2=0x7f6da00cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:34.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.155+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 shutdown_connections 2024-05-08T04:38:34.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.155+0000 7f6da4a00640 1 -- 172.21.15.164:0/3713048733 wait complete. 2024-05-08T04:38:34.175 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T04:38:33.431834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.722938"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201972+0000","last_change":"2024-05-08T04:38:29.759413+0000","last_active":"2024-05-08T04:38:31.201972+0000","last_peered":"2024-05-08T04:38:31.201972+0000","last_clean":"2024-05-08T04:38:31.201972+0000","last_became_active":"2024-05-08T04:38:29.756280+0000","last_became_peered":"2024-05-08T04:38:29.756280+0000","last_unstale":"2024-05-08T04:38:31.201972+0000","last_undegraded":"2024-05-08T04:38:31.201972+0000","last_fullsized":"2024-05-08T04:38:31.201972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739754+0000","last_change":"2024-05-08T04:38:29.729346+0000","last_active":"2024-05-08T04:38:30.739754+0000","last_peered":"2024-05-08T04:38:30.739754+0000","last_clean":"2024-05-08T04:38:30.739754+0000","last_became_active":"2024-05-08T04:38:29.728984+0000","last_became_peered":"2024-05-08T04:38:29.728984+0000","last_unstale":"2024-05-08T04:38:30.739754+0000","last_undegraded":"2024-05-08T04:38:30.739754+0000","last_fullsized":"2024-05-08T04:38:30.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200859+0000","last_change":"2024-05-08T04:38:29.735729+0000","last_active":"2024-05-08T04:38:31.200859+0000","last_peered":"2024-05-08T04:38:31.200859+0000","last_clean":"2024-05-08T04:38:31.200859+0000","last_became_active":"2024-05-08T04:38:29.735510+0000","last_became_peered":"2024-05-08T04:38:29.735510+0000","last_unstale":"2024-05-08T04:38:31.200859+0000","last_undegraded":"2024-05-08T04:38:31.200859+0000","last_fullsized":"2024-05-08T04:38:31.200859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741119+0000","last_change":"2024-05-08T04:38:29.746582+0000","last_active":"2024-05-08T04:38:30.741119+0000","last_peered":"2024-05-08T04:38:30.741119+0000","last_clean":"2024-05-08T04:38:30.741119+0000","last_became_active":"2024-05-08T04:38:29.746344+0000","last_became_peered":"2024-05-08T04:38:29.746344+0000","last_unstale":"2024-05-08T04:38:30.741119+0000","last_undegraded":"2024-05-08T04:38:30.741119+0000","last_fullsized":"2024-05-08T04:38:30.741119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201549+0000","last_change":"2024-05-08T04:38:29.751201+0000","last_active":"2024-05-08T04:38:31.201549+0000","last_peered":"2024-05-08T04:38:31.201549+0000","last_clean":"2024-05-08T04:38:31.201549+0000","last_became_active":"2024-05-08T04:38:29.743374+0000","last_became_peered":"2024-05-08T04:38:29.743374+0000","last_unstale":"2024-05-08T04:38:31.201549+0000","last_undegraded":"2024-05-08T04:38:31.201549+0000","last_fullsized":"2024-05-08T04:38:31.201549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740625+0000","last_change":"2024-05-08T04:38:29.743465+0000","last_active":"2024-05-08T04:38:30.740625+0000","last_peered":"2024-05-08T04:38:30.740625+0000","last_clean":"2024-05-08T04:38:30.740625+0000","last_became_active":"2024-05-08T04:38:29.742816+0000","last_became_peered":"2024-05-08T04:38:29.742816+0000","last_unstale":"2024-05-08T04:38:30.740625+0000","last_undegraded":"2024-05-08T04:38:30.740625+0000","last_fullsized":"2024-05-08T04:38:30.740625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201320+0000","last_change":"2024-05-08T04:38:29.753051+0000","last_active":"2024-05-08T04:38:31.201320+0000","last_peered":"2024-05-08T04:38:31.201320+0000","last_clean":"2024-05-08T04:38:31.201320+0000","last_became_active":"2024-05-08T04:38:29.743659+0000","last_became_peered":"2024-05-08T04:38:29.743659+0000","last_unstale":"2024-05-08T04:38:31.201320+0000","last_undegraded":"2024-05-08T04:38:31.201320+0000","last_fullsized":"2024-05-08T04:38:31.201320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.736688+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.736468+0000","last_became_peered":"2024-05-08T04:38:29.736468+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201990+0000","last_change":"2024-05-08T04:38:29.767705+0000","last_active":"2024-05-08T04:38:31.201990+0000","last_peered":"2024-05-08T04:38:31.201990+0000","last_clean":"2024-05-08T04:38:31.201990+0000","last_became_active":"2024-05-08T04:38:29.767379+0000","last_became_peered":"2024-05-08T04:38:29.767379+0000","last_unstale":"2024-05-08T04:38:31.201990+0000","last_undegraded":"2024-05-08T04:38:31.201990+0000","last_fullsized":"2024-05-08T04:38:31.201990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:59.130461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746455+0000","last_change":"2024-05-08T04:38:29.752398+0000","last_active":"2024-05-08T04:38:30.746455+0000","last_peered":"2024-05-08T04:38:30.746455+0000","last_clean":"2024-05-08T04:38:30.746455+0000","last_became_active":"2024-05-08T04:38:29.751505+0000","last_became_peered":"2024-05-08T04:38:29.751505+0000","last_unstale":"2024-05-08T04:38:30.746455+0000","last_undegraded":"2024-05-08T04:38:30.746455+0000","last_fullsized":"2024-05-08T04:38:30.746455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746387+0000","last_change":"2024-05-08T04:38:29.762618+0000","last_active":"2024-05-08T04:38:30.746387+0000","last_peered":"2024-05-08T04:38:30.746387+0000","last_clean":"2024-05-08T04:38:30.746387+0000","last_became_active":"2024-05-08T04:38:29.762321+0000","last_became_peered":"2024-05-08T04:38:29.762321+0000","last_unstale":"2024-05-08T04:38:30.746387+0000","last_undegraded":"2024-05-08T04:38:30.746387+0000","last_fullsized":"2024-05-08T04:38:30.746387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740525+0000","last_change":"2024-05-08T04:38:29.756698+0000","last_active":"2024-05-08T04:38:30.740525+0000","last_peered":"2024-05-08T04:38:30.740525+0000","last_clean":"2024-05-08T04:38:30.740525+0000","last_became_active":"2024-05-08T04:38:29.756073+0000","last_became_peered":"2024-05-08T04:38:29.756073+0000","last_unstale":"2024-05-08T04:38:30.740525+0000","last_undegraded":"2024-05-08T04:38:30.740525+0000","last_fullsized":"2024-05-08T04:38:30.740525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877713+0000","last_change":"2024-05-08T04:38:29.751753+0000","last_active":"2024-05-08T04:38:30.877713+0000","last_peered":"2024-05-08T04:38:30.877713+0000","last_clean":"2024-05-08T04:38:30.877713+0000","last_became_active":"2024-05-08T04:38:29.750686+0000","last_became_peered":"2024-05-08T04:38:29.750686+0000","last_unstale":"2024-05-08T04:38:30.877713+0000","last_undegraded":"2024-05-08T04:38:30.877713+0000","last_fullsized":"2024-05-08T04:38:30.877713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.730987+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.730744+0000","last_became_peered":"2024-05-08T04:38:29.730744+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:55:21.772046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739297+0000","last_change":"2024-05-08T04:38:29.739983+0000","last_active":"2024-05-08T04:38:30.739297+0000","last_peered":"2024-05-08T04:38:30.739297+0000","last_clean":"2024-05-08T04:38:30.739297+0000","last_became_active":"2024-05-08T04:38:29.738511+0000","last_became_peered":"2024-05-08T04:38:29.738511+0000","last_unstale":"2024-05-08T04:38:30.739297+0000","last_undegraded":"2024-05-08T04:38:30.739297+0000","last_fullsized":"2024-05-08T04:38:30.739297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740242+0000","last_change":"2024-05-08T04:38:29.747325+0000","last_active":"2024-05-08T04:38:30.740242+0000","last_peered":"2024-05-08T04:38:30.740242+0000","last_clean":"2024-05-08T04:38:30.740242+0000","last_became_active":"2024-05-08T04:38:29.744848+0000","last_became_peered":"2024-05-08T04:38:29.744848+0000","last_unstale":"2024-05-08T04:38:30.740242+0000","last_undegraded":"2024-05-08T04:38:30.740242+0000","last_fullsized":"2024-05-08T04:38:30.740242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741529+0000","last_change":"2024-05-08T04:38:29.748850+0000","last_active":"2024-05-08T04:38:30.741529+0000","last_peered":"2024-05-08T04:38:30.741529+0000","last_clean":"2024-05-08T04:38:30.741529+0000","last_became_active":"2024-05-08T04:38:29.748560+0000","last_became_peered":"2024-05-08T04:38:29.748560+0000","last_unstale":"2024-05-08T04:38:30.741529+0000","last_undegraded":"2024-05-08T04:38:30.741529+0000","last_fullsized":"2024-05-08T04:38:30.741529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200471+0000","last_change":"2024-05-08T04:38:29.756416+0000","last_active":"2024-05-08T04:38:31.200471+0000","last_peered":"2024-05-08T04:38:31.200471+0000","last_clean":"2024-05-08T04:38:31.200471+0000","last_became_active":"2024-05-08T04:38:29.756194+0000","last_became_peered":"2024-05-08T04:38:29.756194+0000","last_unstale":"2024-05-08T04:38:31.200471+0000","last_undegraded":"2024-05-08T04:38:31.200471+0000","last_fullsized":"2024-05-08T04:38:31.200471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201900+0000","last_change":"2024-05-08T04:38:29.752005+0000","last_active":"2024-05-08T04:38:31.201900+0000","last_peered":"2024-05-08T04:38:31.201900+0000","last_clean":"2024-05-08T04:38:31.201900+0000","last_became_active":"2024-05-08T04:38:29.743651+0000","last_became_peered":"2024-05-08T04:38:29.743651+0000","last_unstale":"2024-05-08T04:38:31.201900+0000","last_undegraded":"2024-05-08T04:38:31.201900+0000","last_fullsized":"2024-05-08T04:38:31.201900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746095+0000","last_change":"2024-05-08T04:38:29.757006+0000","last_active":"2024-05-08T04:38:30.746095+0000","last_peered":"2024-05-08T04:38:30.746095+0000","last_clean":"2024-05-08T04:38:30.746095+0000","last_became_active":"2024-05-08T04:38:29.756756+0000","last_became_peered":"2024-05-08T04:38:29.756756+0000","last_unstale":"2024-05-08T04:38:30.746095+0000","last_undegraded":"2024-05-08T04:38:30.746095+0000","last_fullsized":"2024-05-08T04:38:30.746095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200787+0000","last_change":"2024-05-08T04:38:29.735804+0000","last_active":"2024-05-08T04:38:31.200787+0000","last_peered":"2024-05-08T04:38:31.200787+0000","last_clean":"2024-05-08T04:38:31.200787+0000","last_became_active":"2024-05-08T04:38:29.735527+0000","last_became_peered":"2024-05-08T04:38:29.735527+0000","last_unstale":"2024-05-08T04:38:31.200787+0000","last_undegraded":"2024-05-08T04:38:31.200787+0000","last_fullsized":"2024-05-08T04:38:31.200787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:18.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740806+0000","last_change":"2024-05-08T04:38:29.762810+0000","last_active":"2024-05-08T04:38:30.740806+0000","last_peered":"2024-05-08T04:38:30.740806+0000","last_clean":"2024-05-08T04:38:30.740806+0000","last_became_active":"2024-05-08T04:38:29.762532+0000","last_became_peered":"2024-05-08T04:38:29.762532+0000","last_unstale":"2024-05-08T04:38:30.740806+0000","last_undegraded":"2024-05-08T04:38:30.740806+0000","last_fullsized":"2024-05-08T04:38:30.740806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.756285+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.756024+0000","last_became_peered":"2024-05-08T04:38:29.756024+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740734+0000","last_change":"2024-05-08T04:38:29.747323+0000","last_active":"2024-05-08T04:38:30.740734+0000","last_peered":"2024-05-08T04:38:30.740734+0000","last_clean":"2024-05-08T04:38:30.740734+0000","last_became_active":"2024-05-08T04:38:29.747077+0000","last_became_peered":"2024-05-08T04:38:29.747077+0000","last_unstale":"2024-05-08T04:38:30.740734+0000","last_undegraded":"2024-05-08T04:38:30.740734+0000","last_fullsized":"2024-05-08T04:38:30.740734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878039+0000","last_change":"2024-05-08T04:38:29.748090+0000","last_active":"2024-05-08T04:38:30.878039+0000","last_peered":"2024-05-08T04:38:30.878039+0000","last_clean":"2024-05-08T04:38:30.878039+0000","last_became_active":"2024-05-08T04:38:29.747823+0000","last_became_peered":"2024-05-08T04:38:29.747823+0000","last_unstale":"2024-05-08T04:38:30.878039+0000","last_undegraded":"2024-05-08T04:38:30.878039+0000","last_fullsized":"2024-05-08T04:38:30.878039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746543+0000","last_change":"2024-05-08T04:38:29.766531+0000","last_active":"2024-05-08T04:38:30.746543+0000","last_peered":"2024-05-08T04:38:30.746543+0000","last_clean":"2024-05-08T04:38:30.746543+0000","last_became_active":"2024-05-08T04:38:29.765752+0000","last_became_peered":"2024-05-08T04:38:29.765752+0000","last_unstale":"2024-05-08T04:38:30.746543+0000","last_undegraded":"2024-05-08T04:38:30.746543+0000","last_fullsized":"2024-05-08T04:38:30.746543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:29.741993+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:29.740916+0000","last_became_peered":"2024-05-08T04:38:29.740916+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877964+0000","last_change":"2024-05-08T04:38:29.747721+0000","last_active":"2024-05-08T04:38:30.877964+0000","last_peered":"2024-05-08T04:38:30.877964+0000","last_clean":"2024-05-08T04:38:30.877964+0000","last_became_active":"2024-05-08T04:38:29.747443+0000","last_became_peered":"2024-05-08T04:38:29.747443+0000","last_unstale":"2024-05-08T04:38:30.877964+0000","last_undegraded":"2024-05-08T04:38:30.877964+0000","last_fullsized":"2024-05-08T04:38:30.877964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200927+0000","last_change":"2024-05-08T04:38:29.731332+0000","last_active":"2024-05-08T04:38:31.200927+0000","last_peered":"2024-05-08T04:38:31.200927+0000","last_clean":"2024-05-08T04:38:31.200927+0000","last_became_active":"2024-05-08T04:38:29.731075+0000","last_became_peered":"2024-05-08T04:38:29.731075+0000","last_unstale":"2024-05-08T04:38:31.200927+0000","last_undegraded":"2024-05-08T04:38:31.200927+0000","last_fullsized":"2024-05-08T04:38:31.200927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740872+0000","last_change":"2024-05-08T04:38:29.743680+0000","last_active":"2024-05-08T04:38:30.740872+0000","last_peered":"2024-05-08T04:38:30.740872+0000","last_clean":"2024-05-08T04:38:30.740872+0000","last_became_active":"2024-05-08T04:38:29.743208+0000","last_became_peered":"2024-05-08T04:38:29.743208+0000","last_unstale":"2024-05-08T04:38:30.740872+0000","last_undegraded":"2024-05-08T04:38:30.740872+0000","last_fullsized":"2024-05-08T04:38:30.740872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200255+0000","last_change":"2024-05-08T04:38:29.738213+0000","last_active":"2024-05-08T04:38:31.200255+0000","last_peered":"2024-05-08T04:38:31.200255+0000","last_clean":"2024-05-08T04:38:31.200255+0000","last_became_active":"2024-05-08T04:38:29.737935+0000","last_became_peered":"2024-05-08T04:38:29.737935+0000","last_unstale":"2024-05-08T04:38:31.200255+0000","last_undegraded":"2024-05-08T04:38:31.200255+0000","last_fullsized":"2024-05-08T04:38:31.200255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746039+0000","last_change":"2024-05-08T04:38:29.750290+0000","last_active":"2024-05-08T04:38:30.746039+0000","last_peered":"2024-05-08T04:38:30.746039+0000","last_clean":"2024-05-08T04:38:30.746039+0000","last_became_active":"2024-05-08T04:38:29.749840+0000","last_became_peered":"2024-05-08T04:38:29.749840+0000","last_unstale":"2024-05-08T04:38:30.746039+0000","last_undegraded":"2024-05-08T04:38:30.746039+0000","last_fullsized":"2024-05-08T04:38:30.746039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740040+0000","last_change":"2024-05-08T04:38:29.734132+0000","last_active":"2024-05-08T04:38:30.740040+0000","last_peered":"2024-05-08T04:38:30.740040+0000","last_clean":"2024-05-08T04:38:30.740040+0000","last_became_active":"2024-05-08T04:38:29.733869+0000","last_became_peered":"2024-05-08T04:38:29.733869+0000","last_unstale":"2024-05-08T04:38:30.740040+0000","last_undegraded":"2024-05-08T04:38:30.740040+0000","last_fullsized":"2024-05-08T04:38:30.740040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:28.424596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877812+0000","last_change":"2024-05-08T04:38:28.741244+0000","last_active":"2024-05-08T04:38:30.877812+0000","last_peered":"2024-05-08T04:38:30.877812+0000","last_clean":"2024-05-08T04:38:30.877812+0000","last_became_active":"2024-05-08T04:38:28.740897+0000","last_became_peered":"2024-05-08T04:38:28.740897+0000","last_unstale":"2024-05-08T04:38:30.877812+0000","last_undegraded":"2024-05-08T04:38:30.877812+0000","last_fullsized":"2024-05-08T04:38:30.877812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200940+0000","last_change":"2024-05-08T04:38:29.731992+0000","last_active":"2024-05-08T04:38:31.200940+0000","last_peered":"2024-05-08T04:38:31.200940+0000","last_clean":"2024-05-08T04:38:31.200940+0000","last_became_active":"2024-05-08T04:38:29.731683+0000","last_became_peered":"2024-05-08T04:38:29.731683+0000","last_unstale":"2024-05-08T04:38:31.200940+0000","last_undegraded":"2024-05-08T04:38:31.200940+0000","last_fullsized":"2024-05-08T04:38:31.200940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746000+0000","last_change":"2024-05-08T04:38:28.738380+0000","last_active":"2024-05-08T04:38:30.746000+0000","last_peered":"2024-05-08T04:38:30.746000+0000","last_clean":"2024-05-08T04:38:30.746000+0000","last_became_active":"2024-05-08T04:38:28.738109+0000","last_became_peered":"2024-05-08T04:38:28.738109+0000","last_unstale":"2024-05-08T04:38:30.746000+0000","last_undegraded":"2024-05-08T04:38:30.746000+0000","last_fullsized":"2024-05-08T04:38:30.746000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877587+0000","last_change":"2024-05-08T04:38:28.743973+0000","last_active":"2024-05-08T04:38:30.877587+0000","last_peered":"2024-05-08T04:38:30.877587+0000","last_clean":"2024-05-08T04:38:30.877587+0000","last_became_active":"2024-05-08T04:38:28.743755+0000","last_became_peered":"2024-05-08T04:38:28.743755+0000","last_unstale":"2024-05-08T04:38:30.877587+0000","last_undegraded":"2024-05-08T04:38:30.877587+0000","last_fullsized":"2024-05-08T04:38:30.877587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877578+0000","last_change":"2024-05-08T04:38:29.737294+0000","last_active":"2024-05-08T04:38:30.877578+0000","last_peered":"2024-05-08T04:38:30.877578+0000","last_clean":"2024-05-08T04:38:30.877578+0000","last_became_active":"2024-05-08T04:38:29.737004+0000","last_became_peered":"2024-05-08T04:38:29.737004+0000","last_unstale":"2024-05-08T04:38:30.877578+0000","last_undegraded":"2024-05-08T04:38:30.877578+0000","last_fullsized":"2024-05-08T04:38:30.877578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740741+0000","last_change":"2024-05-08T04:38:28.737564+0000","last_active":"2024-05-08T04:38:30.740741+0000","last_peered":"2024-05-08T04:38:30.740741+0000","last_clean":"2024-05-08T04:38:30.740741+0000","last_became_active":"2024-05-08T04:38:28.737280+0000","last_became_peered":"2024-05-08T04:38:28.737280+0000","last_unstale":"2024-05-08T04:38:30.740741+0000","last_undegraded":"2024-05-08T04:38:30.740741+0000","last_fullsized":"2024-05-08T04:38:30.740741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739295+0000","last_change":"2024-05-08T04:38:29.739772+0000","last_active":"2024-05-08T04:38:30.739295+0000","last_peered":"2024-05-08T04:38:30.739295+0000","last_clean":"2024-05-08T04:38:30.739295+0000","last_became_active":"2024-05-08T04:38:29.738258+0000","last_became_peered":"2024-05-08T04:38:29.738258+0000","last_unstale":"2024-05-08T04:38:30.739295+0000","last_undegraded":"2024-05-08T04:38:30.739295+0000","last_fullsized":"2024-05-08T04:38:30.739295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746413+0000","last_change":"2024-05-08T04:38:28.742877+0000","last_active":"2024-05-08T04:38:30.746413+0000","last_peered":"2024-05-08T04:38:30.746413+0000","last_clean":"2024-05-08T04:38:30.746413+0000","last_became_active":"2024-05-08T04:38:28.742495+0000","last_became_peered":"2024-05-08T04:38:28.742495+0000","last_unstale":"2024-05-08T04:38:30.746413+0000","last_undegraded":"2024-05-08T04:38:30.746413+0000","last_fullsized":"2024-05-08T04:38:30.746413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741690+0000","last_change":"2024-05-08T04:38:29.759053+0000","last_active":"2024-05-08T04:38:30.741690+0000","last_peered":"2024-05-08T04:38:30.741690+0000","last_clean":"2024-05-08T04:38:30.741690+0000","last_became_active":"2024-05-08T04:38:29.758357+0000","last_became_peered":"2024-05-08T04:38:29.758357+0000","last_unstale":"2024-05-08T04:38:30.741690+0000","last_undegraded":"2024-05-08T04:38:30.741690+0000","last_fullsized":"2024-05-08T04:38:30.741690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741139+0000","last_change":"2024-05-08T04:38:28.748874+0000","last_active":"2024-05-08T04:38:30.741139+0000","last_peered":"2024-05-08T04:38:30.741139+0000","last_clean":"2024-05-08T04:38:30.741139+0000","last_became_active":"2024-05-08T04:38:28.748071+0000","last_became_peered":"2024-05-08T04:38:28.748071+0000","last_unstale":"2024-05-08T04:38:30.741139+0000","last_undegraded":"2024-05-08T04:38:30.741139+0000","last_fullsized":"2024-05-08T04:38:30.741139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201063+0000","last_change":"2024-05-08T04:38:29.731689+0000","last_active":"2024-05-08T04:38:31.201063+0000","last_peered":"2024-05-08T04:38:31.201063+0000","last_clean":"2024-05-08T04:38:31.201063+0000","last_became_active":"2024-05-08T04:38:29.731433+0000","last_became_peered":"2024-05-08T04:38:29.731433+0000","last_unstale":"2024-05-08T04:38:31.201063+0000","last_undegraded":"2024-05-08T04:38:31.201063+0000","last_fullsized":"2024-05-08T04:38:31.201063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:28.752051+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:28.751792+0000","last_became_peered":"2024-05-08T04:38:28.751792+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740885+0000","last_change":"2024-05-08T04:38:29.756998+0000","last_active":"2024-05-08T04:38:30.740885+0000","last_peered":"2024-05-08T04:38:30.740885+0000","last_clean":"2024-05-08T04:38:30.740885+0000","last_became_active":"2024-05-08T04:38:29.756395+0000","last_became_peered":"2024-05-08T04:38:29.756395+0000","last_unstale":"2024-05-08T04:38:30.740885+0000","last_undegraded":"2024-05-08T04:38:30.740885+0000","last_fullsized":"2024-05-08T04:38:30.740885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746251+0000","last_change":"2024-05-08T04:38:28.744776+0000","last_active":"2024-05-08T04:38:30.746251+0000","last_peered":"2024-05-08T04:38:30.746251+0000","last_clean":"2024-05-08T04:38:30.746251+0000","last_became_active":"2024-05-08T04:38:28.744410+0000","last_became_peered":"2024-05-08T04:38:28.744410+0000","last_unstale":"2024-05-08T04:38:30.746251+0000","last_undegraded":"2024-05-08T04:38:30.746251+0000","last_fullsized":"2024-05-08T04:38:30.746251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201683+0000","last_change":"2024-05-08T04:38:29.755661+0000","last_active":"2024-05-08T04:38:31.201683+0000","last_peered":"2024-05-08T04:38:31.201683+0000","last_clean":"2024-05-08T04:38:31.201683+0000","last_became_active":"2024-05-08T04:38:29.754401+0000","last_became_peered":"2024-05-08T04:38:29.754401+0000","last_unstale":"2024-05-08T04:38:31.201683+0000","last_undegraded":"2024-05-08T04:38:31.201683+0000","last_fullsized":"2024-05-08T04:38:31.201683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201037+0000","last_change":"2024-05-08T04:38:28.742800+0000","last_active":"2024-05-08T04:38:31.201037+0000","last_peered":"2024-05-08T04:38:31.201037+0000","last_clean":"2024-05-08T04:38:31.201037+0000","last_became_active":"2024-05-08T04:38:28.742491+0000","last_became_peered":"2024-05-08T04:38:28.742491+0000","last_unstale":"2024-05-08T04:38:31.201037+0000","last_undegraded":"2024-05-08T04:38:31.201037+0000","last_fullsized":"2024-05-08T04:38:31.201037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:53.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200760+0000","last_change":"2024-05-08T04:38:29.740544+0000","last_active":"2024-05-08T04:38:31.200760+0000","last_peered":"2024-05-08T04:38:31.200760+0000","last_clean":"2024-05-08T04:38:31.200760+0000","last_became_active":"2024-05-08T04:38:29.740262+0000","last_became_peered":"2024-05-08T04:38:29.740262+0000","last_unstale":"2024-05-08T04:38:31.200760+0000","last_undegraded":"2024-05-08T04:38:31.200760+0000","last_fullsized":"2024-05-08T04:38:31.200760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:15.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740918+0000","last_change":"2024-05-08T04:38:28.751437+0000","last_active":"2024-05-08T04:38:30.740918+0000","last_peered":"2024-05-08T04:38:30.740918+0000","last_clean":"2024-05-08T04:38:30.740918+0000","last_became_active":"2024-05-08T04:38:28.751039+0000","last_became_peered":"2024-05-08T04:38:28.751039+0000","last_unstale":"2024-05-08T04:38:30.740918+0000","last_undegraded":"2024-05-08T04:38:30.740918+0000","last_fullsized":"2024-05-08T04:38:30.740918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201608+0000","last_change":"2024-05-08T04:38:29.756359+0000","last_active":"2024-05-08T04:38:31.201608+0000","last_peered":"2024-05-08T04:38:31.201608+0000","last_clean":"2024-05-08T04:38:31.201608+0000","last_became_active":"2024-05-08T04:38:29.756087+0000","last_became_peered":"2024-05-08T04:38:29.756087+0000","last_unstale":"2024-05-08T04:38:31.201608+0000","last_undegraded":"2024-05-08T04:38:31.201608+0000","last_fullsized":"2024-05-08T04:38:31.201608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.745976+0000","last_change":"2024-05-08T04:38:28.739635+0000","last_active":"2024-05-08T04:38:30.745976+0000","last_peered":"2024-05-08T04:38:30.745976+0000","last_clean":"2024-05-08T04:38:30.745976+0000","last_became_active":"2024-05-08T04:38:28.739255+0000","last_became_peered":"2024-05-08T04:38:28.739255+0000","last_unstale":"2024-05-08T04:38:30.745976+0000","last_undegraded":"2024-05-08T04:38:30.745976+0000","last_fullsized":"2024-05-08T04:38:30.745976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746057+0000","last_change":"2024-05-08T04:38:29.750751+0000","last_active":"2024-05-08T04:38:30.746057+0000","last_peered":"2024-05-08T04:38:30.746057+0000","last_clean":"2024-05-08T04:38:30.746057+0000","last_became_active":"2024-05-08T04:38:29.749978+0000","last_became_peered":"2024-05-08T04:38:29.749978+0000","last_unstale":"2024-05-08T04:38:30.746057+0000","last_undegraded":"2024-05-08T04:38:30.746057+0000","last_fullsized":"2024-05-08T04:38:30.746057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200130+0000","last_change":"2024-05-08T04:38:28.738083+0000","last_active":"2024-05-08T04:38:31.200130+0000","last_peered":"2024-05-08T04:38:31.200130+0000","last_clean":"2024-05-08T04:38:31.200130+0000","last_became_active":"2024-05-08T04:38:28.736994+0000","last_became_peered":"2024-05-08T04:38:28.736994+0000","last_unstale":"2024-05-08T04:38:31.200130+0000","last_undegraded":"2024-05-08T04:38:31.200130+0000","last_fullsized":"2024-05-08T04:38:31.200130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201379+0000","last_change":"2024-05-08T04:38:29.758699+0000","last_active":"2024-05-08T04:38:31.201379+0000","last_peered":"2024-05-08T04:38:31.201379+0000","last_clean":"2024-05-08T04:38:31.201379+0000","last_became_active":"2024-05-08T04:38:29.758439+0000","last_became_peered":"2024-05-08T04:38:29.758439+0000","last_unstale":"2024-05-08T04:38:31.201379+0000","last_undegraded":"2024-05-08T04:38:31.201379+0000","last_fullsized":"2024-05-08T04:38:31.201379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741696+0000","last_change":"2024-05-08T04:38:28.738978+0000","last_active":"2024-05-08T04:38:30.741696+0000","last_peered":"2024-05-08T04:38:30.741696+0000","last_clean":"2024-05-08T04:38:30.741696+0000","last_became_active":"2024-05-08T04:38:28.738688+0000","last_became_peered":"2024-05-08T04:38:28.738688+0000","last_unstale":"2024-05-08T04:38:30.741696+0000","last_undegraded":"2024-05-08T04:38:30.741696+0000","last_fullsized":"2024-05-08T04:38:30.741696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201862+0000","last_change":"2024-05-08T04:38:29.757772+0000","last_active":"2024-05-08T04:38:31.201862+0000","last_peered":"2024-05-08T04:38:31.201862+0000","last_clean":"2024-05-08T04:38:31.201862+0000","last_became_active":"2024-05-08T04:38:29.756730+0000","last_became_peered":"2024-05-08T04:38:29.756730+0000","last_unstale":"2024-05-08T04:38:31.201862+0000","last_undegraded":"2024-05-08T04:38:31.201862+0000","last_fullsized":"2024-05-08T04:38:31.201862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200418+0000","last_change":"2024-05-08T04:38:28.734548+0000","last_active":"2024-05-08T04:38:31.200418+0000","last_peered":"2024-05-08T04:38:31.200418+0000","last_clean":"2024-05-08T04:38:31.200418+0000","last_became_active":"2024-05-08T04:38:28.734179+0000","last_became_peered":"2024-05-08T04:38:28.734179+0000","last_unstale":"2024-05-08T04:38:31.200418+0000","last_undegraded":"2024-05-08T04:38:31.200418+0000","last_fullsized":"2024-05-08T04:38:31.200418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.202085+0000","last_change":"2024-05-08T04:38:29.758477+0000","last_active":"2024-05-08T04:38:31.202085+0000","last_peered":"2024-05-08T04:38:31.202085+0000","last_clean":"2024-05-08T04:38:31.202085+0000","last_became_active":"2024-05-08T04:38:29.756388+0000","last_became_peered":"2024-05-08T04:38:29.756388+0000","last_unstale":"2024-05-08T04:38:31.202085+0000","last_undegraded":"2024-05-08T04:38:31.202085+0000","last_fullsized":"2024-05-08T04:38:31.202085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:06.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739628+0000","last_change":"2024-05-08T04:38:28.739369+0000","last_active":"2024-05-08T04:38:30.739628+0000","last_peered":"2024-05-08T04:38:30.739628+0000","last_clean":"2024-05-08T04:38:30.739628+0000","last_became_active":"2024-05-08T04:38:28.739079+0000","last_became_peered":"2024-05-08T04:38:28.739079+0000","last_unstale":"2024-05-08T04:38:30.739628+0000","last_undegraded":"2024-05-08T04:38:30.739628+0000","last_fullsized":"2024-05-08T04:38:30.739628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741244+0000","last_change":"2024-05-08T04:38:29.748638+0000","last_active":"2024-05-08T04:38:30.741244+0000","last_peered":"2024-05-08T04:38:30.741244+0000","last_clean":"2024-05-08T04:38:30.741244+0000","last_became_active":"2024-05-08T04:38:29.748277+0000","last_became_peered":"2024-05-08T04:38:29.748277+0000","last_unstale":"2024-05-08T04:38:30.741244+0000","last_undegraded":"2024-05-08T04:38:30.741244+0000","last_fullsized":"2024-05-08T04:38:30.741244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200420+0000","last_change":"2024-05-08T04:38:28.737536+0000","last_active":"2024-05-08T04:38:31.200420+0000","last_peered":"2024-05-08T04:38:31.200420+0000","last_clean":"2024-05-08T04:38:31.200420+0000","last_became_active":"2024-05-08T04:38:28.737250+0000","last_became_peered":"2024-05-08T04:38:28.737250+0000","last_unstale":"2024-05-08T04:38:31.200420+0000","last_undegraded":"2024-05-08T04:38:31.200420+0000","last_fullsized":"2024-05-08T04:38:31.200420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201674+0000","last_change":"2024-05-08T04:38:29.762160+0000","last_active":"2024-05-08T04:38:31.201674+0000","last_peered":"2024-05-08T04:38:31.201674+0000","last_clean":"2024-05-08T04:38:31.201674+0000","last_became_active":"2024-05-08T04:38:29.761754+0000","last_became_peered":"2024-05-08T04:38:29.761754+0000","last_unstale":"2024-05-08T04:38:31.201674+0000","last_undegraded":"2024-05-08T04:38:31.201674+0000","last_fullsized":"2024-05-08T04:38:31.201674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201275+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:38:31.201275+0000","last_peered":"2024-05-08T04:38:31.201275+0000","last_clean":"2024-05-08T04:38:31.201275+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:38:31.201275+0000","last_undegraded":"2024-05-08T04:38:31.201275+0000","last_fullsized":"2024-05-08T04:38:31.201275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741367+0000","last_change":"2024-05-08T04:38:28.750045+0000","last_active":"2024-05-08T04:38:30.741367+0000","last_peered":"2024-05-08T04:38:30.741367+0000","last_clean":"2024-05-08T04:38:30.741367+0000","last_became_active":"2024-05-08T04:38:28.749775+0000","last_became_peered":"2024-05-08T04:38:28.749775+0000","last_unstale":"2024-05-08T04:38:30.741367+0000","last_undegraded":"2024-05-08T04:38:30.741367+0000","last_fullsized":"2024-05-08T04:38:30.741367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877692+0000","last_change":"2024-05-08T04:38:29.733287+0000","last_active":"2024-05-08T04:38:30.877692+0000","last_peered":"2024-05-08T04:38:30.877692+0000","last_clean":"2024-05-08T04:38:30.877692+0000","last_became_active":"2024-05-08T04:38:29.733010+0000","last_became_peered":"2024-05-08T04:38:29.733010+0000","last_unstale":"2024-05-08T04:38:30.877692+0000","last_undegraded":"2024-05-08T04:38:30.877692+0000","last_fullsized":"2024-05-08T04:38:30.877692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201257+0000","last_change":"2024-05-08T04:38:28.750803+0000","last_active":"2024-05-08T04:38:31.201257+0000","last_peered":"2024-05-08T04:38:31.201257+0000","last_clean":"2024-05-08T04:38:31.201257+0000","last_became_active":"2024-05-08T04:38:28.750389+0000","last_became_peered":"2024-05-08T04:38:28.750389+0000","last_unstale":"2024-05-08T04:38:31.201257+0000","last_undegraded":"2024-05-08T04:38:31.201257+0000","last_fullsized":"2024-05-08T04:38:31.201257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.734926+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.733112+0000","last_became_peered":"2024-05-08T04:38:29.733112+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200293+0000","last_change":"2024-05-08T04:38:28.733116+0000","last_active":"2024-05-08T04:38:31.200293+0000","last_peered":"2024-05-08T04:38:31.200293+0000","last_clean":"2024-05-08T04:38:31.200293+0000","last_became_active":"2024-05-08T04:38:28.732033+0000","last_became_peered":"2024-05-08T04:38:28.732033+0000","last_unstale":"2024-05-08T04:38:31.200293+0000","last_undegraded":"2024-05-08T04:38:31.200293+0000","last_fullsized":"2024-05-08T04:38:31.200293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739374+0000","last_change":"2024-05-08T04:38:29.740783+0000","last_active":"2024-05-08T04:38:30.739374+0000","last_peered":"2024-05-08T04:38:30.739374+0000","last_clean":"2024-05-08T04:38:30.739374+0000","last_became_active":"2024-05-08T04:38:29.738992+0000","last_became_peered":"2024-05-08T04:38:29.738992+0000","last_unstale":"2024-05-08T04:38:30.739374+0000","last_undegraded":"2024-05-08T04:38:30.739374+0000","last_fullsized":"2024-05-08T04:38:30.739374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740989+0000","last_change":"2024-05-08T04:38:28.744976+0000","last_active":"2024-05-08T04:38:30.740989+0000","last_peered":"2024-05-08T04:38:30.740989+0000","last_clean":"2024-05-08T04:38:30.740989+0000","last_became_active":"2024-05-08T04:38:28.744674+0000","last_became_peered":"2024-05-08T04:38:28.744674+0000","last_unstale":"2024-05-08T04:38:30.740989+0000","last_undegraded":"2024-05-08T04:38:30.740989+0000","last_fullsized":"2024-05-08T04:38:30.740989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740393+0000","last_change":"2024-05-08T04:38:29.743310+0000","last_active":"2024-05-08T04:38:30.740393+0000","last_peered":"2024-05-08T04:38:30.740393+0000","last_clean":"2024-05-08T04:38:30.740393+0000","last_became_active":"2024-05-08T04:38:29.741857+0000","last_became_peered":"2024-05-08T04:38:29.741857+0000","last_unstale":"2024-05-08T04:38:30.740393+0000","last_undegraded":"2024-05-08T04:38:30.740393+0000","last_fullsized":"2024-05-08T04:38:30.740393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:09.896832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739510+0000","last_change":"2024-05-08T04:38:28.735199+0000","last_active":"2024-05-08T04:38:30.739510+0000","last_peered":"2024-05-08T04:38:30.739510+0000","last_clean":"2024-05-08T04:38:30.739510+0000","last_became_active":"2024-05-08T04:38:28.734839+0000","last_became_peered":"2024-05-08T04:38:28.734839+0000","last_unstale":"2024-05-08T04:38:30.739510+0000","last_undegraded":"2024-05-08T04:38:30.739510+0000","last_fullsized":"2024-05-08T04:38:30.739510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741496+0000","last_change":"2024-05-08T04:38:29.743096+0000","last_active":"2024-05-08T04:38:30.741496+0000","last_peered":"2024-05-08T04:38:30.741496+0000","last_clean":"2024-05-08T04:38:30.741496+0000","last_became_active":"2024-05-08T04:38:29.742795+0000","last_became_peered":"2024-05-08T04:38:29.742795+0000","last_unstale":"2024-05-08T04:38:30.741496+0000","last_undegraded":"2024-05-08T04:38:30.741496+0000","last_fullsized":"2024-05-08T04:38:30.741496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201931+0000","last_change":"2024-05-08T04:38:28.749845+0000","last_active":"2024-05-08T04:38:31.201931+0000","last_peered":"2024-05-08T04:38:31.201931+0000","last_clean":"2024-05-08T04:38:31.201931+0000","last_became_active":"2024-05-08T04:38:28.749522+0000","last_became_peered":"2024-05-08T04:38:28.749522+0000","last_unstale":"2024-05-08T04:38:31.201931+0000","last_undegraded":"2024-05-08T04:38:31.201931+0000","last_fullsized":"2024-05-08T04:38:31.201931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:28.101827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739371+0000","last_change":"2024-05-08T04:38:29.733856+0000","last_active":"2024-05-08T04:38:30.739371+0000","last_peered":"2024-05-08T04:38:30.739371+0000","last_clean":"2024-05-08T04:38:30.739371+0000","last_became_active":"2024-05-08T04:38:29.733589+0000","last_became_peered":"2024-05-08T04:38:29.733589+0000","last_unstale":"2024-05-08T04:38:30.739371+0000","last_undegraded":"2024-05-08T04:38:30.739371+0000","last_fullsized":"2024-05-08T04:38:30.739371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.744254+0000","last_change":"2024-05-08T04:38:28.743259+0000","last_active":"2024-05-08T04:38:30.744254+0000","last_peered":"2024-05-08T04:38:30.744254+0000","last_clean":"2024-05-08T04:38:30.744254+0000","last_became_active":"2024-05-08T04:38:28.743022+0000","last_became_peered":"2024-05-08T04:38:28.743022+0000","last_unstale":"2024-05-08T04:38:30.744254+0000","last_undegraded":"2024-05-08T04:38:30.744254+0000","last_fullsized":"2024-05-08T04:38:30.744254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878061+0000","last_change":"2024-05-08T04:38:29.742718+0000","last_active":"2024-05-08T04:38:30.878061+0000","last_peered":"2024-05-08T04:38:30.878061+0000","last_clean":"2024-05-08T04:38:30.878061+0000","last_became_active":"2024-05-08T04:38:29.742135+0000","last_became_peered":"2024-05-08T04:38:29.742135+0000","last_unstale":"2024-05-08T04:38:30.878061+0000","last_undegraded":"2024-05-08T04:38:30.878061+0000","last_fullsized":"2024-05-08T04:38:30.878061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746492+0000","last_change":"2024-05-08T04:38:28.747934+0000","last_active":"2024-05-08T04:38:30.746492+0000","last_peered":"2024-05-08T04:38:30.746492+0000","last_clean":"2024-05-08T04:38:30.746492+0000","last_became_active":"2024-05-08T04:38:28.747524+0000","last_became_peered":"2024-05-08T04:38:28.747524+0000","last_unstale":"2024-05-08T04:38:30.746492+0000","last_undegraded":"2024-05-08T04:38:30.746492+0000","last_fullsized":"2024-05-08T04:38:30.746492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200901+0000","last_change":"2024-05-08T04:38:29.729062+0000","last_active":"2024-05-08T04:38:31.200901+0000","last_peered":"2024-05-08T04:38:31.200901+0000","last_clean":"2024-05-08T04:38:31.200901+0000","last_became_active":"2024-05-08T04:38:29.728777+0000","last_became_peered":"2024-05-08T04:38:29.728777+0000","last_unstale":"2024-05-08T04:38:31.200901+0000","last_undegraded":"2024-05-08T04:38:31.200901+0000","last_fullsized":"2024-05-08T04:38:31.200901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201185+0000","last_change":"2024-05-08T04:38:28.749529+0000","last_active":"2024-05-08T04:38:31.201185+0000","last_peered":"2024-05-08T04:38:31.201185+0000","last_clean":"2024-05-08T04:38:31.201185+0000","last_became_active":"2024-05-08T04:38:28.749167+0000","last_became_peered":"2024-05-08T04:38:28.749167+0000","last_unstale":"2024-05-08T04:38:31.201185+0000","last_undegraded":"2024-05-08T04:38:31.201185+0000","last_fullsized":"2024-05-08T04:38:31.201185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740076+0000","last_change":"2024-05-08T04:38:29.749670+0000","last_active":"2024-05-08T04:38:30.740076+0000","last_peered":"2024-05-08T04:38:30.740076+0000","last_clean":"2024-05-08T04:38:30.740076+0000","last_became_active":"2024-05-08T04:38:29.741613+0000","last_became_peered":"2024-05-08T04:38:29.741613+0000","last_unstale":"2024-05-08T04:38:30.740076+0000","last_undegraded":"2024-05-08T04:38:30.740076+0000","last_fullsized":"2024-05-08T04:38:30.740076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:10.004004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739385+0000","last_change":"2024-05-08T04:38:28.747163+0000","last_active":"2024-05-08T04:38:30.739385+0000","last_peered":"2024-05-08T04:38:30.739385+0000","last_clean":"2024-05-08T04:38:30.739385+0000","last_became_active":"2024-05-08T04:38:28.746783+0000","last_became_peered":"2024-05-08T04:38:28.746783+0000","last_unstale":"2024-05-08T04:38:30.739385+0000","last_undegraded":"2024-05-08T04:38:30.739385+0000","last_fullsized":"2024-05-08T04:38:30.739385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877757+0000","last_change":"2024-05-08T04:38:29.737910+0000","last_active":"2024-05-08T04:38:30.877757+0000","last_peered":"2024-05-08T04:38:30.877757+0000","last_clean":"2024-05-08T04:38:30.877757+0000","last_became_active":"2024-05-08T04:38:29.736005+0000","last_became_peered":"2024-05-08T04:38:29.736005+0000","last_unstale":"2024-05-08T04:38:30.877757+0000","last_undegraded":"2024-05-08T04:38:30.877757+0000","last_fullsized":"2024-05-08T04:38:30.877757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739337+0000","last_change":"2024-05-08T04:38:28.740345+0000","last_active":"2024-05-08T04:38:30.739337+0000","last_peered":"2024-05-08T04:38:30.739337+0000","last_clean":"2024-05-08T04:38:30.739337+0000","last_became_active":"2024-05-08T04:38:28.740070+0000","last_became_peered":"2024-05-08T04:38:28.740070+0000","last_unstale":"2024-05-08T04:38:30.739337+0000","last_undegraded":"2024-05-08T04:38:30.739337+0000","last_fullsized":"2024-05-08T04:38:30.739337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877804+0000","last_change":"2024-05-08T04:38:29.737168+0000","last_active":"2024-05-08T04:38:30.877804+0000","last_peered":"2024-05-08T04:38:30.877804+0000","last_clean":"2024-05-08T04:38:30.877804+0000","last_became_active":"2024-05-08T04:38:29.736816+0000","last_became_peered":"2024-05-08T04:38:29.736816+0000","last_unstale":"2024-05-08T04:38:30.877804+0000","last_undegraded":"2024-05-08T04:38:30.877804+0000","last_fullsized":"2024-05-08T04:38:30.877804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201887+0000","last_change":"2024-05-08T04:38:28.754098+0000","last_active":"2024-05-08T04:38:31.201887+0000","last_peered":"2024-05-08T04:38:31.201887+0000","last_clean":"2024-05-08T04:38:31.201887+0000","last_became_active":"2024-05-08T04:38:28.753787+0000","last_became_peered":"2024-05-08T04:38:28.753787+0000","last_unstale":"2024-05-08T04:38:31.201887+0000","last_undegraded":"2024-05-08T04:38:31.201887+0000","last_fullsized":"2024-05-08T04:38:31.201887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:56.969185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746089+0000","last_change":"2024-05-08T04:38:29.747896+0000","last_active":"2024-05-08T04:38:30.746089+0000","last_peered":"2024-05-08T04:38:30.746089+0000","last_clean":"2024-05-08T04:38:30.746089+0000","last_became_active":"2024-05-08T04:38:29.747661+0000","last_became_peered":"2024-05-08T04:38:29.747661+0000","last_unstale":"2024-05-08T04:38:30.746089+0000","last_undegraded":"2024-05-08T04:38:30.746089+0000","last_fullsized":"2024-05-08T04:38:30.746089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741307+0000","last_change":"2024-05-08T04:38:29.742488+0000","last_active":"2024-05-08T04:38:30.741307+0000","last_peered":"2024-05-08T04:38:30.741307+0000","last_clean":"2024-05-08T04:38:30.741307+0000","last_became_active":"2024-05-08T04:38:29.738959+0000","last_became_peered":"2024-05-08T04:38:29.738959+0000","last_unstale":"2024-05-08T04:38:30.741307+0000","last_undegraded":"2024-05-08T04:38:30.741307+0000","last_fullsized":"2024-05-08T04:38:30.741307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201514+0000","last_change":"2024-05-08T04:38:28.758434+0000","last_active":"2024-05-08T04:38:31.201514+0000","last_peered":"2024-05-08T04:38:31.201514+0000","last_clean":"2024-05-08T04:38:31.201514+0000","last_became_active":"2024-05-08T04:38:28.758108+0000","last_became_peered":"2024-05-08T04:38:28.758108+0000","last_unstale":"2024-05-08T04:38:31.201514+0000","last_undegraded":"2024-05-08T04:38:31.201514+0000","last_fullsized":"2024-05-08T04:38:31.201514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741246+0000","last_change":"2024-05-08T04:38:29.740724+0000","last_active":"2024-05-08T04:38:30.741246+0000","last_peered":"2024-05-08T04:38:30.741246+0000","last_clean":"2024-05-08T04:38:30.741246+0000","last_became_active":"2024-05-08T04:38:29.739520+0000","last_became_peered":"2024-05-08T04:38:29.739520+0000","last_unstale":"2024-05-08T04:38:30.741246+0000","last_undegraded":"2024-05-08T04:38:30.741246+0000","last_fullsized":"2024-05-08T04:38:30.741246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201827+0000","last_change":"2024-05-08T04:38:28.753194+0000","last_active":"2024-05-08T04:38:31.201827+0000","last_peered":"2024-05-08T04:38:31.201827+0000","last_clean":"2024-05-08T04:38:31.201827+0000","last_became_active":"2024-05-08T04:38:28.752937+0000","last_became_peered":"2024-05-08T04:38:28.752937+0000","last_unstale":"2024-05-08T04:38:31.201827+0000","last_undegraded":"2024-05-08T04:38:31.201827+0000","last_fullsized":"2024-05-08T04:38:31.201827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201009+0000","last_change":"2024-05-08T04:38:29.738092+0000","last_active":"2024-05-08T04:38:31.201009+0000","last_peered":"2024-05-08T04:38:31.201009+0000","last_clean":"2024-05-08T04:38:31.201009+0000","last_became_active":"2024-05-08T04:38:29.737806+0000","last_became_peered":"2024-05-08T04:38:29.737806+0000","last_unstale":"2024-05-08T04:38:31.201009+0000","last_undegraded":"2024-05-08T04:38:31.201009+0000","last_fullsized":"2024-05-08T04:38:31.201009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878056+0000","last_change":"2024-05-08T04:38:28.732592+0000","last_active":"2024-05-08T04:38:30.878056+0000","last_peered":"2024-05-08T04:38:30.878056+0000","last_clean":"2024-05-08T04:38:30.878056+0000","last_became_active":"2024-05-08T04:38:28.732270+0000","last_became_peered":"2024-05-08T04:38:28.732270+0000","last_unstale":"2024-05-08T04:38:30.878056+0000","last_undegraded":"2024-05-08T04:38:30.878056+0000","last_fullsized":"2024-05-08T04:38:30.878056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878079+0000","last_change":"2024-05-08T04:38:29.751389+0000","last_active":"2024-05-08T04:38:30.878079+0000","last_peered":"2024-05-08T04:38:30.878079+0000","last_clean":"2024-05-08T04:38:30.878079+0000","last_became_active":"2024-05-08T04:38:29.751025+0000","last_became_peered":"2024-05-08T04:38:29.751025+0000","last_unstale":"2024-05-08T04:38:30.878079+0000","last_undegraded":"2024-05-08T04:38:30.878079+0000","last_fullsized":"2024-05-08T04:38:30.878079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201069+0000","last_change":"2024-05-08T04:38:28.745873+0000","last_active":"2024-05-08T04:38:31.201069+0000","last_peered":"2024-05-08T04:38:31.201069+0000","last_clean":"2024-05-08T04:38:31.201069+0000","last_became_active":"2024-05-08T04:38:28.745607+0000","last_became_peered":"2024-05-08T04:38:28.745607+0000","last_unstale":"2024-05-08T04:38:31.201069+0000","last_undegraded":"2024-05-08T04:38:31.201069+0000","last_fullsized":"2024-05-08T04:38:31.201069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:38:34.179 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:38:34.179 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T04:38:34.179 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T04:38:34.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T04:38:34.179 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T04:38:34.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 Processor -- start 2024-05-08T04:38:34.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 -- start start 2024-05-08T04:38:34.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5d40b26c0 0x7fc5d40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 0x7fc5d40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d406c770 con 0x7fc5d415ab60 2024-05-08T04:38:34.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40d0060 con 0x7fc5d40b26c0 2024-05-08T04:38:34.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5da600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d406ca50 con 0x7fc5d40cfbd0 2024-05-08T04:38:34.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5caa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5d40b26c0 0x7fc5d40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5d3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc5d415ab60 0x7fc5d4072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57964/0 (socket says 172.21.15.164:57964) 2024-05-08T04:38:34.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.351+0000 7fc5d3400640 1 -- 172.21.15.164:0/1325649121 learned_addr learned my addr 172.21.15.164:0/1325649121 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:34.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d3e00640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 0x7fc5d40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5caa00640 1 -- 172.21.15.164:0/1325649121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 msgr2=0x7fc5d40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5caa00640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 0x7fc5d40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5caa00640 1 -- 172.21.15.164:0/1325649121 >> v1:172.21.15.164:6789/0 conn(0x7fc5d415ab60 legacy=0x7fc5d4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5caa00640 1 -- 172.21.15.164:0/1325649121 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d406cd30 con 0x7fc5d40b26c0 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d2a00640 1 -- 172.21.15.164:0/1325649121 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180984226 0 0) 0x7fc5d406c770 con 0x7fc5d415ab60 2024-05-08T04:38:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d3e00640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 0x7fc5d40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:34.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5caa00640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d40cf690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097c0 tx=0x7fc5bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e94953f5f5303e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d2a00640 1 -- 172.21.15.164:0/1325649121 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0b0070 con 0x7fc5d40b26c0 2024-05-08T04:38:34.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d2a00640 1 -- 172.21.15.164:0/1325649121 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0ab600 con 0x7fc5d40b26c0 2024-05-08T04:38:34.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5d2a00640 1 -- 172.21.15.164:0/1325649121 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0ab9d0 con 0x7fc5d40b26c0 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 -- 172.21.15.164:0/1325649121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 msgr2=0x7fc5d40cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d40cf690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097c0 tx=0x7fc5bc01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 -- 172.21.15.164:0/1325649121 shutdown_connections 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d40cfbd0 0x7fc5d40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 --2- 172.21.15.164:0/1325649121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d40cf690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 -- 172.21.15.164:0/1325649121 >> 172.21.15.164:0/1325649121 conn(0x7fc5d406ac90 msgr2=0x7fc5d406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 -- 172.21.15.164:0/1325649121 shutdown_connections 2024-05-08T04:38:34.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.355+0000 7fc5da600640 1 -- 172.21.15.164:0/1325649121 wait complete. 2024-05-08T04:38:34.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 Processor -- start 2024-05-08T04:38:34.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- start start 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5d40cfbd0 0x7fc5d41490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d415ab60 0x7fc5d414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40aeeb0 con 0x7fc5d40b26c0 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40aed30 con 0x7fc5d40cfbd0 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4167970 con 0x7fc5d415ab60 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41194/0 (socket says 172.21.15.164:41194) 2024-05-08T04:38:34.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 -- 172.21.15.164:0/1887872302 learned_addr learned my addr 172.21.15.164:0/1887872302 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 -- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d415ab60 msgr2=0x7fc5d414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d415ab60 0x7fc5d414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 -- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5d40cfbd0 msgr2=0x7fc5d41490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5d40cfbd0 0x7fc5d41490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 -- 172.21.15.164:0/1887872302 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d414a1d0 con 0x7fc5d40b26c0 2024-05-08T04:38:34.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5d3400640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c7c0 tx=0x7fc5c400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4018070 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c400a640 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400aa30 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d414a4b0 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.359+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d414aa70 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d4069d60 con 0x7fc5d40b26c0 2024-05-08T04:38:34.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5c400abd0 con 0x7fc5d40b26c0 2024-05-08T04:38:34.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5cbe00640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 0x7fc5b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c4084730 con 0x7fc5d40b26c0 2024-05-08T04:38:34.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5caa00640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 0x7fc5b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.363+0000 7fc5caa00640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 0x7fc5b0069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ad000 tx=0x7fc5bc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.367+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5c4055740 con 0x7fc5d40b26c0 2024-05-08T04:38:34.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.559+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc5d406d980 con 0x7fc5d40b26c0 2024-05-08T04:38:34.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.559+0000 7fc5cbe00640 1 -- 172.21.15.164:0/1887872302 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165734 (secure 0 0 0) 0x7fc5c4055040 con 0x7fc5d40b26c0 2024-05-08T04:38:34.565 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:34.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 msgr2=0x7fc5b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 0x7fc5b0069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ad000 tx=0x7fc5bc004740 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 msgr2=0x7fc5d415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c7c0 tx=0x7fc5c400cc80 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 shutdown_connections 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5b0066f60 0x7fc5b0069420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5d415ab60 0x7fc5d414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5d40cfbd0 0x7fc5d41490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 --2- 172.21.15.164:0/1887872302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5d40b26c0 0x7fc5d415b050 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 >> 172.21.15.164:0/1887872302 conn(0x7fc5d406ac90 msgr2=0x7fc5d40b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 shutdown_connections 2024-05-08T04:38:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.563+0000 7fc5da600640 1 -- 172.21.15.164:0/1887872302 wait complete. 2024-05-08T04:38:34.585 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":572599534},{"type":"v1","addr":"172.21.15.164:6833","nonce":572599534}]},"active_addr":"172.21.15.164:6833/572599534","active_change":"2024-05-08T04:38:09.418600+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2921427729}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":897774693}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":142317874}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":972950359}]}]} 2024-05-08T04:38:34.589 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T04:38:34.589 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T04:38:34.589 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:34.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 Processor -- start 2024-05-08T04:38:34.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 -- start start 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f62180d0380 0x7f62180ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180ce6f0 0x7f621806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180d0810 con 0x7f621815db90 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180ceb60 con 0x7f62180d0380 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621815df70 con 0x7f62180ce6f0 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f62180d0380 0x7f62180ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f621815db90 0x7f62180cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57976/0 (socket says 172.21.15.164:57976) 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216a00640 1 -- 172.21.15.164:0/4007723925 learned_addr learned my addr 172.21.15.164:0/4007723925 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6217400640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180ce6f0 0x7f621806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216000640 1 -- 172.21.15.164:0/4007723925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180ce6f0 msgr2=0x7f621806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216000640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180ce6f0 0x7f621806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216000640 1 -- 172.21.15.164:0/4007723925 >> v1:172.21.15.164:6789/0 conn(0x7f621815db90 legacy=0x7f62180cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.723+0000 7f6216000640 1 -- 172.21.15.164:0/4007723925 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180cf5c0 con 0x7f62180d0380 2024-05-08T04:38:34.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6216000640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180d0380 0x7f62180ce1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62040097c0 tx=0x7f620401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46902d882cfb104a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6215600640 1 -- 172.21.15.164:0/4007723925 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040b2070 con 0x7f62180d0380 2024-05-08T04:38:34.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6215600640 1 -- 172.21.15.164:0/4007723925 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621806f210 con 0x7f62180d0380 2024-05-08T04:38:34.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6215600640 1 -- 172.21.15.164:0/4007723925 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040ad860 con 0x7f62180d0380 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- 172.21.15.164:0/4007723925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180d0380 msgr2=0x7f62180ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180d0380 0x7f62180ce1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62040097c0 tx=0x7f620401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- 172.21.15.164:0/4007723925 shutdown_connections 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180ce6f0 0x7f621806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 --2- 172.21.15.164:0/4007723925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180d0380 0x7f62180ce1b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- 172.21.15.164:0/4007723925 >> 172.21.15.164:0/4007723925 conn(0x7f621809aa20 msgr2=0x7f62180b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- 172.21.15.164:0/4007723925 shutdown_connections 2024-05-08T04:38:34.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- 172.21.15.164:0/4007723925 wait complete. 2024-05-08T04:38:34.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 Processor -- start 2024-05-08T04:38:34.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.727+0000 7f6217e00640 1 -- start start 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180d0380 0x7f621806bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f621815db90 0x7f621806c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180b3e20 con 0x7f62180ce6f0 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180cd580 con 0x7f621815db90 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62180b3fa0 con 0x7f62180d0380 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41212/0 (socket says 172.21.15.164:41212) 2024-05-08T04:38:34.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 -- 172.21.15.164:0/1231636856 learned_addr learned my addr 172.21.15.164:0/1231636856 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216000640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180d0380 0x7f621806bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217400640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f621815db90 0x7f621806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 -- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180d0380 msgr2=0x7f621806bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180d0380 0x7f621806bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 -- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f621815db90 msgr2=0x7f621806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f621815db90 0x7f621806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 -- 172.21.15.164:0/1231636856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180b5c10 con 0x7f62180ce6f0 2024-05-08T04:38:34.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6216a00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f620800c420 tx=0x7f620800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208017070 con 0x7f62180ce6f0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208005cf0 con 0x7f62180ce6f0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62080072f0 con 0x7f62180ce6f0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62180b5ef0 con 0x7f62180ce6f0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62180b64b0 con 0x7f62180ce6f0 2024-05-08T04:38:34.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621804ef30 con 0x7f62180ce6f0 2024-05-08T04:38:34.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.731+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f620801a070 con 0x7f62180ce6f0 2024-05-08T04:38:34.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.735+0000 7f620f400640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 0x7f61f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:34.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.735+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6208084340 con 0x7f62180ce6f0 2024-05-08T04:38:34.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.735+0000 7f6216000640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 0x7f61f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:34.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.735+0000 7f6216000640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 0x7f61f8069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62040af000 tx=0x7f6204008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:34.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.739+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62180d0810 con 0x7f62180ce6f0 2024-05-08T04:38:34.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.895+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62181687e0 con 0x7f62180ce6f0 2024-05-08T04:38:34.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.895+0000 7f620f400640 1 -- 172.21.15.164:0/1231636856 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f6208054d00 con 0x7f62180ce6f0 2024-05-08T04:38:34.899 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:34.899 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 msgr2=0x7f61f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 0x7f61f8069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62040af000 tx=0x7f6204008040 comp rx=0 tx=0).stop 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 msgr2=0x7f62181583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f620800c420 tx=0x7f620800c8e0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 shutdown_connections 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f61f8066f60 0x7f61f8069420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62180d0380 0x7f621806bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f621815db90 0x7f621806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 --2- 172.21.15.164:0/1231636856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62180ce6f0 0x7f62181583e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:34.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 >> 172.21.15.164:0/1231636856 conn(0x7f621809aa20 msgr2=0x7f6218161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:34.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 shutdown_connections 2024-05-08T04:38:34.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:34.899+0000 7f6217e00640 1 -- 172.21.15.164:0/1231636856 wait complete. 2024-05-08T04:38:34.919 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T04:38:34.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:35.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.051+0000 7f1734200640 1 Processor -- start 2024-05-08T04:38:35.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.051+0000 7f1734200640 1 -- start start 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1734200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f172c0aee30 0x7f172c0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1734200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1734200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c06e0e0 con 0x7f172c15b390 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1734200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c15e8f0 con 0x7f172c0aee30 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1734200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15a8c0 con 0x7f172c0cef80 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1731800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f172c15b390 0x7f172c06d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:57982/0 (socket says 172.21.15.164:57982) 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1730e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f172c0aee30 0x7f172c0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1731800640 1 -- 172.21.15.164:0/510048111 learned_addr learned my addr 172.21.15.164:0/510048111 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1732200640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1730e00640 1 -- 172.21.15.164:0/510048111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 msgr2=0x7f172c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1730e00640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c06c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1730e00640 1 -- 172.21.15.164:0/510048111 >> v1:172.21.15.164:6789/0 conn(0x7f172c15b390 legacy=0x7f172c06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1730e00640 1 -- 172.21.15.164:0/510048111 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c15ed80 con 0x7f172c0aee30 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f172be00640 1 -- 172.21.15.164:0/510048111 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360629687 0 0) 0x7f172c06e0e0 con 0x7f172c15b390 2024-05-08T04:38:35.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.055+0000 7f1732200640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c06c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:35.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1730e00640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c0aee30 0x7f172c0cea40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17180097c0 tx=0x7f171801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9925113640207 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f172be00640 1 -- 172.21.15.164:0/510048111 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180b0070 con 0x7f172c0aee30 2024-05-08T04:38:35.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f172be00640 1 -- 172.21.15.164:0/510048111 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17180ab600 con 0x7f172c0aee30 2024-05-08T04:38:35.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f172be00640 1 -- 172.21.15.164:0/510048111 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180ab9d0 con 0x7f172c0aee30 2024-05-08T04:38:35.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- 172.21.15.164:0/510048111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c0aee30 msgr2=0x7f172c0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c0aee30 0x7f172c0cea40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17180097c0 tx=0x7f171801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- 172.21.15.164:0/510048111 shutdown_connections 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c06c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 --2- 172.21.15.164:0/510048111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c0aee30 0x7f172c0cea40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- 172.21.15.164:0/510048111 >> 172.21.15.164:0/510048111 conn(0x7f172c06ac90 msgr2=0x7f172c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- 172.21.15.164:0/510048111 shutdown_connections 2024-05-08T04:38:35.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- 172.21.15.164:0/510048111 wait complete. 2024-05-08T04:38:35.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 Processor -- start 2024-05-08T04:38:35.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.059+0000 7f1734200640 1 -- start start 2024-05-08T04:38:35.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f172c0aee30 0x7f172c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c06c910 con 0x7f172c15b390 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15b770 con 0x7f172c0aee30 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c06ca90 con 0x7f172c0cef80 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41234/0 (socket says 172.21.15.164:41234) 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 -- 172.21.15.164:0/2965522990 learned_addr learned my addr 172.21.15.164:0/2965522990 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1730e00640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1731800640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f172c0aee30 0x7f172c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 -- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 msgr2=0x7f172c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 -- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f172c0aee30 msgr2=0x7f172c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f172c0aee30 0x7f172c15bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 -- 172.21.15.164:0/2965522990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0a98d0 con 0x7f172c15b390 2024-05-08T04:38:35.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1730e00640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1732200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f172400e840 tx=0x7f172400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724017070 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172400c6d0 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172400f040 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c0a9bb0 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c0aa170 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c069d60 con 0x7f172c15b390 2024-05-08T04:38:35.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.063+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f172401a070 con 0x7f172c15b390 2024-05-08T04:38:35.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.067+0000 7f1729600640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 0x7f170c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.067+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1724084c30 con 0x7f172c15b390 2024-05-08T04:38:35.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.067+0000 7f1731800640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 0x7f170c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.067+0000 7f1731800640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 0x7f170c069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720007ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.071+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1724055c30 con 0x7f172c15b390 2024-05-08T04:38:35.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.223+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f172c0d0580 con 0x7f172c15b390 2024-05-08T04:38:35.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.223+0000 7f1729600640 1 -- 172.21.15.164:0/2965522990 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f1724055530 con 0x7f172c15b390 2024-05-08T04:38:35.226 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:35.226 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:35.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 msgr2=0x7f170c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 0x7f170c069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720007ee0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 msgr2=0x7f172c141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f172400e840 tx=0x7f172400ed00 comp rx=0 tx=0).stop 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 shutdown_connections 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f170c066f60 0x7f170c069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f172c0cef80 0x7f172c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f172c15b390 0x7f172c141dc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 --2- 172.21.15.164:0/2965522990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f172c0aee30 0x7f172c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 >> 172.21.15.164:0/2965522990 conn(0x7f172c06ac90 msgr2=0x7f172c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 shutdown_connections 2024-05-08T04:38:35.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.227+0000 7f1734200640 1 -- 172.21.15.164:0/2965522990 wait complete. 2024-05-08T04:38:35.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T04:38:35.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T04:38:35.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T04:38:35.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T04:38:35.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T04:38:35.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T04:38:35.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T04:38:35.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T04:38:35.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f1ff1e00640 1 Processor -- start 2024-05-08T04:38:35.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 Processor -- start 2024-05-08T04:38:35.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f1ff1e00640 1 -- start start 2024-05-08T04:38:35.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 -- start start 2024-05-08T04:38:35.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f238415de30 0x7f23840ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 0x7f23840ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2384162750 con 0x7f23840cab70 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840c5580 con 0x7f238415de30 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.491+0000 7f238a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23840cb000 con 0x7f23840ce6b0 2024-05-08T04:38:35.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1febe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1ff0a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1fec159d00 0x7f1fec070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:60208/0 (socket says 172.21.15.164:60208) 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1ff0a00640 1 -- 172.21.15.164:0/869123599 learned_addr learned my addr 172.21.15.164:0/869123599 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fec0cdb80 con 0x7f1fec0ce660 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f238415de30 0x7f23840ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f238415de30 0x7f23840ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41236/0 (socket says 172.21.15.164:41236) 2024-05-08T04:38:35.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 -- 172.21.15.164:0/180004047 learned_addr learned my addr 172.21.15.164:0/180004047 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fec0cde60 con 0x7f1fec159d00 2024-05-08T04:38:35.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0ce140 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2383e00640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 0x7f23840ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1febe00640 1 -- 172.21.15.164:0/869123599 >> v1:172.21.15.172:6789/0 conn(0x7f1fec159d00 legacy=0x7f1fec070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 -- 172.21.15.164:0/180004047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 msgr2=0x7f23840ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1febe00640 1 -- 172.21.15.164:0/869123599 >> v1:172.21.15.164:6789/0 conn(0x7f1fec0ce660 legacy=0x7f1fec0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f1febe00640 1 -- 172.21.15.164:0/869123599 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec15a0e0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 0x7f23840ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 -- 172.21.15.164:0/180004047 >> v1:172.21.15.164:6789/0 conn(0x7f23840cab70 legacy=0x7f2384161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2388c00640 1 -- 172.21.15.164:0/180004047 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384162470 con 0x7f238415de30 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.495+0000 7f2383e00640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 0x7f23840ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 Processor -- start 2024-05-08T04:38:35.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 -- start start 2024-05-08T04:38:35.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ab40e3310 0x7f3ab403e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f1febe00640 1 --2- 172.21.15.164:0/869123599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec0ca300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40097f0 tx=0x7f1fe401ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0e5004e5167052 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 0x7f3ab40540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab40e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab403c000 con 0x7f3ab4053c50 2024-05-08T04:38:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f14ec200640 1 Processor -- start 2024-05-08T04:38:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f2c66200640 1 Processor -- start 2024-05-08T04:38:35.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f1feb400640 1 -- 172.21.15.164:0/869123599 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40b5070 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab4057e20 con 0x7f3ab40e3310 2024-05-08T04:38:35.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ac0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4058100 con 0x7f3ab40e6c70 2024-05-08T04:38:35.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ab9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 0x7f3ab40540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f1feb400640 1 -- 172.21.15.164:0/869123599 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe40a89d0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f1feb400640 1 -- 172.21.15.164:0/869123599 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40a8da0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f2388c00640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f238415de30 0x7f23840ce170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f237c009e50 tx=0x7f237c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b499564eab93205 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ab9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 0x7f3ab40540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41252/0 (socket says 172.21.15.164:41252) 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ab9400640 1 -- 172.21.15.164:0/4186347915 learned_addr learned my addr 172.21.15.164:0/4186347915 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab40e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f14ec200640 1 -- start start 2024-05-08T04:38:35.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f2c66200640 1 -- start start 2024-05-08T04:38:35.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3ab9e00640 1 --2- 172.21.15.164:0/4186347915 >> v2:172.21.15.172:3300/0 conn(0x7f3ab40e3310 0x7f3ab403e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 -- 172.21.15.164:0/4186347915 >> v2:172.21.15.172:3300/0 conn(0x7f3ab40e3310 msgr2=0x7f3ab403e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 --2- 172.21.15.164:0/4186347915 >> v2:172.21.15.172:3300/0 conn(0x7f3ab40e3310 0x7f3ab403e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 -- 172.21.15.164:0/4186347915 >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 msgr2=0x7f3ab40540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 --2- 172.21.15.164:0/4186347915 >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 0x7f3ab40540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.499+0000 7f3aba800640 1 -- 172.21.15.164:0/4186347915 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40583e0 con 0x7f3ab40e6c70 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 msgr2=0x7f1fec0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 --2- 172.21.15.164:0/869123599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec0ca300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40097f0 tx=0x7f1fe401ecd0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7fed69e00640 1 Processor -- start 2024-05-08T04:38:35.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 shutdown_connections 2024-05-08T04:38:35.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 --2- 172.21.15.164:0/869123599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec0ca300 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 >> 172.21.15.164:0/869123599 conn(0x7f1fec0b1380 msgr2=0x7f1fec0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2383400640 1 -- 172.21.15.164:0/180004047 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0a55e0 con 0x7f238415de30 2024-05-08T04:38:35.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2383400640 1 -- 172.21.15.164:0/180004047 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0a5780 con 0x7f238415de30 2024-05-08T04:38:35.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2383400640 1 -- 172.21.15.164:0/180004047 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0a5b50 con 0x7f238415de30 2024-05-08T04:38:35.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7fed69e00640 1 -- start start 2024-05-08T04:38:35.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c66200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c600c9ec0 0x7f2c600ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c66200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600cd8a0 con 0x7f2c600cc430 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c66200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600cdb80 con 0x7f2c60159f60 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c66200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600cde60 con 0x7f2c600c9ec0 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c64e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2c600cc430 0x7f2c600cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58010/0 (socket says 172.21.15.164:58010) 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c64e00640 1 -- 172.21.15.164:0/3398281233 learned_addr learned my addr 172.21.15.164:0/3398281233 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c65800640 1 --2- 172.21.15.164:0/3398281233 >> v2:172.21.15.172:3300/0 conn(0x7f2c600c9ec0 0x7f2c600ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ec200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 -- 172.21.15.164:0/180004047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f238415de30 msgr2=0x7f23840ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f238415de30 0x7f23840ce170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f237c009e50 tx=0x7f237c01c960 comp rx=0 tx=0).stop 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ec200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e40cdd30 0x7f14e40cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ec200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e40ce7d0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ec200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40ceab0 con 0x7f14e40cdd30 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ec200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40ced90 con 0x7f14e406b3a0 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e40cdd30 0x7f14e40cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 shutdown_connections 2024-05-08T04:38:35.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14e9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e40cdd30 0x7f14e40cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38586/0 (socket says 172.21.15.164:38586) 2024-05-08T04:38:35.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 -- 172.21.15.164:0/3009692254 learned_addr learned my addr 172.21.15.164:0/3009692254 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 -- 172.21.15.164:0/180004047 shutdown_connections 2024-05-08T04:38:35.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23840ce6b0 0x7f23840ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 --2- 172.21.15.164:0/180004047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f238415de30 0x7f23840ce170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f238a000640 1 -- 172.21.15.164:0/180004047 >> 172.21.15.164:0/180004047 conn(0x7f23840b1380 msgr2=0x7f23840b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f238a000640 1 -- 172.21.15.164:0/180004047 shutdown_connections 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7fed69e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fed64161f40 0x7fed640b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7fed69e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fed640af040 0x7fed640cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7fed62a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fed64161f40 0x7fed640b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed69e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed64162320 con 0x7fed640cc720 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed69e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6406c0f0 con 0x7fed640af040 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed69e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640b4550 con 0x7fed64161f40 2024-05-08T04:38:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fed64161f40 0x7fed640b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38602/0 (socket says 172.21.15.164:38602) 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 -- 172.21.15.164:0/1121152889 learned_addr learned my addr 172.21.15.164:0/1121152889 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62000640 1 --2- 172.21.15.164:0/1121152889 >> v2:172.21.15.164:3300/0 conn(0x7fed640af040 0x7fed640cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 -- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 msgr2=0x7f14e40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e40b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f14ea200640 1 -- 172.21.15.164:0/3009692254 >> v1:172.21.15.164:6789/0 conn(0x7f14e40cd7e0 legacy=0x7f14e40cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f14ea200640 1 -- 172.21.15.164:0/3009692254 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40cf070 con 0x7f14e40cdd30 2024-05-08T04:38:35.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f14e9800640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c65800640 1 -- 172.21.15.164:0/3398281233 >> v1:172.21.15.172:6789/0 conn(0x7f2c60159f60 legacy=0x7f2c60070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f238a000640 1 -- 172.21.15.164:0/180004047 wait complete. 2024-05-08T04:38:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 -- 172.21.15.164:0/1121152889 >> v2:172.21.15.164:3300/0 conn(0x7fed640af040 msgr2=0x7fed640cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 --2- 172.21.15.164:0/1121152889 >> v2:172.21.15.164:3300/0 conn(0x7fed640af040 0x7fed640cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 -- 172.21.15.164:0/1121152889 >> v1:172.21.15.172:6789/0 conn(0x7fed640cc720 legacy=0x7fed6406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fed62a00640 1 -- 172.21.15.164:0/1121152889 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640b4830 con 0x7fed64161f40 2024-05-08T04:38:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3aba800640 1 --2- 172.21.15.164:0/4186347915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab40e7100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4056930 tx=0x7f3aa801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77479c0e9f904a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f1ff1e00640 1 -- 172.21.15.164:0/869123599 wait complete. 2024-05-08T04:38:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c65800640 1 -- 172.21.15.164:0/3398281233 >> v1:172.21.15.164:6789/0 conn(0x7f2c600cc430 legacy=0x7f2c600cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.503+0000 7f2c65800640 1 -- 172.21.15.164:0/3398281233 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600ce140 con 0x7f2c600c9ec0 2024-05-08T04:38:35.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3ab8a00640 1 -- 172.21.15.164:0/4186347915 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b0070 con 0x7f3ab40e6c70 2024-05-08T04:38:35.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3ab8a00640 1 -- 172.21.15.164:0/4186347915 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80ab600 con 0x7f3ab40e6c70 2024-05-08T04:38:35.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3ab8a00640 1 -- 172.21.15.164:0/4186347915 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b0070 con 0x7f3ab40e6c70 2024-05-08T04:38:35.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f1ff1e00640 1 Processor -- start 2024-05-08T04:38:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f238a000640 1 Processor -- start 2024-05-08T04:38:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fab7c800640 1 Processor -- start 2024-05-08T04:38:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f1ff1e00640 1 -- start start 2024-05-08T04:38:35.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4186347915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 msgr2=0x7f3ab40e7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4186347915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab40e7100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4056930 tx=0x7f3aa801c480 comp rx=0 tx=0).stop 2024-05-08T04:38:35.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.507+0000 7fab7c800640 1 -- start start 2024-05-08T04:38:35.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 -- start start 2024-05-08T04:38:35.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4186347915 shutdown_connections 2024-05-08T04:38:35.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4186347915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab40e7100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4186347915 >> v2:172.21.15.164:3300/0 conn(0x7f3ab4053c50 0x7f3ab40540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4186347915 >> v2:172.21.15.172:3300/0 conn(0x7f3ab40e3310 0x7f3ab403e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4186347915 >> 172.21.15.164:0/4186347915 conn(0x7f3ab4055a20 msgr2=0x7f3ab4055e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fec0ce660 0x7f1fec1f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fec159d00 0x7f1fec0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0cad90 con 0x7f1fec0ce660 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0cac10 con 0x7f1fec159d00 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0ca840 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4186347915 shutdown_connections 2024-05-08T04:38:35.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c65800640 1 --2- 172.21.15.164:0/3398281233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c600c9ec0 0x7f2c600ca370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d90 tx=0x7f2c5801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7258cc3a7e2eb5f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f14ea200640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cdd30 0x7f14e40cfa60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e50 tx=0x7f14dc01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4988dc4c4e2cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1febe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fec0ce660 0x7f1fec1f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7c800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab7415be50 0x7fab74160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7c800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab740cd9f0 con 0x7fab7415ba70 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7c800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab740cdcd0 con 0x7fab740cfda0 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7c800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7415fe70 con 0x7fab7415be50 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fed62a00640 1 --2- 172.21.15.164:0/1121152889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed64161f40 0x7fed640b27c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed6406bbe0 tx=0x7fed4c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2613d27f29d43e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1febe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fec0ce660 0x7f1fec1f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41262/0 (socket says 172.21.15.164:41262) 2024-05-08T04:38:35.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1febe00640 1 -- 172.21.15.164:0/1681234656 learned_addr learned my addr 172.21.15.164:0/1681234656 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 0x7f238415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23840ce6b0 0x7f2384154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f14e3e00640 1 -- 172.21.15.164:0/3009692254 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0a5810 con 0x7f14e40cdd30 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f14e3e00640 1 -- 172.21.15.164:0/3009692254 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc0a59b0 con 0x7f14e40cdd30 2024-05-08T04:38:35.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f14e3e00640 1 -- 172.21.15.164:0/3009692254 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0a5810 con 0x7f14e40cdd30 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c5f400640 1 -- 172.21.15.164:0/3398281233 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580b2070 con 0x7f2c600c9ec0 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c5f400640 1 -- 172.21.15.164:0/3398281233 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c580a5860 con 0x7f2c600c9ec0 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c5f400640 1 -- 172.21.15.164:0/3398281233 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580a5c30 con 0x7f2c600c9ec0 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab79400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fab740cfda0 0x7fab740b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:60226/0 (socket says 172.21.15.164:60226) 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab79400640 1 -- 172.21.15.164:0/2201317227 learned_addr learned my addr 172.21.15.164:0/2201317227 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7faa4e000640 1 Processor -- start 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 -- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fec159d00 msgr2=0x7f1fec0c6630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fec159d00 0x7f1fec0c6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 -- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fec0ce660 msgr2=0x7f1fec1f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fec0ce660 0x7f1fec1f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec1f90e0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f238415de30 0x7f238414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238416af50 con 0x7f23840ce6b0 2024-05-08T04:38:35.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238416add0 con 0x7f23840cab70 2024-05-08T04:38:35.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f238a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238416b0d0 con 0x7f238415de30 2024-05-08T04:38:35.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fed61600640 1 -- 172.21.15.164:0/1121152889 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0aa610 con 0x7fed64161f40 2024-05-08T04:38:35.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7a800640 1 --2- 172.21.15.164:0/2201317227 >> v2:172.21.15.164:3300/0 conn(0x7fab7415be50 0x7fab74160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fed61600640 1 -- 172.21.15.164:0/1121152889 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0af070 con 0x7fed64161f40 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fed61600640 1 -- 172.21.15.164:0/1121152889 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0aa9d0 con 0x7fed64161f40 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2388c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 0x7f238415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2388c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 0x7f238415b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38624/0 (socket says 172.21.15.164:38624) 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2388c00640 1 -- 172.21.15.164:0/326027315 learned_addr learned my addr 172.21.15.164:0/326027315 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2389600640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f238415de30 0x7f238414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c66200640 1 -- 172.21.15.164:0/3398281233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c600c9ec0 msgr2=0x7f2c600ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f2c66200640 1 --2- 172.21.15.164:0/3398281233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c600c9ec0 0x7f2c600ca370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d90 tx=0x7f2c5801c800 comp rx=0 tx=0).stop 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7a800640 1 -- 172.21.15.164:0/2201317227 >> v1:172.21.15.172:6789/0 conn(0x7fab740cfda0 legacy=0x7fab740b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7a800640 1 -- 172.21.15.164:0/2201317227 >> v1:172.21.15.164:6789/0 conn(0x7fab7415ba70 legacy=0x7fab740cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7fab7a800640 1 -- 172.21.15.164:0/2201317227 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7415f820 con 0x7fab7415be50 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7faa4e000640 1 -- start start 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1ff0a00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec1561f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00ca70 tx=0x7f1fdc00ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc017070 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00ab10 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc012400 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 -- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f238415de30 msgr2=0x7f238414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4186347915 wait complete. 2024-05-08T04:38:35.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.511+0000 7f3ac0e00640 1 Processor -- start 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- 172.21.15.164:0/1121152889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed64161f40 msgr2=0x7fed640b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 --2- 172.21.15.164:0/1121152889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed64161f40 0x7fed640b27c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed6406bbe0 tx=0x7fed4c01c4d0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- 172.21.15.164:0/1121152889 shutdown_connections 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 --2- 172.21.15.164:0/1121152889 >> v2:172.21.15.164:3300/0 conn(0x7fed640af040 0x7fed640cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 --2- 172.21.15.164:0/1121152889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed64161f40 0x7fed640b27c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- 172.21.15.164:0/1121152889 >> 172.21.15.164:0/1121152889 conn(0x7fed640b1690 msgr2=0x7fed640b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f238415de30 0x7f238414d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 -- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23840ce6b0 msgr2=0x7f2384154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23840ce6b0 0x7f2384154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238415bf60 con 0x7f23840cab70 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- 172.21.15.164:0/1121152889 shutdown_connections 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 -- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cdd30 msgr2=0x7f14e40cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cdd30 0x7f14e40cfa60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e50 tx=0x7f14dc01ca30 comp rx=0 tx=0).stop 2024-05-08T04:38:35.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec0c7cb0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa4e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faa480aedb0 0x7faa480cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa4e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480ceac0 con 0x7faa48163540 2024-05-08T04:38:35.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa4e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480cc1e0 con 0x7faa480cf6c0 2024-05-08T04:38:35.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa4e000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa480ce860 con 0x7faa480aedb0 2024-05-08T04:38:35.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f3ac0e00640 1 -- start start 2024-05-08T04:38:35.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2388c00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 0x7f238415b860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f237c01b010 tx=0x7f237c019700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 -- 172.21.15.164:0/3009692254 shutdown_connections 2024-05-08T04:38:35.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- 172.21.15.164:0/1121152889 wait complete. 2024-05-08T04:38:35.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec0c8270 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cdd30 0x7f14e40cfa60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2c66200640 1 -- 172.21.15.164:0/3398281233 shutdown_connections 2024-05-08T04:38:35.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2c66200640 1 --2- 172.21.15.164:0/3398281233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c600c9ec0 0x7f2c600ca370 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2c66200640 1 -- 172.21.15.164:0/3398281233 >> 172.21.15.164:0/3398281233 conn(0x7f2c600b1380 msgr2=0x7f2c600b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa47e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7faa480cf6c0 0x7faa4806cfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:60232/0 (socket says 172.21.15.164:60232) 2024-05-08T04:38:35.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa47e00640 1 -- 172.21.15.164:0/2765895935 learned_addr learned my addr 172.21.15.164:0/2765895935 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 --2- 172.21.15.164:0/3009692254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 -- 172.21.15.164:0/3009692254 >> 172.21.15.164:0/3009692254 conn(0x7f14e4066fa0 msgr2=0x7f14e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2c66200640 1 -- 172.21.15.164:0/3398281233 shutdown_connections 2024-05-08T04:38:35.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0bc020 con 0x7f23840cab70 2024-05-08T04:38:35.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f238a000640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238415c240 con 0x7f23840cab70 2024-05-08T04:38:35.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fec04ef30 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa46000640 1 -- 172.21.15.164:0/2765895935 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971845843 0 0) 0x7faa480cc1e0 con 0x7faa480cf6c0 2024-05-08T04:38:35.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 -- 172.21.15.164:0/3009692254 shutdown_connections 2024-05-08T04:38:35.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f14ec200640 1 -- 172.21.15.164:0/3009692254 wait complete. 2024-05-08T04:38:35.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa46000640 1 -- 172.21.15.164:0/2765895935 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2c003530 con 0x7faa480cf6c0 2024-05-08T04:38:35.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7faa47400640 1 --2- 172.21.15.164:0/2765895935 >> v2:172.21.15.164:3300/0 conn(0x7faa480aedb0 0x7faa480cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 Processor -- start 2024-05-08T04:38:35.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fab7a800640 1 --2- 172.21.15.164:0/2201317227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415be50 0x7fab74160300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab70009d90 tx=0x7fab7001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5901499e630cc536 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7fed69e00640 1 -- start start 2024-05-08T04:38:35.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2c66200640 1 -- 172.21.15.164:0/3398281233 wait complete. 2024-05-08T04:38:35.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f238a000640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238415c800 con 0x7f23840cab70 2024-05-08T04:38:35.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0a5740 con 0x7f23840cab70 2024-05-08T04:38:35.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0bc020 con 0x7f23840cab70 2024-05-08T04:38:35.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f238a000640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f235c001040 con 0x7f23840cab70 2024-05-08T04:38:35.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f3ac0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 0x7f3ab40e2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.515+0000 7f3ac0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab403d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab78a00640 1 -- 172.21.15.164:0/2201317227 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700b2070 con 0x7fab7415be50 2024-05-08T04:38:35.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 0x7f3ab40e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 0x7f3ab40e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38632/0 (socket says 172.21.15.164:38632) 2024-05-08T04:38:35.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab78a00640 1 -- 172.21.15.164:0/2201317227 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab700a58d0 con 0x7fab7415be50 2024-05-08T04:38:35.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab78a00640 1 -- 172.21.15.164:0/2201317227 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700a5ca0 con 0x7fab7415be50 2024-05-08T04:38:35.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7faa47400640 1 -- 172.21.15.164:0/2765895935 >> v1:172.21.15.172:6789/0 conn(0x7faa480cf6c0 legacy=0x7faa4806cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fdc01a070 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 -- 172.21.15.164:0/4009920216 learned_addr learned my addr 172.21.15.164:0/4009920216 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ab40e0fa0 0x7f3ab403daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab403ef40 con 0x7f3ab40e0fa0 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab403edc0 con 0x7f3ab4053c50 2024-05-08T04:38:35.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4007040 con 0x7f3ab40e6c70 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 0x7fed64163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed64161f40 0x7fed640ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed64155b40 0x7fed641572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406cf70 con 0x7fed64161f40 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640b2d00 con 0x7fed640cc720 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406d0f0 con 0x7fed64155b40 2024-05-08T04:38:35.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9400640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab403d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 0x7fed64163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 0x7fed64163ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38634/0 (socket says 172.21.15.164:38634) 2024-05-08T04:38:35.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 -- 172.21.15.164:0/3209526361 learned_addr learned my addr 172.21.15.164:0/3209526361 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed63e00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed64155b40 0x7fed641572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 -- 172.21.15.164:0/2201317227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415be50 msgr2=0x7fab74160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 --2- 172.21.15.164:0/2201317227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415be50 0x7fab74160300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab70009d90 tx=0x7fab7001c920 comp rx=0 tx=0).stop 2024-05-08T04:38:35.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 Processor -- start 2024-05-08T04:38:35.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 -- 172.21.15.164:0/2201317227 shutdown_connections 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 --2- 172.21.15.164:0/2201317227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415be50 0x7fab74160300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 -- 172.21.15.164:0/2201317227 >> 172.21.15.164:0/2201317227 conn(0x7fab7406ac90 msgr2=0x7fab7406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 -- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 msgr2=0x7f3ab403d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ab40e6c70 0x7f3ab403d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 -- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed64155b40 msgr2=0x7fed641572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed64155b40 0x7fed641572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 -- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed64161f40 msgr2=0x7fed640ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed64161f40 0x7fed640ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640ce620 con 0x7fed640cc720 2024-05-08T04:38:35.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 -- 172.21.15.164:0/2201317227 shutdown_connections 2024-05-08T04:38:35.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 -- start start 2024-05-08T04:38:35.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 -- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ab40e0fa0 msgr2=0x7f3ab403daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ab40e0fa0 0x7f3ab403daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40e1480 con 0x7f3ab4053c50 2024-05-08T04:38:35.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1fcc066fe0 0x7f1fcc0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fdc0849c0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab9e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 0x7f3ab40e2520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0693f0 tx=0x7f3abc0680b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1fcc06f200 0x7f1fcc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fcc071cd0 con 0x7f1fcc06f200 2024-05-08T04:38:35.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fdc0892c0 con 0x7f1fec0c9ec0 2024-05-08T04:38:35.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed62a00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 0x7fed64163ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed4c01a100 tx=0x7fed4c01ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f237c0bf020 con 0x7f23840cab70 2024-05-08T04:38:35.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fab7c800640 1 -- 172.21.15.164:0/2201317227 wait complete. 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc098070 con 0x7f3ab4053c50 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc093630 con 0x7f3ab4053c50 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0939e0 con 0x7f3ab4053c50 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab40e1760 con 0x7f3ab4053c50 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0bd5f0 con 0x7fed640cc720 2024-05-08T04:38:35.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0bd790 con 0x7fed640cc720 2024-05-08T04:38:35.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0bdb40 con 0x7fed640cc720 2024-05-08T04:38:35.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e414a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab40e1d20 con 0x7f3ab4053c50 2024-05-08T04:38:35.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed64164530 con 0x7fed640cc720 2024-05-08T04:38:35.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 0x7f14e415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 Processor -- start 2024-05-08T04:38:35.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f1ff1400640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1fcc06f200 0x7f1fcc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f2380c00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2364067160 0x7f2364069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f237c1200f0 con 0x7f23840cab70 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2380c00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f236406f380 0x7f23640717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f237c0c9e80 con 0x7f236406f380 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237c1204e0 con 0x7f23840cab70 2024-05-08T04:38:35.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7fed69e00640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64164af0 con 0x7fed640cc720 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14e40cdd30 0x7f14e414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4072570 con 0x7f14e40cdd30 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40723f0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7f14ec200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4160a80 con 0x7f14e406b3a0 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 0x7f14e415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 0x7f14e415c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38644/0 (socket says 172.21.15.164:38644) 2024-05-08T04:38:35.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 -- 172.21.15.164:0/3678835497 learned_addr learned my addr 172.21.15.164:0/3678835497 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a94001040 con 0x7f3ab4053c50 2024-05-08T04:38:35.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed64069d60 con 0x7fed640cc720 2024-05-08T04:38:35.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2389600640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f236406f380 0x7f23640717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f1ff1400640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1fcc06f200 0x7f1fcc071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 -- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 msgr2=0x7f14e414a390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14e406b3a0 0x7f14e414a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 -- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14e40cdd30 msgr2=0x7f14e414e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14e40cdd30 0x7f14e414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e41492f0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 -- start start 2024-05-08T04:38:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7faa47400640 1 -- 172.21.15.164:0/2765895935 >> v1:172.21.15.164:6789/0 conn(0x7faa48163540 legacy=0x7faa480ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.519+0000 7faa47400640 1 -- 172.21.15.164:0/2765895935 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480cc4c0 con 0x7faa480aedb0 2024-05-08T04:38:35.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2383e00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2364067160 0x7f2364069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f1febe00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1fcc066fe0 0x7f1fcc0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2389600640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f236406f380 0x7f23640717a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3abc09b070 con 0x7f3ab4053c50 2024-05-08T04:38:35.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e8e00640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 0x7f14e415c200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c500 tx=0x7f14d000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== osd.0 v2:172.21.15.164:6800/2287928810 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1fcc071cd0 con 0x7f1fcc06f200 2024-05-08T04:38:35.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0016020 con 0x7f14e40cd7e0 2024-05-08T04:38:35.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0005220 con 0x7f14e40cd7e0 2024-05-08T04:38:35.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab740cfda0 0x7fab741484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415ba70 0x7fab742075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 0x7fab7414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74070740 con 0x7fab7415ba70 2024-05-08T04:38:35.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740705c0 con 0x7fab740cfda0 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fab7c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740708c0 con 0x7fab7415be50 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed4c0b33f0 con 0x7fed640cc720 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fed40066fb0 0x7fed40069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed4c11fc00 con 0x7fed640cc720 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7fed4006f1d0 0x7fed400715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed40071ca0 con 0x7fed4006f1d0 2024-05-08T04:38:35.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed4c1200e0 con 0x7fed640cc720 2024-05-08T04:38:35.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0007030 con 0x7f14e40cd7e0 2024-05-08T04:38:35.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e41495d0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4149b90 con 0x7f14e40cd7e0 2024-05-08T04:38:35.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.523+0000 7f2c66200640 1 Processor -- start 2024-05-08T04:38:35.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 0x7fab7414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 0x7fab7414c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54148/0 (socket says 172.21.15.164:54148) 2024-05-08T04:38:35.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 -- 172.21.15.164:0/1514187380 learned_addr learned my addr 172.21.15.164:0/1514187380 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab79e00640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab740cfda0 0x7fab741484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f1febe00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1fcc066fe0 0x7f1fcc0694a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009740 tx=0x7f1fe40aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab2000640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a9c067030 0x7f3a9c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 -- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab740cfda0 msgr2=0x7fab741484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3abc0ffb60 con 0x7f3ab4053c50 2024-05-08T04:38:35.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab2000640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7f3a9c06f250 0x7f3a9c071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 -- start start 2024-05-08T04:38:35.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed63e00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7fed4006f1d0 0x7fed400715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3abc070e50 con 0x7f3a9c06f250 2024-05-08T04:38:35.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3abc1042a0 con 0x7f3ab4053c50 2024-05-08T04:38:35.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed62000640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fed40066fb0 0x7fed40069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3aba800640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7f3a9c06f250 0x7f3a9c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed63e00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7fed4006f1d0 0x7fed400715f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab79400640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415ba70 0x7fab742075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7faa47400640 1 --2- 172.21.15.164:0/2765895935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa480aedb0 0x7faa480cccc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa30009740 tx=0x7faa4806df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f13620066cbfe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== osd.1 v2:172.21.15.164:6808/2349768482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f237c0c9e80 con 0x7f236406f380 2024-05-08T04:38:35.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab740cfda0 0x7fab741484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 -- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415ba70 msgr2=0x7fab742075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415ba70 0x7fab742075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74148d30 con 0x7fab7415be50 2024-05-08T04:38:35.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14c0001040 con 0x7f14e40cd7e0 2024-05-08T04:38:35.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab79400640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7415ba70 0x7fab742075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3aba800640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7f3a9c06f250 0x7f3a9c071670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f3ab9400640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a9c067030 0x7f3a9c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab79e00640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab740cfda0 0x7fab741484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c600c9ec0 0x7f2c60151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 0x7f2c60203940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c6015c9b0 0x7f2c60167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed62000640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fed40066fb0 0x7fed40069470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed640ce000 tx=0x7fed54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== osd.3 v2:172.21.15.164:6809/2335523075 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fed40071ca0 con 0x7fed4006f1d0 2024-05-08T04:38:35.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2383e00640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2364067160 0x7f2364069620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23840b1150 tx=0x7f2370008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== osd.2 v2:172.21.15.164:6824/4253785060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3abc070e50 con 0x7f3a9c06f250 2024-05-08T04:38:35.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f3ab9400640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a9c067030 0x7f3a9c0694f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ab417f8b0 tx=0x7f3aac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14d00071f0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f14c4066f60 0x7f14c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600cf200 con 0x7f2c600c9ec0 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600cf080 con 0x7f2c6015c9b0 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c66200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60168cd0 con 0x7f2c600cc430 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c5fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 0x7f2c60203940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c5fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 0x7f2c60203940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54156/0 (socket says 172.21.15.164:54156) 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c5fe00640 1 -- 172.21.15.164:0/4032727173 learned_addr learned my addr 172.21.15.164:0/4032727173 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7f2c65800640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c6015c9b0 0x7f2c60167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 -- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c6015c9b0 msgr2=0x7f2c60167bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c6015c9b0 0x7f2c60167bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 -- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c600c9ec0 msgr2=0x7f2c60151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c600c9ec0 0x7f2c60151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6015d1b0 con 0x7f2c600cc430 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c65800640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c6015c9b0 0x7f2c60167bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5fe00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 0x7f2c60203940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00c3a0 tx=0x7f2c4c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab7a800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 0x7fab7414c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab700b0040 tx=0x7fab700196c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700bd020 con 0x7fab7415be50 2024-05-08T04:38:35.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab700b8670 con 0x7fab7415be50 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7415dcc0 con 0x7fab7415be50 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700b2040 con 0x7fab7415be50 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7415df80 con 0x7fab7415be50 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7faa46000640 1 -- 172.21.15.164:0/2765895935 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300b0070 con 0x7faa480aedb0 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7faa46000640 1 -- 172.21.15.164:0/2765895935 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa300ab400 con 0x7faa480aedb0 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.527+0000 7faa46000640 1 -- 172.21.15.164:0/2765895935 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38001680 con 0x7faa480aedb0 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 -- 172.21.15.164:0/2765895935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa480aedb0 msgr2=0x7faa480cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 --2- 172.21.15.164:0/2765895935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa480aedb0 0x7faa480cccc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa30009740 tx=0x7faa4806df00 comp rx=0 tx=0).stop 2024-05-08T04:38:35.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab7415e1c0 con 0x7fab7415be50 2024-05-08T04:38:35.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14d0084af0 con 0x7f14e40cd7e0 2024-05-08T04:38:35.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f14c406f180 0x7f14c40715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14d0008e50 con 0x7f14c406f180 2024-05-08T04:38:35.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d008b050 con 0x7f14e40cd7e0 2024-05-08T04:38:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0052d0 con 0x7f2c600cc430 2024-05-08T04:38:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c005470 con 0x7f2c600cc430 2024-05-08T04:38:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c007260 con 0x7f2c600cc430 2024-05-08T04:38:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 -- 172.21.15.164:0/2765895935 shutdown_connections 2024-05-08T04:38:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 --2- 172.21.15.164:0/2765895935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa480aedb0 0x7faa480cccc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 -- 172.21.15.164:0/2765895935 >> 172.21.15.164:0/2765895935 conn(0x7faa4806ac90 msgr2=0x7faa4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 -- 172.21.15.164:0/2765895935 shutdown_connections 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7faa4e000640 1 -- 172.21.15.164:0/2765895935 wait complete. 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14ea200640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f14c406f180 0x7f14c40715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c60203e80 con 0x7f2c600cc430 2024-05-08T04:38:35.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c60204eb0 con 0x7f2c600cc430 2024-05-08T04:38:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c6004ef30 con 0x7f2c600cc430 2024-05-08T04:38:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7f14e9800640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f14c4066f60 0x7f14c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.531+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab700c0020 con 0x7fab7415be50 2024-05-08T04:38:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab6e000640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab58066fe0 0x7fab580694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 Processor -- start 2024-05-08T04:38:35.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 -- start start 2024-05-08T04:38:35.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab7011fdc0 con 0x7fab7415be50 2024-05-08T04:38:35.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab6e000640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7fab5806f180 0x7fab580715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab700a58d0 con 0x7fab5806f180 2024-05-08T04:38:35.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 0x7faa48157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab79400640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7fab5806f180 0x7fab580715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab700ec830 con 0x7fab7415be50 2024-05-08T04:38:35.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7fab79400640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7fab5806f180 0x7fab580715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa480cf6c0 0x7faa48163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 0x7faa48154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa47400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 0x7faa48157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa47400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 0x7faa48157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54160/0 (socket says 172.21.15.164:54160) 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa47e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 0x7faa48154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa47e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 0x7faa48154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41306/0 (socket says 172.21.15.164:41306) 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48170350 con 0x7faa48163540 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481701d0 con 0x7faa480cf6c0 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa4e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa481704d0 con 0x7faa480aedb0 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa46a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa480cf6c0 0x7faa48163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7faa47400640 1 -- 172.21.15.164:0/3842981803 learned_addr learned my addr 172.21.15.164:0/3842981803 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.535+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c4c007400 con 0x7f2c600cc430 2024-05-08T04:38:35.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f14ea200640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f14c406f180 0x7f14c40715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== osd.6 v2:172.21.15.172:6824/2968599637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab700a58d0 con 0x7fab5806f180 2024-05-08T04:38:35.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 -- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 msgr2=0x7faa48157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 0x7faa48157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 -- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa480cf6c0 msgr2=0x7faa48163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa480cf6c0 0x7faa48163d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48164570 con 0x7faa48163540 2024-05-08T04:38:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47400640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa480aedb0 0x7faa48157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa47e00640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 0x7faa48154e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c10 tx=0x7faa3800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== osd.4 v2:172.21.15.172:6800/3510673411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f14d0008e50 con 0x7f14c406f180 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38018070 con 0x7faa48163540 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa380072d0 con 0x7faa48163540 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa48164850 con 0x7faa48163540 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380047f0 con 0x7faa48163540 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48164e10 con 0x7faa48163540 2024-05-08T04:38:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7fab79e00640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab58066fe0 0x7fab580694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa481651e0 con 0x7faa48163540 2024-05-08T04:38:35.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7f14e9800640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f14c4066f60 0x7f14c4069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009a20 tx=0x7f14d8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa3801b070 con 0x7faa48163540 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa24066fb0 0x7faa24069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa38084630 con 0x7faa48163540 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7faa2406f1d0 0x7faa240715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa38017ec0 con 0x7faa2406f1d0 2024-05-08T04:38:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa38089050 con 0x7faa48163540 2024-05-08T04:38:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7fab79e00640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab58066fe0 0x7fab580694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab680016e0 tx=0x7fab68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa46a00640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7faa2406f1d0 0x7faa240715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa47400640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa24066fb0 0x7faa24069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7faa46a00640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7faa2406f1d0 0x7faa240715f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c5cc00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c40066fb0 0x7f2c40069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c4c084c00 con 0x7f2c600cc430 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c5cc00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f2c4006f1d0 0x7f2c400715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c40071ca0 con 0x7f2c4006f1d0 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c4c084fb0 con 0x7f2c600cc430 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c65800640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f2c4006f1d0 0x7f2c400715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.539+0000 7f2c64e00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c40066fb0 0x7f2c40069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7f2c65800640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f2c4006f1d0 0x7f2c400715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.543+0000 7f2c64e00640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c40066fb0 0x7f2c40069470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c60165b80 tx=0x7f2c500079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.547+0000 7faa47400640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa24066fb0 0x7faa24069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa30009740 tx=0x7faa30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.547+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== osd.7 v2:172.21.15.172:6808/1836955626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faa38017ec0 con 0x7faa2406f1d0 2024-05-08T04:38:35.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.551+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== osd.5 v2:172.21.15.172:6814/3686545583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c40071ca0 con 0x7f2c4006f1d0 2024-05-08T04:38:35.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 --> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed64069d60 con 0x7fed4006f1d0 2024-05-08T04:38:35.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed5b400640 1 -- 172.21.15.164:0/3209526361 <== osd.3 v2:172.21.15.164:6809/2335523075 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed64069d60 con 0x7fed4006f1d0 2024-05-08T04:38:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.555+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 --> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1fec068170 con 0x7f1fcc06f200 2024-05-08T04:38:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1fe8c00640 1 -- 172.21.15.164:0/1681234656 <== osd.0 v2:172.21.15.164:6800/2287928810 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1fec068170 con 0x7f1fcc06f200 2024-05-08T04:38:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1fcc06f200 msgr2=0x7f1fcc071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1fcc06f200 0x7f1fcc071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1fcc066fe0 msgr2=0x7f1fcc0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1fcc066fe0 0x7f1fcc0694a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009740 tx=0x7f1fe40aa070 comp rx=0 tx=0).stop 2024-05-08T04:38:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 msgr2=0x7f1fec1561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 --2- 172.21.15.164:0/1681234656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1fec0c9ec0 0x7f1fec1561f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00ca70 tx=0x7f1fdc00ce70 comp rx=0 tx=0).stop 2024-05-08T04:38:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1400640 1 -- 172.21.15.164:0/1681234656 reap_dead start 2024-05-08T04:38:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7fed4006f1d0 msgr2=0x7fed400715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7fed4006f1d0 0x7fed400715f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fed40066fb0 msgr2=0x7fed40069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fed40066fb0 0x7fed40069470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed640ce000 tx=0x7fed54008040 comp rx=0 tx=0).stop 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 msgr2=0x7fed64163ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 --2- 172.21.15.164:0/3209526361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed640cc720 0x7fed64163ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed4c01a100 tx=0x7fed4c01ad10 comp rx=0 tx=0).stop 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed63e00640 1 -- 172.21.15.164:0/3209526361 reap_dead start 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 shutdown_connections 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 >> 172.21.15.164:0/3209526361 conn(0x7fed640b1690 msgr2=0x7fed6415f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 shutdown_connections 2024-05-08T04:38:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 --> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a94002de0 con 0x7f3a9c06f250 2024-05-08T04:38:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.559+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 shutdown_connections 2024-05-08T04:38:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 >> 172.21.15.164:0/1681234656 conn(0x7f1fec0b1380 msgr2=0x7f1fec0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 shutdown_connections 2024-05-08T04:38:35.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f1ff1e00640 1 -- 172.21.15.164:0/1681234656 wait complete. 2024-05-08T04:38:35.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ab2000640 1 -- 172.21.15.164:0/4009920216 <== osd.2 v2:172.21.15.164:6824/4253785060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a94002de0 con 0x7f3a9c06f250 2024-05-08T04:38:35.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7fed58c00640 1 -- 172.21.15.164:0/3209526361 wait complete. 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f238a000640 1 -- 172.21.15.164:0/326027315 --> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f235c002de0 con 0x7f236406f380 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f2380c00640 1 -- 172.21.15.164:0/326027315 <== osd.1 v2:172.21.15.164:6808/2349768482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f235c002de0 con 0x7f236406f380 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7f3a9c06f250 msgr2=0x7f3a9c071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7f3a9c06f250 0x7f3a9c071670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a9c067030 msgr2=0x7f3a9c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a9c067030 0x7f3a9c0694f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ab417f8b0 tx=0x7f3aac008040 comp rx=0 tx=0).stop 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 msgr2=0x7f3ab40e2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 --2- 172.21.15.164:0/4009920216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ab4053c50 0x7f3ab40e2520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0693f0 tx=0x7f3abc0680b0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3aba800640 1 -- 172.21.15.164:0/4009920216 reap_dead start 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 --> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa48069d60 con 0x7faa2406f1d0 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 shutdown_connections 2024-05-08T04:38:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 >> 172.21.15.164:0/4009920216 conn(0x7f3ab4055a20 msgr2=0x7f3ab40d9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7faa3f400640 1 -- 172.21.15.164:0/3842981803 <== osd.7 v2:172.21.15.172:6808/1836955626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faa48069d60 con 0x7faa2406f1d0 2024-05-08T04:38:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 shutdown_connections 2024-05-08T04:38:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f236406f380 msgr2=0x7f23640717a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f236406f380 0x7f23640717a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f3ac0e00640 1 -- 172.21.15.164:0/4009920216 wait complete. 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2364067160 msgr2=0x7f2364069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2364067160 0x7f2364069620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23840b1150 tx=0x7f2370008040 comp rx=0 tx=0).stop 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 msgr2=0x7f238415b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f237a000640 1 --2- 172.21.15.164:0/326027315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23840cab70 0x7f238415b860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f237c01b010 tx=0x7f237c019700 comp rx=0 tx=0).stop 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.563+0000 7f2389600640 1 -- 172.21.15.164:0/326027315 reap_dead start 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 shutdown_connections 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 >> 172.21.15.164:0/326027315 conn(0x7f23840b1380 msgr2=0x7f23840c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 shutdown_connections 2024-05-08T04:38:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7f237a000640 1 -- 172.21.15.164:0/326027315 wait complete. 2024-05-08T04:38:35.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7faa2406f1d0 msgr2=0x7faa240715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7faa2406f1d0 0x7faa240715f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa24066fb0 msgr2=0x7faa24069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa24066fb0 0x7faa24069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa30009740 tx=0x7faa30008040 comp rx=0 tx=0).stop 2024-05-08T04:38:35.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 msgr2=0x7faa48154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 --2- 172.21.15.164:0/3842981803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa48163540 0x7faa48154e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c10 tx=0x7faa3800cbe0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa47e00640 1 -- 172.21.15.164:0/3842981803 reap_dead start 2024-05-08T04:38:35.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 --> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f14c0002de0 con 0x7f14c406f180 2024-05-08T04:38:35.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 shutdown_connections 2024-05-08T04:38:35.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 >> 172.21.15.164:0/3842981803 conn(0x7faa4806ac90 msgr2=0x7faa4806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 shutdown_connections 2024-05-08T04:38:35.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.567+0000 7faa4e000640 1 -- 172.21.15.164:0/3842981803 wait complete. 2024-05-08T04:38:35.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14e1600640 1 -- 172.21.15.164:0/3678835497 <== osd.4 v2:172.21.15.172:6800/3510673411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f14c0002de0 con 0x7f14c406f180 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f14c406f180 msgr2=0x7f14c40715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f14c406f180 0x7f14c40715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f14c4066f60 msgr2=0x7f14c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f14c4066f60 0x7f14c4069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009a20 tx=0x7f14d8009190 comp rx=0 tx=0).stop 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 msgr2=0x7f14e415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 --2- 172.21.15.164:0/3678835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e40cd7e0 0x7f14e415c200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c500 tx=0x7f14d000c9c0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ea200640 1 -- 172.21.15.164:0/3678835497 reap_dead start 2024-05-08T04:38:35.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 --> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab74069d60 con 0x7fab5806f180 2024-05-08T04:38:35.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 shutdown_connections 2024-05-08T04:38:35.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.571+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 >> 172.21.15.164:0/3678835497 conn(0x7f14e4066fa0 msgr2=0x7f14e4159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 shutdown_connections 2024-05-08T04:38:35.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab6e000640 1 -- 172.21.15.164:0/1514187380 <== osd.6 v2:172.21.15.172:6824/2968599637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab74069d60 con 0x7fab5806f180 2024-05-08T04:38:35.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7f14ec200640 1 -- 172.21.15.164:0/3678835497 wait complete. 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7fab5806f180 msgr2=0x7fab580715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7fab5806f180 0x7fab580715a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab58066fe0 msgr2=0x7fab580694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab58066fe0 0x7fab580694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab680016e0 tx=0x7fab68009040 comp rx=0 tx=0).stop 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 msgr2=0x7fab7414c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.575+0000 7fab7c800640 1 --2- 172.21.15.164:0/1514187380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7415be50 0x7fab7414c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab700b0040 tx=0x7fab700196c0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.579+0000 7fab7a800640 1 -- 172.21.15.164:0/1514187380 reap_dead start 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.579+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 shutdown_connections 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.579+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 >> 172.21.15.164:0/1514187380 conn(0x7fab7406ac90 msgr2=0x7fab74070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.579+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 shutdown_connections 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.579+0000 7fab7c800640 1 -- 172.21.15.164:0/1514187380 wait complete. 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.583+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 --> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c60068170 con 0x7f2c4006f1d0 2024-05-08T04:38:35.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.583+0000 7f2c5cc00640 1 -- 172.21.15.164:0/4032727173 <== osd.5 v2:172.21.15.172:6814/3686545583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c60068170 con 0x7f2c4006f1d0 2024-05-08T04:38:35.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f2c4006f1d0 msgr2=0x7f2c400715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f2c4006f1d0 0x7f2c400715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c40066fb0 msgr2=0x7f2c40069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c40066fb0 0x7f2c40069470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c60165b80 tx=0x7f2c500079b0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 msgr2=0x7f2c60203940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c66200640 1 --2- 172.21.15.164:0/4032727173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c600cc430 0x7f2c60203940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00c3a0 tx=0x7f2c4c00c860 comp rx=0 tx=0).stop 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.587+0000 7f2c65800640 1 -- 172.21.15.164:0/4032727173 reap_dead start 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.591+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 shutdown_connections 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.591+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 >> 172.21.15.164:0/4032727173 conn(0x7f2c600b1380 msgr2=0x7f2c6015dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.591+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 shutdown_connections 2024-05-08T04:38:35.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.591+0000 7f2c66200640 1 -- 172.21.15.164:0/4032727173 wait complete. 2024-05-08T04:38:35.615 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-05-08T04:38:35.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T04:38:35.616 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:35.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T04:38:35.617 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:35.618 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T04:38:35.620 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:35.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T04:38:35.623 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:35.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T04:38:35.625 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:35.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T04:38:35.628 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:35.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T04:38:35.634 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:35.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T04:38:35.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.855+0000 7f8684800640 1 Processor -- start 2024-05-08T04:38:35.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.855+0000 7f8684800640 1 -- start start 2024-05-08T04:38:35.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.855+0000 7f6c75a00640 1 Processor -- start 2024-05-08T04:38:35.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.855+0000 7f6c75a00640 1 -- start start 2024-05-08T04:38:35.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8684800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8684800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f867c0cdd30 0x7f867c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8684800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0ce7d0 con 0x7f867c06b3a0 2024-05-08T04:38:35.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8684800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0ceab0 con 0x7f867c0cdd30 2024-05-08T04:38:35.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8684800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0ced90 con 0x7f867c0cd7e0 2024-05-08T04:38:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c75a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c68033d00 0x7f6c6804faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c75a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c68054400 con 0x7f6c6800aae0 2024-05-08T04:38:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c75a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680546e0 con 0x7f6c6804ffe0 2024-05-08T04:38:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c75a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680e6030 con 0x7f6c68033d00 2024-05-08T04:38:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c6ea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6c6800aae0 0x7f6c680df050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58058/0 (socket says 172.21.15.164:58058) 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8682800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f867c0cdd30 0x7f867c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8681e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f867c06b3a0 0x7f867c0cf350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58046/0 (socket says 172.21.15.164:58046) 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8681e00640 1 -- 172.21.15.164:0/1749114887 learned_addr learned my addr 172.21.15.164:0/1749114887 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8681400640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c6ea00640 1 -- 172.21.15.164:0/3288129543 learned_addr learned my addr 172.21.15.164:0/3288129543 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8680a00640 1 -- 172.21.15.164:0/1749114887 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349717688 0 0) 0x7f867c0ce7d0 con 0x7f867c06b3a0 2024-05-08T04:38:35.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973445054 0 0) 0x7f6c68054400 con 0x7f6c6800aae0 2024-05-08T04:38:35.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c54003530 con 0x7f6c6800aae0 2024-05-08T04:38:35.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8680a00640 1 -- 172.21.15.164:0/1749114887 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8664003530 con 0x7f867c06b3a0 2024-05-08T04:38:35.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8682800640 1 -- 172.21.15.164:0/1749114887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 msgr2=0x7f867c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8682800640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c0cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8682800640 1 -- 172.21.15.164:0/1749114887 >> v1:172.21.15.164:6789/0 conn(0x7f867c06b3a0 legacy=0x7f867c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f8682800640 1 -- 172.21.15.164:0/1749114887 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0cf070 con 0x7f867c0cdd30 2024-05-08T04:38:35.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.859+0000 7f6c6e000640 1 --2- 172.21.15.164:0/3288129543 >> v2:172.21.15.164:3300/0 conn(0x7f6c68033d00 0x7f6c6804faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7f6c6e000640 1 -- 172.21.15.164:0/3288129543 >> v1:172.21.15.172:6789/0 conn(0x7f6c6804ffe0 legacy=0x7f6c68006540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7fc357e00640 1 Processor -- start 2024-05-08T04:38:35.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7fc357e00640 1 -- start start 2024-05-08T04:38:35.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7f1e49a00640 1 Processor -- start 2024-05-08T04:38:35.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7f1e49a00640 1 -- start start 2024-05-08T04:38:35.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7f8682800640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cdd30 0x7f867c0b26c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f86700097c0 tx=0x7f867001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1b9632ddd94811 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f6c6e000640 1 -- 172.21.15.164:0/3288129543 >> v1:172.21.15.164:6789/0 conn(0x7f6c6800aae0 legacy=0x7f6c680df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700245866 0 0) 0x7f6c54003530 con 0x7f6c6800aae0 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8680a00640 1 -- 172.21.15.164:0/1749114887 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700b0070 con 0x7f867c0cdd30 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8680a00640 1 -- 172.21.15.164:0/1749114887 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700ab600 con 0x7f867c0cdd30 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8680a00640 1 -- 172.21.15.164:0/1749114887 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700ab9d0 con 0x7f867c0cdd30 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 -- 172.21.15.164:0/1749114887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cdd30 msgr2=0x7f867c0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cdd30 0x7f867c0b26c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f86700097c0 tx=0x7f867001c460 comp rx=0 tx=0).stop 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.863+0000 7fc357e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc358162200 0x7fc358166680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc357e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc358166bc0 con 0x7fc3580cb630 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc357e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3580cece0 con 0x7fc3580cc680 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc357e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580cd130 con 0x7fc358162200 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc356000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc3580cb630 0x7fc3580cbf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58068/0 (socket says 172.21.15.164:58068) 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc356000640 1 -- 172.21.15.164:0/2678395047 learned_addr learned my addr 172.21.15.164:0/2678395047 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e440b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e440cd7e0 0x7f1e44159d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 0x7f1e440cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4415a2a0 con 0x7f1e440cdd30 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440ce7d0 con 0x7f1e440cd7e0 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e49a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440ceab0 con 0x7f1e4406b3a0 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e440cd7e0 0x7f1e44159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e440cd7e0 0x7f1e44159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38684/0 (socket says 172.21.15.164:38684) 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 -- 172.21.15.164:0/2466643005 learned_addr learned my addr 172.21.15.164:0/2466643005 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e43400640 1 --2- 172.21.15.164:0/2466643005 >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 0x7f1e440cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42a00640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e440b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 -- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 msgr2=0x7f1e440b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e440b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 -- 172.21.15.164:0/2466643005 >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 msgr2=0x7f1e440cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 --2- 172.21.15.164:0/2466643005 >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 0x7f1e440cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42000640 1 -- 172.21.15.164:0/2466643005 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440ced90 con 0x7f1e440cd7e0 2024-05-08T04:38:35.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f6c6e000640 1 -- 172.21.15.164:0/3288129543 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68006020 con 0x7f6c68033d00 2024-05-08T04:38:35.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc356a00640 1 --2- 172.21.15.164:0/2678395047 >> v2:172.21.15.172:3300/0 conn(0x7fc358162200 0x7fc358166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e42a00640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e440b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f1e43400640 1 --2- 172.21.15.164:0/2466643005 >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 0x7f1e440cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 -- 172.21.15.164:0/1749114887 shutdown_connections 2024-05-08T04:38:35.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cdd30 0x7f867c0b26c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157313632 0 0) 0x7fc358166bc0 con 0x7fc3580cb630 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c003530 con 0x7fc3580cb630 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 --2- 172.21.15.164:0/1749114887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 -- 172.21.15.164:0/1749114887 >> 172.21.15.164:0/1749114887 conn(0x7f867c066fa0 msgr2=0x7f867c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 -- 172.21.15.164:0/1749114887 shutdown_connections 2024-05-08T04:38:35.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7f8684800640 1 -- 172.21.15.164:0/1749114887 wait complete. 2024-05-08T04:38:35.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fe61e400640 1 Processor -- start 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263097643 0 0) 0x7fc33c003530 con 0x7fc3580cb630 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 >> v2:172.21.15.172:3300/0 conn(0x7fc358162200 msgr2=0x7fc358166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 --2- 172.21.15.164:0/2678395047 >> v2:172.21.15.172:3300/0 conn(0x7fc358162200 0x7fc358166680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 >> v1:172.21.15.172:6789/0 conn(0x7fc3580cc680 legacy=0x7fc3580cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580cbbc0 con 0x7fc3580cb630 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc344001680 con 0x7fc3580cb630 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 >> v1:172.21.15.164:6789/0 conn(0x7fc3580cb630 legacy=0x7fc3580cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc33c0039c0 0x7fc33c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc33c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc33c0051d0 0x7fc33c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580cd130 con 0x7fc33c0051d0 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c00c070 con 0x7fc33c004810 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c00c270 con 0x7fc33c0039c0 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c6e000640 1 --2- 172.21.15.164:0/3288129543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68033d00 0x7f6c6804faa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0097c0 tx=0x7f6c5c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f389950dcd115ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.867+0000 7fe61e400640 1 -- start start 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe61e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 0x7fe6180cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe61e400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe6180d2070 0x7fe6180d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe61e400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180cf930 con 0x7fe6180d0620 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe61e400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180d45e0 con 0x7fe6180d2070 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe61e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180b6a60 con 0x7fe6180d1c90 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0b0070 con 0x7f6c68033d00 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70069200 con 0x7f6c68033d00 2024-05-08T04:38:35.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c6d600640 1 -- 172.21.15.164:0/3288129543 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0b0070 con 0x7f6c68033d00 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356a00640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc33c0039c0 0x7fc33c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc33c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 -- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc33c0039c0 msgr2=0x7fc33c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc33c0039c0 0x7fc33c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 -- 172.21.15.164:0/2678395047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc33c0051d0 msgr2=0x7fc33c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc33c0051d0 0x7fc33c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc356000640 1 -- 172.21.15.164:0/2678395047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580cbbc0 con 0x7fc33c004810 2024-05-08T04:38:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fc357400640 1 -- 172.21.15.164:0/2678395047 reap_dead start 2024-05-08T04:38:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe616a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe6180d2070 0x7fe6180d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe616a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe6180d2070 0x7fe6180d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41354/0 (socket says 172.21.15.164:41354) 2024-05-08T04:38:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe616a00640 1 -- 172.21.15.164:0/1750545507 learned_addr learned my addr 172.21.15.164:0/1750545507 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f9fae600640 1 Processor -- start 2024-05-08T04:38:35.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f9fae600640 1 -- start start 2024-05-08T04:38:35.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c75a00640 1 -- 172.21.15.164:0/3288129543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68033d00 msgr2=0x7f6c6804faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6c75a00640 1 --2- 172.21.15.164:0/3288129543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68033d00 0x7f6c6804faa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0097c0 tx=0x7f6c5c01c3f0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7fe617400640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 0x7fe6180cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f8684800640 1 Processor -- start 2024-05-08T04:38:35.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f8684800640 1 -- start start 2024-05-08T04:38:35.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fe616a00640 1 -- 172.21.15.164:0/1750545507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 msgr2=0x7fe6180cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f1e42000640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cd7e0 0x7f1e44159d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e44068990 tx=0x7f1e3401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e39b4bda42c8e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 0x7f867c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6c75a00640 1 -- 172.21.15.164:0/3288129543 shutdown_connections 2024-05-08T04:38:35.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6c75a00640 1 --2- 172.21.15.164:0/3288129543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68033d00 0x7f6c6804faa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6c75a00640 1 -- 172.21.15.164:0/3288129543 >> 172.21.15.164:0/3288129543 conn(0x7f6c68009fc0 msgr2=0x7f6c6800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fe616a00640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 0x7fe6180cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e41600640 1 -- 172.21.15.164:0/2466643005 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340aa660 con 0x7f1e440cd7e0 2024-05-08T04:38:35.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e41600640 1 -- 172.21.15.164:0/2466643005 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340af070 con 0x7f1e440cd7e0 2024-05-08T04:38:35.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e41600640 1 -- 172.21.15.164:0/2466643005 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340aaa20 con 0x7f1e440cd7e0 2024-05-08T04:38:35.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fe616a00640 1 -- 172.21.15.164:0/1750545507 >> v1:172.21.15.164:6789/0 conn(0x7fe6180d0620 legacy=0x7fe6180d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 0x7f867c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c072570 con 0x7f867c06b3a0 2024-05-08T04:38:35.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0723f0 con 0x7f867c0cdd30 2024-05-08T04:38:35.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.871+0000 7f6a3ca00640 1 Processor -- start 2024-05-08T04:38:35.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a3ca00640 1 -- start start 2024-05-08T04:38:35.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c160a80 con 0x7f867c0cd7e0 2024-05-08T04:38:35.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fae600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 0x7f9fa80d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fe616a00640 1 -- 172.21.15.164:0/1750545507 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180b6cb0 con 0x7fe6180d2070 2024-05-08T04:38:35.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fe617400640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 0x7fe6180cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8681e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 0x7f867c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 0x7f867c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8681e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 0x7f867c141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41362/0 (socket says 172.21.15.164:41362) 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8681e00640 1 -- 172.21.15.164:0/425103120 learned_addr learned my addr 172.21.15.164:0/425103120 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8681400640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6c75a00640 1 -- 172.21.15.164:0/3288129543 shutdown_connections 2024-05-08T04:38:35.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6c75a00640 1 -- 172.21.15.164:0/3288129543 wait complete. 2024-05-08T04:38:35.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fad200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 0x7f9fa80d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fad200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 0x7f9fa80d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38718/0 (socket says 172.21.15.164:38718) 2024-05-08T04:38:35.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54218/0 (socket says 172.21.15.164:54218) 2024-05-08T04:38:35.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 -- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cd7e0 msgr2=0x7f1e44159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cd7e0 0x7f1e44159d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e44068990 tx=0x7f1e3401c560 comp rx=0 tx=0).stop 2024-05-08T04:38:35.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fae600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa80cf550 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 -- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 msgr2=0x7f867c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 -- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 msgr2=0x7f867c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 0x7f867c141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 -- 172.21.15.164:0/425103120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0ce350 con 0x7f867c0cdd30 2024-05-08T04:38:35.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 -- 172.21.15.164:0/2466643005 shutdown_connections 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2466643005 >> v2:172.21.15.164:3300/0 conn(0x7f1e440cdd30 0x7f1e440cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cd7e0 0x7f1e44159d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2466643005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e440b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fae600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa815ff00 con 0x7f9fa815fb20 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fad200640 1 -- 172.21.15.164:0/2512898363 learned_addr learned my addr 172.21.15.164:0/2512898363 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80cf830 con 0x7f9fa8162ef0 2024-05-08T04:38:35.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f1e49a00640 1 -- 172.21.15.164:0/2466643005 >> 172.21.15.164:0/2466643005 conn(0x7f1e44066fa0 msgr2=0x7f1e4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8682800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 0x7f867c15e040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86700ab000 tx=0x7f8670019200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a3ca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fc356000640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc33c004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c260 tx=0x7fc34401cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f933d5678f56dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700b0040 con 0x7f867c0cdd30 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a3ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 0x7f6a30104cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a3ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a37400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344007d40 con 0x7fc33c004810 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3440a9020 con 0x7fc33c004810 2024-05-08T04:38:35.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7fc355600640 1 -- 172.21.15.164:0/2678395047 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3440a5be0 con 0x7fc33c004810 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 -- 172.21.15.164:0/2512898363 >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 msgr2=0x7f9fa80d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 --2- 172.21.15.164:0/2512898363 >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 0x7f9fa80d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 -- 172.21.15.164:0/2512898363 >> v1:172.21.15.164:6789/0 conn(0x7f9fa80c4cf0 legacy=0x7f9fa80ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f9fa7e00640 1 -- 172.21.15.164:0/2512898363 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80cfb10 con 0x7f9fa8162ef0 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 Processor -- start 2024-05-08T04:38:35.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 -- start start 2024-05-08T04:38:35.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700abd50 con 0x7f867c0cdd30 2024-05-08T04:38:35.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700b6690 con 0x7f867c0cdd30 2024-05-08T04:38:35.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c1424c0 con 0x7f867c0cdd30 2024-05-08T04:38:35.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c142a80 con 0x7f867c0cdd30 2024-05-08T04:38:35.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a37400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41370/0 (socket says 172.21.15.164:41370) 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a37e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a37e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54232/0 (socket says 172.21.15.164:54232) 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a36a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 0x7f6a30104cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a36a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 0x7f6a30104cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38732/0 (socket says 172.21.15.164:38732) 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.875+0000 7f6a36a00640 1 -- 172.21.15.164:0/2830580880 learned_addr learned my addr 172.21.15.164:0/2830580880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a30005ea0 con 0x7f6a30054460 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a30006180 con 0x7f6a30104850 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a30101180 con 0x7f6a301051f0 2024-05-08T04:38:35.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 -- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 msgr2=0x7fc33c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc33c004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c260 tx=0x7fc34401cce0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37e00640 1 -- 172.21.15.164:0/2830580880 >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 msgr2=0x7f6a30104cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37e00640 1 --2- 172.21.15.164:0/2830580880 >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 0x7f6a30104cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37e00640 1 -- 172.21.15.164:0/2830580880 >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 msgr2=0x7f6a30053420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37e00640 1 --2- 172.21.15.164:0/2830580880 >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37e00640 1 -- 172.21.15.164:0/2830580880 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30105680 con 0x7f6a301051f0 2024-05-08T04:38:35.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6a37400640 1 --2- 172.21.15.164:0/2830580880 >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 -- 172.21.15.164:0/2678395047 shutdown_connections 2024-05-08T04:38:35.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 --2- 172.21.15.164:0/2678395047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc33c004c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 -- 172.21.15.164:0/2678395047 >> 172.21.15.164:0/2678395047 conn(0x7fc35806b670 msgr2=0x7fc3580b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 -- 172.21.15.164:0/2678395047 shutdown_connections 2024-05-08T04:38:35.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 0x7f6c680d63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f1e49a00640 1 -- 172.21.15.164:0/2466643005 shutdown_connections 2024-05-08T04:38:35.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f1e49a00640 1 -- 172.21.15.164:0/2466643005 wait complete. 2024-05-08T04:38:35.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fc357e00640 1 -- 172.21.15.164:0/2678395047 wait complete. 2024-05-08T04:38:35.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6804ffe0 0x7f6c6804e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 0x7f6c680d63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 0x7f6c680d63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54240/0 (socket says 172.21.15.164:54240) 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38746/0 (socket says 172.21.15.164:38746) 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 -- 172.21.15.164:0/1578617540 learned_addr learned my addr 172.21.15.164:0/1578617540 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680075e0 con 0x7f6c6804ffe0 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68007460 con 0x7f6c68033d00 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68007760 con 0x7f6c6800aae0 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6f400640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6804ffe0 0x7f6c6804e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 -- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 msgr2=0x7f6c680d63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 0x7f6c680d63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7fe616a00640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d2070 0x7fe6180d24b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02cab8048af168c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 -- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6804ffe0 msgr2=0x7f6c6804e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6804ffe0 0x7f6c6804e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.879+0000 7f6c6e000640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680dfa00 con 0x7f6c68033d00 2024-05-08T04:38:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c069d60 con 0x7f867c0cdd30 2024-05-08T04:38:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe616000640 1 -- 172.21.15.164:0/1750545507 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080ab650 con 0x7fe6180d2070 2024-05-08T04:38:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6c6e000640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c01cc90 tx=0x7f6c5c019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe616000640 1 -- 172.21.15.164:0/1750545507 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080b0070 con 0x7fe6180d2070 2024-05-08T04:38:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe616000640 1 -- 172.21.15.164:0/1750545507 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080aba10 con 0x7fe6180d2070 2024-05-08T04:38:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86700bf030 con 0x7f867c0cdd30 2024-05-08T04:38:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0b9020 con 0x7f6c68033d00 2024-05-08T04:38:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fa7e00640 1 --2- 172.21.15.164:0/2512898363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f940097c0 tx=0x7f9f9401c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3086d2108477df58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 Processor -- start 2024-05-08T04:38:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fa7400640 1 -- 172.21.15.164:0/2512898363 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940b0070 con 0x7f9fa8162ef0 2024-05-08T04:38:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fa7400640 1 -- 172.21.15.164:0/2512898363 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940ab600 con 0x7f9fa8162ef0 2024-05-08T04:38:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 -- 172.21.15.164:0/1750545507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d2070 msgr2=0x7fe6180d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d2070 0x7fe6180d24b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60801c560 comp rx=0 tx=0).stop 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 -- 172.21.15.164:0/1750545507 shutdown_connections 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d2070 0x7fe6180d24b0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60801c560 comp rx=0 tx=0).stop 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 --2- 172.21.15.164:0/1750545507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180d1c90 0x7fe6180cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f8676000640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 0x7f8660069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 -- 172.21.15.164:0/1750545507 >> 172.21.15.164:0/1750545507 conn(0x7fe61809cef0 msgr2=0x7fe6180d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fc357e00640 1 Processor -- start 2024-05-08T04:38:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a37e00640 1 --2- 172.21.15.164:0/2830580880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6a30054a30 tx=0x7f6a2c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b090086abf0657 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fa7400640 1 -- 172.21.15.164:0/2512898363 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940ab9d0 con 0x7f9fa8162ef0 2024-05-08T04:38:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fc357e00640 1 -- start start 2024-05-08T04:38:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f867011eb70 con 0x7f867c0cdd30 2024-05-08T04:38:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f8681e00640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 0x7f8660069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a36000640 1 -- 172.21.15.164:0/2830580880 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0b0070 con 0x7f6a301051f0 2024-05-08T04:38:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a36000640 1 -- 172.21.15.164:0/2830580880 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0ab600 con 0x7f6a301051f0 2024-05-08T04:38:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a36000640 1 -- 172.21.15.164:0/2830580880 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0ab9d0 con 0x7f6a301051f0 2024-05-08T04:38:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c003a30 con 0x7f6c68033d00 2024-05-08T04:38:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0b0040 con 0x7f6c68033d00 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 msgr2=0x7f6a3004f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 --2- 172.21.15.164:0/2830580880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6a30054a30 tx=0x7f6a2c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 -- start start 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e4414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e440cd7e0 0x7f1e4413e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44072360 con 0x7f1e440cd7e0 2024-05-08T04:38:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440721e0 con 0x7f1e440cdd30 2024-05-08T04:38:35.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e49a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440724e0 con 0x7f1e4406b3a0 2024-05-08T04:38:35.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 msgr2=0x7f9fa80c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 --2- 172.21.15.164:0/2512898363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f940097c0 tx=0x7f9f9401c460 comp rx=0 tx=0).stop 2024-05-08T04:38:35.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680dfce0 con 0x7f6c68033d00 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 shutdown_connections 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 --2- 172.21.15.164:0/2830580880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a301051f0 0x7f6a3004f3e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 --2- 172.21.15.164:0/2830580880 >> v2:172.21.15.172:3300/0 conn(0x7f6a30104850 0x7f6a30104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 --2- 172.21.15.164:0/2830580880 >> v2:172.21.15.164:3300/0 conn(0x7f6a30054460 0x7f6a30053420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 >> 172.21.15.164:0/2830580880 conn(0x7f6a30009fc0 msgr2=0x7f6a3000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 shutdown_connections 2024-05-08T04:38:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 --2- 172.21.15.164:0/2512898363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa8162ef0 0x7f9fa80c47b0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f940097c0 tx=0x7f9f9401c460 comp rx=0 tx=0).stop 2024-05-08T04:38:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 --2- 172.21.15.164:0/2512898363 >> v2:172.21.15.172:3300/0 conn(0x7f9fa815fb20 0x7f9fa80d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 >> 172.21.15.164:0/2512898363 conn(0x7f9fa809aa20 msgr2=0x7f9fa80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 -- 172.21.15.164:0/1750545507 shutdown_connections 2024-05-08T04:38:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7fe61e400640 1 -- 172.21.15.164:0/1750545507 wait complete. 2024-05-08T04:38:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.883+0000 7f1e42000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e440cd7e0 0x7f1e4413e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 shutdown_connections 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6a3ca00640 1 -- 172.21.15.164:0/2830580880 wait complete. 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38756/0 (socket says 172.21.15.164:38756) 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 -- 172.21.15.164:0/2146448618 learned_addr learned my addr 172.21.15.164:0/2146448618 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e42a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e4414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 shutdown_connections 2024-05-08T04:38:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c680e02a0 con 0x7f6c68033d00 2024-05-08T04:38:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f8681e00640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 0x7f8660069470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f866c0011f0 tx=0x7f866c007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f9fae600640 1 -- 172.21.15.164:0/2512898363 wait complete. 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 -- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 msgr2=0x7f1e4414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e4414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 -- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e440cd7e0 msgr2=0x7f1e4413e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e440cd7e0 0x7f1e4413e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 -- 172.21.15.164:0/2146448618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4415c5a0 con 0x7f1e440cdd30 2024-05-08T04:38:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e43400640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00cef0 tx=0x7f1e3c00a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc358160a70 0x7fc35815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 0x7fc35815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c9e70 con 0x7fc358160ef0 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580cf260 con 0x7fc33c004810 2024-05-08T04:38:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc358166680 con 0x7fc358160a70 2024-05-08T04:38:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38766/0 (socket says 172.21.15.164:38766) 2024-05-08T04:38:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 -- 172.21.15.164:0/2929878540 learned_addr learned my addr 172.21.15.164:0/2929878540 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c680089e0 con 0x7f6c68033d00 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357400640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 0x7fc35815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 -- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc358160a70 msgr2=0x7fc35815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc358160a70 0x7fc35815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 -- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 msgr2=0x7fc35815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 0x7fc35815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 -- 172.21.15.164:0/2929878540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358154e20 con 0x7fc33c004810 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc357400640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 0x7fc35815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f9fae600640 1 Processor -- start 2024-05-08T04:38:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fc356a00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc348011930 tx=0x7fc348011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f9fae600640 1 -- start start 2024-05-08T04:38:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fe61e400640 1 Processor -- start 2024-05-08T04:38:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7fe61e400640 1 -- start start 2024-05-08T04:38:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6a3ca00640 1 Processor -- start 2024-05-08T04:38:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c01a070 con 0x7f1e440cdd30 2024-05-08T04:38:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.887+0000 7f6a3ca00640 1 -- start start 2024-05-08T04:38:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4415b110 con 0x7f1e440cdd30 2024-05-08T04:38:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348019070 con 0x7fc33c004810 2024-05-08T04:38:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc348004240 con 0x7fc33c004810 2024-05-08T04:38:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800f8e0 con 0x7fc33c004810 2024-05-08T04:38:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc358155100 con 0x7fc33c004810 2024-05-08T04:38:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3581556c0 con 0x7fc33c004810 2024-05-08T04:38:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4415ae30 con 0x7f1e440cdd30 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a30104850 0x7f6a30107690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a301051f0 0x7f6a30105b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3010db90 con 0x7f6a301051f0 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a30007720 con 0x7f6a30054460 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a3ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3010dd10 con 0x7f6a30104850 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6180d1c90 0x7fe6180cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180cb6f0 0x7fe61814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61816b1e0 con 0x7fe6180d0620 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180d29f0 con 0x7fe6180d1c90 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe61e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61816b360 con 0x7fe6180cb6f0 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35804ef30 con 0x7fc33c004810 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e44069d60 con 0x7f1e440cdd30 2024-05-08T04:38:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c00d840 con 0x7f1e440cdd30 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00dc30 con 0x7f1e440cdd30 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38778/0 (socket says 172.21.15.164:38778) 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 -- 172.21.15.164:0/3360134857 learned_addr learned my addr 172.21.15.164:0/3360134857 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a36a00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a30104850 0x7f6a30107690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 -- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a30104850 msgr2=0x7f6a30107690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a30104850 0x7f6a30107690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 -- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a301051f0 msgr2=0x7f6a30105b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a301051f0 0x7f6a30105b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 -- 172.21.15.164:0/3360134857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30107ee0 con 0x7f6a30054460 2024-05-08T04:38:35.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41410/0 (socket says 172.21.15.164:41410) 2024-05-08T04:38:35.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6a37400640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e980 tx=0x7f6a2400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe616a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6180d1c90 0x7fe6180cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 -- 172.21.15.164:0/3316929238 learned_addr learned my addr 172.21.15.164:0/3316929238 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617e00640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180cb6f0 0x7fe61814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c5c0bc020 con 0x7f6c68033d00 2024-05-08T04:38:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 -- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180cb6f0 msgr2=0x7fe61814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180cb6f0 0x7fe61814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 -- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6180d1c90 msgr2=0x7fe6180cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6180d1c90 0x7fe6180cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 -- 172.21.15.164:0/3316929238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180cbf90 con 0x7fe6180d0620 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7fe617400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9b0 tx=0x7fe60000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.891+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86700c6060 con 0x7f867c0cdd30 2024-05-08T04:38:35.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600017070 con 0x7fe6180d0620 2024-05-08T04:38:35.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60000c9a0 con 0x7fe6180d0620 2024-05-08T04:38:35.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6c66a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 0x7f6c500694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000cd50 con 0x7fe6180d0620 2024-05-08T04:38:35.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180cbcb0 con 0x7fe6180d0620 2024-05-08T04:38:35.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61814e2c0 con 0x7fe6180d0620 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24024070 con 0x7f6a30054460 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a3ca00640 1 -- 172.21.15.164:0/3360134857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a301081c0 con 0x7f6a30054460 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a3ca00640 1 -- 172.21.15.164:0/3360134857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a30109e80 con 0x7f6a30054460 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24017d20 con 0x7f6a30054460 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400c930 con 0x7f6a30054460 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c5c11f610 con 0x7f6c68033d00 2024-05-08T04:38:35.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c5c0c3030 con 0x7f6c68033d00 2024-05-08T04:38:35.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6c6ea00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 0x7f6c500694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6181616a0 con 0x7fe6180d0620 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa815fb20 0x7f9fa81589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa80afc10 0x7f9fa80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa816ee40 con 0x7f9fa80afc10 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa816ecc0 con 0x7f9fa815fb20 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa816efc0 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54284/0 (socket says 172.21.15.164:54284) 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 -- 172.21.15.164:0/1898175263 learned_addr learned my addr 172.21.15.164:0/1898175263 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fadc00640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa80afc10 0x7f9fa80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 -- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa815fb20 msgr2=0x7f9fa81589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa815fb20 0x7f9fa81589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 -- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa80afc10 msgr2=0x7f9fa80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa80afc10 0x7f9fa80b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 -- 172.21.15.164:0/1898175263 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa806ca10 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6c6ea00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 0x7f6c500694f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c70050240 tx=0x7f6c7006d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fad200640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e840 tx=0x7f9f9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc34801c070 con 0x7fc33c004810 2024-05-08T04:38:35.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c0052a0 con 0x7f6a30054460 2024-05-08T04:38:35.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e3c01d070 con 0x7f1e440cdd30 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98017070 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9800c8d0 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800f040 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa8159080 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.895+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa80b48e0 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7fc34ea00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 0x7fc338069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc348085930 con 0x7fc33c004810 2024-05-08T04:38:35.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7f1e3aa00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 0x7f1e20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e3c085e20 con 0x7f1e440cdd30 2024-05-08T04:38:35.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7f1e42a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 0x7f1e20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7f1e42a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 0x7f1e20069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ccb90 tx=0x7f1e2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe60001a070 con 0x7fe6180d0620 2024-05-08T04:38:35.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3480569b0 con 0x7fc33c004810 2024-05-08T04:38:35.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7fc356000640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 0x7fc338069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7fc356000640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 0x7fc338069450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3440aa000 tx=0x7fc344000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7fe60f400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 0x7fe5f40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6000851e0 con 0x7fe6180d0620 2024-05-08T04:38:35.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.899+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f9801a070 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7fe616a00640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 0x7fe5f40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7f9fa4c00640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 0x7f9f880694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f98084ad0 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f800052a0 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7f9fa7e00640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 0x7f9f880694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.903+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e3c056a10 con 0x7f1e440cdd30 2024-05-08T04:38:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f9fa7e00640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 0x7f9f880694a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f940ad000 tx=0x7f9f94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe600056260 con 0x7fe6180d0620 2024-05-08T04:38:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7fe616a00640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 0x7fe5f40694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe6080af000 tx=0x7fe6080a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a24027070 con 0x7f6a30054460 2024-05-08T04:38:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f6a2b400640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 0x7f6a18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a2408d6b0 con 0x7f6a30054460 2024-05-08T04:38:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f6a36a00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 0x7f6a18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.911+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f98055b50 con 0x7f9fa80c4cf0 2024-05-08T04:38:35.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.907+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a2402e020 con 0x7f6a30054460 2024-05-08T04:38:35.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.911+0000 7f6a36a00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 0x7f6a18069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000b70 tx=0x7f6a3806d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.939+0000 7fc20d400640 1 Processor -- start 2024-05-08T04:38:35.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.939+0000 7fc20d400640 1 -- start start 2024-05-08T04:38:35.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.939+0000 7fc20d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc2080b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.939+0000 7fc20d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 0x7fc2080afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20d400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080aefe0 con 0x7fc2080b1b70 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080b3890 con 0x7fc2080b4b60 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 0x7fc2080afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 0x7fc2080afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38806/0 (socket says 172.21.15.164:38806) 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20ca00640 1 -- 172.21.15.164:0/492505716 learned_addr learned my addr 172.21.15.164:0/492505716 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc207e00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc2080b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080b3b70 con 0x7fc2081599d0 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959374416 0 0) 0x7fc2080aefe0 con 0x7fc2080b1b70 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ec003530 con 0x7fc2080b1b70 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361666744 0 0) 0x7fc1ec003530 con 0x7fc2080b1b70 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 msgr2=0x7fc2080b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.943+0000 7fc206a00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc2080b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 msgr2=0x7fc2080afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 --2- 172.21.15.164:0/492505716 >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 0x7fc2080afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080af280 con 0x7fc2080b1b70 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc200001a20 con 0x7fc2080b1b70 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 --2- 172.21.15.164:0/492505716 >> v2:172.21.15.172:3300/0 conn(0x7fc2080b4b60 0x7fc2080afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc207e00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc2080b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 >> v1:172.21.15.164:6789/0 conn(0x7fc2080b1b70 legacy=0x7fc2080b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc1ec003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1ec004160 0x7fc1ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc1ec0079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080b3890 con 0x7fc1ec004160 2024-05-08T04:38:35.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080b3b70 con 0x7fc1ec004e70 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec005630 con 0x7fc2081599d0 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc1ec0079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc207400640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1ec004160 0x7fc1ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc207e00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc1ec003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 -- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 msgr2=0x7fc1ec003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2081599d0 0x7fc1ec003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 -- 172.21.15.164:0/492505716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1ec004160 msgr2=0x7fc1ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1ec004160 0x7fc1ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc20ca00640 1 -- 172.21.15.164:0/492505716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec003530 con 0x7fc1ec004e70 2024-05-08T04:38:35.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.947+0000 7fc207400640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1ec004160 0x7fc1ec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20ca00640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc1ec0079b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc20815e760 tx=0x7fc1f401c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b30d30db84a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400a730 con 0x7fc1ec004e70 2024-05-08T04:38:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40a8630 con 0x7fc1ec004e70 2024-05-08T04:38:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc206a00640 1 -- 172.21.15.164:0/492505716 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40a8a20 con 0x7fc1ec004e70 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 msgr2=0x7fc1ec0079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 --2- 172.21.15.164:0/492505716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc1ec0079b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc20815e760 tx=0x7fc1f401c6e0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20ca00640 1 -- 172.21.15.164:0/492505716 reap_dead start 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 shutdown_connections 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 >> 172.21.15.164:0/492505716 conn(0x7fc20806ac90 msgr2=0x7fc20806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:35.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 shutdown_connections 2024-05-08T04:38:35.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.959+0000 7fc20d400640 1 -- 172.21.15.164:0/492505716 wait complete. 2024-05-08T04:38:35.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.963+0000 7fc20d400640 1 Processor -- start 2024-05-08T04:38:35.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.963+0000 7fc20d400640 1 -- start start 2024-05-08T04:38:35.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc208143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 0x7fc20815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc208143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41444/0 (socket says 172.21.15.164:41444) 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207400640 1 -- 172.21.15.164:0/3098923965 learned_addr learned my addr 172.21.15.164:0/3098923965 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208159f30 con 0x7fc208141710 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208159db0 con 0x7fc1ec004e70 2024-05-08T04:38:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080b07d0 con 0x7fc208142560 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 0x7fc20815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 -- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 msgr2=0x7fc208143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc208143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 -- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 msgr2=0x7fc208142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208142a00 con 0x7fc208142560 2024-05-08T04:38:35.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207400640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc207e00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc208143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20ca00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 0x7fc20815d410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005600 tx=0x7fc1ec005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400a730 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc208142ce0 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.967+0000 7fc20d400640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2081432a0 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.971+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4000e50 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.971+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400a730 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.971+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1f40b0400 con 0x7fc208142560 2024-05-08T04:38:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.975+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc0052a0 con 0x7fc208142560 2024-05-08T04:38:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.975+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 0x7fc1e8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.975+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1f411f080 con 0x7fc208142560 2024-05-08T04:38:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.975+0000 7fc207e00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 0x7fc1e8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:35.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.975+0000 7fc207e00640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 0x7fc1e8069620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc20806b5f0 tx=0x7fc1f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:35.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:35.979+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1f40bf060 con 0x7fc208142560 2024-05-08T04:38:36.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.219+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c680053a0 con 0x7f6c68033d00 2024-05-08T04:38:36.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.223+0000 7f6c66a00640 1 -- 172.21.15.164:0/1578617540 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c5c016070 con 0x7f6c68033d00 2024-05-08T04:38:36.234 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:36.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 msgr2=0x7f6c500694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 0x7f6c500694f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c70050240 tx=0x7f6c7006d040 comp rx=0 tx=0).stop 2024-05-08T04:38:36.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 msgr2=0x7f6c6804dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c01cc90 tx=0x7f6c5c019190 comp rx=0 tx=0).stop 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 shutdown_connections 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c50067030 0x7f6c500694f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c6804ffe0 0x7f6c6804e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c68033d00 0x7f6c6804dfb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 --2- 172.21.15.164:0/1578617540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6800aae0 0x7f6c680d63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 >> 172.21.15.164:0/1578617540 conn(0x7f6c68009fc0 msgr2=0x7f6c6800a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 shutdown_connections 2024-05-08T04:38:36.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.235+0000 7f6c75a00640 1 -- 172.21.15.164:0/1578617540 wait complete. 2024-05-08T04:38:36.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f867c1513f0 con 0x7f867c0cdd30 2024-05-08T04:38:36.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9f80005590 con 0x7f9fa80c4cf0 2024-05-08T04:38:36.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc358068170 con 0x7fc33c004810 2024-05-08T04:38:36.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7f8676000640 1 -- 172.21.15.164:0/425103120 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8670016050 con 0x7f867c0cdd30 2024-05-08T04:38:36.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7f9fa4c00640 1 -- 172.21.15.164:0/1898175263 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f98055450 con 0x7f9fa80c4cf0 2024-05-08T04:38:36.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6a0c005860 con 0x7f6a30054460 2024-05-08T04:38:36.244 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:36.244 INFO:teuthology.orchestra.run.smithi164.stdout:55834574853 2024-05-08T04:38:36.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.239+0000 7fc34ea00640 1 -- 172.21.15.164:0/2929878540 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3480562b0 con 0x7fc33c004810 2024-05-08T04:38:36.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.243+0000 7f6a2b400640 1 -- 172.21.15.164:0/3360134857 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a2405e620 con 0x7f6a30054460 2024-05-08T04:38:36.245 INFO:teuthology.orchestra.run.smithi164.stdout:55834574853 2024-05-08T04:38:36.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.243+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe61806b1b0 con 0x7fe6180d0620 2024-05-08T04:38:36.248 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 msgr2=0x7f9f880694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 0x7f9f880694a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f940ad000 tx=0x7f9f94008040 comp rx=0 tx=0).stop 2024-05-08T04:38:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 msgr2=0x7f9fa81579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e840 tx=0x7f9f9800ed00 comp rx=0 tx=0).stop 2024-05-08T04:38:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 shutdown_connections 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f88066fe0 0x7f9f880694a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fa80afc10 0x7f9fa80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa815fb20 0x7f9fa81589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 --2- 172.21.15.164:0/1898175263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fa80c4cf0 0x7f9fa81579a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 >> 172.21.15.164:0/1898175263 conn(0x7f9fa809aa20 msgr2=0x7f9fa806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 shutdown_connections 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f9fae600640 1 -- 172.21.15.164:0/1898175263 wait complete. 2024-05-08T04:38:36.250 INFO:teuthology.orchestra.run.smithi164.stdout:55834574853 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fe60f400640 1 -- 172.21.15.164:0/3316929238 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe600055b60 con 0x7fe6180d0620 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 msgr2=0x7f8660069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 0x7f8660069470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f866c0011f0 tx=0x7f866c007cb0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 msgr2=0x7f867c15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 0x7f867c15e040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86700ab000 tx=0x7f8670019200 comp rx=0 tx=0).stop 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 shutdown_connections 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8660066fb0 0x7f8660069470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c0cd7e0 0x7f867c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c0cdd30 0x7f867c15e040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 --2- 172.21.15.164:0/425103120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c06b3a0 0x7f867c141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 >> 172.21.15.164:0/425103120 conn(0x7f867c066fa0 msgr2=0x7f867c06c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 shutdown_connections 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f8684800640 1 -- 172.21.15.164:0/425103120 wait complete. 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 msgr2=0x7f6a18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 0x7f6a18069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000b70 tx=0x7f6a3806d040 comp rx=0 tx=0).stop 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 msgr2=0x7f6a300360b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e980 tx=0x7f6a2400ee40 comp rx=0 tx=0).stop 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 shutdown_connections 2024-05-08T04:38:36.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6a18066f60 0x7f6a18069420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a30104850 0x7f6a30107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a301051f0 0x7f6a30105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 --2- 172.21.15.164:0/3360134857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a30054460 0x7f6a300360b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 >> 172.21.15.164:0/3360134857 conn(0x7f6a30009fc0 msgr2=0x7f6a30052ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 shutdown_connections 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 msgr2=0x7fc338069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 0x7fc338069450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3440aa000 tx=0x7fc344000ff0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 msgr2=0x7fc3581faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc348011930 tx=0x7fc348011df0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.254 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1e4413ef10 con 0x7f1e440cdd30 2024-05-08T04:38:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7f1e3aa00640 1 -- 172.21.15.164:0/2146448618 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e3c05aab0 con 0x7f1e440cdd30 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 shutdown_connections 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc338066f90 0x7fc338069450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc358160a70 0x7fc35815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc358160ef0 0x7fc35815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 --2- 172.21.15.164:0/2929878540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc33c004810 0x7fc3581faec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 >> 172.21.15.164:0/2929878540 conn(0x7fc35806b670 msgr2=0x7fc358070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 shutdown_connections 2024-05-08T04:38:36.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.247+0000 7fc357e00640 1 -- 172.21.15.164:0/2929878540 wait complete. 2024-05-08T04:38:36.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f6a28c00640 1 -- 172.21.15.164:0/3360134857 wait complete. 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 msgr2=0x7fe5f40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 0x7fe5f40694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe6080af000 tx=0x7fe6080a7000 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 msgr2=0x7fe6181555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9b0 tx=0x7fe60000ee70 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 shutdown_connections 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5f4067030 0x7fe5f40694f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6180cb6f0 0x7fe61814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6180d1c90 0x7fe6180cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 --2- 172.21.15.164:0/3316929238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6180d0620 0x7fe6181555c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 msgr2=0x7f1e20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 0x7f1e20069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ccb90 tx=0x7f1e2c008040 comp rx=0 tx=0).stop 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 msgr2=0x7f1e4415cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00cef0 tx=0x7f1e3c00a710 comp rx=0 tx=0).stop 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 >> 172.21.15.164:0/3316929238 conn(0x7fe61809cef0 msgr2=0x7fe6180cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 shutdown_connections 2024-05-08T04:38:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7fe61e400640 1 -- 172.21.15.164:0/3316929238 wait complete. 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 shutdown_connections 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e20066f60 0x7f1e20069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e440cdd30 0x7f1e4415cd60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e440cd7e0 0x7f1e4413e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 --2- 172.21.15.164:0/2146448618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4406b3a0 0x7f1e4414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 >> 172.21.15.164:0/2146448618 conn(0x7f1e44066fa0 msgr2=0x7f1e440af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 shutdown_connections 2024-05-08T04:38:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.251+0000 7f1e49a00640 1 -- 172.21.15.164:0/2146448618 wait complete. 2024-05-08T04:38:36.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-08T04:38:36.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-08T04:38:36.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-05-08T04:38:36.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-08T04:38:36.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-05-08T04:38:36.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-05-08T04:38:36.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-05-08T04:38:36.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.303+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc1dc005590 con 0x7fc208142560 2024-05-08T04:38:36.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1ffe00640 1 -- 172.21.15.164:0/3098923965 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1f40ac030 con 0x7fc208142560 2024-05-08T04:38:36.308 INFO:teuthology.orchestra.run.smithi164.stdout:55834574853 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 msgr2=0x7fc1e8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 0x7fc1e8069620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc20806b5f0 tx=0x7fc1f8008040 comp rx=0 tx=0).stop 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 msgr2=0x7fc20815d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 0x7fc20815d410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005600 tx=0x7fc1ec005840 comp rx=0 tx=0).stop 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 shutdown_connections 2024-05-08T04:38:36.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e8067160 0x7fc1e8069620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc208142560 0x7fc20815d410 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc208141710 0x7fc208142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 --2- 172.21.15.164:0/3098923965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1ec004e70 0x7fc208143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 >> 172.21.15.164:0/3098923965 conn(0x7fc20806ac90 msgr2=0x7fc2080ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 shutdown_connections 2024-05-08T04:38:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:36.307+0000 7fc1fd600640 1 -- 172.21.15.164:0/3098923965 wait complete. 2024-05-08T04:38:36.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-05-08T04:38:37.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T04:38:37.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T04:38:37.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T04:38:37.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T04:38:37.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T04:38:37.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T04:38:37.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T04:38:37.331 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T04:38:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 Processor -- start 2024-05-08T04:38:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 -- start start 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1000071640 0x7f10000ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f10000aea10 0x7f10000ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100006d930 con 0x7f10000cce60 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100015e8f0 con 0x7f10000aea10 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f1006200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100015dd70 con 0x7f1000071640 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f0fff400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f10000cce60 0x7f100015adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:60260/0 (socket says 172.21.15.164:60260) 2024-05-08T04:38:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.503+0000 7f0fff400640 1 -- 172.21.15.164:0/2639808567 learned_addr learned my addr 172.21.15.164:0/2639808567 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0fffe00640 1 --2- 172.21.15.164:0/2639808567 >> v2:172.21.15.164:3300/0 conn(0x7f10000aea10 0x7f10000ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0ffea00640 1 --2- 172.21.15.164:0/2639808567 >> v2:172.21.15.172:3300/0 conn(0x7f1000071640 0x7f10000ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0fffe00640 1 -- 172.21.15.164:0/2639808567 >> v2:172.21.15.172:3300/0 conn(0x7f1000071640 msgr2=0x7f10000ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0fffe00640 1 --2- 172.21.15.164:0/2639808567 >> v2:172.21.15.172:3300/0 conn(0x7f1000071640 0x7f10000ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0fffe00640 1 -- 172.21.15.164:0/2639808567 >> v1:172.21.15.172:6789/0 conn(0x7f10000cce60 legacy=0x7f100015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0fffe00640 1 -- 172.21.15.164:0/2639808567 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100015e050 con 0x7f10000aea10 2024-05-08T04:38:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.507+0000 7f0ffe000640 1 -- 172.21.15.164:0/2639808567 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126293367 0 0) 0x7f100006d930 con 0x7f10000cce60 2024-05-08T04:38:37.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.515+0000 7f0fffe00640 1 --2- 172.21.15.164:0/2639808567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f10000ce750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10000b30b0 tx=0x7f0ff801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855bd100887ffffb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.515+0000 7f0ffe000640 1 -- 172.21.15.164:0/2639808567 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80b2070 con 0x7f10000aea10 2024-05-08T04:38:37.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.515+0000 7f0ffe000640 1 -- 172.21.15.164:0/2639808567 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff80a57f0 con 0x7f10000aea10 2024-05-08T04:38:37.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.515+0000 7f0ffe000640 1 -- 172.21.15.164:0/2639808567 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80a5bc0 con 0x7f10000aea10 2024-05-08T04:38:37.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 -- 172.21.15.164:0/2639808567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 msgr2=0x7f10000ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 --2- 172.21.15.164:0/2639808567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f10000ce750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10000b30b0 tx=0x7f0ff801c930 comp rx=0 tx=0).stop 2024-05-08T04:38:37.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 -- 172.21.15.164:0/2639808567 shutdown_connections 2024-05-08T04:38:37.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 --2- 172.21.15.164:0/2639808567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f10000ce750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 --2- 172.21.15.164:0/2639808567 >> v2:172.21.15.172:3300/0 conn(0x7f1000071640 0x7f10000ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 -- 172.21.15.164:0/2639808567 >> 172.21.15.164:0/2639808567 conn(0x7f10000cc340 msgr2=0x7f10000cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 -- 172.21.15.164:0/2639808567 shutdown_connections 2024-05-08T04:38:37.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.519+0000 7f1006200640 1 -- 172.21.15.164:0/2639808567 wait complete. 2024-05-08T04:38:37.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.523+0000 7f1006200640 1 Processor -- start 2024-05-08T04:38:37.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.523+0000 7f1006200640 1 -- start start 2024-05-08T04:38:37.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.523+0000 7f1006200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.523+0000 7f1006200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f1000154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.523+0000 7f1006200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10000cce60 0x7f100015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f1006200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100015d920 con 0x7f10000aea10 2024-05-08T04:38:37.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f1006200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000166dd0 con 0x7f10000cce60 2024-05-08T04:38:37.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f1006200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100015daa0 con 0x7f1000071640 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54316/0 (socket says 172.21.15.164:54316) 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 -- 172.21.15.164:0/1003227325 learned_addr learned my addr 172.21.15.164:0/1003227325 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0ffea00640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f1000154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fffe00640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10000cce60 0x7f100015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 -- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10000cce60 msgr2=0x7f100015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10000cce60 0x7f100015bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 -- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 msgr2=0x7f1000154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f1000154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 -- 172.21.15.164:0/1003227325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100015ad50 con 0x7f1000071640 2024-05-08T04:38:37.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0ffea00640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f1000154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:37.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0fff400640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0015d70 tx=0x7f0ff0013520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0013e40 con 0x7f1000071640 2024-05-08T04:38:37.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100015b030 con 0x7f1000071640 2024-05-08T04:38:37.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100015b5f0 con 0x7f1000071640 2024-05-08T04:38:37.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff00193f0 con 0x7f1000071640 2024-05-08T04:38:37.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.527+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00197a0 con 0x7f1000071640 2024-05-08T04:38:37.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.531+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd80052a0 con 0x7f1000071640 2024-05-08T04:38:37.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.531+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ff0019940 con 0x7f1000071640 2024-05-08T04:38:37.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.535+0000 7f0ff7400640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 0x7f0fdc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.535+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ff008e320 con 0x7f1000071640 2024-05-08T04:38:37.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.535+0000 7f0ffea00640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 0x7f0fdc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.535+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ff0010030 con 0x7f1000071640 2024-05-08T04:38:37.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.539+0000 7f0ffea00640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 0x7f0fdc0694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1000070ea0 tx=0x7f0fe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.571+0000 7f524ea00640 1 Processor -- start 2024-05-08T04:38:37.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.571+0000 7f524ea00640 1 -- start start 2024-05-08T04:38:37.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f524ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52481441d0 0x7f5248162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f524ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5248071580 0x7f52480ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f524ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524806ec90 con 0x7f52480cec90 2024-05-08T04:38:37.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f524ea00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524806d810 con 0x7f5248071580 2024-05-08T04:38:37.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f524ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52481627c0 con 0x7f52481441d0 2024-05-08T04:38:37.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5248071580 0x7f52480ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5248071580 0x7f52480ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41476/0 (socket says 172.21.15.164:41476) 2024-05-08T04:38:37.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 -- 172.21.15.164:0/924409519 learned_addr learned my addr 172.21.15.164:0/924409519 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064120740 0 0) 0x7f524806ec90 con 0x7f52480cec90 2024-05-08T04:38:37.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522c003530 con 0x7f52480cec90 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 -- 172.21.15.164:0/924409519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52481441d0 msgr2=0x7f5248162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247e00640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52481441d0 0x7f5248162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4102604524 0 0) 0x7f522c003530 con 0x7f52480cec90 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52481441d0 0x7f5248162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 -- 172.21.15.164:0/924409519 >> v1:172.21.15.172:6789/0 conn(0x7f52480cec90 legacy=0x7f52480702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f5247400640 1 -- 172.21.15.164:0/924409519 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248071a80 con 0x7f5248071580 2024-05-08T04:38:37.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 Processor -- start 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 -- start start 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 0x7f0bb40b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb40b3650 con 0x7f0bb415a120 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb40b3930 con 0x7f0bb4070dc0 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb9400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40b4260 con 0x7f0bb41436d0 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0bb415a120 0x7f0bb40706b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58114/0 (socket says 172.21.15.164:58114) 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3400640 1 -- 172.21.15.164:0/3410114271 learned_addr learned my addr 172.21.15.164:0/3410114271 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3e00640 1 --2- 172.21.15.164:0/3410114271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 0x7f0bb40b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb2a00640 1 -- 172.21.15.164:0/3410114271 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617192278 0 0) 0x7f0bb40b3930 con 0x7f0bb4070dc0 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb2a00640 1 -- 172.21.15.164:0/3410114271 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b98005740 con 0x7f0bb4070dc0 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3e00640 1 -- 172.21.15.164:0/3410114271 >> v1:172.21.15.172:6789/0 conn(0x7f0bb4070dc0 legacy=0x7f0bb406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3e00640 1 -- 172.21.15.164:0/3410114271 >> v1:172.21.15.164:6789/0 conn(0x7f0bb415a120 legacy=0x7f0bb40706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.575+0000 7f0bb3e00640 1 -- 172.21.15.164:0/3410114271 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40b3c10 con 0x7f0bb41436d0 2024-05-08T04:38:37.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.579+0000 7f5247400640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f52480ce750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52340098a0 tx=0x7f523401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f32860edf6bf17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.579+0000 7f153b000640 1 Processor -- start 2024-05-08T04:38:37.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.579+0000 7f153b000640 1 -- start start 2024-05-08T04:38:37.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340b0070 con 0x7f5248071580 2024-05-08T04:38:37.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52340ab400 con 0x7f5248071580 2024-05-08T04:38:37.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f5246a00640 1 -- 172.21.15.164:0/924409519 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340b0070 con 0x7f5248071580 2024-05-08T04:38:37.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f153b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f153b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15340b7790 0x7f15340b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f153b000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15340cf930 con 0x7f15340d2070 2024-05-08T04:38:37.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f153b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15340d45e0 con 0x7f15340b7790 2024-05-08T04:38:37.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f153b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340b6a60 con 0x7f15340d1c90 2024-05-08T04:38:37.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 -- 172.21.15.164:0/924409519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 msgr2=0x7f52480ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f52480ce750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52340098a0 tx=0x7f523401c460 comp rx=0 tx=0).stop 2024-05-08T04:38:37.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54336/0 (socket says 172.21.15.164:54336) 2024-05-08T04:38:37.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 -- 172.21.15.164:0/2372212702 learned_addr learned my addr 172.21.15.164:0/2372212702 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7eff4b400640 1 Processor -- start 2024-05-08T04:38:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 -- 172.21.15.164:0/924409519 shutdown_connections 2024-05-08T04:38:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f52480ce750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 --2- 172.21.15.164:0/924409519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52481441d0 0x7f5248162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 -- 172.21.15.164:0/924409519 >> 172.21.15.164:0/924409519 conn(0x7f52480cc340 msgr2=0x7f52480cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1539000640 1 --2- 172.21.15.164:0/2372212702 >> v2:172.21.15.172:3300/0 conn(0x7f15340b7790 0x7f15340b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 -- 172.21.15.164:0/924409519 shutdown_connections 2024-05-08T04:38:37.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7eff4b400640 1 -- start start 2024-05-08T04:38:37.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 -- 172.21.15.164:0/2372212702 >> v2:172.21.15.172:3300/0 conn(0x7f15340b7790 msgr2=0x7f15340b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 --2- 172.21.15.164:0/2372212702 >> v2:172.21.15.172:3300/0 conn(0x7f15340b7790 0x7f15340b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 -- 172.21.15.164:0/2372212702 >> v1:172.21.15.172:6789/0 conn(0x7f15340d2070 legacy=0x7f15340d24b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f1533e00640 1 -- 172.21.15.164:0/2372212702 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340b6cb0 con 0x7f15340d1c90 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.583+0000 7f524ea00640 1 -- 172.21.15.164:0/924409519 wait complete. 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 Processor -- start 2024-05-08T04:38:37.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 -- start start 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 0x7eff440cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7eff44163540 0x7eff440ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff440cf6c0 0x7eff440cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440ceac0 con 0x7eff44163540 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440cc1e0 con 0x7eff440cf6c0 2024-05-08T04:38:37.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff4b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff440cc4c0 con 0x7eff440aedb0 2024-05-08T04:38:37.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb3e00640 1 --2- 172.21.15.164:0/3410114271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 0x7f0bb40b3070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40090e0 tx=0x7f0ba401c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bf4c628f629f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb2a00640 1 -- 172.21.15.164:0/3410114271 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40a9670 con 0x7f0bb41436d0 2024-05-08T04:38:37.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 Processor -- start 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff48a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 0x7eff440cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff48a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 0x7eff440cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54348/0 (socket says 172.21.15.164:54348) 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff48a00640 1 -- 172.21.15.164:0/2096196702 learned_addr learned my addr 172.21.15.164:0/2096196702 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 --2- 172.21.15.164:0/2096196702 >> v2:172.21.15.164:3300/0 conn(0x7eff44163540 0x7eff440ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 Processor -- start 2024-05-08T04:38:37.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 -- start start 2024-05-08T04:38:37.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff49400640 1 --2- 172.21.15.164:0/2096196702 >> v2:172.21.15.172:3300/0 conn(0x7eff440cf6c0 0x7eff440cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb2a00640 1 -- 172.21.15.164:0/3410114271 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba40a9810 con 0x7f0bb41436d0 2024-05-08T04:38:37.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb2a00640 1 -- 172.21.15.164:0/3410114271 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40a9be0 con 0x7f0bb41436d0 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd6a0053c20 0x7fd6a00540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a0057df0 con 0x7fd6a00e32e0 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a00580d0 con 0x7fd6a0006800 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ae800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a003bfd0 con 0x7fd6a0053c20 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 -- 172.21.15.164:0/3410114271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 msgr2=0x7f0bb40b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 --2- 172.21.15.164:0/3410114271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 0x7f0bb40b3070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40090e0 tx=0x7f0ba401c630 comp rx=0 tx=0).stop 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52480cec90 0x7f524814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 0x7f52481611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52480ad890 con 0x7f5248071580 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5248072600 con 0x7f52481441d0 2024-05-08T04:38:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f524ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52481445b0 con 0x7f52480cec90 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 -- 172.21.15.164:0/3410114271 shutdown_connections 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 --2- 172.21.15.164:0/3410114271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb41436d0 0x7f0bb40b3070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 -- 172.21.15.164:0/3410114271 >> 172.21.15.164:0/3410114271 conn(0x7f0bb40cfaf0 msgr2=0x7f0bb40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f1533e00640 1 --2- 172.21.15.164:0/2372212702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1524009870 tx=0x7f152401c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e863720b901e9a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ade00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd6a0006800 0x7fd6a0006c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:60304/0 (socket says 172.21.15.164:60304) 2024-05-08T04:38:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fd6ade00640 1 -- 172.21.15.164:0/884729264 learned_addr learned my addr 172.21.15.164:0/884729264 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 -- 172.21.15.164:0/2096196702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 msgr2=0x7eff440cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 --2- 172.21.15.164:0/2096196702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 0x7eff440cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 -- 172.21.15.164:0/2096196702 >> v2:172.21.15.172:3300/0 conn(0x7eff440cf6c0 msgr2=0x7eff440cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 --2- 172.21.15.164:0/2096196702 >> v2:172.21.15.172:3300/0 conn(0x7eff440cf6c0 0x7eff440cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7eff43e00640 1 -- 172.21.15.164:0/2096196702 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440cc7a0 con 0x7eff44163540 2024-05-08T04:38:37.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 -- 172.21.15.164:0/3410114271 shutdown_connections 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7f0bb9400640 1 -- 172.21.15.164:0/3410114271 wait complete. 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 -- start start 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2d40cdd30 0x7fb2d40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d406cd30 con 0x7fb2d4159fc0 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d40ce7d0 con 0x7fb2d40cdd30 2024-05-08T04:38:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40ceab0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589370258 0 0) 0x7fd6a00580d0 con 0x7fd6a0006800 2024-05-08T04:38:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd690003530 con 0x7fd6a0006800 2024-05-08T04:38:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.587+0000 7fb2d1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54368/0 (socket says 172.21.15.164:54368) 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 -- 172.21.15.164:0/2062379204 learned_addr learned my addr 172.21.15.164:0/2062379204 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41494/0 (socket says 172.21.15.164:41494) 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 -- 172.21.15.164:0/236375372 learned_addr learned my addr 172.21.15.164:0/236375372 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f524ca00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 0x7f52481611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 -- 172.21.15.164:0/2062379204 >> v2:172.21.15.164:3300/0 conn(0x7fb2d40cdd30 msgr2=0x7fb2d40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 --2- 172.21.15.164:0/2062379204 >> v2:172.21.15.164:3300/0 conn(0x7fb2d40cdd30 0x7fb2d40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 -- 172.21.15.164:0/2062379204 >> v1:172.21.15.164:6789/0 conn(0x7fb2d4159fc0 legacy=0x7fb2d40cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fb2d1400640 1 -- 172.21.15.164:0/2062379204 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d40ced90 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f1532a00640 1 -- 172.21.15.164:0/2372212702 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240b0070 con 0x7f15340d1c90 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f1532a00640 1 -- 172.21.15.164:0/2372212702 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15240ab600 con 0x7f15340d1c90 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f1532a00640 1 -- 172.21.15.164:0/2372212702 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240ab9d0 con 0x7f15340d1c90 2024-05-08T04:38:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247400640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52480cec90 0x7f524814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 -- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52480cec90 msgr2=0x7f524814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52480cec90 0x7f524814d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 -- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 msgr2=0x7f52481611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 0x7f52481611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 -- 172.21.15.164:0/236375372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524815fec0 con 0x7f5248071580 2024-05-08T04:38:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f0bb9400640 1 Processor -- start 2024-05-08T04:38:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f5247e00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f523800c7f0 tx=0x7f523800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f524ca00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 0x7f52481611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f0bb9400640 1 -- start start 2024-05-08T04:38:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187110761 0 0) 0x7fd690003530 con 0x7fd6a0006800 2024-05-08T04:38:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 >> v2:172.21.15.164:3300/0 conn(0x7fd6a0053c20 msgr2=0x7fd6a00540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 --2- 172.21.15.164:0/884729264 >> v2:172.21.15.164:3300/0 conn(0x7fd6a0053c20 0x7fd6a00540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 >> v1:172.21.15.164:6789/0 conn(0x7fd6a00e32e0 legacy=0x7fd6a003e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00583b0 con 0x7fd6a0006800 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd6a804f450 con 0x7fd6a0006800 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 >> v1:172.21.15.172:6789/0 conn(0x7fd6a0006800 legacy=0x7fd6a0006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6900039c0 0x7fd690003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 0x7fd690004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6900051d0 0x7fd6900141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900148b0 con 0x7fd6900051d0 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690014a60 con 0x7fd690004810 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd690014c60 con 0x7fd6900039c0 2024-05-08T04:38:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6aca00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 0x7fd690004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 -- 172.21.15.164:0/2372212702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 msgr2=0x7f15340cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 --2- 172.21.15.164:0/2372212702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1524009870 tx=0x7f152401c520 comp rx=0 tx=0).stop 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 -- 172.21.15.164:0/2372212702 shutdown_connections 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 --2- 172.21.15.164:0/2372212702 >> v2:172.21.15.172:3300/0 conn(0x7f15340b7790 0x7f15340b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 --2- 172.21.15.164:0/2372212702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d1c90 0x7f15340cf1e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 -- 172.21.15.164:0/2372212702 >> 172.21.15.164:0/2372212702 conn(0x7f153409cef0 msgr2=0x7f15340d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 -- 172.21.15.164:0/2372212702 shutdown_connections 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238015070 con 0x7f5248071580 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238007820 con 0x7f5248071580 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238007bf0 con 0x7f5248071580 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f524ea00640 1 -- 172.21.15.164:0/236375372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5248160130 con 0x7f5248071580 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f524ea00640 1 -- 172.21.15.164:0/236375372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52481606f0 con 0x7f5248071580 2024-05-08T04:38:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6ad400640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6900039c0 0x7fd690003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bb41436d0 0x7f0bb40aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bb415a120 0x7f0bb40a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb406dae0 con 0x7f0bb415a120 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb406d960 con 0x7f0bb41436d0 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb4167ac0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7fd6ade00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6900051d0 0x7fd6900141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7f153b000640 1 -- 172.21.15.164:0/2372212702 wait complete. 2024-05-08T04:38:37.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54394/0 (socket says 172.21.15.164:54394) 2024-05-08T04:38:37.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 -- 172.21.15.164:0/797040065 learned_addr learned my addr 172.21.15.164:0/797040065 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb8a00640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bb415a120 0x7f0bb40a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fd6aca00640 1 -- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6900039c0 msgr2=0x7fd690003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bb41436d0 0x7f0bb40aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 -- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bb41436d0 msgr2=0x7f0bb40aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bb41436d0 0x7f0bb40aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 -- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bb415a120 msgr2=0x7f0bb40a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bb415a120 0x7f0bb40a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 -- 172.21.15.164:0/797040065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40af390 con 0x7f0bb4070dc0 2024-05-08T04:38:37.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.591+0000 7eff43e00640 1 --2- 172.21.15.164:0/2096196702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44163540 0x7eff440ce150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff4406e860 tx=0x7eff3c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1f71ede46477b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7eff43400640 1 -- 172.21.15.164:0/2096196702 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0b2070 con 0x7eff44163540 2024-05-08T04:38:37.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb3e00640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ab040 tx=0x7f0ba401ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fb2d1400640 1 --2- 172.21.15.164:0/2062379204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007360 tx=0x7fb2c001d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8c30889b5ce716 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7eff43400640 1 -- 172.21.15.164:0/2096196702 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c0a5830 con 0x7eff44163540 2024-05-08T04:38:37.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7eff43400640 1 -- 172.21.15.164:0/2096196702 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0a5c00 con 0x7eff44163540 2024-05-08T04:38:37.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524804ef30 con 0x7f5248071580 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40b1800 con 0x7f0bb4070dc0 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba40b19c0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40b1db0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fb2d0a00640 1 -- 172.21.15.164:0/2062379204 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0008be0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fb2d0a00640 1 -- 172.21.15.164:0/2062379204 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c00af070 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fb2d0a00640 1 -- 172.21.15.164:0/2062379204 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000a0d0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f153b000640 1 Processor -- start 2024-05-08T04:38:37.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb41566b0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb4156c70 con 0x7f0bb4070dc0 2024-05-08T04:38:37.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fd6aca00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6900039c0 0x7fd690003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fd6aca00640 1 -- 172.21.15.164:0/884729264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6900051d0 msgr2=0x7fd6900141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fd6aca00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6900051d0 0x7fd6900141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7fd6aca00640 1 -- 172.21.15.164:0/884729264 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00583b0 con 0x7fd690004810 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7eff4b400640 1 -- 172.21.15.164:0/2096196702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44163540 msgr2=0x7eff440ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.595+0000 7eff4b400640 1 --2- 172.21.15.164:0/2096196702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44163540 0x7eff440ce150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff4406e860 tx=0x7eff3c01c8e0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6ade00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6900051d0 0x7fd6900141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:37.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6ade00640 1 -- 172.21.15.164:0/884729264 reap_dead start 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 -- start start 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 -- 172.21.15.164:0/2096196702 shutdown_connections 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 --2- 172.21.15.164:0/2096196702 >> v2:172.21.15.172:3300/0 conn(0x7eff440cf6c0 0x7eff440cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 --2- 172.21.15.164:0/2096196702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44163540 0x7eff440ce150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 --2- 172.21.15.164:0/2096196702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440aedb0 0x7eff440cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 -- 172.21.15.164:0/2096196702 >> 172.21.15.164:0/2096196702 conn(0x7eff4406ac90 msgr2=0x7eff4406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 -- 172.21.15.164:0/2096196702 shutdown_connections 2024-05-08T04:38:37.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 -- 172.21.15.164:0/2096196702 wait complete. 2024-05-08T04:38:37.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb404ef30 con 0x7f0bb4070dc0 2024-05-08T04:38:37.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ba40c73f0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5238018070 con 0x7f5248071580 2024-05-08T04:38:37.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6aca00640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 0x7fd690004c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd69401a880 tx=0x7fd69402d470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802251b119bb7f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 -- 172.21.15.164:0/2062379204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 msgr2=0x7fb2d40cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 --2- 172.21.15.164:0/2062379204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007360 tx=0x7fb2c001d4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940c1070 con 0x7fd690004810 2024-05-08T04:38:37.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f0babe00640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6940bc600 con 0x7fd690004810 2024-05-08T04:38:37.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fd6a7e00640 1 -- 172.21.15.164:0/884729264 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69401a480 con 0x7fd690004810 2024-05-08T04:38:37.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 -- 172.21.15.164:0/2062379204 shutdown_connections 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 --2- 172.21.15.164:0/2062379204 >> v2:172.21.15.164:3300/0 conn(0x7fb2d40cdd30 0x7fb2d40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 --2- 172.21.15.164:0/2062379204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d40cfa60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7fb2d9000640 1 -- 172.21.15.164:0/2062379204 >> 172.21.15.164:0/2062379204 conn(0x7fb2d4066fa0 msgr2=0x7fb2d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f523fe00640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 0x7f5228069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52380854b0 con 0x7f5248071580 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f5247400640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 0x7f5228069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ba411fe60 con 0x7f0bb4070dc0 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f0bb3400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7eff4b400640 1 Processor -- start 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15340b7790 0x7f1534154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d2070 0x7f15340cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153416ab50 con 0x7f15340d1c90 2024-05-08T04:38:37.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153416a9d0 con 0x7f15340b7790 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f153b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153416acd0 con 0x7f15340d2070 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41518/0 (socket says 172.21.15.164:41518) 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 -- 172.21.15.164:0/1574240724 learned_addr learned my addr 172.21.15.164:0/1574240724 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 -- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d2070 msgr2=0x7f15340cbd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d2070 0x7f15340cbd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 -- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15340b7790 msgr2=0x7f1534154d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15340b7790 0x7f1534154d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 -- 172.21.15.164:0/1574240724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15341586c0 con 0x7f15340d1c90 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.599+0000 7f1533400640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f152000b4a0 tx=0x7f152000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fb2d9000640 1 -- 172.21.15.164:0/2062379204 shutdown_connections 2024-05-08T04:38:37.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 -- start start 2024-05-08T04:38:37.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 -- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 msgr2=0x7fd690004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 0x7fd690004c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd69401a880 tx=0x7fd69402d470 comp rx=0 tx=0).stop 2024-05-08T04:38:37.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 0x7eff44152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 0x7eff44165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f0bb3400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 0x7f0b94069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4154760 tx=0x7f0ba0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff44163540 0x7eff4415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7f5247400640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 0x7f5228069520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52340ad000 tx=0x7f52340a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff48a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 0x7eff44152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff48a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 0x7eff44152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41532/0 (socket says 172.21.15.164:41532) 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff48a00640 1 -- 172.21.15.164:0/11878965 learned_addr learned my addr 172.21.15.164:0/11878965 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 0x7eff44165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406db80 con 0x7eff440aedb0 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406da00 con 0x7eff44163540 2024-05-08T04:38:37.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406dd00 con 0x7eff440cf6c0 2024-05-08T04:38:37.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 -- 172.21.15.164:0/884729264 shutdown_connections 2024-05-08T04:38:37.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 --2- 172.21.15.164:0/884729264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd690004810 0x7fd690004c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 -- 172.21.15.164:0/884729264 >> 172.21.15.164:0/884729264 conn(0x7fd6a00559f0 msgr2=0x7fd6a0055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fb2d9000640 1 -- 172.21.15.164:0/2062379204 wait complete. 2024-05-08T04:38:37.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff49400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff44163540 0x7eff4415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 -- 172.21.15.164:0/884729264 shutdown_connections 2024-05-08T04:38:37.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7fd6ae800640 1 -- 172.21.15.164:0/884729264 wait complete. 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 -- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff44163540 msgr2=0x7eff4415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff44163540 0x7eff4415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 -- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 msgr2=0x7eff44152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 0x7eff44152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44164090 con 0x7eff440cf6c0 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.603+0000 7eff43e00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 0x7eff44165130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff440cd490 tx=0x7eff3c019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520014070 con 0x7f15340d1c90 2024-05-08T04:38:37.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1520010610 con 0x7f15340d1c90 2024-05-08T04:38:37.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ba40ad020 con 0x7f0bb4070dc0 2024-05-08T04:38:37.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520014070 con 0x7f15340d1c90 2024-05-08T04:38:37.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fb2d9000640 1 Processor -- start 2024-05-08T04:38:37.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0bd020 con 0x7eff440cf6c0 2024-05-08T04:38:37.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 Processor -- start 2024-05-08T04:38:37.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c0b3040 con 0x7eff440cf6c0 2024-05-08T04:38:37.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5238055ee0 con 0x7f5248071580 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff44164370 con 0x7eff440cf6c0 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff44164930 con 0x7eff440cf6c0 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0b2070 con 0x7eff440cf6c0 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 -- start start 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 0x7fd6a00e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 0x7fd6a00dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0057bd0 con 0x7fd6a00e0f90 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0057a50 con 0x7fd6a0177c70 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ae800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a00540b0 con 0x7fd690004810 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ad400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ad400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54420/0 (socket says 172.21.15.164:54420) 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ad400640 1 -- 172.21.15.164:0/99777855 learned_addr learned my addr 172.21.15.164:0/99777855 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fd6ade00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 0x7fd6a00dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 -- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 msgr2=0x7fd6a00dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 0x7fd6a00dfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6aca00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 0x7fd6a00e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 -- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 msgr2=0x7fd6a00e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7fb2d9000640 1 -- start start 2024-05-08T04:38:37.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1534153b50 con 0x7f15340d1c90 2024-05-08T04:38:37.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 0x7fd6a00e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 -- 172.21.15.164:0/99777855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00e3c10 con 0x7fd690004810 2024-05-08T04:38:37.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ade00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 0x7fd6a00dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:37.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6aca00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 0x7fd6a00e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:37.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1534157c90 con 0x7f15340d1c90 2024-05-08T04:38:37.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ad400640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00f7a0 tx=0x7fd69c00fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 0x7fb2d415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d40afbe0 0x7fb2d4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4072570 con 0x7fb2d40cdd30 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40723f0 con 0x7fb2d40afbe0 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4166ab0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54432/0 (socket says 172.21.15.164:54432) 2024-05-08T04:38:37.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 -- 172.21.15.164:0/892377214 learned_addr learned my addr 172.21.15.164:0/892377214 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0050d0 con 0x7fd690004810 2024-05-08T04:38:37.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f4b77800640 1 Processor -- start 2024-05-08T04:38:37.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c018070 con 0x7fd690004810 2024-05-08T04:38:37.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 -- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d40afbe0 msgr2=0x7fb2d4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d40afbe0 0x7fb2d4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 -- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 msgr2=0x7fb2d415c5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.607+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44069d60 con 0x7eff440cf6c0 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00d730 con 0x7fd690004810 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a00e3eb0 con 0x7fd690004810 2024-05-08T04:38:37.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a00e4470 con 0x7fd690004810 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1400640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 0x7fb2d415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 0x7fb2d415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 -- 172.21.15.164:0/892377214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d415a9b0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1400640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 0x7fb2d415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7fb2d1e00640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c450 tx=0x7fb2c400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff3c0c0020 con 0x7eff440cf6c0 2024-05-08T04:38:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7eff40c00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 0x7eff200694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1534157ed0 con 0x7f15340d1c90 2024-05-08T04:38:37.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1520017070 con 0x7f15340d1c90 2024-05-08T04:38:37.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f152be00640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 0x7f1514069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f4b77800640 1 -- start start 2024-05-08T04:38:37.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1520052490 con 0x7f15340d1c90 2024-05-08T04:38:37.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.611+0000 7f1533e00640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 0x7f1514069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7eff48a00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 0x7eff200694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff3c11fcb0 con 0x7eff440cf6c0 2024-05-08T04:38:37.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b77800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b700cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b77800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4b700cca00 0x7f4b7015fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b77800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b700d0720 con 0x7f4b700cda70 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b77800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b70070210 con 0x7f4b700cca00 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b77800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700705b0 con 0x7f4b7006fa90 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f1533e00640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 0x7f1514069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15240ad000 tx=0x7f15240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b74e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4b700cda70 0x7f4b700cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58142/0 (socket says 172.21.15.164:58142) 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b74e00640 1 -- 172.21.15.164:0/606269723 learned_addr learned my addr 172.21.15.164:0/606269723 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:37.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/606269723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b700cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a00057f0 con 0x7fd690004810 2024-05-08T04:38:37.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7eff48a00640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 0x7eff200694f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006cd0 tx=0x7eff2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012830 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d415ac90 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41fb1f0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4017070 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012830 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0052a0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b6fe00640 1 -- 172.21.15.164:0/606269723 >> v2:172.21.15.172:3300/0 conn(0x7f4b700cca00 msgr2=0x7f4b7015fb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:38:37.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/606269723 >> v2:172.21.15.172:3300/0 conn(0x7f4b700cca00 0x7f4b7015fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b6fe00640 1 -- 172.21.15.164:0/606269723 >> v1:172.21.15.164:6789/0 conn(0x7f4b700cda70 legacy=0x7f4b700cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.615+0000 7f4b6fe00640 1 -- 172.21.15.164:0/606269723 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700d0160 con 0x7f4b7006fa90 2024-05-08T04:38:37.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff3c0c7030 con 0x7eff440cf6c0 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd69c00d8d0 con 0x7fd690004810 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7fd6a5600640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 0x7fd688069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd69c0859d0 con 0x7fd690004810 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.619+0000 7fd6aca00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 0x7fd688069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7fd6aca00640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 0x7fd688069520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6940be000 tx=0x7fd694019040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd69c056930 con 0x7fd690004810 2024-05-08T04:38:37.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2c40129d0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/606269723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b700cef30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00f9b0 tx=0x7f4b5c0207f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9c15014de13a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7fb2ca000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 0x7fb2b4069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.623+0000 7f4b6f400640 1 -- 172.21.15.164:0/606269723 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0aa4b0 con 0x7f4b7006fa90 2024-05-08T04:38:37.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b6f400640 1 -- 172.21.15.164:0/606269723 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0aa650 con 0x7f4b7006fa90 2024-05-08T04:38:37.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b6f400640 1 -- 172.21.15.164:0/606269723 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0aaa20 con 0x7f4b7006fa90 2024-05-08T04:38:37.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7fb2d1400640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 0x7fb2b4069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2c4012630 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2c4021020 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1520054af0 con 0x7f15340d1c90 2024-05-08T04:38:37.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7fb2d1400640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 0x7fb2b4069820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00ae000 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 -- 172.21.15.164:0/606269723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 msgr2=0x7f4b700cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 --2- 172.21.15.164:0/606269723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b700cef30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00f9b0 tx=0x7f4b5c0207f0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 -- 172.21.15.164:0/606269723 shutdown_connections 2024-05-08T04:38:37.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 --2- 172.21.15.164:0/606269723 >> v2:172.21.15.172:3300/0 conn(0x7f4b700cca00 0x7f4b7015fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 --2- 172.21.15.164:0/606269723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b700cef30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 -- 172.21.15.164:0/606269723 >> 172.21.15.164:0/606269723 conn(0x7f4b700b34f0 msgr2=0x7f4b700b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 -- 172.21.15.164:0/606269723 shutdown_connections 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.627+0000 7f4b77800640 1 -- 172.21.15.164:0/606269723 wait complete. 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.631+0000 7f4b77800640 1 Processor -- start 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.631+0000 7f4b77800640 1 -- start start 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b7015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b700cda70 0x7f4b701fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b74e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b7015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b74e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b7015ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54440/0 (socket says 172.21.15.164:54440) 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:38908/0 (socket says 172.21.15.164:38908) 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b74e00640 1 -- 172.21.15.164:0/3926577924 learned_addr learned my addr 172.21.15.164:0/3926577924 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70166e70 con 0x7f4b700cda70 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700cd080 con 0x7f4b700cca00 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70166ff0 con 0x7f4b7006fa90 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 -- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 msgr2=0x7f4b7015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b7015ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 -- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b700cda70 msgr2=0x7f4b701fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b700cda70 0x7f4b701fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b701ff5a0 con 0x7f4b700cca00 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6fe00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b0000 tx=0x7f4b5c01d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0d0070 con 0x7f4b700cca00 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b700c5860 con 0x7f4b700cca00 2024-05-08T04:38:37.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.635+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b700c5e20 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.639+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0b2620 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.639+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0b2a10 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.639+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b5c0d3070 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.643+0000 7f4b6cc00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 0x7f4b50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.643+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b5c1301c0 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.643+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b480052a0 con 0x7f4b700cca00 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.643+0000 7f4b74e00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 0x7f4b50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.643+0000 7f4b74e00640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 0x7f4b50069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b700b3fe0 tx=0x7f4b60007f20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:37.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.651+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b5c00d040 con 0x7f4b700cca00 2024-05-08T04:38:37.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.883+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0fd8005860 con 0x7f1000071640 2024-05-08T04:38:37.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.887+0000 7f0ff7400640 1 -- 172.21.15.164:0/1003227325 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ff005ed50 con 0x7f1000071640 2024-05-08T04:38:37.888 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:37.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.887+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 msgr2=0x7f0fdc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 0x7f0fdc0694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1000070ea0 tx=0x7f0fe8008040 comp rx=0 tx=0).stop 2024-05-08T04:38:37.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 msgr2=0x7f100013f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0015d70 tx=0x7f0ff0013520 comp rx=0 tx=0).stop 2024-05-08T04:38:37.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 shutdown_connections 2024-05-08T04:38:37.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0fdc067030 0x7f0fdc0694f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10000cce60 0x7f100015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10000aea10 0x7f1000154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 --2- 172.21.15.164:0/1003227325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1000071640 0x7f100013f110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 >> 172.21.15.164:0/1003227325 conn(0x7f10000cc340 msgr2=0x7f10000b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 shutdown_connections 2024-05-08T04:38:37.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.891+0000 7f1006200640 1 -- 172.21.15.164:0/1003227325 wait complete. 2024-05-08T04:38:37.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-08T04:38:37.925 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.923+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5248068160 con 0x7f5248071580 2024-05-08T04:38:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.927+0000 7f523fe00640 1 -- 172.21.15.164:0/236375372 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5238059f80 con 0x7f5248071580 2024-05-08T04:38:37.929 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 msgr2=0x7f5228069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 0x7f5228069520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52340ad000 tx=0x7f52340a6000 comp rx=0 tx=0).stop 2024-05-08T04:38:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 msgr2=0x7f524815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f523800c7f0 tx=0x7f523800ccb0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 shutdown_connections 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5228067060 0x7f5228069520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52480cec90 0x7f524814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52481441d0 0x7f52481611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 --2- 172.21.15.164:0/236375372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5248071580 0x7f524815e280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 >> 172.21.15.164:0/236375372 conn(0x7f52480cc340 msgr2=0x7f52480b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 shutdown_connections 2024-05-08T04:38:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.931+0000 7f523d600640 1 -- 172.21.15.164:0/236375372 wait complete. 2024-05-08T04:38:37.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.943+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0bb41504b0 con 0x7f0bb4070dc0 2024-05-08T04:38:37.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.943+0000 7f0babe00640 1 -- 172.21.15.164:0/797040065 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ba4016220 con 0x7f0bb4070dc0 2024-05-08T04:38:37.946 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 msgr2=0x7f0b94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 0x7f0b94069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4154760 tx=0x7f0ba0009040 comp rx=0 tx=0).stop 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 msgr2=0x7f0bb4153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ab040 tx=0x7f0ba401ae70 comp rx=0 tx=0).stop 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 shutdown_connections 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bb415a120 0x7f0bb40a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bb41436d0 0x7f0bb40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 --2- 172.21.15.164:0/797040065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bb4070dc0 0x7f0bb4153dd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 >> 172.21.15.164:0/797040065 conn(0x7f0bb40cfaf0 msgr2=0x7f0bb40afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 shutdown_connections 2024-05-08T04:38:37.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.947+0000 7f0bb9400640 1 -- 172.21.15.164:0/797040065 wait complete. 2024-05-08T04:38:37.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.951+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb2ac005860 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2ca000640 1 -- 172.21.15.164:0/892377214 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb2c4055fd0 con 0x7fb2d40cd7e0 2024-05-08T04:38:37.957 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:37.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 msgr2=0x7fb2b4069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 0x7fb2b4069820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00ae000 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2024-05-08T04:38:37.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 msgr2=0x7fb2d415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c450 tx=0x7fb2c400c910 comp rx=0 tx=0).stop 2024-05-08T04:38:37.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 shutdown_connections 2024-05-08T04:38:37.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2b4067360 0x7fb2b4069820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d40afbe0 0x7fb2d4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2d40cdd30 0x7fb2d415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 --2- 172.21.15.164:0/892377214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2d40cd7e0 0x7fb2d415a3b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.955+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 >> 172.21.15.164:0/892377214 conn(0x7fb2d4066fa0 msgr2=0x7fb2d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.959+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 shutdown_connections 2024-05-08T04:38:37.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.959+0000 7fb2d9000640 1 -- 172.21.15.164:0/892377214 wait complete. 2024-05-08T04:38:37.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.959+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6a00ddac0 con 0x7fd690004810 2024-05-08T04:38:37.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.963+0000 7fd6a5600640 1 -- 172.21.15.164:0/99777855 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd69c056230 con 0x7fd690004810 2024-05-08T04:38:37.964 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:37.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.963+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 msgr2=0x7fd688069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.963+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 0x7fd688069520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6940be000 tx=0x7fd694019040 comp rx=0 tx=0).stop 2024-05-08T04:38:37.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.963+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 msgr2=0x7fd6a00e4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.963+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00f7a0 tx=0x7fd69c00fc60 comp rx=0 tx=0).stop 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 shutdown_connections 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd688067060 0x7fd688069520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0177c70 0x7fd6a00dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a00e0f90 0x7fd6a00e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 --2- 172.21.15.164:0/99777855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd690004810 0x7fd6a00e4f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 >> 172.21.15.164:0/99777855 conn(0x7fd6a00559f0 msgr2=0x7fd6a003bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 shutdown_connections 2024-05-08T04:38:37.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.967+0000 7fd6ae800640 1 -- 172.21.15.164:0/99777855 wait complete. 2024-05-08T04:38:37.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-08T04:38:37.971 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:37.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.971+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f15341613c0 con 0x7f15340d1c90 2024-05-08T04:38:37.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.971+0000 7f152be00640 1 -- 172.21.15.164:0/1574240724 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15200543f0 con 0x7f15340d1c90 2024-05-08T04:38:37.974 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 msgr2=0x7f1514069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 0x7f1514069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15240ad000 tx=0x7f15240a6000 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 msgr2=0x7f1534153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f152000b4a0 tx=0x7f152000b960 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 shutdown_connections 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1514067160 0x7f1514069620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15340d2070 0x7f15340cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15340d1c90 0x7f1534153610 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f152000b4a0 tx=0x7f152000b960 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 --2- 172.21.15.164:0/1574240724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15340b7790 0x7f1534154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 >> 172.21.15.164:0/1574240724 conn(0x7f153409cef0 msgr2=0x7f15340b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 shutdown_connections 2024-05-08T04:38:37.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f153b000640 1 -- 172.21.15.164:0/1574240724 wait complete. 2024-05-08T04:38:37.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4b48005590 con 0x7f4b700cca00 2024-05-08T04:38:37.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.975+0000 7f4b6cc00640 1 -- 172.21.15.164:0/3926577924 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b5c01a3e0 con 0x7f4b700cca00 2024-05-08T04:38:37.979 INFO:teuthology.orchestra.run.smithi164.stdout:55834574854 2024-05-08T04:38:37.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-05-08T04:38:37.981 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 msgr2=0x7f4b50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 0x7f4b50069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b700b3fe0 tx=0x7f4b60007f20 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 msgr2=0x7f4b700c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b0000 tx=0x7f4b5c01d590 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 shutdown_connections 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b50066f60 0x7f4b50069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b700cda70 0x7f4b701fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b700cca00 0x7f4b700c6900 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 --2- 172.21.15.164:0/3926577924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b7006fa90 0x7f4b7015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 >> 172.21.15.164:0/3926577924 conn(0x7f4b700b34f0 msgr2=0x7f4b700b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 shutdown_connections 2024-05-08T04:38:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7f4b77800640 1 -- 172.21.15.164:0/3926577924 wait complete. 2024-05-08T04:38:37.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7eff44159950 con 0x7eff440cf6c0 2024-05-08T04:38:37.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.979+0000 7eff40c00640 1 -- 172.21.15.164:0/11878965 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff3c016560 con 0x7eff440cf6c0 2024-05-08T04:38:37.984 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 msgr2=0x7eff200694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 0x7eff200694f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006cd0 tx=0x7eff2c008040 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 msgr2=0x7eff44165130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 0x7eff44165130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff440cd490 tx=0x7eff3c019680 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 shutdown_connections 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff20067030 0x7eff200694f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff440cf6c0 0x7eff44165130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff44163540 0x7eff4415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 --2- 172.21.15.164:0/11878965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440aedb0 0x7eff44152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 >> 172.21.15.164:0/11878965 conn(0x7eff4406ac90 msgr2=0x7eff441531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:37.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.983+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 shutdown_connections 2024-05-08T04:38:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:37.987+0000 7eff4b400640 1 -- 172.21.15.164:0/11878965 wait complete. 2024-05-08T04:38:37.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-05-08T04:38:37.991 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:38.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-08T04:38:38.011 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:38.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-08T04:38:38.012 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:38.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-08T04:38:38.015 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:38.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-05-08T04:38:38.016 DEBUG:teuthology.parallel:result is None 2024-05-08T04:38:38.016 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T04:38:38.016 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:38:38.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 Processor -- start 2024-05-08T04:38:38.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 -- start start 2024-05-08T04:38:38.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73b015ab60 0x7f73b0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73b00b26c0 0x7f73b00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b0072640 con 0x7f73b00cfbd0 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b00d0060 con 0x7f73b00b26c0 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.191+0000 7f73b8000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b006c770 con 0x7f73b015ab60 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73b00b26c0 0x7f73b00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73b00b26c0 0x7f73b00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41570/0 (socket says 172.21.15.164:41570) 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 -- 172.21.15.164:0/1237030614 learned_addr learned my addr 172.21.15.164:0/1237030614 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b5600640 1 --2- 172.21.15.164:0/1237030614 >> v2:172.21.15.172:3300/0 conn(0x7f73b015ab60 0x7f73b0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 -- 172.21.15.164:0/1237030614 >> v2:172.21.15.172:3300/0 conn(0x7f73b015ab60 msgr2=0x7f73b0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 --2- 172.21.15.164:0/1237030614 >> v2:172.21.15.172:3300/0 conn(0x7f73b015ab60 0x7f73b0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 -- 172.21.15.164:0/1237030614 >> v1:172.21.15.172:6789/0 conn(0x7f73b00cfbd0 legacy=0x7f73b00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 -- 172.21.15.164:0/1237030614 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b006ca50 con 0x7f73b00b26c0 2024-05-08T04:38:38.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73b4c00640 1 --2- 172.21.15.164:0/1237030614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b00b26c0 0x7f73b00cf690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73b00cd420 tx=0x7f739c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eecfe9b96f4843 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73afe00640 1 -- 172.21.15.164:0/1237030614 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0b0070 con 0x7f73b00b26c0 2024-05-08T04:38:38.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73afe00640 1 -- 172.21.15.164:0/1237030614 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0ab600 con 0x7f73b00b26c0 2024-05-08T04:38:38.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.195+0000 7f73afe00640 1 -- 172.21.15.164:0/1237030614 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0ab9d0 con 0x7f73b00b26c0 2024-05-08T04:38:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- 172.21.15.164:0/1237030614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b00b26c0 msgr2=0x7f73b00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- 172.21.15.164:0/1237030614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b00b26c0 0x7f73b00cf690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73b00cd420 tx=0x7f739c01c520 comp rx=0 tx=0).stop 2024-05-08T04:38:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- 172.21.15.164:0/1237030614 shutdown_connections 2024-05-08T04:38:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- 172.21.15.164:0/1237030614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b00b26c0 0x7f73b00cf690 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- 172.21.15.164:0/1237030614 >> v2:172.21.15.172:3300/0 conn(0x7f73b015ab60 0x7f73b0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- 172.21.15.164:0/1237030614 >> 172.21.15.164:0/1237030614 conn(0x7f73b006ac90 msgr2=0x7f73b006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:38.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- 172.21.15.164:0/1237030614 shutdown_connections 2024-05-08T04:38:38.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- 172.21.15.164:0/1237030614 wait complete. 2024-05-08T04:38:38.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 Processor -- start 2024-05-08T04:38:38.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- start start 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b00b26c0 0x7f73b014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73b00cfbd0 0x7f73b01377f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b006c550 con 0x7f73b015ab60 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b006c3d0 con 0x7f73b00b26c0 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b8000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00aed30 con 0x7f73b00cfbd0 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41572/0 (socket says 172.21.15.164:41572) 2024-05-08T04:38:38.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 -- 172.21.15.164:0/528231700 learned_addr learned my addr 172.21.15.164:0/528231700 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b4c00640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73b00cfbd0 0x7f73b01377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b5600640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b00b26c0 0x7f73b014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 -- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73b00cfbd0 msgr2=0x7f73b01377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73b00cfbd0 0x7f73b01377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 -- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b00b26c0 msgr2=0x7f73b014c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b00b26c0 0x7f73b014c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.199+0000 7f73b6000640 1 -- 172.21.15.164:0/528231700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b01fa770 con 0x7f73b015ab60 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73b6000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c770 tx=0x7f73a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8003b80 con 0x7f73b015ab60 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8003d20 con 0x7f73b015ab60 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800aa90 con 0x7f73b015ab60 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b01faa50 con 0x7f73b015ab60 2024-05-08T04:38:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b013a1c0 con 0x7f73b015ab60 2024-05-08T04:38:38.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0069d60 con 0x7f73b015ab60 2024-05-08T04:38:38.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73a800ac30 con 0x7f73b015ab60 2024-05-08T04:38:38.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 0x7f7390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73a8084ec0 con 0x7f73b015ab60 2024-05-08T04:38:38.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.203+0000 7f73b5600640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 0x7f7390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.207+0000 7f73b5600640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 0x7f7390069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ba70 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.207+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73a80557e0 con 0x7f73b015ab60 2024-05-08T04:38:38.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.359+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73b01549a0 con 0x7f7390066f60 2024-05-08T04:38:38.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.363+0000 7f73ad600640 1 -- 172.21.15.164:0/528231700 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f73b01549a0 con 0x7f7390066f60 2024-05-08T04:38:38.369 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:38.371 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 msgr2=0x7f7390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 0x7f7390069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ba70 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 msgr2=0x7f73b0138090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c770 tx=0x7f73a800cc30 comp rx=0 tx=0).stop 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 shutdown_connections 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7390066f60 0x7f7390069420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73b00cfbd0 0x7f73b01377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73b015ab60 0x7f73b0138090 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 --2- 172.21.15.164:0/528231700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b00b26c0 0x7f73b014c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 >> 172.21.15.164:0/528231700 conn(0x7f73b006ac90 msgr2=0x7f73b00cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:38.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 shutdown_connections 2024-05-08T04:38:38.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.367+0000 7f73b8000640 1 -- 172.21.15.164:0/528231700 wait complete. 2024-05-08T04:38:38.389 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-08T04:38:37.433091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201972+0000","last_change":"2024-05-08T04:38:29.759413+0000","last_active":"2024-05-08T04:38:31.201972+0000","last_peered":"2024-05-08T04:38:31.201972+0000","last_clean":"2024-05-08T04:38:31.201972+0000","last_became_active":"2024-05-08T04:38:29.756280+0000","last_became_peered":"2024-05-08T04:38:29.756280+0000","last_unstale":"2024-05-08T04:38:31.201972+0000","last_undegraded":"2024-05-08T04:38:31.201972+0000","last_fullsized":"2024-05-08T04:38:31.201972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739754+0000","last_change":"2024-05-08T04:38:29.729346+0000","last_active":"2024-05-08T04:38:30.739754+0000","last_peered":"2024-05-08T04:38:30.739754+0000","last_clean":"2024-05-08T04:38:30.739754+0000","last_became_active":"2024-05-08T04:38:29.728984+0000","last_became_peered":"2024-05-08T04:38:29.728984+0000","last_unstale":"2024-05-08T04:38:30.739754+0000","last_undegraded":"2024-05-08T04:38:30.739754+0000","last_fullsized":"2024-05-08T04:38:30.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200859+0000","last_change":"2024-05-08T04:38:29.735729+0000","last_active":"2024-05-08T04:38:31.200859+0000","last_peered":"2024-05-08T04:38:31.200859+0000","last_clean":"2024-05-08T04:38:31.200859+0000","last_became_active":"2024-05-08T04:38:29.735510+0000","last_became_peered":"2024-05-08T04:38:29.735510+0000","last_unstale":"2024-05-08T04:38:31.200859+0000","last_undegraded":"2024-05-08T04:38:31.200859+0000","last_fullsized":"2024-05-08T04:38:31.200859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741119+0000","last_change":"2024-05-08T04:38:29.746582+0000","last_active":"2024-05-08T04:38:30.741119+0000","last_peered":"2024-05-08T04:38:30.741119+0000","last_clean":"2024-05-08T04:38:30.741119+0000","last_became_active":"2024-05-08T04:38:29.746344+0000","last_became_peered":"2024-05-08T04:38:29.746344+0000","last_unstale":"2024-05-08T04:38:30.741119+0000","last_undegraded":"2024-05-08T04:38:30.741119+0000","last_fullsized":"2024-05-08T04:38:30.741119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201549+0000","last_change":"2024-05-08T04:38:29.751201+0000","last_active":"2024-05-08T04:38:31.201549+0000","last_peered":"2024-05-08T04:38:31.201549+0000","last_clean":"2024-05-08T04:38:31.201549+0000","last_became_active":"2024-05-08T04:38:29.743374+0000","last_became_peered":"2024-05-08T04:38:29.743374+0000","last_unstale":"2024-05-08T04:38:31.201549+0000","last_undegraded":"2024-05-08T04:38:31.201549+0000","last_fullsized":"2024-05-08T04:38:31.201549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740625+0000","last_change":"2024-05-08T04:38:29.743465+0000","last_active":"2024-05-08T04:38:30.740625+0000","last_peered":"2024-05-08T04:38:30.740625+0000","last_clean":"2024-05-08T04:38:30.740625+0000","last_became_active":"2024-05-08T04:38:29.742816+0000","last_became_peered":"2024-05-08T04:38:29.742816+0000","last_unstale":"2024-05-08T04:38:30.740625+0000","last_undegraded":"2024-05-08T04:38:30.740625+0000","last_fullsized":"2024-05-08T04:38:30.740625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201320+0000","last_change":"2024-05-08T04:38:29.753051+0000","last_active":"2024-05-08T04:38:31.201320+0000","last_peered":"2024-05-08T04:38:31.201320+0000","last_clean":"2024-05-08T04:38:31.201320+0000","last_became_active":"2024-05-08T04:38:29.743659+0000","last_became_peered":"2024-05-08T04:38:29.743659+0000","last_unstale":"2024-05-08T04:38:31.201320+0000","last_undegraded":"2024-05-08T04:38:31.201320+0000","last_fullsized":"2024-05-08T04:38:31.201320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.736688+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.736468+0000","last_became_peered":"2024-05-08T04:38:29.736468+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201990+0000","last_change":"2024-05-08T04:38:29.767705+0000","last_active":"2024-05-08T04:38:31.201990+0000","last_peered":"2024-05-08T04:38:31.201990+0000","last_clean":"2024-05-08T04:38:31.201990+0000","last_became_active":"2024-05-08T04:38:29.767379+0000","last_became_peered":"2024-05-08T04:38:29.767379+0000","last_unstale":"2024-05-08T04:38:31.201990+0000","last_undegraded":"2024-05-08T04:38:31.201990+0000","last_fullsized":"2024-05-08T04:38:31.201990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:59.130461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746455+0000","last_change":"2024-05-08T04:38:29.752398+0000","last_active":"2024-05-08T04:38:30.746455+0000","last_peered":"2024-05-08T04:38:30.746455+0000","last_clean":"2024-05-08T04:38:30.746455+0000","last_became_active":"2024-05-08T04:38:29.751505+0000","last_became_peered":"2024-05-08T04:38:29.751505+0000","last_unstale":"2024-05-08T04:38:30.746455+0000","last_undegraded":"2024-05-08T04:38:30.746455+0000","last_fullsized":"2024-05-08T04:38:30.746455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746387+0000","last_change":"2024-05-08T04:38:29.762618+0000","last_active":"2024-05-08T04:38:30.746387+0000","last_peered":"2024-05-08T04:38:30.746387+0000","last_clean":"2024-05-08T04:38:30.746387+0000","last_became_active":"2024-05-08T04:38:29.762321+0000","last_became_peered":"2024-05-08T04:38:29.762321+0000","last_unstale":"2024-05-08T04:38:30.746387+0000","last_undegraded":"2024-05-08T04:38:30.746387+0000","last_fullsized":"2024-05-08T04:38:30.746387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740525+0000","last_change":"2024-05-08T04:38:29.756698+0000","last_active":"2024-05-08T04:38:30.740525+0000","last_peered":"2024-05-08T04:38:30.740525+0000","last_clean":"2024-05-08T04:38:30.740525+0000","last_became_active":"2024-05-08T04:38:29.756073+0000","last_became_peered":"2024-05-08T04:38:29.756073+0000","last_unstale":"2024-05-08T04:38:30.740525+0000","last_undegraded":"2024-05-08T04:38:30.740525+0000","last_fullsized":"2024-05-08T04:38:30.740525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877713+0000","last_change":"2024-05-08T04:38:29.751753+0000","last_active":"2024-05-08T04:38:30.877713+0000","last_peered":"2024-05-08T04:38:30.877713+0000","last_clean":"2024-05-08T04:38:30.877713+0000","last_became_active":"2024-05-08T04:38:29.750686+0000","last_became_peered":"2024-05-08T04:38:29.750686+0000","last_unstale":"2024-05-08T04:38:30.877713+0000","last_undegraded":"2024-05-08T04:38:30.877713+0000","last_fullsized":"2024-05-08T04:38:30.877713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.730987+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.730744+0000","last_became_peered":"2024-05-08T04:38:29.730744+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:55:21.772046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739297+0000","last_change":"2024-05-08T04:38:29.739983+0000","last_active":"2024-05-08T04:38:30.739297+0000","last_peered":"2024-05-08T04:38:30.739297+0000","last_clean":"2024-05-08T04:38:30.739297+0000","last_became_active":"2024-05-08T04:38:29.738511+0000","last_became_peered":"2024-05-08T04:38:29.738511+0000","last_unstale":"2024-05-08T04:38:30.739297+0000","last_undegraded":"2024-05-08T04:38:30.739297+0000","last_fullsized":"2024-05-08T04:38:30.739297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740242+0000","last_change":"2024-05-08T04:38:29.747325+0000","last_active":"2024-05-08T04:38:30.740242+0000","last_peered":"2024-05-08T04:38:30.740242+0000","last_clean":"2024-05-08T04:38:30.740242+0000","last_became_active":"2024-05-08T04:38:29.744848+0000","last_became_peered":"2024-05-08T04:38:29.744848+0000","last_unstale":"2024-05-08T04:38:30.740242+0000","last_undegraded":"2024-05-08T04:38:30.740242+0000","last_fullsized":"2024-05-08T04:38:30.740242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741529+0000","last_change":"2024-05-08T04:38:29.748850+0000","last_active":"2024-05-08T04:38:30.741529+0000","last_peered":"2024-05-08T04:38:30.741529+0000","last_clean":"2024-05-08T04:38:30.741529+0000","last_became_active":"2024-05-08T04:38:29.748560+0000","last_became_peered":"2024-05-08T04:38:29.748560+0000","last_unstale":"2024-05-08T04:38:30.741529+0000","last_undegraded":"2024-05-08T04:38:30.741529+0000","last_fullsized":"2024-05-08T04:38:30.741529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200471+0000","last_change":"2024-05-08T04:38:29.756416+0000","last_active":"2024-05-08T04:38:31.200471+0000","last_peered":"2024-05-08T04:38:31.200471+0000","last_clean":"2024-05-08T04:38:31.200471+0000","last_became_active":"2024-05-08T04:38:29.756194+0000","last_became_peered":"2024-05-08T04:38:29.756194+0000","last_unstale":"2024-05-08T04:38:31.200471+0000","last_undegraded":"2024-05-08T04:38:31.200471+0000","last_fullsized":"2024-05-08T04:38:31.200471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201900+0000","last_change":"2024-05-08T04:38:29.752005+0000","last_active":"2024-05-08T04:38:31.201900+0000","last_peered":"2024-05-08T04:38:31.201900+0000","last_clean":"2024-05-08T04:38:31.201900+0000","last_became_active":"2024-05-08T04:38:29.743651+0000","last_became_peered":"2024-05-08T04:38:29.743651+0000","last_unstale":"2024-05-08T04:38:31.201900+0000","last_undegraded":"2024-05-08T04:38:31.201900+0000","last_fullsized":"2024-05-08T04:38:31.201900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746095+0000","last_change":"2024-05-08T04:38:29.757006+0000","last_active":"2024-05-08T04:38:30.746095+0000","last_peered":"2024-05-08T04:38:30.746095+0000","last_clean":"2024-05-08T04:38:30.746095+0000","last_became_active":"2024-05-08T04:38:29.756756+0000","last_became_peered":"2024-05-08T04:38:29.756756+0000","last_unstale":"2024-05-08T04:38:30.746095+0000","last_undegraded":"2024-05-08T04:38:30.746095+0000","last_fullsized":"2024-05-08T04:38:30.746095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200787+0000","last_change":"2024-05-08T04:38:29.735804+0000","last_active":"2024-05-08T04:38:31.200787+0000","last_peered":"2024-05-08T04:38:31.200787+0000","last_clean":"2024-05-08T04:38:31.200787+0000","last_became_active":"2024-05-08T04:38:29.735527+0000","last_became_peered":"2024-05-08T04:38:29.735527+0000","last_unstale":"2024-05-08T04:38:31.200787+0000","last_undegraded":"2024-05-08T04:38:31.200787+0000","last_fullsized":"2024-05-08T04:38:31.200787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:18.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740806+0000","last_change":"2024-05-08T04:38:29.762810+0000","last_active":"2024-05-08T04:38:30.740806+0000","last_peered":"2024-05-08T04:38:30.740806+0000","last_clean":"2024-05-08T04:38:30.740806+0000","last_became_active":"2024-05-08T04:38:29.762532+0000","last_became_peered":"2024-05-08T04:38:29.762532+0000","last_unstale":"2024-05-08T04:38:30.740806+0000","last_undegraded":"2024-05-08T04:38:30.740806+0000","last_fullsized":"2024-05-08T04:38:30.740806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.756285+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.756024+0000","last_became_peered":"2024-05-08T04:38:29.756024+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740734+0000","last_change":"2024-05-08T04:38:29.747323+0000","last_active":"2024-05-08T04:38:30.740734+0000","last_peered":"2024-05-08T04:38:30.740734+0000","last_clean":"2024-05-08T04:38:30.740734+0000","last_became_active":"2024-05-08T04:38:29.747077+0000","last_became_peered":"2024-05-08T04:38:29.747077+0000","last_unstale":"2024-05-08T04:38:30.740734+0000","last_undegraded":"2024-05-08T04:38:30.740734+0000","last_fullsized":"2024-05-08T04:38:30.740734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878039+0000","last_change":"2024-05-08T04:38:29.748090+0000","last_active":"2024-05-08T04:38:30.878039+0000","last_peered":"2024-05-08T04:38:30.878039+0000","last_clean":"2024-05-08T04:38:30.878039+0000","last_became_active":"2024-05-08T04:38:29.747823+0000","last_became_peered":"2024-05-08T04:38:29.747823+0000","last_unstale":"2024-05-08T04:38:30.878039+0000","last_undegraded":"2024-05-08T04:38:30.878039+0000","last_fullsized":"2024-05-08T04:38:30.878039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746543+0000","last_change":"2024-05-08T04:38:29.766531+0000","last_active":"2024-05-08T04:38:30.746543+0000","last_peered":"2024-05-08T04:38:30.746543+0000","last_clean":"2024-05-08T04:38:30.746543+0000","last_became_active":"2024-05-08T04:38:29.765752+0000","last_became_peered":"2024-05-08T04:38:29.765752+0000","last_unstale":"2024-05-08T04:38:30.746543+0000","last_undegraded":"2024-05-08T04:38:30.746543+0000","last_fullsized":"2024-05-08T04:38:30.746543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:29.741993+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:29.740916+0000","last_became_peered":"2024-05-08T04:38:29.740916+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877964+0000","last_change":"2024-05-08T04:38:29.747721+0000","last_active":"2024-05-08T04:38:30.877964+0000","last_peered":"2024-05-08T04:38:30.877964+0000","last_clean":"2024-05-08T04:38:30.877964+0000","last_became_active":"2024-05-08T04:38:29.747443+0000","last_became_peered":"2024-05-08T04:38:29.747443+0000","last_unstale":"2024-05-08T04:38:30.877964+0000","last_undegraded":"2024-05-08T04:38:30.877964+0000","last_fullsized":"2024-05-08T04:38:30.877964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200927+0000","last_change":"2024-05-08T04:38:29.731332+0000","last_active":"2024-05-08T04:38:31.200927+0000","last_peered":"2024-05-08T04:38:31.200927+0000","last_clean":"2024-05-08T04:38:31.200927+0000","last_became_active":"2024-05-08T04:38:29.731075+0000","last_became_peered":"2024-05-08T04:38:29.731075+0000","last_unstale":"2024-05-08T04:38:31.200927+0000","last_undegraded":"2024-05-08T04:38:31.200927+0000","last_fullsized":"2024-05-08T04:38:31.200927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740872+0000","last_change":"2024-05-08T04:38:29.743680+0000","last_active":"2024-05-08T04:38:30.740872+0000","last_peered":"2024-05-08T04:38:30.740872+0000","last_clean":"2024-05-08T04:38:30.740872+0000","last_became_active":"2024-05-08T04:38:29.743208+0000","last_became_peered":"2024-05-08T04:38:29.743208+0000","last_unstale":"2024-05-08T04:38:30.740872+0000","last_undegraded":"2024-05-08T04:38:30.740872+0000","last_fullsized":"2024-05-08T04:38:30.740872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200255+0000","last_change":"2024-05-08T04:38:29.738213+0000","last_active":"2024-05-08T04:38:31.200255+0000","last_peered":"2024-05-08T04:38:31.200255+0000","last_clean":"2024-05-08T04:38:31.200255+0000","last_became_active":"2024-05-08T04:38:29.737935+0000","last_became_peered":"2024-05-08T04:38:29.737935+0000","last_unstale":"2024-05-08T04:38:31.200255+0000","last_undegraded":"2024-05-08T04:38:31.200255+0000","last_fullsized":"2024-05-08T04:38:31.200255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746039+0000","last_change":"2024-05-08T04:38:29.750290+0000","last_active":"2024-05-08T04:38:30.746039+0000","last_peered":"2024-05-08T04:38:30.746039+0000","last_clean":"2024-05-08T04:38:30.746039+0000","last_became_active":"2024-05-08T04:38:29.749840+0000","last_became_peered":"2024-05-08T04:38:29.749840+0000","last_unstale":"2024-05-08T04:38:30.746039+0000","last_undegraded":"2024-05-08T04:38:30.746039+0000","last_fullsized":"2024-05-08T04:38:30.746039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740040+0000","last_change":"2024-05-08T04:38:29.734132+0000","last_active":"2024-05-08T04:38:30.740040+0000","last_peered":"2024-05-08T04:38:30.740040+0000","last_clean":"2024-05-08T04:38:30.740040+0000","last_became_active":"2024-05-08T04:38:29.733869+0000","last_became_peered":"2024-05-08T04:38:29.733869+0000","last_unstale":"2024-05-08T04:38:30.740040+0000","last_undegraded":"2024-05-08T04:38:30.740040+0000","last_fullsized":"2024-05-08T04:38:30.740040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:28.424596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877812+0000","last_change":"2024-05-08T04:38:28.741244+0000","last_active":"2024-05-08T04:38:30.877812+0000","last_peered":"2024-05-08T04:38:30.877812+0000","last_clean":"2024-05-08T04:38:30.877812+0000","last_became_active":"2024-05-08T04:38:28.740897+0000","last_became_peered":"2024-05-08T04:38:28.740897+0000","last_unstale":"2024-05-08T04:38:30.877812+0000","last_undegraded":"2024-05-08T04:38:30.877812+0000","last_fullsized":"2024-05-08T04:38:30.877812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200940+0000","last_change":"2024-05-08T04:38:29.731992+0000","last_active":"2024-05-08T04:38:31.200940+0000","last_peered":"2024-05-08T04:38:31.200940+0000","last_clean":"2024-05-08T04:38:31.200940+0000","last_became_active":"2024-05-08T04:38:29.731683+0000","last_became_peered":"2024-05-08T04:38:29.731683+0000","last_unstale":"2024-05-08T04:38:31.200940+0000","last_undegraded":"2024-05-08T04:38:31.200940+0000","last_fullsized":"2024-05-08T04:38:31.200940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746000+0000","last_change":"2024-05-08T04:38:28.738380+0000","last_active":"2024-05-08T04:38:30.746000+0000","last_peered":"2024-05-08T04:38:30.746000+0000","last_clean":"2024-05-08T04:38:30.746000+0000","last_became_active":"2024-05-08T04:38:28.738109+0000","last_became_peered":"2024-05-08T04:38:28.738109+0000","last_unstale":"2024-05-08T04:38:30.746000+0000","last_undegraded":"2024-05-08T04:38:30.746000+0000","last_fullsized":"2024-05-08T04:38:30.746000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877587+0000","last_change":"2024-05-08T04:38:28.743973+0000","last_active":"2024-05-08T04:38:30.877587+0000","last_peered":"2024-05-08T04:38:30.877587+0000","last_clean":"2024-05-08T04:38:30.877587+0000","last_became_active":"2024-05-08T04:38:28.743755+0000","last_became_peered":"2024-05-08T04:38:28.743755+0000","last_unstale":"2024-05-08T04:38:30.877587+0000","last_undegraded":"2024-05-08T04:38:30.877587+0000","last_fullsized":"2024-05-08T04:38:30.877587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877578+0000","last_change":"2024-05-08T04:38:29.737294+0000","last_active":"2024-05-08T04:38:30.877578+0000","last_peered":"2024-05-08T04:38:30.877578+0000","last_clean":"2024-05-08T04:38:30.877578+0000","last_became_active":"2024-05-08T04:38:29.737004+0000","last_became_peered":"2024-05-08T04:38:29.737004+0000","last_unstale":"2024-05-08T04:38:30.877578+0000","last_undegraded":"2024-05-08T04:38:30.877578+0000","last_fullsized":"2024-05-08T04:38:30.877578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740741+0000","last_change":"2024-05-08T04:38:28.737564+0000","last_active":"2024-05-08T04:38:30.740741+0000","last_peered":"2024-05-08T04:38:30.740741+0000","last_clean":"2024-05-08T04:38:30.740741+0000","last_became_active":"2024-05-08T04:38:28.737280+0000","last_became_peered":"2024-05-08T04:38:28.737280+0000","last_unstale":"2024-05-08T04:38:30.740741+0000","last_undegraded":"2024-05-08T04:38:30.740741+0000","last_fullsized":"2024-05-08T04:38:30.740741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739295+0000","last_change":"2024-05-08T04:38:29.739772+0000","last_active":"2024-05-08T04:38:30.739295+0000","last_peered":"2024-05-08T04:38:30.739295+0000","last_clean":"2024-05-08T04:38:30.739295+0000","last_became_active":"2024-05-08T04:38:29.738258+0000","last_became_peered":"2024-05-08T04:38:29.738258+0000","last_unstale":"2024-05-08T04:38:30.739295+0000","last_undegraded":"2024-05-08T04:38:30.739295+0000","last_fullsized":"2024-05-08T04:38:30.739295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746413+0000","last_change":"2024-05-08T04:38:28.742877+0000","last_active":"2024-05-08T04:38:30.746413+0000","last_peered":"2024-05-08T04:38:30.746413+0000","last_clean":"2024-05-08T04:38:30.746413+0000","last_became_active":"2024-05-08T04:38:28.742495+0000","last_became_peered":"2024-05-08T04:38:28.742495+0000","last_unstale":"2024-05-08T04:38:30.746413+0000","last_undegraded":"2024-05-08T04:38:30.746413+0000","last_fullsized":"2024-05-08T04:38:30.746413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741690+0000","last_change":"2024-05-08T04:38:29.759053+0000","last_active":"2024-05-08T04:38:30.741690+0000","last_peered":"2024-05-08T04:38:30.741690+0000","last_clean":"2024-05-08T04:38:30.741690+0000","last_became_active":"2024-05-08T04:38:29.758357+0000","last_became_peered":"2024-05-08T04:38:29.758357+0000","last_unstale":"2024-05-08T04:38:30.741690+0000","last_undegraded":"2024-05-08T04:38:30.741690+0000","last_fullsized":"2024-05-08T04:38:30.741690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741139+0000","last_change":"2024-05-08T04:38:28.748874+0000","last_active":"2024-05-08T04:38:30.741139+0000","last_peered":"2024-05-08T04:38:30.741139+0000","last_clean":"2024-05-08T04:38:30.741139+0000","last_became_active":"2024-05-08T04:38:28.748071+0000","last_became_peered":"2024-05-08T04:38:28.748071+0000","last_unstale":"2024-05-08T04:38:30.741139+0000","last_undegraded":"2024-05-08T04:38:30.741139+0000","last_fullsized":"2024-05-08T04:38:30.741139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201063+0000","last_change":"2024-05-08T04:38:29.731689+0000","last_active":"2024-05-08T04:38:31.201063+0000","last_peered":"2024-05-08T04:38:31.201063+0000","last_clean":"2024-05-08T04:38:31.201063+0000","last_became_active":"2024-05-08T04:38:29.731433+0000","last_became_peered":"2024-05-08T04:38:29.731433+0000","last_unstale":"2024-05-08T04:38:31.201063+0000","last_undegraded":"2024-05-08T04:38:31.201063+0000","last_fullsized":"2024-05-08T04:38:31.201063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:28.752051+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:28.751792+0000","last_became_peered":"2024-05-08T04:38:28.751792+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740885+0000","last_change":"2024-05-08T04:38:29.756998+0000","last_active":"2024-05-08T04:38:30.740885+0000","last_peered":"2024-05-08T04:38:30.740885+0000","last_clean":"2024-05-08T04:38:30.740885+0000","last_became_active":"2024-05-08T04:38:29.756395+0000","last_became_peered":"2024-05-08T04:38:29.756395+0000","last_unstale":"2024-05-08T04:38:30.740885+0000","last_undegraded":"2024-05-08T04:38:30.740885+0000","last_fullsized":"2024-05-08T04:38:30.740885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746251+0000","last_change":"2024-05-08T04:38:28.744776+0000","last_active":"2024-05-08T04:38:30.746251+0000","last_peered":"2024-05-08T04:38:30.746251+0000","last_clean":"2024-05-08T04:38:30.746251+0000","last_became_active":"2024-05-08T04:38:28.744410+0000","last_became_peered":"2024-05-08T04:38:28.744410+0000","last_unstale":"2024-05-08T04:38:30.746251+0000","last_undegraded":"2024-05-08T04:38:30.746251+0000","last_fullsized":"2024-05-08T04:38:30.746251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201683+0000","last_change":"2024-05-08T04:38:29.755661+0000","last_active":"2024-05-08T04:38:31.201683+0000","last_peered":"2024-05-08T04:38:31.201683+0000","last_clean":"2024-05-08T04:38:31.201683+0000","last_became_active":"2024-05-08T04:38:29.754401+0000","last_became_peered":"2024-05-08T04:38:29.754401+0000","last_unstale":"2024-05-08T04:38:31.201683+0000","last_undegraded":"2024-05-08T04:38:31.201683+0000","last_fullsized":"2024-05-08T04:38:31.201683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201037+0000","last_change":"2024-05-08T04:38:28.742800+0000","last_active":"2024-05-08T04:38:31.201037+0000","last_peered":"2024-05-08T04:38:31.201037+0000","last_clean":"2024-05-08T04:38:31.201037+0000","last_became_active":"2024-05-08T04:38:28.742491+0000","last_became_peered":"2024-05-08T04:38:28.742491+0000","last_unstale":"2024-05-08T04:38:31.201037+0000","last_undegraded":"2024-05-08T04:38:31.201037+0000","last_fullsized":"2024-05-08T04:38:31.201037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:53.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200760+0000","last_change":"2024-05-08T04:38:29.740544+0000","last_active":"2024-05-08T04:38:31.200760+0000","last_peered":"2024-05-08T04:38:31.200760+0000","last_clean":"2024-05-08T04:38:31.200760+0000","last_became_active":"2024-05-08T04:38:29.740262+0000","last_became_peered":"2024-05-08T04:38:29.740262+0000","last_unstale":"2024-05-08T04:38:31.200760+0000","last_undegraded":"2024-05-08T04:38:31.200760+0000","last_fullsized":"2024-05-08T04:38:31.200760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:15.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740918+0000","last_change":"2024-05-08T04:38:28.751437+0000","last_active":"2024-05-08T04:38:30.740918+0000","last_peered":"2024-05-08T04:38:30.740918+0000","last_clean":"2024-05-08T04:38:30.740918+0000","last_became_active":"2024-05-08T04:38:28.751039+0000","last_became_peered":"2024-05-08T04:38:28.751039+0000","last_unstale":"2024-05-08T04:38:30.740918+0000","last_undegraded":"2024-05-08T04:38:30.740918+0000","last_fullsized":"2024-05-08T04:38:30.740918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201608+0000","last_change":"2024-05-08T04:38:29.756359+0000","last_active":"2024-05-08T04:38:31.201608+0000","last_peered":"2024-05-08T04:38:31.201608+0000","last_clean":"2024-05-08T04:38:31.201608+0000","last_became_active":"2024-05-08T04:38:29.756087+0000","last_became_peered":"2024-05-08T04:38:29.756087+0000","last_unstale":"2024-05-08T04:38:31.201608+0000","last_undegraded":"2024-05-08T04:38:31.201608+0000","last_fullsized":"2024-05-08T04:38:31.201608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.745976+0000","last_change":"2024-05-08T04:38:28.739635+0000","last_active":"2024-05-08T04:38:30.745976+0000","last_peered":"2024-05-08T04:38:30.745976+0000","last_clean":"2024-05-08T04:38:30.745976+0000","last_became_active":"2024-05-08T04:38:28.739255+0000","last_became_peered":"2024-05-08T04:38:28.739255+0000","last_unstale":"2024-05-08T04:38:30.745976+0000","last_undegraded":"2024-05-08T04:38:30.745976+0000","last_fullsized":"2024-05-08T04:38:30.745976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746057+0000","last_change":"2024-05-08T04:38:29.750751+0000","last_active":"2024-05-08T04:38:30.746057+0000","last_peered":"2024-05-08T04:38:30.746057+0000","last_clean":"2024-05-08T04:38:30.746057+0000","last_became_active":"2024-05-08T04:38:29.749978+0000","last_became_peered":"2024-05-08T04:38:29.749978+0000","last_unstale":"2024-05-08T04:38:30.746057+0000","last_undegraded":"2024-05-08T04:38:30.746057+0000","last_fullsized":"2024-05-08T04:38:30.746057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200130+0000","last_change":"2024-05-08T04:38:28.738083+0000","last_active":"2024-05-08T04:38:31.200130+0000","last_peered":"2024-05-08T04:38:31.200130+0000","last_clean":"2024-05-08T04:38:31.200130+0000","last_became_active":"2024-05-08T04:38:28.736994+0000","last_became_peered":"2024-05-08T04:38:28.736994+0000","last_unstale":"2024-05-08T04:38:31.200130+0000","last_undegraded":"2024-05-08T04:38:31.200130+0000","last_fullsized":"2024-05-08T04:38:31.200130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201379+0000","last_change":"2024-05-08T04:38:29.758699+0000","last_active":"2024-05-08T04:38:31.201379+0000","last_peered":"2024-05-08T04:38:31.201379+0000","last_clean":"2024-05-08T04:38:31.201379+0000","last_became_active":"2024-05-08T04:38:29.758439+0000","last_became_peered":"2024-05-08T04:38:29.758439+0000","last_unstale":"2024-05-08T04:38:31.201379+0000","last_undegraded":"2024-05-08T04:38:31.201379+0000","last_fullsized":"2024-05-08T04:38:31.201379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741696+0000","last_change":"2024-05-08T04:38:28.738978+0000","last_active":"2024-05-08T04:38:30.741696+0000","last_peered":"2024-05-08T04:38:30.741696+0000","last_clean":"2024-05-08T04:38:30.741696+0000","last_became_active":"2024-05-08T04:38:28.738688+0000","last_became_peered":"2024-05-08T04:38:28.738688+0000","last_unstale":"2024-05-08T04:38:30.741696+0000","last_undegraded":"2024-05-08T04:38:30.741696+0000","last_fullsized":"2024-05-08T04:38:30.741696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201862+0000","last_change":"2024-05-08T04:38:29.757772+0000","last_active":"2024-05-08T04:38:31.201862+0000","last_peered":"2024-05-08T04:38:31.201862+0000","last_clean":"2024-05-08T04:38:31.201862+0000","last_became_active":"2024-05-08T04:38:29.756730+0000","last_became_peered":"2024-05-08T04:38:29.756730+0000","last_unstale":"2024-05-08T04:38:31.201862+0000","last_undegraded":"2024-05-08T04:38:31.201862+0000","last_fullsized":"2024-05-08T04:38:31.201862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200418+0000","last_change":"2024-05-08T04:38:28.734548+0000","last_active":"2024-05-08T04:38:31.200418+0000","last_peered":"2024-05-08T04:38:31.200418+0000","last_clean":"2024-05-08T04:38:31.200418+0000","last_became_active":"2024-05-08T04:38:28.734179+0000","last_became_peered":"2024-05-08T04:38:28.734179+0000","last_unstale":"2024-05-08T04:38:31.200418+0000","last_undegraded":"2024-05-08T04:38:31.200418+0000","last_fullsized":"2024-05-08T04:38:31.200418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.202085+0000","last_change":"2024-05-08T04:38:29.758477+0000","last_active":"2024-05-08T04:38:31.202085+0000","last_peered":"2024-05-08T04:38:31.202085+0000","last_clean":"2024-05-08T04:38:31.202085+0000","last_became_active":"2024-05-08T04:38:29.756388+0000","last_became_peered":"2024-05-08T04:38:29.756388+0000","last_unstale":"2024-05-08T04:38:31.202085+0000","last_undegraded":"2024-05-08T04:38:31.202085+0000","last_fullsized":"2024-05-08T04:38:31.202085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:06.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739628+0000","last_change":"2024-05-08T04:38:28.739369+0000","last_active":"2024-05-08T04:38:30.739628+0000","last_peered":"2024-05-08T04:38:30.739628+0000","last_clean":"2024-05-08T04:38:30.739628+0000","last_became_active":"2024-05-08T04:38:28.739079+0000","last_became_peered":"2024-05-08T04:38:28.739079+0000","last_unstale":"2024-05-08T04:38:30.739628+0000","last_undegraded":"2024-05-08T04:38:30.739628+0000","last_fullsized":"2024-05-08T04:38:30.739628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741244+0000","last_change":"2024-05-08T04:38:29.748638+0000","last_active":"2024-05-08T04:38:30.741244+0000","last_peered":"2024-05-08T04:38:30.741244+0000","last_clean":"2024-05-08T04:38:30.741244+0000","last_became_active":"2024-05-08T04:38:29.748277+0000","last_became_peered":"2024-05-08T04:38:29.748277+0000","last_unstale":"2024-05-08T04:38:30.741244+0000","last_undegraded":"2024-05-08T04:38:30.741244+0000","last_fullsized":"2024-05-08T04:38:30.741244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200420+0000","last_change":"2024-05-08T04:38:28.737536+0000","last_active":"2024-05-08T04:38:31.200420+0000","last_peered":"2024-05-08T04:38:31.200420+0000","last_clean":"2024-05-08T04:38:31.200420+0000","last_became_active":"2024-05-08T04:38:28.737250+0000","last_became_peered":"2024-05-08T04:38:28.737250+0000","last_unstale":"2024-05-08T04:38:31.200420+0000","last_undegraded":"2024-05-08T04:38:31.200420+0000","last_fullsized":"2024-05-08T04:38:31.200420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201674+0000","last_change":"2024-05-08T04:38:29.762160+0000","last_active":"2024-05-08T04:38:31.201674+0000","last_peered":"2024-05-08T04:38:31.201674+0000","last_clean":"2024-05-08T04:38:31.201674+0000","last_became_active":"2024-05-08T04:38:29.761754+0000","last_became_peered":"2024-05-08T04:38:29.761754+0000","last_unstale":"2024-05-08T04:38:31.201674+0000","last_undegraded":"2024-05-08T04:38:31.201674+0000","last_fullsized":"2024-05-08T04:38:31.201674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201275+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:38:31.201275+0000","last_peered":"2024-05-08T04:38:31.201275+0000","last_clean":"2024-05-08T04:38:31.201275+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:38:31.201275+0000","last_undegraded":"2024-05-08T04:38:31.201275+0000","last_fullsized":"2024-05-08T04:38:31.201275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741367+0000","last_change":"2024-05-08T04:38:28.750045+0000","last_active":"2024-05-08T04:38:30.741367+0000","last_peered":"2024-05-08T04:38:30.741367+0000","last_clean":"2024-05-08T04:38:30.741367+0000","last_became_active":"2024-05-08T04:38:28.749775+0000","last_became_peered":"2024-05-08T04:38:28.749775+0000","last_unstale":"2024-05-08T04:38:30.741367+0000","last_undegraded":"2024-05-08T04:38:30.741367+0000","last_fullsized":"2024-05-08T04:38:30.741367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877692+0000","last_change":"2024-05-08T04:38:29.733287+0000","last_active":"2024-05-08T04:38:30.877692+0000","last_peered":"2024-05-08T04:38:30.877692+0000","last_clean":"2024-05-08T04:38:30.877692+0000","last_became_active":"2024-05-08T04:38:29.733010+0000","last_became_peered":"2024-05-08T04:38:29.733010+0000","last_unstale":"2024-05-08T04:38:30.877692+0000","last_undegraded":"2024-05-08T04:38:30.877692+0000","last_fullsized":"2024-05-08T04:38:30.877692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201257+0000","last_change":"2024-05-08T04:38:28.750803+0000","last_active":"2024-05-08T04:38:31.201257+0000","last_peered":"2024-05-08T04:38:31.201257+0000","last_clean":"2024-05-08T04:38:31.201257+0000","last_became_active":"2024-05-08T04:38:28.750389+0000","last_became_peered":"2024-05-08T04:38:28.750389+0000","last_unstale":"2024-05-08T04:38:31.201257+0000","last_undegraded":"2024-05-08T04:38:31.201257+0000","last_fullsized":"2024-05-08T04:38:31.201257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.734926+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.733112+0000","last_became_peered":"2024-05-08T04:38:29.733112+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200293+0000","last_change":"2024-05-08T04:38:28.733116+0000","last_active":"2024-05-08T04:38:31.200293+0000","last_peered":"2024-05-08T04:38:31.200293+0000","last_clean":"2024-05-08T04:38:31.200293+0000","last_became_active":"2024-05-08T04:38:28.732033+0000","last_became_peered":"2024-05-08T04:38:28.732033+0000","last_unstale":"2024-05-08T04:38:31.200293+0000","last_undegraded":"2024-05-08T04:38:31.200293+0000","last_fullsized":"2024-05-08T04:38:31.200293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739374+0000","last_change":"2024-05-08T04:38:29.740783+0000","last_active":"2024-05-08T04:38:30.739374+0000","last_peered":"2024-05-08T04:38:30.739374+0000","last_clean":"2024-05-08T04:38:30.739374+0000","last_became_active":"2024-05-08T04:38:29.738992+0000","last_became_peered":"2024-05-08T04:38:29.738992+0000","last_unstale":"2024-05-08T04:38:30.739374+0000","last_undegraded":"2024-05-08T04:38:30.739374+0000","last_fullsized":"2024-05-08T04:38:30.739374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740989+0000","last_change":"2024-05-08T04:38:28.744976+0000","last_active":"2024-05-08T04:38:30.740989+0000","last_peered":"2024-05-08T04:38:30.740989+0000","last_clean":"2024-05-08T04:38:30.740989+0000","last_became_active":"2024-05-08T04:38:28.744674+0000","last_became_peered":"2024-05-08T04:38:28.744674+0000","last_unstale":"2024-05-08T04:38:30.740989+0000","last_undegraded":"2024-05-08T04:38:30.740989+0000","last_fullsized":"2024-05-08T04:38:30.740989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740393+0000","last_change":"2024-05-08T04:38:29.743310+0000","last_active":"2024-05-08T04:38:30.740393+0000","last_peered":"2024-05-08T04:38:30.740393+0000","last_clean":"2024-05-08T04:38:30.740393+0000","last_became_active":"2024-05-08T04:38:29.741857+0000","last_became_peered":"2024-05-08T04:38:29.741857+0000","last_unstale":"2024-05-08T04:38:30.740393+0000","last_undegraded":"2024-05-08T04:38:30.740393+0000","last_fullsized":"2024-05-08T04:38:30.740393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:09.896832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739510+0000","last_change":"2024-05-08T04:38:28.735199+0000","last_active":"2024-05-08T04:38:30.739510+0000","last_peered":"2024-05-08T04:38:30.739510+0000","last_clean":"2024-05-08T04:38:30.739510+0000","last_became_active":"2024-05-08T04:38:28.734839+0000","last_became_peered":"2024-05-08T04:38:28.734839+0000","last_unstale":"2024-05-08T04:38:30.739510+0000","last_undegraded":"2024-05-08T04:38:30.739510+0000","last_fullsized":"2024-05-08T04:38:30.739510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741496+0000","last_change":"2024-05-08T04:38:29.743096+0000","last_active":"2024-05-08T04:38:30.741496+0000","last_peered":"2024-05-08T04:38:30.741496+0000","last_clean":"2024-05-08T04:38:30.741496+0000","last_became_active":"2024-05-08T04:38:29.742795+0000","last_became_peered":"2024-05-08T04:38:29.742795+0000","last_unstale":"2024-05-08T04:38:30.741496+0000","last_undegraded":"2024-05-08T04:38:30.741496+0000","last_fullsized":"2024-05-08T04:38:30.741496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201931+0000","last_change":"2024-05-08T04:38:28.749845+0000","last_active":"2024-05-08T04:38:31.201931+0000","last_peered":"2024-05-08T04:38:31.201931+0000","last_clean":"2024-05-08T04:38:31.201931+0000","last_became_active":"2024-05-08T04:38:28.749522+0000","last_became_peered":"2024-05-08T04:38:28.749522+0000","last_unstale":"2024-05-08T04:38:31.201931+0000","last_undegraded":"2024-05-08T04:38:31.201931+0000","last_fullsized":"2024-05-08T04:38:31.201931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:28.101827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739371+0000","last_change":"2024-05-08T04:38:29.733856+0000","last_active":"2024-05-08T04:38:30.739371+0000","last_peered":"2024-05-08T04:38:30.739371+0000","last_clean":"2024-05-08T04:38:30.739371+0000","last_became_active":"2024-05-08T04:38:29.733589+0000","last_became_peered":"2024-05-08T04:38:29.733589+0000","last_unstale":"2024-05-08T04:38:30.739371+0000","last_undegraded":"2024-05-08T04:38:30.739371+0000","last_fullsized":"2024-05-08T04:38:30.739371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.744254+0000","last_change":"2024-05-08T04:38:28.743259+0000","last_active":"2024-05-08T04:38:30.744254+0000","last_peered":"2024-05-08T04:38:30.744254+0000","last_clean":"2024-05-08T04:38:30.744254+0000","last_became_active":"2024-05-08T04:38:28.743022+0000","last_became_peered":"2024-05-08T04:38:28.743022+0000","last_unstale":"2024-05-08T04:38:30.744254+0000","last_undegraded":"2024-05-08T04:38:30.744254+0000","last_fullsized":"2024-05-08T04:38:30.744254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878061+0000","last_change":"2024-05-08T04:38:29.742718+0000","last_active":"2024-05-08T04:38:30.878061+0000","last_peered":"2024-05-08T04:38:30.878061+0000","last_clean":"2024-05-08T04:38:30.878061+0000","last_became_active":"2024-05-08T04:38:29.742135+0000","last_became_peered":"2024-05-08T04:38:29.742135+0000","last_unstale":"2024-05-08T04:38:30.878061+0000","last_undegraded":"2024-05-08T04:38:30.878061+0000","last_fullsized":"2024-05-08T04:38:30.878061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746492+0000","last_change":"2024-05-08T04:38:28.747934+0000","last_active":"2024-05-08T04:38:30.746492+0000","last_peered":"2024-05-08T04:38:30.746492+0000","last_clean":"2024-05-08T04:38:30.746492+0000","last_became_active":"2024-05-08T04:38:28.747524+0000","last_became_peered":"2024-05-08T04:38:28.747524+0000","last_unstale":"2024-05-08T04:38:30.746492+0000","last_undegraded":"2024-05-08T04:38:30.746492+0000","last_fullsized":"2024-05-08T04:38:30.746492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200901+0000","last_change":"2024-05-08T04:38:29.729062+0000","last_active":"2024-05-08T04:38:31.200901+0000","last_peered":"2024-05-08T04:38:31.200901+0000","last_clean":"2024-05-08T04:38:31.200901+0000","last_became_active":"2024-05-08T04:38:29.728777+0000","last_became_peered":"2024-05-08T04:38:29.728777+0000","last_unstale":"2024-05-08T04:38:31.200901+0000","last_undegraded":"2024-05-08T04:38:31.200901+0000","last_fullsized":"2024-05-08T04:38:31.200901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201185+0000","last_change":"2024-05-08T04:38:28.749529+0000","last_active":"2024-05-08T04:38:31.201185+0000","last_peered":"2024-05-08T04:38:31.201185+0000","last_clean":"2024-05-08T04:38:31.201185+0000","last_became_active":"2024-05-08T04:38:28.749167+0000","last_became_peered":"2024-05-08T04:38:28.749167+0000","last_unstale":"2024-05-08T04:38:31.201185+0000","last_undegraded":"2024-05-08T04:38:31.201185+0000","last_fullsized":"2024-05-08T04:38:31.201185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740076+0000","last_change":"2024-05-08T04:38:29.749670+0000","last_active":"2024-05-08T04:38:30.740076+0000","last_peered":"2024-05-08T04:38:30.740076+0000","last_clean":"2024-05-08T04:38:30.740076+0000","last_became_active":"2024-05-08T04:38:29.741613+0000","last_became_peered":"2024-05-08T04:38:29.741613+0000","last_unstale":"2024-05-08T04:38:30.740076+0000","last_undegraded":"2024-05-08T04:38:30.740076+0000","last_fullsized":"2024-05-08T04:38:30.740076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:10.004004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739385+0000","last_change":"2024-05-08T04:38:28.747163+0000","last_active":"2024-05-08T04:38:30.739385+0000","last_peered":"2024-05-08T04:38:30.739385+0000","last_clean":"2024-05-08T04:38:30.739385+0000","last_became_active":"2024-05-08T04:38:28.746783+0000","last_became_peered":"2024-05-08T04:38:28.746783+0000","last_unstale":"2024-05-08T04:38:30.739385+0000","last_undegraded":"2024-05-08T04:38:30.739385+0000","last_fullsized":"2024-05-08T04:38:30.739385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877757+0000","last_change":"2024-05-08T04:38:29.737910+0000","last_active":"2024-05-08T04:38:30.877757+0000","last_peered":"2024-05-08T04:38:30.877757+0000","last_clean":"2024-05-08T04:38:30.877757+0000","last_became_active":"2024-05-08T04:38:29.736005+0000","last_became_peered":"2024-05-08T04:38:29.736005+0000","last_unstale":"2024-05-08T04:38:30.877757+0000","last_undegraded":"2024-05-08T04:38:30.877757+0000","last_fullsized":"2024-05-08T04:38:30.877757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739337+0000","last_change":"2024-05-08T04:38:28.740345+0000","last_active":"2024-05-08T04:38:30.739337+0000","last_peered":"2024-05-08T04:38:30.739337+0000","last_clean":"2024-05-08T04:38:30.739337+0000","last_became_active":"2024-05-08T04:38:28.740070+0000","last_became_peered":"2024-05-08T04:38:28.740070+0000","last_unstale":"2024-05-08T04:38:30.739337+0000","last_undegraded":"2024-05-08T04:38:30.739337+0000","last_fullsized":"2024-05-08T04:38:30.739337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877804+0000","last_change":"2024-05-08T04:38:29.737168+0000","last_active":"2024-05-08T04:38:30.877804+0000","last_peered":"2024-05-08T04:38:30.877804+0000","last_clean":"2024-05-08T04:38:30.877804+0000","last_became_active":"2024-05-08T04:38:29.736816+0000","last_became_peered":"2024-05-08T04:38:29.736816+0000","last_unstale":"2024-05-08T04:38:30.877804+0000","last_undegraded":"2024-05-08T04:38:30.877804+0000","last_fullsized":"2024-05-08T04:38:30.877804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201887+0000","last_change":"2024-05-08T04:38:28.754098+0000","last_active":"2024-05-08T04:38:31.201887+0000","last_peered":"2024-05-08T04:38:31.201887+0000","last_clean":"2024-05-08T04:38:31.201887+0000","last_became_active":"2024-05-08T04:38:28.753787+0000","last_became_peered":"2024-05-08T04:38:28.753787+0000","last_unstale":"2024-05-08T04:38:31.201887+0000","last_undegraded":"2024-05-08T04:38:31.201887+0000","last_fullsized":"2024-05-08T04:38:31.201887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:56.969185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746089+0000","last_change":"2024-05-08T04:38:29.747896+0000","last_active":"2024-05-08T04:38:30.746089+0000","last_peered":"2024-05-08T04:38:30.746089+0000","last_clean":"2024-05-08T04:38:30.746089+0000","last_became_active":"2024-05-08T04:38:29.747661+0000","last_became_peered":"2024-05-08T04:38:29.747661+0000","last_unstale":"2024-05-08T04:38:30.746089+0000","last_undegraded":"2024-05-08T04:38:30.746089+0000","last_fullsized":"2024-05-08T04:38:30.746089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741307+0000","last_change":"2024-05-08T04:38:29.742488+0000","last_active":"2024-05-08T04:38:30.741307+0000","last_peered":"2024-05-08T04:38:30.741307+0000","last_clean":"2024-05-08T04:38:30.741307+0000","last_became_active":"2024-05-08T04:38:29.738959+0000","last_became_peered":"2024-05-08T04:38:29.738959+0000","last_unstale":"2024-05-08T04:38:30.741307+0000","last_undegraded":"2024-05-08T04:38:30.741307+0000","last_fullsized":"2024-05-08T04:38:30.741307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201514+0000","last_change":"2024-05-08T04:38:28.758434+0000","last_active":"2024-05-08T04:38:31.201514+0000","last_peered":"2024-05-08T04:38:31.201514+0000","last_clean":"2024-05-08T04:38:31.201514+0000","last_became_active":"2024-05-08T04:38:28.758108+0000","last_became_peered":"2024-05-08T04:38:28.758108+0000","last_unstale":"2024-05-08T04:38:31.201514+0000","last_undegraded":"2024-05-08T04:38:31.201514+0000","last_fullsized":"2024-05-08T04:38:31.201514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741246+0000","last_change":"2024-05-08T04:38:29.740724+0000","last_active":"2024-05-08T04:38:30.741246+0000","last_peered":"2024-05-08T04:38:30.741246+0000","last_clean":"2024-05-08T04:38:30.741246+0000","last_became_active":"2024-05-08T04:38:29.739520+0000","last_became_peered":"2024-05-08T04:38:29.739520+0000","last_unstale":"2024-05-08T04:38:30.741246+0000","last_undegraded":"2024-05-08T04:38:30.741246+0000","last_fullsized":"2024-05-08T04:38:30.741246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201827+0000","last_change":"2024-05-08T04:38:28.753194+0000","last_active":"2024-05-08T04:38:31.201827+0000","last_peered":"2024-05-08T04:38:31.201827+0000","last_clean":"2024-05-08T04:38:31.201827+0000","last_became_active":"2024-05-08T04:38:28.752937+0000","last_became_peered":"2024-05-08T04:38:28.752937+0000","last_unstale":"2024-05-08T04:38:31.201827+0000","last_undegraded":"2024-05-08T04:38:31.201827+0000","last_fullsized":"2024-05-08T04:38:31.201827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201009+0000","last_change":"2024-05-08T04:38:29.738092+0000","last_active":"2024-05-08T04:38:31.201009+0000","last_peered":"2024-05-08T04:38:31.201009+0000","last_clean":"2024-05-08T04:38:31.201009+0000","last_became_active":"2024-05-08T04:38:29.737806+0000","last_became_peered":"2024-05-08T04:38:29.737806+0000","last_unstale":"2024-05-08T04:38:31.201009+0000","last_undegraded":"2024-05-08T04:38:31.201009+0000","last_fullsized":"2024-05-08T04:38:31.201009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878056+0000","last_change":"2024-05-08T04:38:28.732592+0000","last_active":"2024-05-08T04:38:30.878056+0000","last_peered":"2024-05-08T04:38:30.878056+0000","last_clean":"2024-05-08T04:38:30.878056+0000","last_became_active":"2024-05-08T04:38:28.732270+0000","last_became_peered":"2024-05-08T04:38:28.732270+0000","last_unstale":"2024-05-08T04:38:30.878056+0000","last_undegraded":"2024-05-08T04:38:30.878056+0000","last_fullsized":"2024-05-08T04:38:30.878056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878079+0000","last_change":"2024-05-08T04:38:29.751389+0000","last_active":"2024-05-08T04:38:30.878079+0000","last_peered":"2024-05-08T04:38:30.878079+0000","last_clean":"2024-05-08T04:38:30.878079+0000","last_became_active":"2024-05-08T04:38:29.751025+0000","last_became_peered":"2024-05-08T04:38:29.751025+0000","last_unstale":"2024-05-08T04:38:30.878079+0000","last_undegraded":"2024-05-08T04:38:30.878079+0000","last_fullsized":"2024-05-08T04:38:30.878079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201069+0000","last_change":"2024-05-08T04:38:28.745873+0000","last_active":"2024-05-08T04:38:31.201069+0000","last_peered":"2024-05-08T04:38:31.201069+0000","last_clean":"2024-05-08T04:38:31.201069+0000","last_became_active":"2024-05-08T04:38:28.745607+0000","last_became_peered":"2024-05-08T04:38:28.745607+0000","last_unstale":"2024-05-08T04:38:31.201069+0000","last_undegraded":"2024-05-08T04:38:31.201069+0000","last_fullsized":"2024-05-08T04:38:31.201069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:38:38.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:38:38.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 Processor -- start 2024-05-08T04:38:38.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 -- start start 2024-05-08T04:38:38.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc15b390 0x7f88fc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88fc0aee30 0x7f88fc0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88fc0cef80 0x7f88fc0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc15e8f0 con 0x7f88fc0cef80 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc15a8c0 con 0x7f88fc0aee30 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8903800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06e0e0 con 0x7f88fc15b390 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88fc0cef80 0x7f88fc0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8900e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc15b390 0x7f88fc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88fc0cef80 0x7f88fc0cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41588/0 (socket says 172.21.15.164:41588) 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 -- 172.21.15.164:0/709488232 learned_addr learned my addr 172.21.15.164:0/709488232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f88fbe00640 1 --2- 172.21.15.164:0/709488232 >> v2:172.21.15.172:3300/0 conn(0x7f88fc0aee30 0x7f88fc0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 -- 172.21.15.164:0/709488232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc15b390 msgr2=0x7f88fc06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 --2- 172.21.15.164:0/709488232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc15b390 0x7f88fc06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 -- 172.21.15.164:0/709488232 >> v2:172.21.15.172:3300/0 conn(0x7f88fc0aee30 msgr2=0x7f88fc0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 --2- 172.21.15.164:0/709488232 >> v2:172.21.15.172:3300/0 conn(0x7f88fc0aee30 0x7f88fc0cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.563+0000 7f8901800640 1 -- 172.21.15.164:0/709488232 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc15ed80 con 0x7f88fc0cef80 2024-05-08T04:38:38.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8901800640 1 --2- 172.21.15.164:0/709488232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc0cef80 0x7f88fc0cd630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88fc067430 tx=0x7f88f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef9a193f1d93b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f88fb400640 1 -- 172.21.15.164:0/709488232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40b0070 con 0x7f88fc0cef80 2024-05-08T04:38:38.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f88fb400640 1 -- 172.21.15.164:0/709488232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f40ab600 con 0x7f88fc0cef80 2024-05-08T04:38:38.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f88fb400640 1 -- 172.21.15.164:0/709488232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40ab9d0 con 0x7f88fc0cef80 2024-05-08T04:38:38.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 -- 172.21.15.164:0/709488232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc0cef80 msgr2=0x7f88fc0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 --2- 172.21.15.164:0/709488232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc0cef80 0x7f88fc0cd630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88fc067430 tx=0x7f88f401c520 comp rx=0 tx=0).stop 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 -- 172.21.15.164:0/709488232 shutdown_connections 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 --2- 172.21.15.164:0/709488232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc0cef80 0x7f88fc0cd630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 --2- 172.21.15.164:0/709488232 >> v2:172.21.15.172:3300/0 conn(0x7f88fc0aee30 0x7f88fc0cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 --2- 172.21.15.164:0/709488232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc15b390 0x7f88fc06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 -- 172.21.15.164:0/709488232 >> 172.21.15.164:0/709488232 conn(0x7f88fc06ac90 msgr2=0x7f88fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 -- 172.21.15.164:0/709488232 shutdown_connections 2024-05-08T04:38:38.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.567+0000 7f8903800640 1 -- 172.21.15.164:0/709488232 wait complete. 2024-05-08T04:38:38.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 Processor -- start 2024-05-08T04:38:38.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- start start 2024-05-08T04:38:38.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc0aee30 0x7f88fc1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88fc0cef80 0x7f88fc1f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15b770 con 0x7f88fc15b390 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06ddf0 con 0x7f88fc0cef80 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cdb70 con 0x7f88fc0aee30 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41596/0 (socket says 172.21.15.164:41596) 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 -- 172.21.15.164:0/2959960812 learned_addr learned my addr 172.21.15.164:0/2959960812 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8900e00640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc0aee30 0x7f88fc1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f88fbe00640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88fc0cef80 0x7f88fc1f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 -- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc0aee30 msgr2=0x7f88fc1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc0aee30 0x7f88fc1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 -- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88fc0cef80 msgr2=0x7f88fc1f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88fc0cef80 0x7f88fc1f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 -- 172.21.15.164:0/2959960812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc1f4ef0 con 0x7f88fc15b390 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8901800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88f401abd0 tx=0x7f88f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40ba020 con 0x7f88fc15b390 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f40abe30 con 0x7f88fc15b390 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc1f51d0 con 0x7f88fc15b390 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40b0040 con 0x7f88fc15b390 2024-05-08T04:38:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.571+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc1f5790 con 0x7f88fc15b390 2024-05-08T04:38:38.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc0ce9a0 con 0x7f88fc15b390 2024-05-08T04:38:38.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88f40bd020 con 0x7f88fc15b390 2024-05-08T04:38:38.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f88f8c00640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 0x7f88dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88f411efa0 con 0x7f88fc15b390 2024-05-08T04:38:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f8900e00640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 0x7f88dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.575+0000 7f8900e00640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 0x7f88dc069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00a3d0 tx=0x7f88ec014040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.579+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88f40c4030 con 0x7f88fc15b390 2024-05-08T04:38:38.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.731+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88fc069d60 con 0x7f88dc066f60 2024-05-08T04:38:38.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.739+0000 7f88f8c00640 1 -- 172.21.15.164:0/2959960812 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f88ec05e7d0 con 0x7f88dc066f60 2024-05-08T04:38:38.743 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:38.744 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:38:38.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.739+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 msgr2=0x7f88dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.739+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 0x7f88dc069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00a3d0 tx=0x7f88ec014040 comp rx=0 tx=0).stop 2024-05-08T04:38:38.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 msgr2=0x7f88fc14b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88f401abd0 tx=0x7f88f401ad60 comp rx=0 tx=0).stop 2024-05-08T04:38:38.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 shutdown_connections 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88dc066f60 0x7f88dc069420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88fc15b390 0x7f88fc14b210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88fc0cef80 0x7f88fc1f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 --2- 172.21.15.164:0/2959960812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88fc0aee30 0x7f88fc1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 >> 172.21.15.164:0/2959960812 conn(0x7f88fc06ac90 msgr2=0x7f88fc152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 shutdown_connections 2024-05-08T04:38:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.743+0000 7f8903800640 1 -- 172.21.15.164:0/2959960812 wait complete. 2024-05-08T04:38:38.762 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-08T04:38:37.433091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201972+0000","last_change":"2024-05-08T04:38:29.759413+0000","last_active":"2024-05-08T04:38:31.201972+0000","last_peered":"2024-05-08T04:38:31.201972+0000","last_clean":"2024-05-08T04:38:31.201972+0000","last_became_active":"2024-05-08T04:38:29.756280+0000","last_became_peered":"2024-05-08T04:38:29.756280+0000","last_unstale":"2024-05-08T04:38:31.201972+0000","last_undegraded":"2024-05-08T04:38:31.201972+0000","last_fullsized":"2024-05-08T04:38:31.201972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739754+0000","last_change":"2024-05-08T04:38:29.729346+0000","last_active":"2024-05-08T04:38:30.739754+0000","last_peered":"2024-05-08T04:38:30.739754+0000","last_clean":"2024-05-08T04:38:30.739754+0000","last_became_active":"2024-05-08T04:38:29.728984+0000","last_became_peered":"2024-05-08T04:38:29.728984+0000","last_unstale":"2024-05-08T04:38:30.739754+0000","last_undegraded":"2024-05-08T04:38:30.739754+0000","last_fullsized":"2024-05-08T04:38:30.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200859+0000","last_change":"2024-05-08T04:38:29.735729+0000","last_active":"2024-05-08T04:38:31.200859+0000","last_peered":"2024-05-08T04:38:31.200859+0000","last_clean":"2024-05-08T04:38:31.200859+0000","last_became_active":"2024-05-08T04:38:29.735510+0000","last_became_peered":"2024-05-08T04:38:29.735510+0000","last_unstale":"2024-05-08T04:38:31.200859+0000","last_undegraded":"2024-05-08T04:38:31.200859+0000","last_fullsized":"2024-05-08T04:38:31.200859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741119+0000","last_change":"2024-05-08T04:38:29.746582+0000","last_active":"2024-05-08T04:38:30.741119+0000","last_peered":"2024-05-08T04:38:30.741119+0000","last_clean":"2024-05-08T04:38:30.741119+0000","last_became_active":"2024-05-08T04:38:29.746344+0000","last_became_peered":"2024-05-08T04:38:29.746344+0000","last_unstale":"2024-05-08T04:38:30.741119+0000","last_undegraded":"2024-05-08T04:38:30.741119+0000","last_fullsized":"2024-05-08T04:38:30.741119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201549+0000","last_change":"2024-05-08T04:38:29.751201+0000","last_active":"2024-05-08T04:38:31.201549+0000","last_peered":"2024-05-08T04:38:31.201549+0000","last_clean":"2024-05-08T04:38:31.201549+0000","last_became_active":"2024-05-08T04:38:29.743374+0000","last_became_peered":"2024-05-08T04:38:29.743374+0000","last_unstale":"2024-05-08T04:38:31.201549+0000","last_undegraded":"2024-05-08T04:38:31.201549+0000","last_fullsized":"2024-05-08T04:38:31.201549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740625+0000","last_change":"2024-05-08T04:38:29.743465+0000","last_active":"2024-05-08T04:38:30.740625+0000","last_peered":"2024-05-08T04:38:30.740625+0000","last_clean":"2024-05-08T04:38:30.740625+0000","last_became_active":"2024-05-08T04:38:29.742816+0000","last_became_peered":"2024-05-08T04:38:29.742816+0000","last_unstale":"2024-05-08T04:38:30.740625+0000","last_undegraded":"2024-05-08T04:38:30.740625+0000","last_fullsized":"2024-05-08T04:38:30.740625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201320+0000","last_change":"2024-05-08T04:38:29.753051+0000","last_active":"2024-05-08T04:38:31.201320+0000","last_peered":"2024-05-08T04:38:31.201320+0000","last_clean":"2024-05-08T04:38:31.201320+0000","last_became_active":"2024-05-08T04:38:29.743659+0000","last_became_peered":"2024-05-08T04:38:29.743659+0000","last_unstale":"2024-05-08T04:38:31.201320+0000","last_undegraded":"2024-05-08T04:38:31.201320+0000","last_fullsized":"2024-05-08T04:38:31.201320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.736688+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.736468+0000","last_became_peered":"2024-05-08T04:38:29.736468+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201990+0000","last_change":"2024-05-08T04:38:29.767705+0000","last_active":"2024-05-08T04:38:31.201990+0000","last_peered":"2024-05-08T04:38:31.201990+0000","last_clean":"2024-05-08T04:38:31.201990+0000","last_became_active":"2024-05-08T04:38:29.767379+0000","last_became_peered":"2024-05-08T04:38:29.767379+0000","last_unstale":"2024-05-08T04:38:31.201990+0000","last_undegraded":"2024-05-08T04:38:31.201990+0000","last_fullsized":"2024-05-08T04:38:31.201990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:59.130461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746455+0000","last_change":"2024-05-08T04:38:29.752398+0000","last_active":"2024-05-08T04:38:30.746455+0000","last_peered":"2024-05-08T04:38:30.746455+0000","last_clean":"2024-05-08T04:38:30.746455+0000","last_became_active":"2024-05-08T04:38:29.751505+0000","last_became_peered":"2024-05-08T04:38:29.751505+0000","last_unstale":"2024-05-08T04:38:30.746455+0000","last_undegraded":"2024-05-08T04:38:30.746455+0000","last_fullsized":"2024-05-08T04:38:30.746455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746387+0000","last_change":"2024-05-08T04:38:29.762618+0000","last_active":"2024-05-08T04:38:30.746387+0000","last_peered":"2024-05-08T04:38:30.746387+0000","last_clean":"2024-05-08T04:38:30.746387+0000","last_became_active":"2024-05-08T04:38:29.762321+0000","last_became_peered":"2024-05-08T04:38:29.762321+0000","last_unstale":"2024-05-08T04:38:30.746387+0000","last_undegraded":"2024-05-08T04:38:30.746387+0000","last_fullsized":"2024-05-08T04:38:30.746387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740525+0000","last_change":"2024-05-08T04:38:29.756698+0000","last_active":"2024-05-08T04:38:30.740525+0000","last_peered":"2024-05-08T04:38:30.740525+0000","last_clean":"2024-05-08T04:38:30.740525+0000","last_became_active":"2024-05-08T04:38:29.756073+0000","last_became_peered":"2024-05-08T04:38:29.756073+0000","last_unstale":"2024-05-08T04:38:30.740525+0000","last_undegraded":"2024-05-08T04:38:30.740525+0000","last_fullsized":"2024-05-08T04:38:30.740525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877713+0000","last_change":"2024-05-08T04:38:29.751753+0000","last_active":"2024-05-08T04:38:30.877713+0000","last_peered":"2024-05-08T04:38:30.877713+0000","last_clean":"2024-05-08T04:38:30.877713+0000","last_became_active":"2024-05-08T04:38:29.750686+0000","last_became_peered":"2024-05-08T04:38:29.750686+0000","last_unstale":"2024-05-08T04:38:30.877713+0000","last_undegraded":"2024-05-08T04:38:30.877713+0000","last_fullsized":"2024-05-08T04:38:30.877713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.730987+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.730744+0000","last_became_peered":"2024-05-08T04:38:29.730744+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:55:21.772046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739297+0000","last_change":"2024-05-08T04:38:29.739983+0000","last_active":"2024-05-08T04:38:30.739297+0000","last_peered":"2024-05-08T04:38:30.739297+0000","last_clean":"2024-05-08T04:38:30.739297+0000","last_became_active":"2024-05-08T04:38:29.738511+0000","last_became_peered":"2024-05-08T04:38:29.738511+0000","last_unstale":"2024-05-08T04:38:30.739297+0000","last_undegraded":"2024-05-08T04:38:30.739297+0000","last_fullsized":"2024-05-08T04:38:30.739297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740242+0000","last_change":"2024-05-08T04:38:29.747325+0000","last_active":"2024-05-08T04:38:30.740242+0000","last_peered":"2024-05-08T04:38:30.740242+0000","last_clean":"2024-05-08T04:38:30.740242+0000","last_became_active":"2024-05-08T04:38:29.744848+0000","last_became_peered":"2024-05-08T04:38:29.744848+0000","last_unstale":"2024-05-08T04:38:30.740242+0000","last_undegraded":"2024-05-08T04:38:30.740242+0000","last_fullsized":"2024-05-08T04:38:30.740242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741529+0000","last_change":"2024-05-08T04:38:29.748850+0000","last_active":"2024-05-08T04:38:30.741529+0000","last_peered":"2024-05-08T04:38:30.741529+0000","last_clean":"2024-05-08T04:38:30.741529+0000","last_became_active":"2024-05-08T04:38:29.748560+0000","last_became_peered":"2024-05-08T04:38:29.748560+0000","last_unstale":"2024-05-08T04:38:30.741529+0000","last_undegraded":"2024-05-08T04:38:30.741529+0000","last_fullsized":"2024-05-08T04:38:30.741529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200471+0000","last_change":"2024-05-08T04:38:29.756416+0000","last_active":"2024-05-08T04:38:31.200471+0000","last_peered":"2024-05-08T04:38:31.200471+0000","last_clean":"2024-05-08T04:38:31.200471+0000","last_became_active":"2024-05-08T04:38:29.756194+0000","last_became_peered":"2024-05-08T04:38:29.756194+0000","last_unstale":"2024-05-08T04:38:31.200471+0000","last_undegraded":"2024-05-08T04:38:31.200471+0000","last_fullsized":"2024-05-08T04:38:31.200471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201900+0000","last_change":"2024-05-08T04:38:29.752005+0000","last_active":"2024-05-08T04:38:31.201900+0000","last_peered":"2024-05-08T04:38:31.201900+0000","last_clean":"2024-05-08T04:38:31.201900+0000","last_became_active":"2024-05-08T04:38:29.743651+0000","last_became_peered":"2024-05-08T04:38:29.743651+0000","last_unstale":"2024-05-08T04:38:31.201900+0000","last_undegraded":"2024-05-08T04:38:31.201900+0000","last_fullsized":"2024-05-08T04:38:31.201900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:10.998840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746095+0000","last_change":"2024-05-08T04:38:29.757006+0000","last_active":"2024-05-08T04:38:30.746095+0000","last_peered":"2024-05-08T04:38:30.746095+0000","last_clean":"2024-05-08T04:38:30.746095+0000","last_became_active":"2024-05-08T04:38:29.756756+0000","last_became_peered":"2024-05-08T04:38:29.756756+0000","last_unstale":"2024-05-08T04:38:30.746095+0000","last_undegraded":"2024-05-08T04:38:30.746095+0000","last_fullsized":"2024-05-08T04:38:30.746095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200787+0000","last_change":"2024-05-08T04:38:29.735804+0000","last_active":"2024-05-08T04:38:31.200787+0000","last_peered":"2024-05-08T04:38:31.200787+0000","last_clean":"2024-05-08T04:38:31.200787+0000","last_became_active":"2024-05-08T04:38:29.735527+0000","last_became_peered":"2024-05-08T04:38:29.735527+0000","last_unstale":"2024-05-08T04:38:31.200787+0000","last_undegraded":"2024-05-08T04:38:31.200787+0000","last_fullsized":"2024-05-08T04:38:31.200787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:18.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740806+0000","last_change":"2024-05-08T04:38:29.762810+0000","last_active":"2024-05-08T04:38:30.740806+0000","last_peered":"2024-05-08T04:38:30.740806+0000","last_clean":"2024-05-08T04:38:30.740806+0000","last_became_active":"2024-05-08T04:38:29.762532+0000","last_became_peered":"2024-05-08T04:38:29.762532+0000","last_unstale":"2024-05-08T04:38:30.740806+0000","last_undegraded":"2024-05-08T04:38:30.740806+0000","last_fullsized":"2024-05-08T04:38:30.740806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877550+0000","last_change":"2024-05-08T04:38:29.756285+0000","last_active":"2024-05-08T04:38:30.877550+0000","last_peered":"2024-05-08T04:38:30.877550+0000","last_clean":"2024-05-08T04:38:30.877550+0000","last_became_active":"2024-05-08T04:38:29.756024+0000","last_became_peered":"2024-05-08T04:38:29.756024+0000","last_unstale":"2024-05-08T04:38:30.877550+0000","last_undegraded":"2024-05-08T04:38:30.877550+0000","last_fullsized":"2024-05-08T04:38:30.877550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740734+0000","last_change":"2024-05-08T04:38:29.747323+0000","last_active":"2024-05-08T04:38:30.740734+0000","last_peered":"2024-05-08T04:38:30.740734+0000","last_clean":"2024-05-08T04:38:30.740734+0000","last_became_active":"2024-05-08T04:38:29.747077+0000","last_became_peered":"2024-05-08T04:38:29.747077+0000","last_unstale":"2024-05-08T04:38:30.740734+0000","last_undegraded":"2024-05-08T04:38:30.740734+0000","last_fullsized":"2024-05-08T04:38:30.740734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878039+0000","last_change":"2024-05-08T04:38:29.748090+0000","last_active":"2024-05-08T04:38:30.878039+0000","last_peered":"2024-05-08T04:38:30.878039+0000","last_clean":"2024-05-08T04:38:30.878039+0000","last_became_active":"2024-05-08T04:38:29.747823+0000","last_became_peered":"2024-05-08T04:38:29.747823+0000","last_unstale":"2024-05-08T04:38:30.878039+0000","last_undegraded":"2024-05-08T04:38:30.878039+0000","last_fullsized":"2024-05-08T04:38:30.878039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746543+0000","last_change":"2024-05-08T04:38:29.766531+0000","last_active":"2024-05-08T04:38:30.746543+0000","last_peered":"2024-05-08T04:38:30.746543+0000","last_clean":"2024-05-08T04:38:30.746543+0000","last_became_active":"2024-05-08T04:38:29.765752+0000","last_became_peered":"2024-05-08T04:38:29.765752+0000","last_unstale":"2024-05-08T04:38:30.746543+0000","last_undegraded":"2024-05-08T04:38:30.746543+0000","last_fullsized":"2024-05-08T04:38:30.746543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:29.741993+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:29.740916+0000","last_became_peered":"2024-05-08T04:38:29.740916+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877964+0000","last_change":"2024-05-08T04:38:29.747721+0000","last_active":"2024-05-08T04:38:30.877964+0000","last_peered":"2024-05-08T04:38:30.877964+0000","last_clean":"2024-05-08T04:38:30.877964+0000","last_became_active":"2024-05-08T04:38:29.747443+0000","last_became_peered":"2024-05-08T04:38:29.747443+0000","last_unstale":"2024-05-08T04:38:30.877964+0000","last_undegraded":"2024-05-08T04:38:30.877964+0000","last_fullsized":"2024-05-08T04:38:30.877964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:05:39.774487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200927+0000","last_change":"2024-05-08T04:38:29.731332+0000","last_active":"2024-05-08T04:38:31.200927+0000","last_peered":"2024-05-08T04:38:31.200927+0000","last_clean":"2024-05-08T04:38:31.200927+0000","last_became_active":"2024-05-08T04:38:29.731075+0000","last_became_peered":"2024-05-08T04:38:29.731075+0000","last_unstale":"2024-05-08T04:38:31.200927+0000","last_undegraded":"2024-05-08T04:38:31.200927+0000","last_fullsized":"2024-05-08T04:38:31.200927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740872+0000","last_change":"2024-05-08T04:38:29.743680+0000","last_active":"2024-05-08T04:38:30.740872+0000","last_peered":"2024-05-08T04:38:30.740872+0000","last_clean":"2024-05-08T04:38:30.740872+0000","last_became_active":"2024-05-08T04:38:29.743208+0000","last_became_peered":"2024-05-08T04:38:29.743208+0000","last_unstale":"2024-05-08T04:38:30.740872+0000","last_undegraded":"2024-05-08T04:38:30.740872+0000","last_fullsized":"2024-05-08T04:38:30.740872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200255+0000","last_change":"2024-05-08T04:38:29.738213+0000","last_active":"2024-05-08T04:38:31.200255+0000","last_peered":"2024-05-08T04:38:31.200255+0000","last_clean":"2024-05-08T04:38:31.200255+0000","last_became_active":"2024-05-08T04:38:29.737935+0000","last_became_peered":"2024-05-08T04:38:29.737935+0000","last_unstale":"2024-05-08T04:38:31.200255+0000","last_undegraded":"2024-05-08T04:38:31.200255+0000","last_fullsized":"2024-05-08T04:38:31.200255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:26.928204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746039+0000","last_change":"2024-05-08T04:38:29.750290+0000","last_active":"2024-05-08T04:38:30.746039+0000","last_peered":"2024-05-08T04:38:30.746039+0000","last_clean":"2024-05-08T04:38:30.746039+0000","last_became_active":"2024-05-08T04:38:29.749840+0000","last_became_peered":"2024-05-08T04:38:29.749840+0000","last_unstale":"2024-05-08T04:38:30.746039+0000","last_undegraded":"2024-05-08T04:38:30.746039+0000","last_fullsized":"2024-05-08T04:38:30.746039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:33.447502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740040+0000","last_change":"2024-05-08T04:38:29.734132+0000","last_active":"2024-05-08T04:38:30.740040+0000","last_peered":"2024-05-08T04:38:30.740040+0000","last_clean":"2024-05-08T04:38:30.740040+0000","last_became_active":"2024-05-08T04:38:29.733869+0000","last_became_peered":"2024-05-08T04:38:29.733869+0000","last_unstale":"2024-05-08T04:38:30.740040+0000","last_undegraded":"2024-05-08T04:38:30.740040+0000","last_fullsized":"2024-05-08T04:38:30.740040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:28.424596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877812+0000","last_change":"2024-05-08T04:38:28.741244+0000","last_active":"2024-05-08T04:38:30.877812+0000","last_peered":"2024-05-08T04:38:30.877812+0000","last_clean":"2024-05-08T04:38:30.877812+0000","last_became_active":"2024-05-08T04:38:28.740897+0000","last_became_peered":"2024-05-08T04:38:28.740897+0000","last_unstale":"2024-05-08T04:38:30.877812+0000","last_undegraded":"2024-05-08T04:38:30.877812+0000","last_fullsized":"2024-05-08T04:38:30.877812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200940+0000","last_change":"2024-05-08T04:38:29.731992+0000","last_active":"2024-05-08T04:38:31.200940+0000","last_peered":"2024-05-08T04:38:31.200940+0000","last_clean":"2024-05-08T04:38:31.200940+0000","last_became_active":"2024-05-08T04:38:29.731683+0000","last_became_peered":"2024-05-08T04:38:29.731683+0000","last_unstale":"2024-05-08T04:38:31.200940+0000","last_undegraded":"2024-05-08T04:38:31.200940+0000","last_fullsized":"2024-05-08T04:38:31.200940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746000+0000","last_change":"2024-05-08T04:38:28.738380+0000","last_active":"2024-05-08T04:38:30.746000+0000","last_peered":"2024-05-08T04:38:30.746000+0000","last_clean":"2024-05-08T04:38:30.746000+0000","last_became_active":"2024-05-08T04:38:28.738109+0000","last_became_peered":"2024-05-08T04:38:28.738109+0000","last_unstale":"2024-05-08T04:38:30.746000+0000","last_undegraded":"2024-05-08T04:38:30.746000+0000","last_fullsized":"2024-05-08T04:38:30.746000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877587+0000","last_change":"2024-05-08T04:38:28.743973+0000","last_active":"2024-05-08T04:38:30.877587+0000","last_peered":"2024-05-08T04:38:30.877587+0000","last_clean":"2024-05-08T04:38:30.877587+0000","last_became_active":"2024-05-08T04:38:28.743755+0000","last_became_peered":"2024-05-08T04:38:28.743755+0000","last_unstale":"2024-05-08T04:38:30.877587+0000","last_undegraded":"2024-05-08T04:38:30.877587+0000","last_fullsized":"2024-05-08T04:38:30.877587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877578+0000","last_change":"2024-05-08T04:38:29.737294+0000","last_active":"2024-05-08T04:38:30.877578+0000","last_peered":"2024-05-08T04:38:30.877578+0000","last_clean":"2024-05-08T04:38:30.877578+0000","last_became_active":"2024-05-08T04:38:29.737004+0000","last_became_peered":"2024-05-08T04:38:29.737004+0000","last_unstale":"2024-05-08T04:38:30.877578+0000","last_undegraded":"2024-05-08T04:38:30.877578+0000","last_fullsized":"2024-05-08T04:38:30.877578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740741+0000","last_change":"2024-05-08T04:38:28.737564+0000","last_active":"2024-05-08T04:38:30.740741+0000","last_peered":"2024-05-08T04:38:30.740741+0000","last_clean":"2024-05-08T04:38:30.740741+0000","last_became_active":"2024-05-08T04:38:28.737280+0000","last_became_peered":"2024-05-08T04:38:28.737280+0000","last_unstale":"2024-05-08T04:38:30.740741+0000","last_undegraded":"2024-05-08T04:38:30.740741+0000","last_fullsized":"2024-05-08T04:38:30.740741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739295+0000","last_change":"2024-05-08T04:38:29.739772+0000","last_active":"2024-05-08T04:38:30.739295+0000","last_peered":"2024-05-08T04:38:30.739295+0000","last_clean":"2024-05-08T04:38:30.739295+0000","last_became_active":"2024-05-08T04:38:29.738258+0000","last_became_peered":"2024-05-08T04:38:29.738258+0000","last_unstale":"2024-05-08T04:38:30.739295+0000","last_undegraded":"2024-05-08T04:38:30.739295+0000","last_fullsized":"2024-05-08T04:38:30.739295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746413+0000","last_change":"2024-05-08T04:38:28.742877+0000","last_active":"2024-05-08T04:38:30.746413+0000","last_peered":"2024-05-08T04:38:30.746413+0000","last_clean":"2024-05-08T04:38:30.746413+0000","last_became_active":"2024-05-08T04:38:28.742495+0000","last_became_peered":"2024-05-08T04:38:28.742495+0000","last_unstale":"2024-05-08T04:38:30.746413+0000","last_undegraded":"2024-05-08T04:38:30.746413+0000","last_fullsized":"2024-05-08T04:38:30.746413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741690+0000","last_change":"2024-05-08T04:38:29.759053+0000","last_active":"2024-05-08T04:38:30.741690+0000","last_peered":"2024-05-08T04:38:30.741690+0000","last_clean":"2024-05-08T04:38:30.741690+0000","last_became_active":"2024-05-08T04:38:29.758357+0000","last_became_peered":"2024-05-08T04:38:29.758357+0000","last_unstale":"2024-05-08T04:38:30.741690+0000","last_undegraded":"2024-05-08T04:38:30.741690+0000","last_fullsized":"2024-05-08T04:38:30.741690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:05.508549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741139+0000","last_change":"2024-05-08T04:38:28.748874+0000","last_active":"2024-05-08T04:38:30.741139+0000","last_peered":"2024-05-08T04:38:30.741139+0000","last_clean":"2024-05-08T04:38:30.741139+0000","last_became_active":"2024-05-08T04:38:28.748071+0000","last_became_peered":"2024-05-08T04:38:28.748071+0000","last_unstale":"2024-05-08T04:38:30.741139+0000","last_undegraded":"2024-05-08T04:38:30.741139+0000","last_fullsized":"2024-05-08T04:38:30.741139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201063+0000","last_change":"2024-05-08T04:38:29.731689+0000","last_active":"2024-05-08T04:38:31.201063+0000","last_peered":"2024-05-08T04:38:31.201063+0000","last_clean":"2024-05-08T04:38:31.201063+0000","last_became_active":"2024-05-08T04:38:29.731433+0000","last_became_peered":"2024-05-08T04:38:29.731433+0000","last_unstale":"2024-05-08T04:38:31.201063+0000","last_undegraded":"2024-05-08T04:38:31.201063+0000","last_fullsized":"2024-05-08T04:38:31.201063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:30.376143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739541+0000","last_change":"2024-05-08T04:38:28.752051+0000","last_active":"2024-05-08T04:38:30.739541+0000","last_peered":"2024-05-08T04:38:30.739541+0000","last_clean":"2024-05-08T04:38:30.739541+0000","last_became_active":"2024-05-08T04:38:28.751792+0000","last_became_peered":"2024-05-08T04:38:28.751792+0000","last_unstale":"2024-05-08T04:38:30.739541+0000","last_undegraded":"2024-05-08T04:38:30.739541+0000","last_fullsized":"2024-05-08T04:38:30.739541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740885+0000","last_change":"2024-05-08T04:38:29.756998+0000","last_active":"2024-05-08T04:38:30.740885+0000","last_peered":"2024-05-08T04:38:30.740885+0000","last_clean":"2024-05-08T04:38:30.740885+0000","last_became_active":"2024-05-08T04:38:29.756395+0000","last_became_peered":"2024-05-08T04:38:29.756395+0000","last_unstale":"2024-05-08T04:38:30.740885+0000","last_undegraded":"2024-05-08T04:38:30.740885+0000","last_fullsized":"2024-05-08T04:38:30.740885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:38.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746251+0000","last_change":"2024-05-08T04:38:28.744776+0000","last_active":"2024-05-08T04:38:30.746251+0000","last_peered":"2024-05-08T04:38:30.746251+0000","last_clean":"2024-05-08T04:38:30.746251+0000","last_became_active":"2024-05-08T04:38:28.744410+0000","last_became_peered":"2024-05-08T04:38:28.744410+0000","last_unstale":"2024-05-08T04:38:30.746251+0000","last_undegraded":"2024-05-08T04:38:30.746251+0000","last_fullsized":"2024-05-08T04:38:30.746251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201683+0000","last_change":"2024-05-08T04:38:29.755661+0000","last_active":"2024-05-08T04:38:31.201683+0000","last_peered":"2024-05-08T04:38:31.201683+0000","last_clean":"2024-05-08T04:38:31.201683+0000","last_became_active":"2024-05-08T04:38:29.754401+0000","last_became_peered":"2024-05-08T04:38:29.754401+0000","last_unstale":"2024-05-08T04:38:31.201683+0000","last_undegraded":"2024-05-08T04:38:31.201683+0000","last_fullsized":"2024-05-08T04:38:31.201683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201037+0000","last_change":"2024-05-08T04:38:28.742800+0000","last_active":"2024-05-08T04:38:31.201037+0000","last_peered":"2024-05-08T04:38:31.201037+0000","last_clean":"2024-05-08T04:38:31.201037+0000","last_became_active":"2024-05-08T04:38:28.742491+0000","last_became_peered":"2024-05-08T04:38:28.742491+0000","last_unstale":"2024-05-08T04:38:31.201037+0000","last_undegraded":"2024-05-08T04:38:31.201037+0000","last_fullsized":"2024-05-08T04:38:31.201037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:53.444783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200760+0000","last_change":"2024-05-08T04:38:29.740544+0000","last_active":"2024-05-08T04:38:31.200760+0000","last_peered":"2024-05-08T04:38:31.200760+0000","last_clean":"2024-05-08T04:38:31.200760+0000","last_became_active":"2024-05-08T04:38:29.740262+0000","last_became_peered":"2024-05-08T04:38:29.740262+0000","last_unstale":"2024-05-08T04:38:31.200760+0000","last_undegraded":"2024-05-08T04:38:31.200760+0000","last_fullsized":"2024-05-08T04:38:31.200760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:15.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740918+0000","last_change":"2024-05-08T04:38:28.751437+0000","last_active":"2024-05-08T04:38:30.740918+0000","last_peered":"2024-05-08T04:38:30.740918+0000","last_clean":"2024-05-08T04:38:30.740918+0000","last_became_active":"2024-05-08T04:38:28.751039+0000","last_became_peered":"2024-05-08T04:38:28.751039+0000","last_unstale":"2024-05-08T04:38:30.740918+0000","last_undegraded":"2024-05-08T04:38:30.740918+0000","last_fullsized":"2024-05-08T04:38:30.740918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201608+0000","last_change":"2024-05-08T04:38:29.756359+0000","last_active":"2024-05-08T04:38:31.201608+0000","last_peered":"2024-05-08T04:38:31.201608+0000","last_clean":"2024-05-08T04:38:31.201608+0000","last_became_active":"2024-05-08T04:38:29.756087+0000","last_became_peered":"2024-05-08T04:38:29.756087+0000","last_unstale":"2024-05-08T04:38:31.201608+0000","last_undegraded":"2024-05-08T04:38:31.201608+0000","last_fullsized":"2024-05-08T04:38:31.201608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.745976+0000","last_change":"2024-05-08T04:38:28.739635+0000","last_active":"2024-05-08T04:38:30.745976+0000","last_peered":"2024-05-08T04:38:30.745976+0000","last_clean":"2024-05-08T04:38:30.745976+0000","last_became_active":"2024-05-08T04:38:28.739255+0000","last_became_peered":"2024-05-08T04:38:28.739255+0000","last_unstale":"2024-05-08T04:38:30.745976+0000","last_undegraded":"2024-05-08T04:38:30.745976+0000","last_fullsized":"2024-05-08T04:38:30.745976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746057+0000","last_change":"2024-05-08T04:38:29.750751+0000","last_active":"2024-05-08T04:38:30.746057+0000","last_peered":"2024-05-08T04:38:30.746057+0000","last_clean":"2024-05-08T04:38:30.746057+0000","last_became_active":"2024-05-08T04:38:29.749978+0000","last_became_peered":"2024-05-08T04:38:29.749978+0000","last_unstale":"2024-05-08T04:38:30.746057+0000","last_undegraded":"2024-05-08T04:38:30.746057+0000","last_fullsized":"2024-05-08T04:38:30.746057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200130+0000","last_change":"2024-05-08T04:38:28.738083+0000","last_active":"2024-05-08T04:38:31.200130+0000","last_peered":"2024-05-08T04:38:31.200130+0000","last_clean":"2024-05-08T04:38:31.200130+0000","last_became_active":"2024-05-08T04:38:28.736994+0000","last_became_peered":"2024-05-08T04:38:28.736994+0000","last_unstale":"2024-05-08T04:38:31.200130+0000","last_undegraded":"2024-05-08T04:38:31.200130+0000","last_fullsized":"2024-05-08T04:38:31.200130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201379+0000","last_change":"2024-05-08T04:38:29.758699+0000","last_active":"2024-05-08T04:38:31.201379+0000","last_peered":"2024-05-08T04:38:31.201379+0000","last_clean":"2024-05-08T04:38:31.201379+0000","last_became_active":"2024-05-08T04:38:29.758439+0000","last_became_peered":"2024-05-08T04:38:29.758439+0000","last_unstale":"2024-05-08T04:38:31.201379+0000","last_undegraded":"2024-05-08T04:38:31.201379+0000","last_fullsized":"2024-05-08T04:38:31.201379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:24.562953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741696+0000","last_change":"2024-05-08T04:38:28.738978+0000","last_active":"2024-05-08T04:38:30.741696+0000","last_peered":"2024-05-08T04:38:30.741696+0000","last_clean":"2024-05-08T04:38:30.741696+0000","last_became_active":"2024-05-08T04:38:28.738688+0000","last_became_peered":"2024-05-08T04:38:28.738688+0000","last_unstale":"2024-05-08T04:38:30.741696+0000","last_undegraded":"2024-05-08T04:38:30.741696+0000","last_fullsized":"2024-05-08T04:38:30.741696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201862+0000","last_change":"2024-05-08T04:38:29.757772+0000","last_active":"2024-05-08T04:38:31.201862+0000","last_peered":"2024-05-08T04:38:31.201862+0000","last_clean":"2024-05-08T04:38:31.201862+0000","last_became_active":"2024-05-08T04:38:29.756730+0000","last_became_peered":"2024-05-08T04:38:29.756730+0000","last_unstale":"2024-05-08T04:38:31.201862+0000","last_undegraded":"2024-05-08T04:38:31.201862+0000","last_fullsized":"2024-05-08T04:38:31.201862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200418+0000","last_change":"2024-05-08T04:38:28.734548+0000","last_active":"2024-05-08T04:38:31.200418+0000","last_peered":"2024-05-08T04:38:31.200418+0000","last_clean":"2024-05-08T04:38:31.200418+0000","last_became_active":"2024-05-08T04:38:28.734179+0000","last_became_peered":"2024-05-08T04:38:28.734179+0000","last_unstale":"2024-05-08T04:38:31.200418+0000","last_undegraded":"2024-05-08T04:38:31.200418+0000","last_fullsized":"2024-05-08T04:38:31.200418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:23.550606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.202085+0000","last_change":"2024-05-08T04:38:29.758477+0000","last_active":"2024-05-08T04:38:31.202085+0000","last_peered":"2024-05-08T04:38:31.202085+0000","last_clean":"2024-05-08T04:38:31.202085+0000","last_became_active":"2024-05-08T04:38:29.756388+0000","last_became_peered":"2024-05-08T04:38:29.756388+0000","last_unstale":"2024-05-08T04:38:31.202085+0000","last_undegraded":"2024-05-08T04:38:31.202085+0000","last_fullsized":"2024-05-08T04:38:31.202085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:06.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739628+0000","last_change":"2024-05-08T04:38:28.739369+0000","last_active":"2024-05-08T04:38:30.739628+0000","last_peered":"2024-05-08T04:38:30.739628+0000","last_clean":"2024-05-08T04:38:30.739628+0000","last_became_active":"2024-05-08T04:38:28.739079+0000","last_became_peered":"2024-05-08T04:38:28.739079+0000","last_unstale":"2024-05-08T04:38:30.739628+0000","last_undegraded":"2024-05-08T04:38:30.739628+0000","last_fullsized":"2024-05-08T04:38:30.739628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:08.997664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741244+0000","last_change":"2024-05-08T04:38:29.748638+0000","last_active":"2024-05-08T04:38:30.741244+0000","last_peered":"2024-05-08T04:38:30.741244+0000","last_clean":"2024-05-08T04:38:30.741244+0000","last_became_active":"2024-05-08T04:38:29.748277+0000","last_became_peered":"2024-05-08T04:38:29.748277+0000","last_unstale":"2024-05-08T04:38:30.741244+0000","last_undegraded":"2024-05-08T04:38:30.741244+0000","last_fullsized":"2024-05-08T04:38:30.741244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200420+0000","last_change":"2024-05-08T04:38:28.737536+0000","last_active":"2024-05-08T04:38:31.200420+0000","last_peered":"2024-05-08T04:38:31.200420+0000","last_clean":"2024-05-08T04:38:31.200420+0000","last_became_active":"2024-05-08T04:38:28.737250+0000","last_became_peered":"2024-05-08T04:38:28.737250+0000","last_unstale":"2024-05-08T04:38:31.200420+0000","last_undegraded":"2024-05-08T04:38:31.200420+0000","last_fullsized":"2024-05-08T04:38:31.200420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201674+0000","last_change":"2024-05-08T04:38:29.762160+0000","last_active":"2024-05-08T04:38:31.201674+0000","last_peered":"2024-05-08T04:38:31.201674+0000","last_clean":"2024-05-08T04:38:31.201674+0000","last_became_active":"2024-05-08T04:38:29.761754+0000","last_became_peered":"2024-05-08T04:38:29.761754+0000","last_unstale":"2024-05-08T04:38:31.201674+0000","last_undegraded":"2024-05-08T04:38:31.201674+0000","last_fullsized":"2024-05-08T04:38:31.201674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201275+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:38:31.201275+0000","last_peered":"2024-05-08T04:38:31.201275+0000","last_clean":"2024-05-08T04:38:31.201275+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:38:31.201275+0000","last_undegraded":"2024-05-08T04:38:31.201275+0000","last_fullsized":"2024-05-08T04:38:31.201275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741367+0000","last_change":"2024-05-08T04:38:28.750045+0000","last_active":"2024-05-08T04:38:30.741367+0000","last_peered":"2024-05-08T04:38:30.741367+0000","last_clean":"2024-05-08T04:38:30.741367+0000","last_became_active":"2024-05-08T04:38:28.749775+0000","last_became_peered":"2024-05-08T04:38:28.749775+0000","last_unstale":"2024-05-08T04:38:30.741367+0000","last_undegraded":"2024-05-08T04:38:30.741367+0000","last_fullsized":"2024-05-08T04:38:30.741367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:27.418257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877692+0000","last_change":"2024-05-08T04:38:29.733287+0000","last_active":"2024-05-08T04:38:30.877692+0000","last_peered":"2024-05-08T04:38:30.877692+0000","last_clean":"2024-05-08T04:38:30.877692+0000","last_became_active":"2024-05-08T04:38:29.733010+0000","last_became_peered":"2024-05-08T04:38:29.733010+0000","last_unstale":"2024-05-08T04:38:30.877692+0000","last_undegraded":"2024-05-08T04:38:30.877692+0000","last_fullsized":"2024-05-08T04:38:30.877692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201257+0000","last_change":"2024-05-08T04:38:28.750803+0000","last_active":"2024-05-08T04:38:31.201257+0000","last_peered":"2024-05-08T04:38:31.201257+0000","last_clean":"2024-05-08T04:38:31.201257+0000","last_became_active":"2024-05-08T04:38:28.750389+0000","last_became_peered":"2024-05-08T04:38:28.750389+0000","last_unstale":"2024-05-08T04:38:31.201257+0000","last_undegraded":"2024-05-08T04:38:31.201257+0000","last_fullsized":"2024-05-08T04:38:31.201257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200625+0000","last_change":"2024-05-08T04:38:29.734926+0000","last_active":"2024-05-08T04:38:31.200625+0000","last_peered":"2024-05-08T04:38:31.200625+0000","last_clean":"2024-05-08T04:38:31.200625+0000","last_became_active":"2024-05-08T04:38:29.733112+0000","last_became_peered":"2024-05-08T04:38:29.733112+0000","last_unstale":"2024-05-08T04:38:31.200625+0000","last_undegraded":"2024-05-08T04:38:31.200625+0000","last_fullsized":"2024-05-08T04:38:31.200625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:57.975524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200293+0000","last_change":"2024-05-08T04:38:28.733116+0000","last_active":"2024-05-08T04:38:31.200293+0000","last_peered":"2024-05-08T04:38:31.200293+0000","last_clean":"2024-05-08T04:38:31.200293+0000","last_became_active":"2024-05-08T04:38:28.732033+0000","last_became_peered":"2024-05-08T04:38:28.732033+0000","last_unstale":"2024-05-08T04:38:31.200293+0000","last_undegraded":"2024-05-08T04:38:31.200293+0000","last_fullsized":"2024-05-08T04:38:31.200293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:51.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739374+0000","last_change":"2024-05-08T04:38:29.740783+0000","last_active":"2024-05-08T04:38:30.739374+0000","last_peered":"2024-05-08T04:38:30.739374+0000","last_clean":"2024-05-08T04:38:30.739374+0000","last_became_active":"2024-05-08T04:38:29.738992+0000","last_became_peered":"2024-05-08T04:38:29.738992+0000","last_unstale":"2024-05-08T04:38:30.739374+0000","last_undegraded":"2024-05-08T04:38:30.739374+0000","last_fullsized":"2024-05-08T04:38:30.739374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740989+0000","last_change":"2024-05-08T04:38:28.744976+0000","last_active":"2024-05-08T04:38:30.740989+0000","last_peered":"2024-05-08T04:38:30.740989+0000","last_clean":"2024-05-08T04:38:30.740989+0000","last_became_active":"2024-05-08T04:38:28.744674+0000","last_became_peered":"2024-05-08T04:38:28.744674+0000","last_unstale":"2024-05-08T04:38:30.740989+0000","last_undegraded":"2024-05-08T04:38:30.740989+0000","last_fullsized":"2024-05-08T04:38:30.740989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740393+0000","last_change":"2024-05-08T04:38:29.743310+0000","last_active":"2024-05-08T04:38:30.740393+0000","last_peered":"2024-05-08T04:38:30.740393+0000","last_clean":"2024-05-08T04:38:30.740393+0000","last_became_active":"2024-05-08T04:38:29.741857+0000","last_became_peered":"2024-05-08T04:38:29.741857+0000","last_unstale":"2024-05-08T04:38:30.740393+0000","last_undegraded":"2024-05-08T04:38:30.740393+0000","last_fullsized":"2024-05-08T04:38:30.740393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:09.896832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739510+0000","last_change":"2024-05-08T04:38:28.735199+0000","last_active":"2024-05-08T04:38:30.739510+0000","last_peered":"2024-05-08T04:38:30.739510+0000","last_clean":"2024-05-08T04:38:30.739510+0000","last_became_active":"2024-05-08T04:38:28.734839+0000","last_became_peered":"2024-05-08T04:38:28.734839+0000","last_unstale":"2024-05-08T04:38:30.739510+0000","last_undegraded":"2024-05-08T04:38:30.739510+0000","last_fullsized":"2024-05-08T04:38:30.739510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741496+0000","last_change":"2024-05-08T04:38:29.743096+0000","last_active":"2024-05-08T04:38:30.741496+0000","last_peered":"2024-05-08T04:38:30.741496+0000","last_clean":"2024-05-08T04:38:30.741496+0000","last_became_active":"2024-05-08T04:38:29.742795+0000","last_became_peered":"2024-05-08T04:38:29.742795+0000","last_unstale":"2024-05-08T04:38:30.741496+0000","last_undegraded":"2024-05-08T04:38:30.741496+0000","last_fullsized":"2024-05-08T04:38:30.741496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:53.875495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201931+0000","last_change":"2024-05-08T04:38:28.749845+0000","last_active":"2024-05-08T04:38:31.201931+0000","last_peered":"2024-05-08T04:38:31.201931+0000","last_clean":"2024-05-08T04:38:31.201931+0000","last_became_active":"2024-05-08T04:38:28.749522+0000","last_became_peered":"2024-05-08T04:38:28.749522+0000","last_unstale":"2024-05-08T04:38:31.201931+0000","last_undegraded":"2024-05-08T04:38:31.201931+0000","last_fullsized":"2024-05-08T04:38:31.201931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:28.101827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739371+0000","last_change":"2024-05-08T04:38:29.733856+0000","last_active":"2024-05-08T04:38:30.739371+0000","last_peered":"2024-05-08T04:38:30.739371+0000","last_clean":"2024-05-08T04:38:30.739371+0000","last_became_active":"2024-05-08T04:38:29.733589+0000","last_became_peered":"2024-05-08T04:38:29.733589+0000","last_unstale":"2024-05-08T04:38:30.739371+0000","last_undegraded":"2024-05-08T04:38:30.739371+0000","last_fullsized":"2024-05-08T04:38:30.739371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:07.965810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.744254+0000","last_change":"2024-05-08T04:38:28.743259+0000","last_active":"2024-05-08T04:38:30.744254+0000","last_peered":"2024-05-08T04:38:30.744254+0000","last_clean":"2024-05-08T04:38:30.744254+0000","last_became_active":"2024-05-08T04:38:28.743022+0000","last_became_peered":"2024-05-08T04:38:28.743022+0000","last_unstale":"2024-05-08T04:38:30.744254+0000","last_undegraded":"2024-05-08T04:38:30.744254+0000","last_fullsized":"2024-05-08T04:38:30.744254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878061+0000","last_change":"2024-05-08T04:38:29.742718+0000","last_active":"2024-05-08T04:38:30.878061+0000","last_peered":"2024-05-08T04:38:30.878061+0000","last_clean":"2024-05-08T04:38:30.878061+0000","last_became_active":"2024-05-08T04:38:29.742135+0000","last_became_peered":"2024-05-08T04:38:29.742135+0000","last_unstale":"2024-05-08T04:38:30.878061+0000","last_undegraded":"2024-05-08T04:38:30.878061+0000","last_fullsized":"2024-05-08T04:38:30.878061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:17.223187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746492+0000","last_change":"2024-05-08T04:38:28.747934+0000","last_active":"2024-05-08T04:38:30.746492+0000","last_peered":"2024-05-08T04:38:30.746492+0000","last_clean":"2024-05-08T04:38:30.746492+0000","last_became_active":"2024-05-08T04:38:28.747524+0000","last_became_peered":"2024-05-08T04:38:28.747524+0000","last_unstale":"2024-05-08T04:38:30.746492+0000","last_undegraded":"2024-05-08T04:38:30.746492+0000","last_fullsized":"2024-05-08T04:38:30.746492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.200901+0000","last_change":"2024-05-08T04:38:29.729062+0000","last_active":"2024-05-08T04:38:31.200901+0000","last_peered":"2024-05-08T04:38:31.200901+0000","last_clean":"2024-05-08T04:38:31.200901+0000","last_became_active":"2024-05-08T04:38:29.728777+0000","last_became_peered":"2024-05-08T04:38:29.728777+0000","last_unstale":"2024-05-08T04:38:31.200901+0000","last_undegraded":"2024-05-08T04:38:31.200901+0000","last_fullsized":"2024-05-08T04:38:31.200901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:11.519339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201185+0000","last_change":"2024-05-08T04:38:28.749529+0000","last_active":"2024-05-08T04:38:31.201185+0000","last_peered":"2024-05-08T04:38:31.201185+0000","last_clean":"2024-05-08T04:38:31.201185+0000","last_became_active":"2024-05-08T04:38:28.749167+0000","last_became_peered":"2024-05-08T04:38:28.749167+0000","last_unstale":"2024-05-08T04:38:31.201185+0000","last_undegraded":"2024-05-08T04:38:31.201185+0000","last_fullsized":"2024-05-08T04:38:31.201185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:01:08.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.740076+0000","last_change":"2024-05-08T04:38:29.749670+0000","last_active":"2024-05-08T04:38:30.740076+0000","last_peered":"2024-05-08T04:38:30.740076+0000","last_clean":"2024-05-08T04:38:30.740076+0000","last_became_active":"2024-05-08T04:38:29.741613+0000","last_became_peered":"2024-05-08T04:38:29.741613+0000","last_unstale":"2024-05-08T04:38:30.740076+0000","last_undegraded":"2024-05-08T04:38:30.740076+0000","last_fullsized":"2024-05-08T04:38:30.740076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:10.004004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739385+0000","last_change":"2024-05-08T04:38:28.747163+0000","last_active":"2024-05-08T04:38:30.739385+0000","last_peered":"2024-05-08T04:38:30.739385+0000","last_clean":"2024-05-08T04:38:30.739385+0000","last_became_active":"2024-05-08T04:38:28.746783+0000","last_became_peered":"2024-05-08T04:38:28.746783+0000","last_unstale":"2024-05-08T04:38:30.739385+0000","last_undegraded":"2024-05-08T04:38:30.739385+0000","last_fullsized":"2024-05-08T04:38:30.739385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877757+0000","last_change":"2024-05-08T04:38:29.737910+0000","last_active":"2024-05-08T04:38:30.877757+0000","last_peered":"2024-05-08T04:38:30.877757+0000","last_clean":"2024-05-08T04:38:30.877757+0000","last_became_active":"2024-05-08T04:38:29.736005+0000","last_became_peered":"2024-05-08T04:38:29.736005+0000","last_unstale":"2024-05-08T04:38:30.877757+0000","last_undegraded":"2024-05-08T04:38:30.877757+0000","last_fullsized":"2024-05-08T04:38:30.877757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:23.883020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.739337+0000","last_change":"2024-05-08T04:38:28.740345+0000","last_active":"2024-05-08T04:38:30.739337+0000","last_peered":"2024-05-08T04:38:30.739337+0000","last_clean":"2024-05-08T04:38:30.739337+0000","last_became_active":"2024-05-08T04:38:28.740070+0000","last_became_peered":"2024-05-08T04:38:28.740070+0000","last_unstale":"2024-05-08T04:38:30.739337+0000","last_undegraded":"2024-05-08T04:38:30.739337+0000","last_fullsized":"2024-05-08T04:38:30.739337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:37.721416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.877804+0000","last_change":"2024-05-08T04:38:29.737168+0000","last_active":"2024-05-08T04:38:30.877804+0000","last_peered":"2024-05-08T04:38:30.877804+0000","last_clean":"2024-05-08T04:38:30.877804+0000","last_became_active":"2024-05-08T04:38:29.736816+0000","last_became_peered":"2024-05-08T04:38:29.736816+0000","last_unstale":"2024-05-08T04:38:30.877804+0000","last_undegraded":"2024-05-08T04:38:30.877804+0000","last_fullsized":"2024-05-08T04:38:30.877804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:26.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201887+0000","last_change":"2024-05-08T04:38:28.754098+0000","last_active":"2024-05-08T04:38:31.201887+0000","last_peered":"2024-05-08T04:38:31.201887+0000","last_clean":"2024-05-08T04:38:31.201887+0000","last_became_active":"2024-05-08T04:38:28.753787+0000","last_became_peered":"2024-05-08T04:38:28.753787+0000","last_unstale":"2024-05-08T04:38:31.201887+0000","last_undegraded":"2024-05-08T04:38:31.201887+0000","last_fullsized":"2024-05-08T04:38:31.201887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:56.969185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.746089+0000","last_change":"2024-05-08T04:38:29.747896+0000","last_active":"2024-05-08T04:38:30.746089+0000","last_peered":"2024-05-08T04:38:30.746089+0000","last_clean":"2024-05-08T04:38:30.746089+0000","last_became_active":"2024-05-08T04:38:29.747661+0000","last_became_peered":"2024-05-08T04:38:29.747661+0000","last_unstale":"2024-05-08T04:38:30.746089+0000","last_undegraded":"2024-05-08T04:38:30.746089+0000","last_fullsized":"2024-05-08T04:38:30.746089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:41.816021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741307+0000","last_change":"2024-05-08T04:38:29.742488+0000","last_active":"2024-05-08T04:38:30.741307+0000","last_peered":"2024-05-08T04:38:30.741307+0000","last_clean":"2024-05-08T04:38:30.741307+0000","last_became_active":"2024-05-08T04:38:29.738959+0000","last_became_peered":"2024-05-08T04:38:29.738959+0000","last_unstale":"2024-05-08T04:38:30.741307+0000","last_undegraded":"2024-05-08T04:38:30.741307+0000","last_fullsized":"2024-05-08T04:38:30.741307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:14.693989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201514+0000","last_change":"2024-05-08T04:38:28.758434+0000","last_active":"2024-05-08T04:38:31.201514+0000","last_peered":"2024-05-08T04:38:31.201514+0000","last_clean":"2024-05-08T04:38:31.201514+0000","last_became_active":"2024-05-08T04:38:28.758108+0000","last_became_peered":"2024-05-08T04:38:28.758108+0000","last_unstale":"2024-05-08T04:38:31.201514+0000","last_undegraded":"2024-05-08T04:38:31.201514+0000","last_fullsized":"2024-05-08T04:38:31.201514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:25.921864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.741246+0000","last_change":"2024-05-08T04:38:29.740724+0000","last_active":"2024-05-08T04:38:30.741246+0000","last_peered":"2024-05-08T04:38:30.741246+0000","last_clean":"2024-05-08T04:38:30.741246+0000","last_became_active":"2024-05-08T04:38:29.739520+0000","last_became_peered":"2024-05-08T04:38:29.739520+0000","last_unstale":"2024-05-08T04:38:30.741246+0000","last_undegraded":"2024-05-08T04:38:30.741246+0000","last_fullsized":"2024-05-08T04:38:30.741246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:19:45.256058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201827+0000","last_change":"2024-05-08T04:38:28.753194+0000","last_active":"2024-05-08T04:38:31.201827+0000","last_peered":"2024-05-08T04:38:31.201827+0000","last_clean":"2024-05-08T04:38:31.201827+0000","last_became_active":"2024-05-08T04:38:28.752937+0000","last_became_peered":"2024-05-08T04:38:28.752937+0000","last_unstale":"2024-05-08T04:38:31.201827+0000","last_undegraded":"2024-05-08T04:38:31.201827+0000","last_fullsized":"2024-05-08T04:38:31.201827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.090889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201009+0000","last_change":"2024-05-08T04:38:29.738092+0000","last_active":"2024-05-08T04:38:31.201009+0000","last_peered":"2024-05-08T04:38:31.201009+0000","last_clean":"2024-05-08T04:38:31.201009+0000","last_became_active":"2024-05-08T04:38:29.737806+0000","last_became_peered":"2024-05-08T04:38:29.737806+0000","last_unstale":"2024-05-08T04:38:31.201009+0000","last_undegraded":"2024-05-08T04:38:31.201009+0000","last_fullsized":"2024-05-08T04:38:31.201009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:39.943622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878056+0000","last_change":"2024-05-08T04:38:28.732592+0000","last_active":"2024-05-08T04:38:30.878056+0000","last_peered":"2024-05-08T04:38:30.878056+0000","last_clean":"2024-05-08T04:38:30.878056+0000","last_became_active":"2024-05-08T04:38:28.732270+0000","last_became_peered":"2024-05-08T04:38:28.732270+0000","last_unstale":"2024-05-08T04:38:30.878056+0000","last_undegraded":"2024-05-08T04:38:30.878056+0000","last_fullsized":"2024-05-08T04:38:30.878056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:32.956968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:30.878079+0000","last_change":"2024-05-08T04:38:29.751389+0000","last_active":"2024-05-08T04:38:30.878079+0000","last_peered":"2024-05-08T04:38:30.878079+0000","last_clean":"2024-05-08T04:38:30.878079+0000","last_became_active":"2024-05-08T04:38:29.751025+0000","last_became_peered":"2024-05-08T04:38:29.751025+0000","last_unstale":"2024-05-08T04:38:30.878079+0000","last_undegraded":"2024-05-08T04:38:30.878079+0000","last_fullsized":"2024-05-08T04:38:30.878079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:28.708320+0000","last_clean_scrub_stamp":"2024-05-08T04:38:28.708320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:34:50.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T04:38:31.201069+0000","last_change":"2024-05-08T04:38:28.745873+0000","last_active":"2024-05-08T04:38:31.201069+0000","last_peered":"2024-05-08T04:38:31.201069+0000","last_clean":"2024-05-08T04:38:31.201069+0000","last_became_active":"2024-05-08T04:38:28.745607+0000","last_became_peered":"2024-05-08T04:38:28.745607+0000","last_unstale":"2024-05-08T04:38:31.201069+0000","last_undegraded":"2024-05-08T04:38:31.201069+0000","last_fullsized":"2024-05-08T04:38:31.201069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:27.701981+0000","last_clean_scrub_stamp":"2024-05-08T04:38:27.701981+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:18.516540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:38:38.767 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T04:38:38.767 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T04:38:38.767 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T04:38:38.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T04:38:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.939+0000 7f8719c00640 1 Processor -- start 2024-05-08T04:38:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.939+0000 7f8719c00640 1 -- start start 2024-05-08T04:38:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87140ae930 0x7f87140aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8714161620 0x7f8714161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871415de80 con 0x7f871415daa0 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8714161fd0 con 0x7f87140ae930 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87140ad890 con 0x7f8714161620 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87140ae930 0x7f87140aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87140ae930 0x7f87140aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41602/0 (socket says 172.21.15.164:41602) 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 -- 172.21.15.164:0/16421985 learned_addr learned my addr 172.21.15.164:0/16421985 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f8719200640 1 --2- 172.21.15.164:0/16421985 >> v2:172.21.15.172:3300/0 conn(0x7f8714161620 0x7f8714161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 -- 172.21.15.164:0/16421985 >> v2:172.21.15.172:3300/0 conn(0x7f8714161620 msgr2=0x7f8714161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:38.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 --2- 172.21.15.164:0/16421985 >> v2:172.21.15.172:3300/0 conn(0x7f8714161620 0x7f8714161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 -- 172.21.15.164:0/16421985 >> v1:172.21.15.172:6789/0 conn(0x7f871415daa0 legacy=0x7f871406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:38.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.943+0000 7f870b400640 1 -- 172.21.15.164:0/16421985 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140adb70 con 0x7f87140ae930 2024-05-08T04:38:38.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f870b400640 1 --2- 172.21.15.164:0/16421985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87140ae930 0x7f87140aed90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f87140af7f0 tx=0x7f86fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2dcfadaa857980 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8713400640 1 -- 172.21.15.164:0/16421985 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0ab610 con 0x7f87140ae930 2024-05-08T04:38:38.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8713400640 1 -- 172.21.15.164:0/16421985 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc0b0070 con 0x7f87140ae930 2024-05-08T04:38:38.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8713400640 1 -- 172.21.15.164:0/16421985 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0ab9d0 con 0x7f87140ae930 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- 172.21.15.164:0/16421985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87140ae930 msgr2=0x7f87140aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- 172.21.15.164:0/16421985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87140ae930 0x7f87140aed90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f87140af7f0 tx=0x7f86fc01c520 comp rx=0 tx=0).stop 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- 172.21.15.164:0/16421985 shutdown_connections 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- 172.21.15.164:0/16421985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87140ae930 0x7f87140aed90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- 172.21.15.164:0/16421985 >> v2:172.21.15.172:3300/0 conn(0x7f8714161620 0x7f8714161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- 172.21.15.164:0/16421985 >> 172.21.15.164:0/16421985 conn(0x7f87140d0410 msgr2=0x7f871406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- 172.21.15.164:0/16421985 shutdown_connections 2024-05-08T04:38:38.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- 172.21.15.164:0/16421985 wait complete. 2024-05-08T04:38:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 Processor -- start 2024-05-08T04:38:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- start start 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140ae930 0x7f871415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871415daa0 0x7f8714151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871406d420 con 0x7f8714161620 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871406d2a0 con 0x7f871415daa0 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.947+0000 7f8719c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871416a9c0 con 0x7f87140ae930 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41618/0 (socket says 172.21.15.164:41618) 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 -- 172.21.15.164:0/1526761628 learned_addr learned my addr 172.21.15.164:0/1526761628 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f870b400640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871415daa0 0x7f8714151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8713e00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140ae930 0x7f871415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 -- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140ae930 msgr2=0x7f871415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140ae930 0x7f871415e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 -- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871415daa0 msgr2=0x7f8714151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871415daa0 0x7f8714151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 -- 172.21.15.164:0/1526761628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714151ee0 con 0x7f8714161620 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719200640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f870c007e70 tx=0x7f870c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c014620 con 0x7f8714161620 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c019070 con 0x7f8714161620 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c014a20 con 0x7f8714161620 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87141521c0 con 0x7f8714161620 2024-05-08T04:38:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8714152b80 con 0x7f8714161620 2024-05-08T04:38:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871404ef30 con 0x7f8714161620 2024-05-08T04:38:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f870c014bc0 con 0x7f8714161620 2024-05-08T04:38:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 0x7f86f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.951+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f870c085950 con 0x7f8714161620 2024-05-08T04:38:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.955+0000 7f8713e00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 0x7f86f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.955+0000 7f8713e00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 0x7f86f0069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87040015b0 tx=0x7f8704009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:38.955+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f870c011260 con 0x7f8714161620 2024-05-08T04:38:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f87141610b0 con 0x7f8714161620 2024-05-08T04:38:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8710c00640 1 -- 172.21.15.164:0/1526761628 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f870c056950 con 0x7f8714161620 2024-05-08T04:38:39.146 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:39.146 INFO:teuthology.orchestra.run.smithi164.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 msgr2=0x7f86f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 0x7f86f0069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87040015b0 tx=0x7f8704009040 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 msgr2=0x7f871413f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.143+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f870c007e70 tx=0x7f870c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 shutdown_connections 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86f0066f60 0x7f86f0069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8714161620 0x7f871413f3a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871415daa0 0x7f8714151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 --2- 172.21.15.164:0/1526761628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140ae930 0x7f871415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 >> 172.21.15.164:0/1526761628 conn(0x7f87140d0410 msgr2=0x7f8714159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 shutdown_connections 2024-05-08T04:38:39.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.147+0000 7f8719c00640 1 -- 172.21.15.164:0/1526761628 wait complete. 2024-05-08T04:38:39.167 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T04:38:39.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:39.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.299+0000 7f0367400640 1 Processor -- start 2024-05-08T04:38:39.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.299+0000 7f0367400640 1 -- start start 2024-05-08T04:38:39.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.299+0000 7f0367400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f036015ab60 0x7f0360072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0367400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f03600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0367400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0360072640 con 0x7f03600cfbd0 2024-05-08T04:38:39.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0367400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03600d0060 con 0x7f036015ab60 2024-05-08T04:38:39.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0367400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036006c770 con 0x7f03600b26c0 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f03600cfbd0 0x7f03600cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47514/0 (socket says 172.21.15.164:47514) 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 -- 172.21.15.164:0/1700588985 learned_addr learned my addr 172.21.15.164:0/1700588985 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0364a00640 1 --2- 172.21.15.164:0/1700588985 >> v2:172.21.15.172:3300/0 conn(0x7f036015ab60 0x7f0360072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035fe00640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f03600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133796224 0 0) 0x7f0360072640 con 0x7f03600cfbd0 2024-05-08T04:38:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0344005740 con 0x7f03600cfbd0 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985883634 0 0) 0x7f0344005740 con 0x7f03600cfbd0 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 msgr2=0x7f03600cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f03600cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 >> v2:172.21.15.172:3300/0 conn(0x7f036015ab60 msgr2=0x7f0360072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 --2- 172.21.15.164:0/1700588985 >> v2:172.21.15.172:3300/0 conn(0x7f036015ab60 0x7f0360072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036006ca50 con 0x7f03600cfbd0 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f0358001680 con 0x7f03600cfbd0 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 >> v1:172.21.15.164:6789/0 conn(0x7f03600cfbd0 legacy=0x7f03600cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0344005bd0 0x7f0344005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f0344006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f0344006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036006c770 con 0x7f0344006530 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0344007700 con 0x7f0344005bd0 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034400ffe0 con 0x7f03600b26c0 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f0344006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0364a00640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0344005bd0 0x7f0344005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f035fe00640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f0344006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 -- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 msgr2=0x7f0344006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03600b26c0 0x7f0344006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 -- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0344005bd0 msgr2=0x7f0344005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0344005bd0 0x7f0344005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.303+0000 7f0365400640 1 -- 172.21.15.164:0/1700588985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344005740 con 0x7f0344006530 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0365400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f0344006f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f035800c2c0 tx=0x7f035801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee37e37c75943dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358007d60 con 0x7f0344006530 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03580a9020 con 0x7f0344006530 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f035f400640 1 -- 172.21.15.164:0/1700588985 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580a5bb0 con 0x7f0344006530 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- 172.21.15.164:0/1700588985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 msgr2=0x7f0344006f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 --2- 172.21.15.164:0/1700588985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f0344006f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f035800c2c0 tx=0x7f035801cd40 comp rx=0 tx=0).stop 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0365400640 1 -- 172.21.15.164:0/1700588985 reap_dead start 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- 172.21.15.164:0/1700588985 shutdown_connections 2024-05-08T04:38:39.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- 172.21.15.164:0/1700588985 >> 172.21.15.164:0/1700588985 conn(0x7f036006ac90 msgr2=0x7f036006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- 172.21.15.164:0/1700588985 shutdown_connections 2024-05-08T04:38:39.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- 172.21.15.164:0/1700588985 wait complete. 2024-05-08T04:38:39.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 Processor -- start 2024-05-08T04:38:39.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.307+0000 7f0367400640 1 -- start start 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 0x7f0360155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036015d750 0x7f036013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036006d2f0 con 0x7f0344006530 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036006d010 con 0x7f0360140ba0 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036006c3d0 con 0x7f036015d750 2024-05-08T04:38:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37510/0 (socket says 172.21.15.164:37510) 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 -- 172.21.15.164:0/3106293022 learned_addr learned my addr 172.21.15.164:0/3106293022 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035fe00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 0x7f0360155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0365400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036015d750 0x7f036013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 -- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036015d750 msgr2=0x7f036013ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036015d750 0x7f036013ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 -- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 msgr2=0x7f0360155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 0x7f0360155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 -- 172.21.15.164:0/3106293022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036013dec0 con 0x7f0344006530 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035fe00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 0x7f0360155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:39.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0364a00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0350011e60 tx=0x7f035000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350015630 con 0x7f0344006530 2024-05-08T04:38:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500157d0 con 0x7f0344006530 2024-05-08T04:38:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036013e1a0 con 0x7f0344006530 2024-05-08T04:38:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350015ba0 con 0x7f0344006530 2024-05-08T04:38:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036013e760 con 0x7f0344006530 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360069d60 con 0x7f0344006530 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.311+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f035001d400 con 0x7f0344006530 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.315+0000 7f035cc00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 0x7f0340069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.315+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0350086290 con 0x7f0344006530 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.315+0000 7f035fe00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 0x7f0340069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.315+0000 7f035fe00640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 0x7f0340069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f03600b48f0 tx=0x7f034c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.319+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0350019030 con 0x7f0344006530 2024-05-08T04:38:39.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.475+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03600cedd0 con 0x7f0344006530 2024-05-08T04:38:39.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f035cc00640 1 -- 172.21.15.164:0/3106293022 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f0350010040 con 0x7f0344006530 2024-05-08T04:38:39.480 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:39.480 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:39.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 msgr2=0x7f0340069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 0x7f0340069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f03600b48f0 tx=0x7f034c008040 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 msgr2=0x7f03601505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0350011e60 tx=0x7f035000f580 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 shutdown_connections 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0340066f90 0x7f0340069450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036015d750 0x7f036013ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0360140ba0 0x7f0360155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 --2- 172.21.15.164:0/3106293022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0344006530 0x7f03601505d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 >> 172.21.15.164:0/3106293022 conn(0x7f036006ac90 msgr2=0x7f03600b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 shutdown_connections 2024-05-08T04:38:39.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.479+0000 7f0367400640 1 -- 172.21.15.164:0/3106293022 wait complete. 2024-05-08T04:38:39.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:39.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:39.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.635+0000 7f5419400640 1 Processor -- start 2024-05-08T04:38:39.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.635+0000 7f5419400640 1 -- start start 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5419400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f541415ab10 0x7f541406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5419400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f54140ae6e0 0x7f54140aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5419400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541415e870 con 0x7f54140ce780 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5419400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541406bb80 con 0x7f54140ae6e0 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5419400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541415eb50 con 0x7f541415ab10 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5409e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f54140ae6e0 0x7f54140aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5413200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f54140ce780 0x7f54140c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47530/0 (socket says 172.21.15.164:47530) 2024-05-08T04:38:39.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5413200640 1 -- 172.21.15.164:0/2113284076 learned_addr learned my addr 172.21.15.164:0/2113284076 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:39.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5412800640 1 --2- 172.21.15.164:0/2113284076 >> v2:172.21.15.172:3300/0 conn(0x7f541415ab10 0x7f541406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5409e00640 1 -- 172.21.15.164:0/2113284076 >> v2:172.21.15.172:3300/0 conn(0x7f541415ab10 msgr2=0x7f541406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5409e00640 1 --2- 172.21.15.164:0/2113284076 >> v2:172.21.15.172:3300/0 conn(0x7f541415ab10 0x7f541406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5411e00640 1 -- 172.21.15.164:0/2113284076 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175569286 0 0) 0x7f541415e870 con 0x7f54140ce780 2024-05-08T04:38:39.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5409e00640 1 -- 172.21.15.164:0/2113284076 >> v1:172.21.15.164:6789/0 conn(0x7f54140ce780 legacy=0x7f54140c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.639+0000 7f5409e00640 1 -- 172.21.15.164:0/2113284076 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541415ee30 con 0x7f54140ae6e0 2024-05-08T04:38:39.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5409e00640 1 --2- 172.21.15.164:0/2113284076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54140ae6e0 0x7f54140aeb70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54140c4920 tx=0x7f53fc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1749e4908b525ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5411e00640 1 -- 172.21.15.164:0/2113284076 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b1070 con 0x7f54140ae6e0 2024-05-08T04:38:39.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5411e00640 1 -- 172.21.15.164:0/2113284076 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0ac620 con 0x7f54140ae6e0 2024-05-08T04:38:39.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5411e00640 1 -- 172.21.15.164:0/2113284076 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0ac9f0 con 0x7f54140ae6e0 2024-05-08T04:38:39.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- 172.21.15.164:0/2113284076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54140ae6e0 msgr2=0x7f54140aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 --2- 172.21.15.164:0/2113284076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54140ae6e0 0x7f54140aeb70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54140c4920 tx=0x7f53fc01c3c0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- 172.21.15.164:0/2113284076 shutdown_connections 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 --2- 172.21.15.164:0/2113284076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54140ae6e0 0x7f54140aeb70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 --2- 172.21.15.164:0/2113284076 >> v2:172.21.15.172:3300/0 conn(0x7f541415ab10 0x7f541406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- 172.21.15.164:0/2113284076 >> 172.21.15.164:0/2113284076 conn(0x7f54140cf770 msgr2=0x7f54140cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- 172.21.15.164:0/2113284076 shutdown_connections 2024-05-08T04:38:39.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- 172.21.15.164:0/2113284076 wait complete. 2024-05-08T04:38:39.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 Processor -- start 2024-05-08T04:38:39.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.643+0000 7f5419400640 1 -- start start 2024-05-08T04:38:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 0x7f54141f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 0x7f5414150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54140cefa0 con 0x7f541415ab10 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414167ae0 con 0x7f54140ae6e0 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54141fa510 con 0x7f54140ce780 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37516/0 (socket says 172.21.15.164:37516) 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 -- 172.21.15.164:0/3965061904 learned_addr learned my addr 172.21.15.164:0/3965061904 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5409e00640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 0x7f5414150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5412800640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 0x7f54141f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 -- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 msgr2=0x7f5414150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 0x7f5414150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 -- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 msgr2=0x7f54141f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 0x7f54141f9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 -- 172.21.15.164:0/3965061904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414150fb0 con 0x7f541415ab10 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5409e00640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 0x7f5414150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5412800640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 0x7f54141f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5413200640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c730 tx=0x7f540c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c018070 con 0x7f541415ab10 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c00ce40 con 0x7f541415ab10 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c00a860 con 0x7f541415ab10 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541414f9d0 con 0x7f541415ab10 2024-05-08T04:38:39.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541414ff90 con 0x7f541415ab10 2024-05-08T04:38:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5414072d30 con 0x7f541415ab10 2024-05-08T04:38:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.647+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f540c01b070 con 0x7f541415ab10 2024-05-08T04:38:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.651+0000 7f540b400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 0x7f53f00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.651+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f540c084670 con 0x7f541415ab10 2024-05-08T04:38:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.651+0000 7f5412800640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 0x7f53f00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.651+0000 7f5412800640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 0x7f53f00694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f540400b9f0 tx=0x7f5404008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.655+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f540c0556f0 con 0x7f541415ab10 2024-05-08T04:38:39.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f541404f010 con 0x7f541415ab10 2024-05-08T04:38:39.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f540b400640 1 -- 172.21.15.164:0/3965061904 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f540c054ff0 con 0x7f541415ab10 2024-05-08T04:38:39.813 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 msgr2=0x7f53f00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 0x7f53f00694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f540400b9f0 tx=0x7f5404008040 comp rx=0 tx=0).stop 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 msgr2=0x7f541414ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c730 tx=0x7f540c00cbf0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 shutdown_connections 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f53f0066fe0 0x7f53f00694a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54140ce780 0x7f5414150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f541415ab10 0x7f541414ef80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 --2- 172.21.15.164:0/3965061904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54140ae6e0 0x7f54141f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 >> 172.21.15.164:0/3965061904 conn(0x7f54140cf770 msgr2=0x7f541406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 shutdown_connections 2024-05-08T04:38:39.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.811+0000 7f5419400640 1 -- 172.21.15.164:0/3965061904 wait complete. 2024-05-08T04:38:39.833 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:39.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.967+0000 7fae02200640 1 Processor -- start 2024-05-08T04:38:39.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.967+0000 7fae02200640 1 -- start start 2024-05-08T04:38:39.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fae02200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 0x7fadfc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fae02200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fae02200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc15e870 con 0x7fadfc15ab10 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fae02200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc06bb80 con 0x7fadfc0cecc0 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fae02200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc0cf0f0 con 0x7fadfc06cce0 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37518/0 (socket says 172.21.15.164:37518) 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 -- 172.21.15.164:0/3394900008 learned_addr learned my addr 172.21.15.164:0/3394900008 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfaa00640 1 --2- 172.21.15.164:0/3394900008 >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 0x7fadfc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 -- 172.21.15.164:0/3394900008 >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 msgr2=0x7fadfc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:39.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3394900008 >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 0x7fadfc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 -- 172.21.15.164:0/3394900008 >> v1:172.21.15.172:6789/0 conn(0x7fadfc15ab10 legacy=0x7fadfc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfbe00640 1 -- 172.21.15.164:0/3394900008 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc15eb50 con 0x7fadfc0cecc0 2024-05-08T04:38:39.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.971+0000 7fadfaa00640 1 --2- 172.21.15.164:0/3394900008 >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 0x7fadfc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:39.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3394900008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0cf3c0 tx=0x7fadf401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8285f5ebc792ea00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfa000640 1 -- 172.21.15.164:0/3394900008 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40a5700 con 0x7fadfc0cecc0 2024-05-08T04:38:39.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfa000640 1 -- 172.21.15.164:0/3394900008 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf40a58a0 con 0x7fadfc0cecc0 2024-05-08T04:38:39.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfa000640 1 -- 172.21.15.164:0/3394900008 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40a5c70 con 0x7fadfc0cecc0 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- 172.21.15.164:0/3394900008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0cecc0 msgr2=0x7fadfc0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- 172.21.15.164:0/3394900008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0cf3c0 tx=0x7fadf401c930 comp rx=0 tx=0).stop 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- 172.21.15.164:0/3394900008 shutdown_connections 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- 172.21.15.164:0/3394900008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0cecc0 0x7fadfc0ca6e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- 172.21.15.164:0/3394900008 >> v2:172.21.15.172:3300/0 conn(0x7fadfc06cce0 0x7fadfc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- 172.21.15.164:0/3394900008 >> 172.21.15.164:0/3394900008 conn(0x7fadfc0cf770 msgr2=0x7fadfc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- 172.21.15.164:0/3394900008 shutdown_connections 2024-05-08T04:38:39.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- 172.21.15.164:0/3394900008 wait complete. 2024-05-08T04:38:39.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 Processor -- start 2024-05-08T04:38:39.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- start start 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 0x7fadfc159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cecc0 0x7fadfc153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0aedf0 con 0x7fadfc15ab10 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc15aef0 con 0x7fadfc06cce0 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fae02200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0aef70 con 0x7fadfc0cecc0 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37520/0 (socket says 172.21.15.164:37520) 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.975+0000 7fadfbe00640 1 -- 172.21.15.164:0/3836020783 learned_addr learned my addr 172.21.15.164:0/3836020783 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfaa00640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cecc0 0x7fadfc153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfb400640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 0x7fadfc159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 -- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cecc0 msgr2=0x7fadfc153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cecc0 0x7fadfc153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 -- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 msgr2=0x7fadfc159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 0x7fadfc159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 -- 172.21.15.164:0/3836020783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc154b90 con 0x7fadfc15ab10 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfb400640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 0x7fadfc159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf40af040 tx=0x7fadf40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40bc020 con 0x7fadfc15ab10 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf40b8670 con 0x7fadfc15ab10 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4009510 con 0x7fadfc15ab10 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc15a080 con 0x7fadfc15ab10 2024-05-08T04:38:39.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1553f0 con 0x7fadfc15ab10 2024-05-08T04:38:39.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc154e90 con 0x7fadfc15ab10 2024-05-08T04:38:39.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fadf40bf020 con 0x7fadfc15ab10 2024-05-08T04:38:39.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 0x7faddc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:39.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.979+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadf41200b0 con 0x7fadfc15ab10 2024-05-08T04:38:39.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.983+0000 7fadfb400640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 0x7faddc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:39.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.983+0000 7fadfb400640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 0x7faddc069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadec001670 tx=0x7fadec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:39.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:39.983+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fadfc154e90 con 0x7fadfc15ab10 2024-05-08T04:38:40.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadfc04f010 con 0x7fadfc15ab10 2024-05-08T04:38:40.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fadf3400640 1 -- 172.21.15.164:0/3836020783 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadf40c6030 con 0x7fadfc15ab10 2024-05-08T04:38:40.141 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:40.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 msgr2=0x7faddc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 0x7faddc069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadec001670 tx=0x7fadec009040 comp rx=0 tx=0).stop 2024-05-08T04:38:40.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 msgr2=0x7fadfc0b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf40af040 tx=0x7fadf40196d0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 shutdown_connections 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc066f60 0x7faddc069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cecc0 0x7fadfc153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc15ab10 0x7fadfc0b3040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 --2- 172.21.15.164:0/3836020783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc06cce0 0x7fadfc159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 >> 172.21.15.164:0/3836020783 conn(0x7fadfc0cf770 msgr2=0x7fadfc154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 shutdown_connections 2024-05-08T04:38:40.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.139+0000 7fae02200640 1 -- 172.21.15.164:0/3836020783 wait complete. 2024-05-08T04:38:40.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:40.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 Processor -- start 2024-05-08T04:38:40.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 -- start start 2024-05-08T04:38:40.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f84240aee90 0x7f84240d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84240af9a0 con 0x7f842415e900 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84240afc80 con 0x7f84240d0a90 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f842b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84240af750 con 0x7f84240aee90 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8429000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f842415e900 0x7f842406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47538/0 (socket says 172.21.15.164:47538) 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8429000640 1 -- 172.21.15.164:0/3155091558 learned_addr learned my addr 172.21.15.164:0/3155091558 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:40.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423e00640 1 --2- 172.21.15.164:0/3155091558 >> v2:172.21.15.172:3300/0 conn(0x7f84240aee90 0x7f84240d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423400640 1 -- 172.21.15.164:0/3155091558 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167737581 0 0) 0x7f84240af9a0 con 0x7f842415e900 2024-05-08T04:38:40.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423400640 1 -- 172.21.15.164:0/3155091558 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8408003530 con 0x7f842415e900 2024-05-08T04:38:40.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423e00640 1 -- 172.21.15.164:0/3155091558 >> v1:172.21.15.172:6789/0 conn(0x7f84240d0a90 legacy=0x7f84240b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423e00640 1 -- 172.21.15.164:0/3155091558 >> v1:172.21.15.164:6789/0 conn(0x7f842415e900 legacy=0x7f842406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.299+0000 7f8423e00640 1 -- 172.21.15.164:0/3155091558 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240aff60 con 0x7f84240aee90 2024-05-08T04:38:40.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f8423e00640 1 --2- 172.21.15.164:0/3155091558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240aee90 0x7f84240d0550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097c0 tx=0x7f840c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15ac848d8be5547 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f8423400640 1 -- 172.21.15.164:0/3155091558 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0a7070 con 0x7f84240aee90 2024-05-08T04:38:40.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f8423400640 1 -- 172.21.15.164:0/3155091558 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c0b0070 con 0x7f84240aee90 2024-05-08T04:38:40.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f8423400640 1 -- 172.21.15.164:0/3155091558 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0ab890 con 0x7f84240aee90 2024-05-08T04:38:40.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 -- 172.21.15.164:0/3155091558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240aee90 msgr2=0x7f84240d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 --2- 172.21.15.164:0/3155091558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240aee90 0x7f84240d0550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097c0 tx=0x7f840c01c470 comp rx=0 tx=0).stop 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 -- 172.21.15.164:0/3155091558 shutdown_connections 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 --2- 172.21.15.164:0/3155091558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240aee90 0x7f84240d0550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 -- 172.21.15.164:0/3155091558 >> 172.21.15.164:0/3155091558 conn(0x7f8424067050 msgr2=0x7f842406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 -- 172.21.15.164:0/3155091558 shutdown_connections 2024-05-08T04:38:40.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.303+0000 7f842b000640 1 -- 172.21.15.164:0/3155091558 wait complete. 2024-05-08T04:38:40.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 Processor -- start 2024-05-08T04:38:40.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- start start 2024-05-08T04:38:40.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240aee90 0x7f8424152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f842415e900 0x7f842415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84240720f0 con 0x7f84240d0a90 2024-05-08T04:38:40.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84240af270 con 0x7f842415e900 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8424072270 con 0x7f84240aee90 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37534/0 (socket says 172.21.15.164:37534) 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 -- 172.21.15.164:0/2084354928 learned_addr learned my addr 172.21.15.164:0/2084354928 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f8423e00640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240aee90 0x7f8424152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f8429000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f842415e900 0x7f842415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 -- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240aee90 msgr2=0x7f8424152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240aee90 0x7f8424152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 -- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f842415e900 msgr2=0x7f842415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f842415e900 0x7f842415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 -- 172.21.15.164:0/2084354928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84241515b0 con 0x7f84240d0a90 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f841b400640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f841400c430 tx=0x7f841400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140052d0 con 0x7f84240d0a90 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414005470 con 0x7f84240d0a90 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414007260 con 0x7f84240d0a90 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8424151890 con 0x7f84240d0a90 2024-05-08T04:38:40.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8424152350 con 0x7f84240d0a90 2024-05-08T04:38:40.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.307+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8424152550 con 0x7f84240d0a90 2024-05-08T04:38:40.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.311+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8414007400 con 0x7f84240d0a90 2024-05-08T04:38:40.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.311+0000 7f8420c00640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 0x7f8400069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.311+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8414084460 con 0x7f84240d0a90 2024-05-08T04:38:40.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.311+0000 7f8423e00640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 0x7f8400069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.311+0000 7f8423e00640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 0x7f8400069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f840c0af000 tx=0x7f840c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.315+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8424152550 con 0x7f84240d0a90 2024-05-08T04:38:40.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8424151a60 con 0x7f84240d0a90 2024-05-08T04:38:40.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f8420c00640 1 -- 172.21.15.164:0/2084354928 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8414054e10 con 0x7f84240d0a90 2024-05-08T04:38:40.474 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 msgr2=0x7f8400069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 0x7f8400069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f840c0af000 tx=0x7f840c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 msgr2=0x7f8424150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f841400c430 tx=0x7f841400c8f0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 shutdown_connections 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8400066f90 0x7f8400069450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f842415e900 0x7f842415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84240d0a90 0x7f8424150c60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 --2- 172.21.15.164:0/2084354928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240aee90 0x7f8424152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 >> 172.21.15.164:0/2084354928 conn(0x7f8424067050 msgr2=0x7f842406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 shutdown_connections 2024-05-08T04:38:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.471+0000 7f842b000640 1 -- 172.21.15.164:0/2084354928 wait complete. 2024-05-08T04:38:40.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:40.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 Processor -- start 2024-05-08T04:38:40.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 -- start start 2024-05-08T04:38:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40b26c0 0x7f5cc40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5cc40cfbd0 0x7f5cc40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc406c770 con 0x7f5cc415ab60 2024-05-08T04:38:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc40d0060 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc8c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc406ca50 con 0x7f5cc40b26c0 2024-05-08T04:38:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40b26c0 0x7f5cc40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.627+0000 7f5cc2800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5cc40cfbd0 0x7f5cc40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc1e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5cc415ab60 0x7f5cc4072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:44322/0 (socket says 172.21.15.164:44322) 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc1e00640 1 -- 172.21.15.164:0/2387953428 learned_addr learned my addr 172.21.15.164:0/2387953428 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc0a00640 1 -- 172.21.15.164:0/2387953428 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967155776 0 0) 0x7f5cc406c770 con 0x7f5cc415ab60 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc2800640 1 -- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40b26c0 msgr2=0x7f5cc40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc2800640 1 --2- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40b26c0 0x7f5cc40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc2800640 1 -- 172.21.15.164:0/2387953428 >> v1:172.21.15.172:6789/0 conn(0x7f5cc415ab60 legacy=0x7f5cc4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc2800640 1 -- 172.21.15.164:0/2387953428 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc406cd30 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc2800640 1 --2- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40cfbd0 0x7f5cc40cd910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009d90 tx=0x7f5cbc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ecbc18a6b3bb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc0a00640 1 -- 172.21.15.164:0/2387953428 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0b2070 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc0a00640 1 -- 172.21.15.164:0/2387953428 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0a5830 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.631+0000 7f5cc0a00640 1 -- 172.21.15.164:0/2387953428 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0a5c00 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40cfbd0 msgr2=0x7f5cc40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40cfbd0 0x7f5cc40cd910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009d90 tx=0x7f5cbc01c970 comp rx=0 tx=0).stop 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2387953428 shutdown_connections 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40b26c0 0x7f5cc40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2387953428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40cfbd0 0x7f5cc40cd910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2387953428 >> 172.21.15.164:0/2387953428 conn(0x7f5cc406ac90 msgr2=0x7f5cc406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2387953428 shutdown_connections 2024-05-08T04:38:40.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2387953428 wait complete. 2024-05-08T04:38:40.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 Processor -- start 2024-05-08T04:38:40.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- start start 2024-05-08T04:38:40.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40b26c0 0x7f5cc414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40cfbd0 0x7f5cc415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40aeeb0 con 0x7f5cc415ab60 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40aed30 con 0x7f5cc40b26c0 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc8c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4167970 con 0x7f5cc40cfbd0 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37536/0 (socket says 172.21.15.164:37536) 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 -- 172.21.15.164:0/2183119527 learned_addr learned my addr 172.21.15.164:0/2183119527 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc1400640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40cfbd0 0x7f5cc415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc1e00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40b26c0 0x7f5cc414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 -- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40cfbd0 msgr2=0x7f5cc415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40cfbd0 0x7f5cc415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 -- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40b26c0 msgr2=0x7f5cc414cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40b26c0 0x7f5cc414cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.635+0000 7f5cc2800640 1 -- 172.21.15.164:0/2183119527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40b2c40 con 0x7f5cc415ab60 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cc2800640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0b0040 tx=0x7f5cbc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0b2050 con 0x7f5cc415ab60 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0b3040 con 0x7f5cc415ab60 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0ad650 con 0x7f5cc415ab60 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40b2f20 con 0x7f5cc415ab60 2024-05-08T04:38:40.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc40b34e0 con 0x7f5cc415ab60 2024-05-08T04:38:40.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4069d60 con 0x7f5cc415ab60 2024-05-08T04:38:40.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cbc0c0020 con 0x7f5cc415ab60 2024-05-08T04:38:40.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 0x7f5ca4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.639+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cbc11f800 con 0x7f5cc415ab60 2024-05-08T04:38:40.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.643+0000 7f5cc1e00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 0x7f5ca4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.643+0000 7f5cc1e00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 0x7f5ca4069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000f70 tx=0x7f5cb4007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.647+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cbc016560 con 0x7f5cc415ab60 2024-05-08T04:38:40.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.831+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc406d9a0 con 0x7f5cc415ab60 2024-05-08T04:38:40.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.831+0000 7f5cba000640 1 -- 172.21.15.164:0/2183119527 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f5cbc0f08a0 con 0x7f5cc415ab60 2024-05-08T04:38:40.835 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:40.835 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:38:40.835 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:38:40.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 msgr2=0x7f5ca4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 0x7f5ca4069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000f70 tx=0x7f5cb4007c10 comp rx=0 tx=0).stop 2024-05-08T04:38:40.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 msgr2=0x7f5cc415e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0b0040 tx=0x7f5cbc019710 comp rx=0 tx=0).stop 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 shutdown_connections 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5ca4066f60 0x7f5ca4069420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc40cfbd0 0x7f5cc415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cc415ab60 0x7f5cc415e7d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 --2- 172.21.15.164:0/2183119527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc40b26c0 0x7f5cc414cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 >> 172.21.15.164:0/2183119527 conn(0x7f5cc406ac90 msgr2=0x7f5cc40afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 shutdown_connections 2024-05-08T04:38:40.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.835+0000 7f5cc8c00640 1 -- 172.21.15.164:0/2183119527 wait complete. 2024-05-08T04:38:40.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:40.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 Processor -- start 2024-05-08T04:38:40.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 -- start start 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba00b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 0x7faba00d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faba00b4d50 0x7faba00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00b42a0 con 0x7faba00b4d50 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00cd4e0 con 0x7faba00ccfe0 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba7400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00cd7c0 con 0x7faba015bc50 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faba00b4d50 0x7faba00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faba00b4d50 0x7faba00d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37552/0 (socket says 172.21.15.164:37552) 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 -- 172.21.15.164:0/835487174 learned_addr learned my addr 172.21.15.164:0/835487174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba4a00640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba00b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7fab97e00640 1 --2- 172.21.15.164:0/835487174 >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 0x7faba00d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 -- 172.21.15.164:0/835487174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 msgr2=0x7faba00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba00b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 -- 172.21.15.164:0/835487174 >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 msgr2=0x7faba00d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:40.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 --2- 172.21.15.164:0/835487174 >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 0x7faba00d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba5400640 1 -- 172.21.15.164:0/835487174 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00cf590 con 0x7faba00b4d50 2024-05-08T04:38:40.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7faba4a00640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:40.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.991+0000 7fab97e00640 1 --2- 172.21.15.164:0/835487174 >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 0x7faba00d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:40.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba5400640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00b4d50 0x7faba00d05b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faba0071020 tx=0x7fab9801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220d46f2a3253fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:40.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7fab9fe00640 1 -- 172.21.15.164:0/835487174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980a57b0 con 0x7faba00b4d50 2024-05-08T04:38:40.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7fab9fe00640 1 -- 172.21.15.164:0/835487174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab980a5950 con 0x7faba00b4d50 2024-05-08T04:38:40.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7fab9fe00640 1 -- 172.21.15.164:0/835487174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980a5d20 con 0x7faba00b4d50 2024-05-08T04:38:40.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 -- 172.21.15.164:0/835487174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00b4d50 msgr2=0x7faba00d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00b4d50 0x7faba00d05b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faba0071020 tx=0x7fab9801c930 comp rx=0 tx=0).stop 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 -- 172.21.15.164:0/835487174 shutdown_connections 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00b4d50 0x7faba00d05b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 --2- 172.21.15.164:0/835487174 >> v2:172.21.15.172:3300/0 conn(0x7faba00ccfe0 0x7faba00d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 --2- 172.21.15.164:0/835487174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 -- 172.21.15.164:0/835487174 >> 172.21.15.164:0/835487174 conn(0x7faba00b34f0 msgr2=0x7faba00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 -- 172.21.15.164:0/835487174 shutdown_connections 2024-05-08T04:38:40.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.995+0000 7faba7400640 1 -- 172.21.15.164:0/835487174 wait complete. 2024-05-08T04:38:41.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 Processor -- start 2024-05-08T04:38:41.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- start start 2024-05-08T04:38:41.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 0x7faba0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba0158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00cea90 con 0x7faba00ccfe0 2024-05-08T04:38:41.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00ce910 con 0x7faba00b4d50 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00cec10 con 0x7faba015bc50 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37562/0 (socket says 172.21.15.164:37562) 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 -- 172.21.15.164:0/1645951423 learned_addr learned my addr 172.21.15.164:0/1645951423 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba5400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba0158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba4a00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 0x7faba0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 -- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 msgr2=0x7faba0158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba0158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 -- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 msgr2=0x7faba0154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 0x7faba0154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 -- 172.21.15.164:0/1645951423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba01562b0 con 0x7faba00ccfe0 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba5400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba0158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:41.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba4a00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 0x7faba0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab97e00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab8800c7c0 tx=0x7fab8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88015070 con 0x7faba00ccfe0 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88007a30 con 0x7faba00ccfe0 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba01541f0 con 0x7faba00ccfe0 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88007e00 con 0x7faba00ccfe0 2024-05-08T04:38:41.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba0154490 con 0x7faba00ccfe0 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:40.999+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba0069d60 con 0x7faba00ccfe0 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.003+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab88018070 con 0x7faba00ccfe0 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.003+0000 7fab9d600640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.003+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab88084f30 con 0x7faba00ccfe0 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.003+0000 7faba4a00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.003+0000 7faba4a00640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 0x7fab7c069450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab90008890 tx=0x7fab90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.007+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab88055f30 con 0x7faba00ccfe0 2024-05-08T04:38:41.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.163+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faba00b3a00 con 0x7faba00ccfe0 2024-05-08T04:38:41.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7fab9d600640 1 -- 172.21.15.164:0/1645951423 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fab88055830 con 0x7faba00ccfe0 2024-05-08T04:38:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:41.169 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 msgr2=0x7fab7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 0x7fab7c069450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab90008890 tx=0x7fab90008040 comp rx=0 tx=0).stop 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 msgr2=0x7faba0155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab8800c7c0 tx=0x7fab8800cc80 comp rx=0 tx=0).stop 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 shutdown_connections 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faba015bc50 0x7faba0158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba00ccfe0 0x7faba0155aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 --2- 172.21.15.164:0/1645951423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faba00b4d50 0x7faba0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 >> 172.21.15.164:0/1645951423 conn(0x7faba00b34f0 msgr2=0x7faba006d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 shutdown_connections 2024-05-08T04:38:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.167+0000 7faba7400640 1 -- 172.21.15.164:0/1645951423 wait complete. 2024-05-08T04:38:41.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:41.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:41.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 Processor -- start 2024-05-08T04:38:41.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 -- start start 2024-05-08T04:38:41.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda580ae6e0 0x7fda580aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580c6600 con 0x7fda5815ba20 2024-05-08T04:38:41.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580c6040 con 0x7fda5815ce10 2024-05-08T04:38:41.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda5e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5815fbe0 con 0x7fda580ae6e0 2024-05-08T04:38:41.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda57400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fda5815ba20 0x7fda58160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47542/0 (socket says 172.21.15.164:47542) 2024-05-08T04:38:41.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda57400640 1 -- 172.21.15.164:0/3772080885 learned_addr learned my addr 172.21.15.164:0/3772080885 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605633199 0 0) 0x7fda580c6600 con 0x7fda5815ba20 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda57e00640 1 --2- 172.21.15.164:0/3772080885 >> v2:172.21.15.172:3300/0 conn(0x7fda580ae6e0 0x7fda580aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda3c003530 con 0x7fda5815ba20 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 94946330 0 0) 0x7fda3c003530 con 0x7fda5815ba20 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 >> v2:172.21.15.172:3300/0 conn(0x7fda580ae6e0 msgr2=0x7fda580aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 --2- 172.21.15.164:0/3772080885 >> v2:172.21.15.172:3300/0 conn(0x7fda580ae6e0 0x7fda580aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 >> v1:172.21.15.172:6789/0 conn(0x7fda5815ce10 legacy=0x7fda5806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580c6320 con 0x7fda5815ba20 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fda48001a00 con 0x7fda5815ba20 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 >> v1:172.21.15.164:6789/0 conn(0x7fda5815ba20 legacy=0x7fda58160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c003a10 0x7fda3c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 0x7fda3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda3c0051d0 0x7fda3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5815fbe0 con 0x7fda3c004810 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c00c0d0 con 0x7fda3c003a10 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c00c2d0 con 0x7fda3c0051d0 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 0x7fda3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda57e00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda3c0051d0 0x7fda3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 -- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda3c0051d0 msgr2=0x7fda3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda3c0051d0 0x7fda3c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 -- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c003a10 msgr2=0x7fda3c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c003a10 0x7fda3c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda4ea00640 1 -- 172.21.15.164:0/3772080885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c003530 con 0x7fda3c004810 2024-05-08T04:38:41.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.327+0000 7fda57e00640 1 -- 172.21.15.164:0/3772080885 reap_dead start 2024-05-08T04:38:41.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda4ea00640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 0x7fda3c004c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda40001780 tx=0x7fda4001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859a800cae62fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400013a0 con 0x7fda3c004810 2024-05-08T04:38:41.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda400a9070 con 0x7fda3c004810 2024-05-08T04:38:41.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda56a00640 1 -- 172.21.15.164:0/3772080885 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40003600 con 0x7fda3c004810 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 -- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 msgr2=0x7fda3c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 0x7fda3c004c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda40001780 tx=0x7fda4001c420 comp rx=0 tx=0).stop 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 -- 172.21.15.164:0/3772080885 shutdown_connections 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 --2- 172.21.15.164:0/3772080885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda3c004810 0x7fda3c004c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 -- 172.21.15.164:0/3772080885 >> 172.21.15.164:0/3772080885 conn(0x7fda580cf770 msgr2=0x7fda580cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 -- 172.21.15.164:0/3772080885 shutdown_connections 2024-05-08T04:38:41.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.331+0000 7fda5e200640 1 -- 172.21.15.164:0/3772080885 wait complete. 2024-05-08T04:38:41.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 Processor -- start 2024-05-08T04:38:41.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- start start 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c004810 0x7fda5814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 0x7fda580cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580aeb20 con 0x7fda580ad890 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580cf410 con 0x7fda3c004810 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580aeca0 con 0x7fda580cae10 2024-05-08T04:38:41.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37586/0 (socket says 172.21.15.164:37586) 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 -- 172.21.15.164:0/931427641 learned_addr learned my addr 172.21.15.164:0/931427641 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57400640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c004810 0x7fda5814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda4ea00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 0x7fda580cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 -- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 msgr2=0x7fda580cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 0x7fda580cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 -- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c004810 msgr2=0x7fda5814fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c004810 0x7fda5814fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 -- 172.21.15.164:0/931427641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580b1180 con 0x7fda580ad890 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda4ea00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 0x7fda580cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda57e00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda580cbff0 tx=0x7fda50011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50004440 con 0x7fda580ad890 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5001b070 con 0x7fda580ad890 2024-05-08T04:38:41.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50003860 con 0x7fda580ad890 2024-05-08T04:38:41.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda580af030 con 0x7fda580ad890 2024-05-08T04:38:41.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.335+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda580af5f0 con 0x7fda580ad890 2024-05-08T04:38:41.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda58072d30 con 0x7fda580ad890 2024-05-08T04:38:41.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda50003a00 con 0x7fda580ad890 2024-05-08T04:38:41.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda4fe00640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 0x7fda34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda50085580 con 0x7fda580ad890 2024-05-08T04:38:41.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda57400640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 0x7fda34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.339+0000 7fda57400640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 0x7fda34069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda5806e7b0 tx=0x7fda580c64f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.343+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda50056670 con 0x7fda580ad890 2024-05-08T04:38:41.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.499+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda5804f010 con 0x7fda580ad890 2024-05-08T04:38:41.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.499+0000 7fda4fe00640 1 -- 172.21.15.164:0/931427641 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda50055f70 con 0x7fda580ad890 2024-05-08T04:38:41.504 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 msgr2=0x7fda34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 0x7fda34069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda5806e7b0 tx=0x7fda580c64f0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 msgr2=0x7fda580add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda580cbff0 tx=0x7fda50011920 comp rx=0 tx=0).stop 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 shutdown_connections 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda34066f60 0x7fda34069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda580cae10 0x7fda580cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda580ad890 0x7fda580add40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 --2- 172.21.15.164:0/931427641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda3c004810 0x7fda5814fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 >> 172.21.15.164:0/931427641 conn(0x7fda580cf770 msgr2=0x7fda58073e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:41.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 shutdown_connections 2024-05-08T04:38:41.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.503+0000 7fda5e200640 1 -- 172.21.15.164:0/931427641 wait complete. 2024-05-08T04:38:41.524 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:41.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 Processor -- start 2024-05-08T04:38:41.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 -- start start 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f802415e000 0x7f80240b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f802406c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80240b2be0 con 0x7f802406c740 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80240d0c40 con 0x7f802415e000 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802406cbc0 con 0x7f802406bda0 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f802406c740 0x7f80240ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47546/0 (socket says 172.21.15.164:47546) 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.659+0000 7f802a800640 1 -- 172.21.15.164:0/3233205107 learned_addr learned my addr 172.21.15.164:0/3233205107 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8029e00640 1 --2- 172.21.15.164:0/3233205107 >> v2:172.21.15.172:3300/0 conn(0x7f802415e000 0x7f80240b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8029400640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f802406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583278876 0 0) 0x7f80240b2be0 con 0x7f802406c740 2024-05-08T04:38:41.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f800c003530 con 0x7f802406c740 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248673519 0 0) 0x7f800c003530 con 0x7f802406c740 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 msgr2=0x7f802406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f802406c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 >> v2:172.21.15.172:3300/0 conn(0x7f802415e000 msgr2=0x7f80240b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 --2- 172.21.15.164:0/3233205107 >> v2:172.21.15.172:3300/0 conn(0x7f802415e000 0x7f80240b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240b2eb0 con 0x7f802406c740 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f8020001a00 con 0x7f802406c740 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 >> v1:172.21.15.164:6789/0 conn(0x7f802406c740 legacy=0x7f80240ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c0039c0 0x7f800c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f800c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f800c004d50 0x7f800c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240d0c40 con 0x7f800c004d50 2024-05-08T04:38:41.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802406cbc0 con 0x7f800c0039c0 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c00b9d0 con 0x7f802406bda0 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8029e00640 1 --2- 172.21.15.164:0/3233205107 >> v2:172.21.15.172:3300/0 conn(0x7f802415e000 0x7f80240b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f800c004d50 0x7f800c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8029400640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f800c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f8029e00640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c0039c0 0x7f800c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 -- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 msgr2=0x7f800c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f802406bda0 0x7f800c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 -- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c0039c0 msgr2=0x7f800c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c0039c0 0x7f800c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.663+0000 7f802a800640 1 -- 172.21.15.164:0/3233205107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c003530 con 0x7f800c004d50 2024-05-08T04:38:41.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802a800640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f800c004d50 0x7f800c0051d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f802000a8a0 tx=0x7f802001fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfa3b9f4d0a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000a4c0 con 0x7f800c004d50 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80200ad020 con 0x7f800c004d50 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f8028a00640 1 -- 172.21.15.164:0/3233205107 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80200a9740 con 0x7f800c004d50 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- 172.21.15.164:0/3233205107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f800c004d50 msgr2=0x7f800c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 --2- 172.21.15.164:0/3233205107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f800c004d50 0x7f800c0051d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f802000a8a0 tx=0x7f802001fb20 comp rx=0 tx=0).stop 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802a800640 1 -- 172.21.15.164:0/3233205107 reap_dead start 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- 172.21.15.164:0/3233205107 shutdown_connections 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- 172.21.15.164:0/3233205107 >> 172.21.15.164:0/3233205107 conn(0x7f80240b34f0 msgr2=0x7f80240b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- 172.21.15.164:0/3233205107 shutdown_connections 2024-05-08T04:38:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- 172.21.15.164:0/3233205107 wait complete. 2024-05-08T04:38:41.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 Processor -- start 2024-05-08T04:38:41.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- start start 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c004d50 0x7f80240b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8024160700 0x7f8024160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240d08d0 con 0x7f8024161500 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240d05f0 con 0x7f800c004d50 2024-05-08T04:38:41.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.667+0000 7f802c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802415e3e0 con 0x7f8024160700 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37604/0 (socket says 172.21.15.164:37604) 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 -- 172.21.15.164:0/1090227475 learned_addr learned my addr 172.21.15.164:0/1090227475 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029e00640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c004d50 0x7f80240b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f802a800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8024160700 0x7f8024160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 -- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8024160700 msgr2=0x7f8024160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8024160700 0x7f8024160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 -- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c004d50 msgr2=0x7f80240b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c004d50 0x7f80240b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 -- 172.21.15.164:0/1090227475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80241610c0 con 0x7f8024161500 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f8029400640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8014011930 tx=0x7f8014011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400fa30 con 0x7f8024161500 2024-05-08T04:38:41.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400fbd0 con 0x7f8024161500 2024-05-08T04:38:41.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014014440 con 0x7f8024161500 2024-05-08T04:38:41.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80240b74a0 con 0x7f8024161500 2024-05-08T04:38:41.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80240ca7d0 con 0x7f8024161500 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802409fb50 con 0x7f8024161500 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8014012040 con 0x7f8024161500 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 0x7f8008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.671+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8014085680 con 0x7f8024161500 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.675+0000 7f8029e00640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 0x7f8008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:41.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.675+0000 7f8029e00640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 0x7f8008069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d00 tx=0x7f8018008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:41.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.679+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8014019030 con 0x7f8024161500 2024-05-08T04:38:41.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f802404f010 con 0x7f8024161500 2024-05-08T04:38:41.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f801e000640 1 -- 172.21.15.164:0/1090227475 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80140565f0 con 0x7f8024161500 2024-05-08T04:38:41.838 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 msgr2=0x7f8008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 0x7f8008069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d00 tx=0x7f8018008040 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 msgr2=0x7f80241601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8014011930 tx=0x7f8014011d30 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 shutdown_connections 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8008066f60 0x7f8008069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8024160700 0x7f8024160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8024161500 0x7f80241601c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 --2- 172.21.15.164:0/1090227475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f800c004d50 0x7f80240b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:41.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 >> 172.21.15.164:0/1090227475 conn(0x7f80240b34f0 msgr2=0x7f80240ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:41.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.835+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 shutdown_connections 2024-05-08T04:38:41.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.839+0000 7f802c800640 1 -- 172.21.15.164:0/1090227475 wait complete. 2024-05-08T04:38:41.859 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:41.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 Processor -- start 2024-05-08T04:38:41.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 -- start start 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f222c15f2b0 0x7f222c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0b3fe0 0x7f222c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f222c0cd390 0x7f222c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c15f690 con 0x7f222c0cd390 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0cff80 con 0x7f222c15f2b0 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f2232800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c070280 con 0x7f222c0b3fe0 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f222c0cd390 0x7f222c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f222c0cd390 0x7f222c0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37612/0 (socket says 172.21.15.164:37612) 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 -- 172.21.15.164:0/1660104384 learned_addr learned my addr 172.21.15.164:0/1660104384 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222b400640 1 --2- 172.21.15.164:0/1660104384 >> v2:172.21.15.172:3300/0 conn(0x7f222c15f2b0 0x7f222c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222aa00640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0b3fe0 0x7f222c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 -- 172.21.15.164:0/1660104384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0b3fe0 msgr2=0x7f222c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:42.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0b3fe0 0x7f222c0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 -- 172.21.15.164:0/1660104384 >> v2:172.21.15.172:3300/0 conn(0x7f222c15f2b0 msgr2=0x7f222c0d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:42.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 --2- 172.21.15.164:0/1660104384 >> v2:172.21.15.172:3300/0 conn(0x7f222c15f2b0 0x7f222c0d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.995+0000 7f222be00640 1 -- 172.21.15.164:0/1660104384 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c070560 con 0x7f222c0cd390 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f222be00640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0cd390 0x7f222c0cd820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c070870 tx=0x7f222401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c3bc68320005e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f222a000640 1 -- 172.21.15.164:0/1660104384 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240a5840 con 0x7f222c0cd390 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f222a000640 1 -- 172.21.15.164:0/1660104384 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22240a59e0 con 0x7f222c0cd390 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f222a000640 1 -- 172.21.15.164:0/1660104384 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240a5db0 con 0x7f222c0cd390 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 -- 172.21.15.164:0/1660104384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0cd390 msgr2=0x7f222c0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0cd390 0x7f222c0cd820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c070870 tx=0x7f222401c9c0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 -- 172.21.15.164:0/1660104384 shutdown_connections 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0cd390 0x7f222c0cd820 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 --2- 172.21.15.164:0/1660104384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0b3fe0 0x7f222c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 --2- 172.21.15.164:0/1660104384 >> v2:172.21.15.172:3300/0 conn(0x7f222c15f2b0 0x7f222c0d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 -- 172.21.15.164:0/1660104384 >> 172.21.15.164:0/1660104384 conn(0x7f222c0b34f0 msgr2=0x7f222c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 -- 172.21.15.164:0/1660104384 shutdown_connections 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:41.999+0000 7f2232800640 1 -- 172.21.15.164:0/1660104384 wait complete. 2024-05-08T04:38:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 Processor -- start 2024-05-08T04:38:42.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- start start 2024-05-08T04:38:42.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0cd390 0x7f222c06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222c15f2b0 0x7f222c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c06efb0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c06ee30 con 0x7f222c15f2b0 2024-05-08T04:38:42.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c06f130 con 0x7f222c0cd390 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37624/0 (socket says 172.21.15.164:37624) 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 -- 172.21.15.164:0/3040998412 learned_addr learned my addr 172.21.15.164:0/3040998412 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222be00640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222c15f2b0 0x7f222c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222aa00640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0cd390 0x7f222c06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 -- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0cd390 msgr2=0x7f222c06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0cd390 0x7f222c06bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 -- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222c15f2b0 msgr2=0x7f222c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222c15f2b0 0x7f222c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 -- 172.21.15.164:0/3040998412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0c66c0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f222b400640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e870 tx=0x7f221c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c017070 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c00c940 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00ccf0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c0c69a0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.003+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c0c6f60 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c09fae0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f221c01a070 con 0x7f222c0b3fe0 2024-05-08T04:38:42.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f2223400640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 0x7f220c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f221c085190 con 0x7f222c0b3fe0 2024-05-08T04:38:42.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f222aa00640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 0x7f220c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.007+0000 7f222aa00640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 0x7f220c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f222c1fa0e0 tx=0x7f2218009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.011+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f221c055b50 con 0x7f222c0b3fe0 2024-05-08T04:38:42.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.163+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f222c04f010 con 0x7f222c0b3fe0 2024-05-08T04:38:42.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.163+0000 7f2223400640 1 -- 172.21.15.164:0/3040998412 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f221c059bf0 con 0x7f222c0b3fe0 2024-05-08T04:38:42.167 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 msgr2=0x7f220c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 0x7f220c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f222c1fa0e0 tx=0x7f2218009190 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 msgr2=0x7f222c0c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e870 tx=0x7f221c00ec70 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 shutdown_connections 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f220c066f60 0x7f220c069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f222c0cd390 0x7f222c06bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222c15f2b0 0x7f222c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 --2- 172.21.15.164:0/3040998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f222c0b3fe0 0x7f222c0c7eb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 >> 172.21.15.164:0/3040998412 conn(0x7f222c0b34f0 msgr2=0x7f222c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 shutdown_connections 2024-05-08T04:38:42.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.167+0000 7f2232800640 1 -- 172.21.15.164:0/3040998412 wait complete. 2024-05-08T04:38:42.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:42.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 Processor -- start 2024-05-08T04:38:42.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 -- start start 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a4165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a406e0e0 con 0x7fd1a40aee30 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a415ed80 con 0x7fd1a415b390 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a415e8f0 con 0x7fd1a40cf150 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd19a000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd1a40aee30 0x7fd1a40cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47562/0 (socket says 172.21.15.164:47562) 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd19a000640 1 -- 172.21.15.164:0/3326908603 learned_addr learned my addr 172.21.15.164:0/3326908603 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:42.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a3400640 1 --2- 172.21.15.164:0/3326908603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a4165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347538297 0 0) 0x7fd1a406e0e0 con 0x7fd1a40aee30 2024-05-08T04:38:42.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.323+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188003530 con 0x7fd1a40aee30 2024-05-08T04:38:42.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319755192 0 0) 0x7fd1a415ed80 con 0x7fd1a415b390 2024-05-08T04:38:42.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a406e0e0 con 0x7fd1a415b390 2024-05-08T04:38:42.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112316159 0 0) 0x7fd188003530 con 0x7fd1a40aee30 2024-05-08T04:38:42.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a3400640 1 -- 172.21.15.164:0/3326908603 >> v1:172.21.15.172:6789/0 conn(0x7fd1a415b390 legacy=0x7fd1a406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a3400640 1 -- 172.21.15.164:0/3326908603 >> v1:172.21.15.164:6789/0 conn(0x7fd1a40aee30 legacy=0x7fd1a40cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a3400640 1 -- 172.21.15.164:0/3326908603 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a415dfc0 con 0x7fd1a40cf150 2024-05-08T04:38:42.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a3400640 1 --2- 172.21.15.164:0/3326908603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a4165240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009dc0 tx=0x7fd19c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d790060582b9fbb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0b2070 con 0x7fd1a40cf150 2024-05-08T04:38:42.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c0a5580 con 0x7fd1a40cf150 2024-05-08T04:38:42.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a2000640 1 -- 172.21.15.164:0/3326908603 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0016a0 con 0x7fd1a40cf150 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3326908603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 msgr2=0x7fd1a4165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.327+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3326908603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a4165240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009dc0 tx=0x7fd19c01c8b0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3326908603 shutdown_connections 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3326908603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a4165240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3326908603 >> 172.21.15.164:0/3326908603 conn(0x7fd1a406ac90 msgr2=0x7fd1a406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3326908603 shutdown_connections 2024-05-08T04:38:42.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3326908603 wait complete. 2024-05-08T04:38:42.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 Processor -- start 2024-05-08T04:38:42.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- start start 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 0x7fd1a40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4167790 con 0x7fd1a415b390 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a415a8c0 con 0x7fd1a40aee30 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4167910 con 0x7fd1a40cf150 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37630/0 (socket says 172.21.15.164:37630) 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 -- 172.21.15.164:0/3423101040 learned_addr learned my addr 172.21.15.164:0/3423101040 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd19a000640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a2a00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 0x7fd1a40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 -- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 msgr2=0x7fd1a413ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a413ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 -- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 msgr2=0x7fd1a40ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 0x7fd1a40ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 -- 172.21.15.164:0/3423101040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a41540d0 con 0x7fd1a415b390 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a2a00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 0x7fd1a40ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.331+0000 7fd1a3400640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd19c01a4e0 tx=0x7fd19c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0b2050 con 0x7fd1a415b390 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c0be070 con 0x7fd1a415b390 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c009450 con 0x7fd1a415b390 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a4168440 con 0x7fd1a415b390 2024-05-08T04:38:42.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a4168a00 con 0x7fd1a415b390 2024-05-08T04:38:42.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a4069d60 con 0x7fd1a415b390 2024-05-08T04:38:42.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd19c0ada70 con 0x7fd1a415b390 2024-05-08T04:38:42.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 0x7fd180069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd19c11fe10 con 0x7fd1a415b390 2024-05-08T04:38:42.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd1a2a00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 0x7fd180069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.335+0000 7fd1a2a00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 0x7fd180069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1940011f0 tx=0x7fd194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.339+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd19c123030 con 0x7fd1a415b390 2024-05-08T04:38:42.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a40d0580 con 0x7fd1a415b390 2024-05-08T04:38:42.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd19be00640 1 -- 172.21.15.164:0/3423101040 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7fd19c0164a0 con 0x7fd1a415b390 2024-05-08T04:38:42.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:42.533 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:38:42.533 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 msgr2=0x7fd180069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 0x7fd180069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1940011f0 tx=0x7fd194009040 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 msgr2=0x7fd1a40abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd19c01a4e0 tx=0x7fd19c019650 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 shutdown_connections 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd180066f60 0x7fd180069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1a40cf150 0x7fd1a413ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1a415b390 0x7fd1a40abfa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 --2- 172.21.15.164:0/3423101040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1a40aee30 0x7fd1a40ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 >> 172.21.15.164:0/3423101040 conn(0x7fd1a406ac90 msgr2=0x7fd1a406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 shutdown_connections 2024-05-08T04:38:42.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.531+0000 7fd1a9e00640 1 -- 172.21.15.164:0/3423101040 wait complete. 2024-05-08T04:38:42.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 Processor -- start 2024-05-08T04:38:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 -- start start 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d8806ac90 0x7f4d8806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 0x7f4d88071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d880add10 0x7f4d880719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d88071f00 con 0x7f4d8806ac90 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8806b5b0 con 0x7f4d880ad890 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d8cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88072120 con 0x7f4d880add10 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d8806ac90 0x7f4d8806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d8806ac90 0x7f4d8806b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37644/0 (socket says 172.21.15.164:37644) 2024-05-08T04:38:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 -- 172.21.15.164:0/2101866712 learned_addr learned my addr 172.21.15.164:0/2101866712 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d86800640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d880add10 0x7f4d880719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85400640 1 --2- 172.21.15.164:0/2101866712 >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 0x7f4d88071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 -- 172.21.15.164:0/2101866712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d880add10 msgr2=0x7f4d880719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d880add10 0x7f4d880719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 -- 172.21.15.164:0/2101866712 >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 msgr2=0x7f4d88071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 --2- 172.21.15.164:0/2101866712 >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 0x7f4d88071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85e00640 1 -- 172.21.15.164:0/2101866712 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d88072400 con 0x7f4d8806ac90 2024-05-08T04:38:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.687+0000 7f4d85400640 1 --2- 172.21.15.164:0/2101866712 >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 0x7f4d88071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d85e00640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d8806ac90 0x7f4d8806b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ce160 tx=0x7f4d7801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f95f364dce56c3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d84a00640 1 -- 172.21.15.164:0/2101866712 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780b1070 con 0x7f4d8806ac90 2024-05-08T04:38:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d84a00640 1 -- 172.21.15.164:0/2101866712 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d780ac630 con 0x7f4d8806ac90 2024-05-08T04:38:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d84a00640 1 -- 172.21.15.164:0/2101866712 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780a8050 con 0x7f4d8806ac90 2024-05-08T04:38:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 -- 172.21.15.164:0/2101866712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d8806ac90 msgr2=0x7f4d8806b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d8806ac90 0x7f4d8806b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ce160 tx=0x7f4d7801c520 comp rx=0 tx=0).stop 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 -- 172.21.15.164:0/2101866712 shutdown_connections 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d880add10 0x7f4d880719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/2101866712 >> v2:172.21.15.172:3300/0 conn(0x7f4d880ad890 0x7f4d88071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/2101866712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d8806ac90 0x7f4d8806b070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 -- 172.21.15.164:0/2101866712 >> 172.21.15.164:0/2101866712 conn(0x7f4d88066fa0 msgr2=0x7f4d8806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 -- 172.21.15.164:0/2101866712 shutdown_connections 2024-05-08T04:38:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 -- 172.21.15.164:0/2101866712 wait complete. 2024-05-08T04:38:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.691+0000 7f4d8cc00640 1 Processor -- start 2024-05-08T04:38:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- start start 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 0x7f4d88157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 0x7f4d880cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8815f3a0 con 0x7f4d880add10 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8815f220 con 0x7f4d880ad890 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8806e590 con 0x7f4d8806ac90 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37654/0 (socket says 172.21.15.164:37654) 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 -- 172.21.15.164:0/3956819920 learned_addr learned my addr 172.21.15.164:0/3956819920 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d85e00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 0x7f4d88157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d85400640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 0x7f4d880cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 -- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 msgr2=0x7f4d88157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 0x7f4d88157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 -- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 msgr2=0x7f4d880cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 0x7f4d880cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 -- 172.21.15.164:0/3956819920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8815be60 con 0x7f4d880add10 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d85e00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 0x7f4d88157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d85400640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 0x7f4d880cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d86800640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000cc30 tx=0x7f4d8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000ce40 con 0x7f4d880add10 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d800039a0 con 0x7f4d880add10 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8815c140 con 0x7f4d880add10 2024-05-08T04:38:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d80003d50 con 0x7f4d880add10 2024-05-08T04:38:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8815c700 con 0x7f4d880add10 2024-05-08T04:38:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.695+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d88069d60 con 0x7f4d880add10 2024-05-08T04:38:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.699+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d80003f70 con 0x7f4d880add10 2024-05-08T04:38:42.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.699+0000 7f4d7e000640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 0x7f4d68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:42.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.699+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d800854c0 con 0x7f4d880add10 2024-05-08T04:38:42.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.699+0000 7f4d85e00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 0x7f4d68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:42.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.699+0000 7f4d85e00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 0x7f4d68069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ae000 tx=0x7f4d78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:42.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.703+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d80014030 con 0x7f4d880add10 2024-05-08T04:38:42.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.859+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d880cdd10 con 0x7f4d880add10 2024-05-08T04:38:42.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.859+0000 7f4d7e000640 1 -- 172.21.15.164:0/3956819920 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f4d800564d0 con 0x7f4d880add10 2024-05-08T04:38:42.862 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:42.862 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:42.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 msgr2=0x7f4d68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 0x7f4d68069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ae000 tx=0x7f4d78008040 comp rx=0 tx=0).stop 2024-05-08T04:38:42.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 msgr2=0x7f4d880d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000cc30 tx=0x7f4d8000a5c0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 shutdown_connections 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d68066f60 0x7f4d68069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d880add10 0x7f4d880d04c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d880ad890 0x7f4d880cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 --2- 172.21.15.164:0/3956819920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d8806ac90 0x7f4d88157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 >> 172.21.15.164:0/3956819920 conn(0x7f4d88066fa0 msgr2=0x7f4d880b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 shutdown_connections 2024-05-08T04:38:42.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:42.863+0000 7f4d8cc00640 1 -- 172.21.15.164:0/3956819920 wait complete. 2024-05-08T04:38:42.883 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:38:43.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.015+0000 7fc4c3800640 1 Processor -- start 2024-05-08T04:38:43.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.015+0000 7fc4c3800640 1 -- start start 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c3800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc4bc0b26c0 0x7fc4bc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c3800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc06c770 con 0x7fc4bc15ab60 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c3800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc06ca50 con 0x7fc4bc0cfbd0 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c3800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc0d0060 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c0e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc4bc15ab60 0x7fc4bc072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:47574/0 (socket says 172.21.15.164:47574) 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4c0e00640 1 -- 172.21.15.164:0/1378859131 learned_addr learned my addr 172.21.15.164:0/1378859131 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:43.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4b3e00640 1 --2- 172.21.15.164:0/1378859131 >> v2:172.21.15.164:3300/0 conn(0x7fc4bc0b26c0 0x7fc4bc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4bbe00640 1 -- 172.21.15.164:0/1378859131 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213770872 0 0) 0x7fc4bc06c770 con 0x7fc4bc15ab60 2024-05-08T04:38:43.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4bbe00640 1 -- 172.21.15.164:0/1378859131 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a0003530 con 0x7fc4bc15ab60 2024-05-08T04:38:43.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4b3e00640 1 -- 172.21.15.164:0/1378859131 >> v1:172.21.15.172:6789/0 conn(0x7fc4bc0cfbd0 legacy=0x7fc4bc0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4b3e00640 1 -- 172.21.15.164:0/1378859131 >> v1:172.21.15.164:6789/0 conn(0x7fc4bc15ab60 legacy=0x7fc4bc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.019+0000 7fc4b3e00640 1 -- 172.21.15.164:0/1378859131 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc06cd30 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4b3e00640 1 --2- 172.21.15.164:0/1378859131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc0cf690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2304967ef85724ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4bbe00640 1 -- 172.21.15.164:0/1378859131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b0070 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4bbe00640 1 -- 172.21.15.164:0/1378859131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40ab600 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4bbe00640 1 -- 172.21.15.164:0/1378859131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40ab9d0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- 172.21.15.164:0/1378859131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 msgr2=0x7fc4bc0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 --2- 172.21.15.164:0/1378859131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc0cf690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- 172.21.15.164:0/1378859131 shutdown_connections 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 --2- 172.21.15.164:0/1378859131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc0cf690 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- 172.21.15.164:0/1378859131 >> 172.21.15.164:0/1378859131 conn(0x7fc4bc06ac90 msgr2=0x7fc4bc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- 172.21.15.164:0/1378859131 shutdown_connections 2024-05-08T04:38:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- 172.21.15.164:0/1378859131 wait complete. 2024-05-08T04:38:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 Processor -- start 2024-05-08T04:38:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.023+0000 7fc4c3800640 1 -- start start 2024-05-08T04:38:43.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4bc0cfbd0 0x7fc4bc159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 0x7fc4bc1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0aeeb0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0aed30 con 0x7fc4bc0cfbd0 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc168980 con 0x7fc4bc15ab60 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37676/0 (socket says 172.21.15.164:37676) 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 -- 172.21.15.164:0/2497076633 learned_addr learned my addr 172.21.15.164:0/2497076633 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4b3e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4bc0cfbd0 0x7fc4bc159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c1800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 0x7fc4bc1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 -- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 msgr2=0x7fc4bc1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 0x7fc4bc1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 -- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4bc0cfbd0 msgr2=0x7fc4bc159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4bc0cfbd0 0x7fc4bc159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 -- 172.21.15.164:0/2497076633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc14f340 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c1800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 0x7fc4bc1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c0e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00c420 tx=0x7fc4ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac017070 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac005cf0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc1575a0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0072d0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc158160 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc069d60 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.027+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4ac01a070 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.031+0000 7fc4b9600640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 0x7fc498069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.031+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4ac0843e0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.031+0000 7fc4b3e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 0x7fc498069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.031+0000 7fc4b3e00640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 0x7fc498069450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40ad000 tx=0x7fc4a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.035+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4ac054d00 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.223+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4bc070e30 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.223+0000 7fc4b9600640 1 -- 172.21.15.164:0/2497076633 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7fc4ac058da0 con 0x7fc4bc0b26c0 2024-05-08T04:38:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:43.227 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:38:43.228 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:38:43.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 msgr2=0x7fc498069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 0x7fc498069450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40ad000 tx=0x7fc4a4008040 comp rx=0 tx=0).stop 2024-05-08T04:38:43.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 msgr2=0x7fc4bc156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00c420 tx=0x7fc4ac00c8e0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 shutdown_connections 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc498066f90 0x7fc498069450 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4bc15ab60 0x7fc4bc1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4bc0cfbd0 0x7fc4bc159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 --2- 172.21.15.164:0/2497076633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4bc0b26c0 0x7fc4bc156fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 >> 172.21.15.164:0/2497076633 conn(0x7fc4bc06ac90 msgr2=0x7fc4bc15ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 shutdown_connections 2024-05-08T04:38:43.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.227+0000 7fc4c3800640 1 -- 172.21.15.164:0/2497076633 wait complete. 2024-05-08T04:38:43.650 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:38:29.101740+0000', 'modified': '2024-05-08T04:38:32.110093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4309}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4309': {'gid': 4309, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/3487014303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6833', 'nonce': 3487014303}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 3487014303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:38:43.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:38:43.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T04:38:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 Processor -- start 2024-05-08T04:38:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 -- start start 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08840cfda0 0x7f0884160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415be50 0x7f08840b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cd9f0 con 0x7f088415ba70 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088415fe70 con 0x7f08840cfda0 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.787+0000 7f088a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08840cdcd0 con 0x7f088415be50 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08840cfda0 0x7f0884160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08840cfda0 0x7f0884160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37678/0 (socket says 172.21.15.164:37678) 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 -- 172.21.15.164:0/2796392308 learned_addr learned my addr 172.21.15.164:0/2796392308 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f0883e00640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415be50 0x7f08840b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 -- 172.21.15.164:0/2796392308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415be50 msgr2=0x7f08840b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415be50 0x7f08840b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 -- 172.21.15.164:0/2796392308 >> v1:172.21.15.172:6789/0 conn(0x7f088415ba70 legacy=0x7f08840cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 -- 172.21.15.164:0/2796392308 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088415f820 con 0x7f08840cfda0 2024-05-08T04:38:43.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f087be00640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08840cfda0 0x7f0884160300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ba55b40db93e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f0882a00640 1 -- 172.21.15.164:0/2796392308 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c0ab640 con 0x7f08840cfda0 2024-05-08T04:38:43.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f0882a00640 1 -- 172.21.15.164:0/2796392308 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c0b0070 con 0x7f08840cfda0 2024-05-08T04:38:43.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.791+0000 7f0882a00640 1 -- 172.21.15.164:0/2796392308 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c0aba00 con 0x7f08840cfda0 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- 172.21.15.164:0/2796392308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08840cfda0 msgr2=0x7f0884160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08840cfda0 0x7f0884160300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c01c530 comp rx=0 tx=0).stop 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- 172.21.15.164:0/2796392308 shutdown_connections 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415be50 0x7f08840b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- 172.21.15.164:0/2796392308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08840cfda0 0x7f0884160300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- 172.21.15.164:0/2796392308 >> 172.21.15.164:0/2796392308 conn(0x7f088406ac90 msgr2=0x7f088406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- 172.21.15.164:0/2796392308 shutdown_connections 2024-05-08T04:38:43.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- 172.21.15.164:0/2796392308 wait complete. 2024-05-08T04:38:43.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 Processor -- start 2024-05-08T04:38:43.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- start start 2024-05-08T04:38:43.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08840cfda0 0x7f0884149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415ba70 0x7f08840afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884168d40 con 0x7f088415be50 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884168bc0 con 0x7f08840cfda0 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f088a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0884168ec0 con 0x7f088415ba70 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37686/0 (socket says 172.21.15.164:37686) 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 -- 172.21.15.164:0/2266664959 learned_addr learned my addr 172.21.15.164:0/2266664959 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f087be00640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415ba70 0x7f08840afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883400640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08840cfda0 0x7f0884149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 -- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415ba70 msgr2=0x7f08840afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415ba70 0x7f08840afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 -- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08840cfda0 msgr2=0x7f0884149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08840cfda0 0x7f0884149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.795+0000 7f0883e00640 1 -- 172.21.15.164:0/2266664959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840b0230 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f0883e00640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f087c00efc0 tx=0x7f087c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 -- 172.21.15.164:0/2266664959 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c018020 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 -- 172.21.15.164:0/2266664959 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c014480 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 -- 172.21.15.164:0/2266664959 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c009e60 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08840b0d60 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08840af210 con 0x7f088415be50 2024-05-08T04:38:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 -- 172.21.15.164:0/2266664959 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f087c01b020 con 0x7f088415be50 2024-05-08T04:38:43.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 0x7f0860069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f087b400640 1 -- 172.21.15.164:0/2266664959 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f087c085c20 con 0x7f088415be50 2024-05-08T04:38:43.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.799+0000 7f0883400640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 0x7f0860069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 Processor -- start 2024-05-08T04:38:43.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- start start 2024-05-08T04:38:43.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883400640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 0x7f0860069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0874001810 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x5650036c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x5650036789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650035605f0 con 0x565003678f30 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650034c7da0 con 0x5650036c3240 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x565003450150 con 0x5650036c2920 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37688/0 (socket says 172.21.15.164:37688) 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 -- 172.21.15.164:0/3314468037 learned_addr learned my addr 172.21.15.164:0/3314468037 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f087be00640 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x5650036789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883400640 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x5650036c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 -- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 msgr2=0x5650036c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x5650036c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 -- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 msgr2=0x5650036789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x5650036789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 -- 172.21.15.164:0/3314468037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565003469ca0 con 0x565003678f30 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0883e00640 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f087c056570 tx=0x7f087c056250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2017eb5d845b85bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0873e00640 1 -- 172.21.15.164:0/3314468037 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c052c50 con 0x565003678f30 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0873e00640 1 -- 172.21.15.164:0/3314468037 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c05a160 con 0x565003678f30 2024-05-08T04:38:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f0873e00640 1 -- 172.21.15.164:0/3314468037 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c052df0 con 0x565003678f30 2024-05-08T04:38:43.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- 172.21.15.164:0/3314468037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 msgr2=0x565003679310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f087c056570 tx=0x7f087c056250 comp rx=0 tx=0).stop 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- 172.21.15.164:0/3314468037 shutdown_connections 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x565003679310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x5650036789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 --2- 172.21.15.164:0/3314468037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x5650036c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- 172.21.15.164:0/3314468037 >> 172.21.15.164:0/3314468037 conn(0x56500367f2a0 msgr2=0x5650036772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- 172.21.15.164:0/3314468037 shutdown_connections 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- 172.21.15.164:0/3314468037 wait complete. 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 -- start start 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.803+0000 7f088dd68000 1 Processor -- start 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565003678f30 0x5650036c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c3240 0x56500367caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x565003482990 con 0x5650036c2920 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650034c3e40 con 0x565003678f30 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x565003453900 con 0x5650036c3240 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f088dd68000 10 client.0 _ll_register_callbacks cb 0x56500357b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37694/0 (socket says 172.21.15.164:37694) 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 -- 172.21.15.164:0/3582957245 learned_addr learned my addr 172.21.15.164:0/3582957245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f0883400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565003678f30 0x5650036c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f0883e00640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c3240 0x56500367caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 -- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c3240 msgr2=0x56500367caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c3240 0x56500367caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 -- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565003678f30 msgr2=0x5650036c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565003678f30 0x5650036c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 -- 172.21.15.164:0/3582957245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565003480630 con 0x5650036c2920 2024-05-08T04:38:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f087be00640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f086c01abe0 tx=0x7f086c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8231cdaf1edd67f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.807+0000 7f0870c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c0abb40 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f086c0abce0 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f087c05a160 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f086c0c9050 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 10 client.4635 fetch_fsmap learned FSMap version 8 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5650034503a0 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c05a160 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f086c0cc020 con 0x5650036c2920 2024-05-08T04:38:43.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 20 client.4635 populate_metadata read hostname 'smithi164' 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500367d4b0 0x56500367d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 4 client.4635 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 --> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565003463a50 con 0x56500367d4b0 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0883400640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500367d4b0 0x56500367d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0883400640 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500367d4b0 0x56500367d890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea00ca2b0bb23398 server_cookie=9165da6926373e09 in_seq=0 out_seq=0 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 10 client.4635 ms_handle_connect on v2:172.21.15.172:6833/3487014303 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 1 -- 172.21.15.164:0/3582957245 <== mds.0 v2:172.21.15.172:6833/3487014303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565003463a50 con 0x56500367d4b0 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 10 client.4635 handle_command_reply: tid=0 2024-05-08T04:38:43.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 client.4635 shutdown 2024-05-08T04:38:43.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 msgr2=0x5650036ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f086c01abe0 tx=0x7f086c0046d0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 shutdown_connections 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500367d4b0 0x56500367d890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c3240 0x56500367caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x5650036ca230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 --2- 172.21.15.164:0/3582957245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565003678f30 0x5650036c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 >> 172.21.15.164:0/3582957245 conn(0x56500367f2a0 msgr2=0x5650036779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f0870c00640 0 client.4635 ms_handle_reset on v2:172.21.15.172:6833/3487014303 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 shutdown_connections 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.811+0000 7f088dd68000 1 -- 172.21.15.164:0/3582957245 wait complete. 2024-05-08T04:38:43.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.815+0000 7f088dd68000 20 client.4635 trim_cache size 0 max 16384 2024-05-08T04:38:43.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.827+0000 7f088dd68000 1 Processor -- start 2024-05-08T04:38:43.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.827+0000 7f088dd68000 1 -- start start 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x56500371e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088414fd60 con 0x565003678f30 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088414f160 con 0x5650036c3240 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088414f760 con 0x5650036c2920 2024-05-08T04:38:43.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37698/0 (socket says 172.21.15.164:37698) 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 -- 172.21.15.164:0/2593290647 learned_addr learned my addr 172.21.15.164:0/2593290647 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883e00640 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f087be00640 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x56500371e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 -- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 msgr2=0x56500371e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x56500371e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 -- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 msgr2=0x56500371eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 -- 172.21.15.164:0/2593290647 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650036c3070 con 0x565003678f30 2024-05-08T04:38:43.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0883400640 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x565003682400 tx=0x7f0874000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b221025861c9b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0873400640 1 -- 172.21.15.164:0/2593290647 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874014920 con 0x565003678f30 2024-05-08T04:38:43.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0873400640 1 -- 172.21.15.164:0/2593290647 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874014ac0 con 0x565003678f30 2024-05-08T04:38:43.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f0873400640 1 -- 172.21.15.164:0/2593290647 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874014c70 con 0x565003678f30 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- 172.21.15.164:0/2593290647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 msgr2=0x56500371e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x565003682400 tx=0x7f0874000ee0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- 172.21.15.164:0/2593290647 shutdown_connections 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650036c2920 0x56500371e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 --2- 172.21.15.164:0/2593290647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565003678f30 0x56500371e040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- 172.21.15.164:0/2593290647 >> 172.21.15.164:0/2593290647 conn(0x56500367f2a0 msgr2=0x56500367c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- 172.21.15.164:0/2593290647 shutdown_connections 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- 172.21.15.164:0/2593290647 wait complete. 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 -- start start 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.831+0000 7f088dd68000 1 Processor -- start 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x565003678f30 0x5650036cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650034cc3d0 con 0x5650036c2920 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x565003583d50 con 0x5650036c3240 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x565003462980 con 0x565003678f30 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 10 client.0 _ll_register_callbacks cb 0x7f087c009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37700/0 (socket says 172.21.15.164:37700) 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 -- 172.21.15.164:0/3446679505 learned_addr learned my addr 172.21.15.164:0/3446679505 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0883400640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x565003678f30 0x5650036cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0883e00640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 -- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x565003678f30 msgr2=0x5650036cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x565003678f30 0x5650036cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 -- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 msgr2=0x56500371f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:43.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 -- 172.21.15.164:0/3446679505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565003549600 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f087be00640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f086c097f10 tx=0x7f086c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98906169fd0850 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c0abcc0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f086c0ceaa0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0874014ac0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f086c0ceca0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 10 client.4641 fetch_fsmap learned FSMap version 8 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56500345c9b0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874014ac0 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f086c0cc070 con 0x5650036c2920 2024-05-08T04:38:43.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 10 client.4641 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 20 client.4641 populate_metadata read hostname 'smithi164' 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500371fb80 0x56500371ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 4 client.4641 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 --> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650034d52e0 con 0x56500371fb80 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0883400640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500371fb80 0x56500371ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0883400640 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500371fb80 0x56500371ff60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aed75496baa39ac server_cookie=af8393bb10a42647 in_seq=0 out_seq=0 2024-05-08T04:38:43.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 10 client.4641 ms_handle_connect on v2:172.21.15.172:6833/3487014303 2024-05-08T04:38:43.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 1 -- 172.21.15.164:0/3446679505 <== mds.0 v2:172.21.15.172:6833/3487014303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5650034d52e0 con 0x56500371fb80 2024-05-08T04:38:43.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 10 client.4641 handle_command_reply: tid=0 2024-05-08T04:38:43.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 client.4641 shutdown 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 msgr2=0x56500371f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f086c097f10 tx=0x7f086c01ad70 comp rx=0 tx=0).stop 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 shutdown_connections 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:6833/3487014303,v1:172.21.15.172:6835/3487014303] conn(0x56500371fb80 0x56500371ff60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650036c3240 0x56500371f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650036c2920 0x56500371f100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 --2- 172.21.15.164:0/3446679505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x565003678f30 0x5650036cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f0870c00640 0 client.4641 ms_handle_reset on v2:172.21.15.172:6833/3487014303 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 >> 172.21.15.164:0/3446679505 conn(0x56500367f2a0 msgr2=0x5650036ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 shutdown_connections 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 1 -- 172.21.15.164:0/3446679505 wait complete. 2024-05-08T04:38:43.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.835+0000 7f088dd68000 20 client.4641 trim_cache size 0 max 16384 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4309, 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 14.106601017999999, 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 8, 2024-05-08T04:38:43.845 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 20, 2024-05-08T04:38:43.846 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 20, 2024-05-08T04:38:43.846 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 20.869706163 2024-05-08T04:38:43.846 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 msgr2=0x7f0860069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 0x7f0860069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0874001810 tx=0x7f0874009040 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 msgr2=0x7f08841508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f087c00efc0 tx=0x7f087c00c490 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 shutdown_connections 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0860066f60 0x7f0860069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f088415ba70 0x7f08840afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f088415be50 0x7f08841508c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 --2- 172.21.15.164:0/2266664959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08840cfda0 0x7f0884149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:43.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 >> 172.21.15.164:0/2266664959 conn(0x7f088406ac90 msgr2=0x7f08841554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:43.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 shutdown_connections 2024-05-08T04:38:43.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:43.839+0000 7f088a800640 1 -- 172.21.15.164:0/2266664959 wait complete. 2024-05-08T04:38:43.859 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T04:38:43.887 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T04:38:43.887 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T04:38:43.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:43.887 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:38:43.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:43.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-05-08T04:38:43.961 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-05-08T04:38:43.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:38:43.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:43.963 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:38:44.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-05-08T04:38:44.077 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-05-08T04:38:44.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:38:44.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.079 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:38:44.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.090 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-05-08T04:38:44.158 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-05-08T04:38:44.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:38:44.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.161 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:38:44.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-05-08T04:38:44.277 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-05-08T04:38:44.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:38:44.280 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-08T04:38:44.280 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T04:38:44.280 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:38:44.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:38:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 Processor -- start 2024-05-08T04:38:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 -- start start 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa88815bc50 0x7fa8880b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880ccfe0 0x7fa8880d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 0x7fa8880d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880b42a0 con 0x7fa88815bc50 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880cd4e0 con 0x7fa8880b4d50 2024-05-08T04:38:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880cd7c0 con 0x7fa8880ccfe0 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa88815bc50 0x7fa8880b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa88815bc50 0x7fa8880b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37710/0 (socket says 172.21.15.164:37710) 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 -- 172.21.15.164:0/3017776270 learned_addr learned my addr 172.21.15.164:0/3017776270 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887400640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880ccfe0 0x7fa8880d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88d000640 1 --2- 172.21.15.164:0/3017776270 >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 0x7fa8880d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 -- 172.21.15.164:0/3017776270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880ccfe0 msgr2=0x7fa8880d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:44.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880ccfe0 0x7fa8880d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 -- 172.21.15.164:0/3017776270 >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 msgr2=0x7fa8880d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:44.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 --2- 172.21.15.164:0/3017776270 >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 0x7fa8880d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa887e00640 1 -- 172.21.15.164:0/3017776270 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880cf590 con 0x7fa88815bc50 2024-05-08T04:38:44.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.414+0000 7fa88d000640 1 --2- 172.21.15.164:0/3017776270 >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 0x7fa8880d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:44.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa887e00640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa8880b4810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8880cbfa0 tx=0x7fa87801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d0f95886041f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:44.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa886a00640 1 -- 172.21.15.164:0/3017776270 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780a57b0 con 0x7fa88815bc50 2024-05-08T04:38:44.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa886a00640 1 -- 172.21.15.164:0/3017776270 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8780a5950 con 0x7fa88815bc50 2024-05-08T04:38:44.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa886a00640 1 -- 172.21.15.164:0/3017776270 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780a5d20 con 0x7fa88815bc50 2024-05-08T04:38:44.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 -- 172.21.15.164:0/3017776270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 msgr2=0x7fa8880b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:44.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa8880b4810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8880cbfa0 tx=0x7fa87801c930 comp rx=0 tx=0).stop 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 -- 172.21.15.164:0/3017776270 shutdown_connections 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 --2- 172.21.15.164:0/3017776270 >> v2:172.21.15.172:3300/0 conn(0x7fa8880b4d50 0x7fa8880d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880ccfe0 0x7fa8880d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 --2- 172.21.15.164:0/3017776270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa8880b4810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 -- 172.21.15.164:0/3017776270 >> 172.21.15.164:0/3017776270 conn(0x7fa8880b34f0 msgr2=0x7fa8880b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 -- 172.21.15.164:0/3017776270 shutdown_connections 2024-05-08T04:38:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.418+0000 7fa88f000640 1 -- 172.21.15.164:0/3017776270 wait complete. 2024-05-08T04:38:44.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 Processor -- start 2024-05-08T04:38:44.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- start start 2024-05-08T04:38:44.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880b4d50 0x7fa88806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880ccfe0 0x7fa88815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880cea90 con 0x7fa88815bc50 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880ce910 con 0x7fa8880ccfe0 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880cec10 con 0x7fa8880b4d50 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37714/0 (socket says 172.21.15.164:37714) 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 -- 172.21.15.164:0/2086548128 learned_addr learned my addr 172.21.15.164:0/2086548128 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa887400640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880ccfe0 0x7fa88815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa887e00640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880b4d50 0x7fa88806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 -- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880b4d50 msgr2=0x7fa88806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880b4d50 0x7fa88806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 -- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880ccfe0 msgr2=0x7fa88815c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880ccfe0 0x7fa88815c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 -- 172.21.15.164:0/2086548128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88815c9b0 con 0x7fa88815bc50 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88d000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e610 tx=0x7fa88000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:44.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000f040 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880004230 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88815cc90 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880003620 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88815d250 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.422+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa888069d60 con 0x7fa88815bc50 2024-05-08T04:38:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.426+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8800037c0 con 0x7fa88815bc50 2024-05-08T04:38:44.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.426+0000 7fa87fe00640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:44.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.426+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa880084bd0 con 0x7fa88815bc50 2024-05-08T04:38:44.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.426+0000 7fa887e00640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:44.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.426+0000 7fa887e00640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 0x7fa868069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8780ae000 tx=0x7fa8780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:44.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.430+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa880017020 con 0x7fa88815bc50 2024-05-08T04:38:44.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8880b7940 con 0x7fa88815bc50 2024-05-08T04:38:44.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa87fe00640 1 -- 172.21.15.164:0/2086548128 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa880055590 con 0x7fa88815bc50 2024-05-08T04:38:44.621 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 msgr2=0x7fa868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 0x7fa868069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8780ae000 tx=0x7fa8780a7070 comp rx=0 tx=0).stop 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 msgr2=0x7fa888158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e610 tx=0x7fa88000ead0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 shutdown_connections 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa88815bc50 0x7fa888158160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880ccfe0 0x7fa88815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 --2- 172.21.15.164:0/2086548128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8880b4d50 0x7fa88806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 >> 172.21.15.164:0/2086548128 conn(0x7fa8880b34f0 msgr2=0x7fa8880c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 shutdown_connections 2024-05-08T04:38:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:44.618+0000 7fa88f000640 1 -- 172.21.15.164:0/2086548128 wait complete. 2024-05-08T04:38:44.642 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:38:44.643 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:38:44.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.643 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:38:44.652 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:38:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42342sec preferred_lft 42342sec 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:38:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:44.654 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T04:38:44.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add name ceph-brx type bridge 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr flush dev ceph-brx 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set ceph-brx up 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T04:38:44.655 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:44.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.767 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:38:44.786 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:38:44.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.788 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:44.868 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:38:44.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:38:44.869 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:44.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:44.963 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:38:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:45.012 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-05-08T04:38:45.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:45.064 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:45.064 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:45.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:38:45.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T04:38:45.064 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:45.149 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T04:38:45.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T04:38:45.150 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:45.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:45.306 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:45.307 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:45.307 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:38:45.307 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T04:38:45.307 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:45.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:45.346 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:38:45.399 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:38:45.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:45.400 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:38:45.451 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:38:45.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:45.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:38:45.538 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:38:45.538 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:38:45.539 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:46.155 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:38:46.155 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:38:46.155 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:46.155 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:46.155 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:46.168 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:38:46.169 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:38:46.173 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:38:46.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:38:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.194 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:46.270 INFO:teuthology.orchestra.run.smithi164.stdout:client4511 2024-05-08T04:38:46.270 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:46.270 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:38:46.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.321 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:46.383 INFO:teuthology.orchestra.run.smithi164.stdout:client4511 2024-05-08T04:38:46.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.384 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:46.453 INFO:teuthology.orchestra.run.smithi164.stdout:client4511 2024-05-08T04:38:46.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.453 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6150cf8f-12a4-433b-b39d-bc24e302fd2a.client4511/status') 2024-05-08T04:38:46.521 INFO:teuthology.orchestra.run.smithi164.stdout:instance: client.4511 (3)172.21.15.164:0/3181259819 2024-05-08T04:38:46.522 INFO:teuthology.orchestra.run.smithi164.stdout:blocklisted: false 2024-05-08T04:38:46.522 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records in 2024-05-08T04:38:46.522 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records out 2024-05-08T04:38:46.522 INFO:teuthology.orchestra.run.smithi164.stderr:71 bytes copied, 6.0965e-05 s, 1.2 MB/s 2024-05-08T04:38:46.524 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-08T04:38:46.524 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T04:38:46.524 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T04:38:46.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:38:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 Processor -- start 2024-05-08T04:38:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 -- start start 2024-05-08T04:38:46.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f183c15dea0 0x7f183c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183c0cbbf0 0x7f183c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c0c9ed0 con 0x7f183c15dac0 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c0cc130 con 0x7f183c15dea0 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1842200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c0cc410 con 0x7f183c0cbbf0 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f183be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f183c15dea0 0x7f183c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f183be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f183c15dea0 0x7f183c0cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37728/0 (socket says 172.21.15.164:37728) 2024-05-08T04:38:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f183be00640 1 -- 172.21.15.164:0/1344007391 learned_addr learned my addr 172.21.15.164:0/1344007391 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:46.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.698+0000 7f1841800640 1 --2- 172.21.15.164:0/1344007391 >> v2:172.21.15.172:3300/0 conn(0x7f183c0cbbf0 0x7f183c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183be00640 1 -- 172.21.15.164:0/1344007391 >> v2:172.21.15.172:3300/0 conn(0x7f183c0cbbf0 msgr2=0x7f183c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:46.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183be00640 1 --2- 172.21.15.164:0/1344007391 >> v2:172.21.15.172:3300/0 conn(0x7f183c0cbbf0 0x7f183c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183be00640 1 -- 172.21.15.164:0/1344007391 >> v1:172.21.15.172:6789/0 conn(0x7f183c15dac0 legacy=0x7f183c0c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:46.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183be00640 1 -- 172.21.15.164:0/1344007391 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c0cc6f0 con 0x7f183c15dea0 2024-05-08T04:38:46.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183be00640 1 --2- 172.21.15.164:0/1344007391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dea0 0x7f183c0cb6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f183c0c65c0 tx=0x7f182801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a92532a8edc2d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:46.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183b400640 1 -- 172.21.15.164:0/1344007391 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280b0070 con 0x7f183c15dea0 2024-05-08T04:38:46.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183b400640 1 -- 172.21.15.164:0/1344007391 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18280ab600 con 0x7f183c15dea0 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f183b400640 1 -- 172.21.15.164:0/1344007391 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280ab9d0 con 0x7f183c15dea0 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f1842200640 1 -- 172.21.15.164:0/1344007391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dea0 msgr2=0x7f183c0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.702+0000 7f1842200640 1 --2- 172.21.15.164:0/1344007391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dea0 0x7f183c0cb6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f183c0c65c0 tx=0x7f182801c4c0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- 172.21.15.164:0/1344007391 shutdown_connections 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 --2- 172.21.15.164:0/1344007391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dea0 0x7f183c0cb6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 --2- 172.21.15.164:0/1344007391 >> v2:172.21.15.172:3300/0 conn(0x7f183c0cbbf0 0x7f183c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- 172.21.15.164:0/1344007391 >> 172.21.15.164:0/1344007391 conn(0x7f183c06b670 msgr2=0x7f183c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:46.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- 172.21.15.164:0/1344007391 shutdown_connections 2024-05-08T04:38:46.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- 172.21.15.164:0/1344007391 wait complete. 2024-05-08T04:38:46.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 Processor -- start 2024-05-08T04:38:46.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- start start 2024-05-08T04:38:46.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183c0cbbf0 0x7f183c146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183c15dea0 0x7f183c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0cb230 con 0x7f183c15dac0 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0cb0b0 con 0x7f183c15dea0 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1842200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c16abc0 con 0x7f183c0cbbf0 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37734/0 (socket says 172.21.15.164:37734) 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 -- 172.21.15.164:0/2427916010 learned_addr learned my addr 172.21.15.164:0/2427916010 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1840e00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183c0cbbf0 0x7f183c146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f1841800640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183c15dea0 0x7f183c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 -- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183c0cbbf0 msgr2=0x7f183c146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183c0cbbf0 0x7f183c146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 -- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183c15dea0 msgr2=0x7f183c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:46.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183c15dea0 0x7f183c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 -- 172.21.15.164:0/2427916010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c155910 con 0x7f183c15dac0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.706+0000 7f183be00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f182801ab70 tx=0x7f182801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280b0040 con 0x7f183c15dac0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18280bc070 con 0x7f183c15dac0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280b6640 con 0x7f183c15dac0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c155bf0 con 0x7f183c15dac0 2024-05-08T04:38:46.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c1561b0 con 0x7f183c15dac0 2024-05-08T04:38:46.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c04ef30 con 0x7f183c15dac0 2024-05-08T04:38:46.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18280b67e0 con 0x7f183c15dac0 2024-05-08T04:38:46.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 0x7f181c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:46.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f182811f300 con 0x7f183c15dac0 2024-05-08T04:38:46.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1840e00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 0x7f181c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:46.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.710+0000 7f1840e00640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 0x7f181c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f183c1fa580 tx=0x7f1830009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:46.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.714+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18280160b0 con 0x7f183c15dac0 2024-05-08T04:38:46.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.902+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f183c068170 con 0x7f183c15dac0 2024-05-08T04:38:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1838c00640 1 -- 172.21.15.164:0/2427916010 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18280efce0 con 0x7f183c15dac0 2024-05-08T04:38:46.909 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:38:46.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 msgr2=0x7f181c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:46.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 0x7f181c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f183c1fa580 tx=0x7f1830009060 comp rx=0 tx=0).stop 2024-05-08T04:38:46.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 msgr2=0x7f183c150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:46.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f182801ab70 tx=0x7f182801ad00 comp rx=0 tx=0).stop 2024-05-08T04:38:46.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 shutdown_connections 2024-05-08T04:38:46.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f181c066f60 0x7f181c069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183c15dea0 0x7f183c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183c15dac0 0x7f183c150520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 --2- 172.21.15.164:0/2427916010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183c0cbbf0 0x7f183c146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:46.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 >> 172.21.15.164:0/2427916010 conn(0x7f183c06b670 msgr2=0x7f183c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:46.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 shutdown_connections 2024-05-08T04:38:46.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:46.906+0000 7f1842200640 1 -- 172.21.15.164:0/2427916010 wait complete. 2024-05-08T04:38:46.929 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T04:38:46.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:38:46.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:38:46.929 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T04:38:46.930 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T04:38:46.930 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T04:38:46.930 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T04:38:46.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:38:46.930 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T04:38:46.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.931 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42340sec preferred_lft 42340sec 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:38:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:38:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:38:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:46.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:46.944 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:38:47.000 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:38:47.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.003 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:47.059 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:38:47.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:38:47.060 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:47.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.152 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:38:47.206 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:38:47.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.208 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-05-08T04:38:47.271 INFO:teuthology.orchestra.run.smithi164.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T04:38:47.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.272 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:47.272 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:47.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:38:47.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T04:38:47.272 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:47.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.352 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T04:38:47.419 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:47.419 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:47.419 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:47.420 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.420 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:47.420 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.420 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:47.421 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 56:77:d2:9c:1f:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T04:38:47.421 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T04:38:47.421 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.421 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5477:d2ff:fe9c:1ffb/64 scope link 2024-05-08T04:38:47.421 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.423 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:47.484 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 56:77:d2:9c:1f:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T04:38:47.485 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T04:38:47.485 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.485 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5477:d2ff:fe9c:1ffb/64 scope link 2024-05-08T04:38:47.485 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:47.486 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T04:38:47.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.486 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:47.486 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:47.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T04:38:47.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T04:38:47.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T04:38:47.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T04:38:47.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T04:38:47.487 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:47.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.622 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:38:47.622 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:38:47.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.1 up 2024-05-08T04:38:47.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T04:38:47.622 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:38:47.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:47.659 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T04:38:47.712 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T04:38:47.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:47.712 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T04:38:47.767 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:38:47.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T04:38:47.859 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T04:38:47.859 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T04:38:47.859 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:47.874 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T04:38:47.875 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T04:38:47.875 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:47.875 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:47.875 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:38:47.886 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:38:47.891 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:38:47.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T04:38:47.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:47.911 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:47.962 INFO:teuthology.orchestra.run.smithi164.stdout:client4654 2024-05-08T04:38:47.962 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:38:47.962 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:38:48.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.015 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:48.066 INFO:teuthology.orchestra.run.smithi164.stdout:client4654 2024-05-08T04:38:48.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.066 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:48.118 INFO:teuthology.orchestra.run.smithi164.stdout:client4654 2024-05-08T04:38:48.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.119 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6150cf8f-12a4-433b-b39d-bc24e302fd2a.client4654/status') 2024-05-08T04:38:48.178 INFO:teuthology.orchestra.run.smithi164.stdout:instance: client.4654 (3)172.21.15.164:0/1027483092 2024-05-08T04:38:48.178 INFO:teuthology.orchestra.run.smithi164.stdout:blocklisted: false 2024-05-08T04:38:48.179 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records in 2024-05-08T04:38:48.179 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records out 2024-05-08T04:38:48.179 INFO:teuthology.orchestra.run.smithi164.stderr:71 bytes copied, 5.3265e-05 s, 1.3 MB/s 2024-05-08T04:38:48.180 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-08T04:38:48.180 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T04:38:48.180 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:38:48.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:38:48.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 Processor -- start 2024-05-08T04:38:48.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 -- start start 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ac815ba70 0x7f1ac8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ac80cfda0 0x7f1ac80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80cd9f0 con 0x7f1ac815be50 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac815fe70 con 0x7f1ac80cfda0 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ad0200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80cdcd0 con 0x7f1ac815ba70 2024-05-08T04:38:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ac80cfda0 0x7f1ac80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ac80cfda0 0x7f1ac80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37742/0 (socket says 172.21.15.164:37742) 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1acce00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1ac815be50 0x7f1ac80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:44374/0 (socket says 172.21.15.164:44374) 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 -- 172.21.15.164:0/1264614170 learned_addr learned my addr 172.21.15.164:0/1264614170 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1acd800640 1 --2- 172.21.15.164:0/1264614170 >> v2:172.21.15.172:3300/0 conn(0x7f1ac815ba70 0x7f1ac8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 -- 172.21.15.164:0/1264614170 >> v2:172.21.15.172:3300/0 conn(0x7f1ac815ba70 msgr2=0x7f1ac8160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 --2- 172.21.15.164:0/1264614170 >> v2:172.21.15.172:3300/0 conn(0x7f1ac815ba70 0x7f1ac8160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 -- 172.21.15.164:0/1264614170 >> v1:172.21.15.172:6789/0 conn(0x7f1ac815be50 legacy=0x7f1ac80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ace200640 1 -- 172.21.15.164:0/1264614170 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac815f820 con 0x7f1ac80cfda0 2024-05-08T04:38:48.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.354+0000 7f1ac7e00640 1 -- 172.21.15.164:0/1264614170 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103736856 0 0) 0x7f1ac80cd9f0 con 0x7f1ac815be50 2024-05-08T04:38:48.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ace200640 1 --2- 172.21.15.164:0/1264614170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac80cfda0 0x7f1ac80b31a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806bc70 tx=0x7f1ac001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aba91a6f2785568 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:48.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ac7e00640 1 -- 172.21.15.164:0/1264614170 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a5790 con 0x7f1ac80cfda0 2024-05-08T04:38:48.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ac7e00640 1 -- 172.21.15.164:0/1264614170 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00a5930 con 0x7f1ac80cfda0 2024-05-08T04:38:48.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ac7e00640 1 -- 172.21.15.164:0/1264614170 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a5d00 con 0x7f1ac80cfda0 2024-05-08T04:38:48.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 -- 172.21.15.164:0/1264614170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac80cfda0 msgr2=0x7f1ac80b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 --2- 172.21.15.164:0/1264614170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac80cfda0 0x7f1ac80b31a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806bc70 tx=0x7f1ac001c9c0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 -- 172.21.15.164:0/1264614170 shutdown_connections 2024-05-08T04:38:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 --2- 172.21.15.164:0/1264614170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac80cfda0 0x7f1ac80b31a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 --2- 172.21.15.164:0/1264614170 >> v2:172.21.15.172:3300/0 conn(0x7f1ac815ba70 0x7f1ac8160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 -- 172.21.15.164:0/1264614170 >> 172.21.15.164:0/1264614170 conn(0x7f1ac806ac90 msgr2=0x7f1ac806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 -- 172.21.15.164:0/1264614170 shutdown_connections 2024-05-08T04:38:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.358+0000 7f1ad0200640 1 -- 172.21.15.164:0/1264614170 wait complete. 2024-05-08T04:38:48.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 Processor -- start 2024-05-08T04:38:48.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- start start 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 0x7f1ac815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 0x7f1ac815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8168d40 con 0x7f1ac815be50 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8168bc0 con 0x7f1ac815ba70 2024-05-08T04:38:48.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8168ec0 con 0x7f1ac80cfda0 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37752/0 (socket says 172.21.15.164:37752) 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 -- 172.21.15.164:0/4003003750 learned_addr learned my addr 172.21.15.164:0/4003003750 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1acd800640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 0x7f1ac815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1acce00640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 0x7f1ac815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 -- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 msgr2=0x7f1ac815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 0x7f1ac815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 -- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 msgr2=0x7f1ac815c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 0x7f1ac815c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 -- 172.21.15.164:0/4003003750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac815df30 con 0x7f1ac815be50 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1acce00640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 0x7f1ac815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1acd800640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 0x7f1ac815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ace200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009e20 tx=0x7f1ac0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:48.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b8470 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00b8630 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b8a20 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac815e210 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.362+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac815e7d0 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8069d60 con 0x7f1ac815be50 2024-05-08T04:38:48.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac00c0020 con 0x7f1ac815be50 2024-05-08T04:38:48.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1ac5600640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:48.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ac011fce0 con 0x7f1ac815be50 2024-05-08T04:38:48.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1acd800640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:48.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.366+0000 7f1acd800640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1abc007a70 tx=0x7f1abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:48.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.370+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ac00165e0 con 0x7f1ac815be50 2024-05-08T04:38:48.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.558+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ac806e210 con 0x7f1ac815be50 2024-05-08T04:38:48.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.558+0000 7f1ac5600640 1 -- 172.21.15.164:0/4003003750 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ac00f0ce0 con 0x7f1ac815be50 2024-05-08T04:38:48.563 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 msgr2=0x7f1aa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1abc007a70 tx=0x7f1abc008040 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 msgr2=0x7f1ac814f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009e20 tx=0x7f1ac0019760 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 shutdown_connections 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ac815be50 0x7f1ac814f850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815ba70 0x7f1ac815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 --2- 172.21.15.164:0/4003003750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac80cfda0 0x7f1ac815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:48.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 >> 172.21.15.164:0/4003003750 conn(0x7f1ac806ac90 msgr2=0x7f1ac806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:48.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 shutdown_connections 2024-05-08T04:38:48.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:48.562+0000 7f1ad0200640 1 -- 172.21.15.164:0/4003003750 wait complete. 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:38:48.584 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:38:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.585 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:38:48.594 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:48.594 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:48.594 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:38:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:38:48.596 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:48.597 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T04:38:48.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.597 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:48.597 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:48.597 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-05-08T04:38:48.597 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-05-08T04:38:48.598 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-05-08T04:38:48.598 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T04:38:48.598 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:48.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.711 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:38:48.729 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:38:48.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.731 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:48.797 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:48.798 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:48.798 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:38:48.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:38:48.798 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:48.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.892 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:38:48.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.901 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-05-08T04:38:48.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:48.951 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:48.951 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:48.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:38:48.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T04:38:48.951 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:49.033 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T04:38:49.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T04:38:49.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T04:38:49.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T04:38:49.034 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:49.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:49.195 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:49.195 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:49.195 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:38:49.195 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T04:38:49.195 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:49.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:49.234 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:38:49.286 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:38:49.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:49.286 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:38:49.337 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:38:49.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:49.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:38:49.429 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:38:49.429 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:38:49.429 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:50.022 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:38:50.022 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:38:50.022 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:50.022 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:50.022 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:50.034 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:38:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:38:50.038 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:38:50.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.039 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:38:50.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.058 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:50.149 INFO:teuthology.orchestra.run.smithi172.stdout:client4520 2024-05-08T04:38:50.149 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:50.149 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:38:50.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.202 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:50.256 INFO:teuthology.orchestra.run.smithi172.stdout:client4520 2024-05-08T04:38:50.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.256 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:50.308 INFO:teuthology.orchestra.run.smithi172.stdout:client4520 2024-05-08T04:38:50.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.308 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6150cf8f-12a4-433b-b39d-bc24e302fd2a.client4520/status') 2024-05-08T04:38:50.368 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4520 (3)192.168.0.1:0/2067221894 2024-05-08T04:38:50.368 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-05-08T04:38:50.369 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-05-08T04:38:50.369 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-05-08T04:38:50.369 INFO:teuthology.orchestra.run.smithi172.stderr:69 bytes copied, 4.964e-05 s, 1.4 MB/s 2024-05-08T04:38:50.371 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-08T04:38:50.371 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T04:38:50.371 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T04:38:50.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:38:50.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 Processor -- start 2024-05-08T04:38:50.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 -- start start 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ce415dd70 0x7f4ce41619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ce406c7b0 0x7f4ce4161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce406cc70 con 0x7f4ce4162420 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce406cf50 con 0x7f4ce415dd70 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce9a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce40b1570 con 0x7f4ce406c7b0 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ce415dd70 0x7f4ce41619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4ce4162420 0x7f4ce40cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33026/0 (socket says 172.21.15.164:33026) 2024-05-08T04:38:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce3400640 1 -- 172.21.15.164:0/1175712704 learned_addr learned my addr 172.21.15.164:0/1175712704 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4cdbe00640 1 --2- 172.21.15.164:0/1175712704 >> v2:172.21.15.172:3300/0 conn(0x7f4ce406c7b0 0x7f4ce4161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2000640 1 -- 172.21.15.164:0/1175712704 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688819845 0 0) 0x7f4ce406cc70 con 0x7f4ce4162420 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2000640 1 -- 172.21.15.164:0/1175712704 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc8003530 con 0x7f4ce4162420 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2a00640 1 -- 172.21.15.164:0/1175712704 >> v2:172.21.15.172:3300/0 conn(0x7f4ce406c7b0 msgr2=0x7f4ce4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2a00640 1 --2- 172.21.15.164:0/1175712704 >> v2:172.21.15.172:3300/0 conn(0x7f4ce406c7b0 0x7f4ce4161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2a00640 1 -- 172.21.15.164:0/1175712704 >> v1:172.21.15.164:6789/0 conn(0x7f4ce4162420 legacy=0x7f4ce40cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.506+0000 7f4ce2a00640 1 -- 172.21.15.164:0/1175712704 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40b1850 con 0x7f4ce415dd70 2024-05-08T04:38:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce2a00640 1 --2- 172.21.15.164:0/1175712704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce415dd70 0x7f4ce41619a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4070fa0 tx=0x7f4cd401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60aa610f20ec66bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce2000640 1 -- 172.21.15.164:0/1175712704 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b5070 con 0x7f4ce415dd70 2024-05-08T04:38:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce2000640 1 -- 172.21.15.164:0/1175712704 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd40a8970 con 0x7f4ce415dd70 2024-05-08T04:38:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce2000640 1 -- 172.21.15.164:0/1175712704 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40a8d40 con 0x7f4ce415dd70 2024-05-08T04:38:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 -- 172.21.15.164:0/1175712704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce415dd70 msgr2=0x7f4ce41619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/1175712704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce415dd70 0x7f4ce41619a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4070fa0 tx=0x7f4cd401ec70 comp rx=0 tx=0).stop 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 -- 172.21.15.164:0/1175712704 shutdown_connections 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/1175712704 >> v2:172.21.15.172:3300/0 conn(0x7f4ce406c7b0 0x7f4ce4161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/1175712704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce415dd70 0x7f4ce41619a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 -- 172.21.15.164:0/1175712704 >> 172.21.15.164:0/1175712704 conn(0x7f4ce40b34f0 msgr2=0x7f4ce40b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 -- 172.21.15.164:0/1175712704 shutdown_connections 2024-05-08T04:38:50.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.510+0000 7f4ce9a00640 1 -- 172.21.15.164:0/1175712704 wait complete. 2024-05-08T04:38:50.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 Processor -- start 2024-05-08T04:38:50.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- start start 2024-05-08T04:38:50.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 0x7f4ce40c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 0x7f4ce415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce415e150 con 0x7f4ce406c7b0 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40b4230 con 0x7f4ce4162420 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce415e2d0 con 0x7f4ce415dd70 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49768/0 (socket says 172.21.15.164:49768) 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 -- 172.21.15.164:0/2631690702 learned_addr learned my addr 172.21.15.164:0/2631690702 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce3400640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 0x7f4ce415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4cdbe00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 0x7f4ce40c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 -- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 msgr2=0x7f4ce40c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 0x7f4ce40c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 -- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 msgr2=0x7f4ce415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 0x7f4ce415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 -- 172.21.15.164:0/2631690702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40b4b80 con 0x7f4ce406c7b0 2024-05-08T04:38:50.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce3400640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 0x7f4ce415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4cdbe00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 0x7f4ce40c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce2a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009880 tx=0x7f4cd401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40bf070 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd40b0700 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b5040 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce40b4e60 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.514+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce40b5420 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce4069d60 con 0x7f4ce406c7b0 2024-05-08T04:38:50.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cd40b0ad0 con 0x7f4ce406c7b0 2024-05-08T04:38:50.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4cdaa00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 0x7f4cc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:50.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cd4121790 con 0x7f4ce406c7b0 2024-05-08T04:38:50.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4cdbe00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 0x7f4cc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:50.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.518+0000 7f4cdbe00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 0x7f4cc0069450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40c6990 tx=0x7f4ccc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:50.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.522+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cd40187d0 con 0x7f4ce406c7b0 2024-05-08T04:38:50.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ce40b4010 con 0x7f4ce406c7b0 2024-05-08T04:38:50.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4cdaa00640 1 -- 172.21.15.164:0/2631690702 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4cd40f2790 con 0x7f4ce406c7b0 2024-05-08T04:38:50.718 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 msgr2=0x7f4cc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 0x7f4cc0069450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40c6990 tx=0x7f4ccc008040 comp rx=0 tx=0).stop 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 msgr2=0x7f4ce41509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009880 tx=0x7f4cd401ba10 comp rx=0 tx=0).stop 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 shutdown_connections 2024-05-08T04:38:50.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc0066f90 0x7f4cc0069450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce415dd70 0x7f4ce40c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce4162420 0x7f4ce415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 --2- 172.21.15.164:0/2631690702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce406c7b0 0x7f4ce41509d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.714+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 >> 172.21.15.164:0/2631690702 conn(0x7f4ce40b34f0 msgr2=0x7f4ce406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.718+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 shutdown_connections 2024-05-08T04:38:50.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:50.718+0000 7f4ce9a00640 1 -- 172.21.15.164:0/2631690702 wait complete. 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T04:38:50.739 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:38:50.740 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T04:38:50.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.740 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:38:50.749 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:50.749 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:38:50.750 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42340sec preferred_lft 42340sec 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.751 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:38:50.752 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:38:50.752 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-05-08T04:38:50.752 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:50.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.752 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:38:50.807 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:38:50.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.810 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:38:50.861 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:38:50.861 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:38:50.861 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:38:50.861 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:38:50.861 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:38:50.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.862 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:50.863 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:50.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:38:50.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:38:50.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:38:50.863 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:50.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.950 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:38:50.958 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:38:50.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:50.960 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-05-08T04:38:51.010 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T04:38:51.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.011 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:51.011 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:51.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:38:51.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T04:38:51.011 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:51.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.091 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:51.158 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 1e:a9:5d:21:ca:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T04:38:51.159 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T04:38:51.159 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.159 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::1ca9:5dff:fe21:ca68/64 scope link 2024-05-08T04:38:51.159 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T04:38:51.225 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 1e:a9:5d:21:ca:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::1ca9:5dff:fe21:ca68/64 scope link 2024-05-08T04:38:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:38:51.227 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T04:38:51.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T04:38:51.228 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:51.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.362 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:38:51.362 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:38:51.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2024-05-08T04:38:51.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T04:38:51.362 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:38:51.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:51.401 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T04:38:51.454 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T04:38:51.454 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:38:51.454 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T04:38:51.505 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:38:51.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.505 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T04:38:51.593 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T04:38:51.594 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T04:38:51.594 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:51.609 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T04:38:51.609 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T04:38:51.609 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:51.609 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:51.609 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:38:51.618 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:38:51.622 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:38:51.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T04:38:51.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.641 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:51.696 INFO:teuthology.orchestra.run.smithi172.stdout:client4526 2024-05-08T04:38:51.697 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:38:51.697 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:38:51.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.746 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:51.800 INFO:teuthology.orchestra.run.smithi172.stdout:client4526 2024-05-08T04:38:51.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.801 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:38:51.857 INFO:teuthology.orchestra.run.smithi172.stdout:client4526 2024-05-08T04:38:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:38:51.857 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6150cf8f-12a4-433b-b39d-bc24e302fd2a.client4526/status') 2024-05-08T04:38:51.916 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4526 (3)192.168.0.2:0/3551806643 2024-05-08T04:38:51.917 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-05-08T04:38:51.917 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-05-08T04:38:51.917 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-05-08T04:38:51.917 INFO:teuthology.orchestra.run.smithi172.stderr:69 bytes copied, 5.2963e-05 s, 1.3 MB/s 2024-05-08T04:38:51.919 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T04:38:51.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:52.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 Processor -- start 2024-05-08T04:38:52.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 -- start start 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff92c0ccfe0 0x7ff92c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c0d05b0 0x7ff92c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c0b42a0 con 0x7ff92c15bc50 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0cd4e0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff933e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0cd7c0 con 0x7ff92c0d05b0 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff930a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff92c0ccfe0 0x7ff92c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff930a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff92c0ccfe0 0x7ff92c0d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49774/0 (socket says 172.21.15.164:49774) 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.070+0000 7ff930a00640 1 -- 172.21.15.164:0/1117295782 learned_addr learned my addr 172.21.15.164:0/1117295782 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:52.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff931e00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c0d05b0 0x7ff92c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff930a00640 1 -- 172.21.15.164:0/1117295782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c0d05b0 msgr2=0x7ff92c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:52.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff930a00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c0d05b0 0x7ff92c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff930a00640 1 -- 172.21.15.164:0/1117295782 >> v1:172.21.15.172:6789/0 conn(0x7ff92c15bc50 legacy=0x7ff92c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff930a00640 1 -- 172.21.15.164:0/1117295782 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0cf590 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff930a00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c0d0070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff918009870 tx=0x7ff91801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30a2eb756dbda53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff92be00640 1 -- 172.21.15.164:0/1117295782 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180b1070 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff92be00640 1 -- 172.21.15.164:0/1117295782 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9180ac620 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.074+0000 7ff92be00640 1 -- 172.21.15.164:0/1117295782 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180ac9f0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- 172.21.15.164:0/1117295782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 msgr2=0x7ff92c0d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c0d0070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff918009870 tx=0x7ff91801c4d0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- 172.21.15.164:0/1117295782 shutdown_connections 2024-05-08T04:38:52.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c0d05b0 0x7ff92c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 --2- 172.21.15.164:0/1117295782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c0d0070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- 172.21.15.164:0/1117295782 >> 172.21.15.164:0/1117295782 conn(0x7ff92c0b34f0 msgr2=0x7ff92c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- 172.21.15.164:0/1117295782 shutdown_connections 2024-05-08T04:38:52.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- 172.21.15.164:0/1117295782 wait complete. 2024-05-08T04:38:52.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 Processor -- start 2024-05-08T04:38:52.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.078+0000 7ff933e00640 1 -- start start 2024-05-08T04:38:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0d05b0 0x7ff92c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c15bc50 0x7ff92c165e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0b50a0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0b4f20 con 0x7ff92c0d05b0 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c168ae0 con 0x7ff92c15bc50 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49788/0 (socket says 172.21.15.164:49788) 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 -- 172.21.15.164:0/2325455912 learned_addr learned my addr 172.21.15.164:0/2325455912 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c15bc50 0x7ff92c165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff930a00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0d05b0 0x7ff92c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 -- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c15bc50 msgr2=0x7ff92c165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c15bc50 0x7ff92c165e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 -- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0d05b0 msgr2=0x7ff92c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0d05b0 0x7ff92c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 -- 172.21.15.164:0/2325455912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c157fd0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff931400640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c420 tx=0x7ff92000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200052d0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920005470 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920007260 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c1582b0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c158870 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c069d60 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.082+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff920007400 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.086+0000 7ff929600640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.086+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff920084570 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.086+0000 7ff930a00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.086+0000 7ff930a00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 0x7ff90c069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9180ae000 tx=0x7ff918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.090+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff920055570 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.262+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff92c15b1b0 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.262+0000 7ff929600640 1 -- 172.21.15.164:0/2325455912 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7ff920082e20 con 0x7ff92c0ccfe0 2024-05-08T04:38:52.267 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:52.268 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 msgr2=0x7ff90c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 0x7ff90c069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9180ae000 tx=0x7ff918008040 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 msgr2=0x7ff92c152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c420 tx=0x7ff92000c8e0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 shutdown_connections 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff90c066f60 0x7ff90c069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c15bc50 0x7ff92c165e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0d05b0 0x7ff92c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 --2- 172.21.15.164:0/2325455912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c0ccfe0 0x7ff92c152580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 >> 172.21.15.164:0/2325455912 conn(0x7ff92c0b34f0 msgr2=0x7ff92c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 shutdown_connections 2024-05-08T04:38:52.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.266+0000 7ff933e00640 1 -- 172.21.15.164:0/2325455912 wait complete. 2024-05-08T04:38:52.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:52.290 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:52.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 Processor -- start 2024-05-08T04:38:52.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 -- start start 2024-05-08T04:38:52.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 0x7f40a00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00b2be0 con 0x7f40a006bda0 2024-05-08T04:38:52.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00b2eb0 con 0x7f40a006c910 2024-05-08T04:38:52.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f40a5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00d0c40 con 0x7f40a015e000 2024-05-08T04:38:52.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f4095400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f40a006bda0 0x7f40a006c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33040/0 (socket says 172.21.15.164:33040) 2024-05-08T04:38:52.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f4095400640 1 -- 172.21.15.164:0/1498876922 learned_addr learned my addr 172.21.15.164:0/1498876922 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:52.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409de00640 1 --2- 172.21.15.164:0/1498876922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 0x7f40a00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250176987 0 0) 0x7f40a00b2be0 con 0x7f40a006bda0 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4084003530 con 0x7f40a006bda0 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297624532 0 0) 0x7f40a00b2eb0 con 0x7f40a006c910 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a00b2be0 con 0x7f40a006c910 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022653123 0 0) 0x7f4084003530 con 0x7f40a006bda0 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409de00640 1 -- 172.21.15.164:0/1498876922 >> v1:172.21.15.172:6789/0 conn(0x7f40a006c910 legacy=0x7f40a00cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409de00640 1 -- 172.21.15.164:0/1498876922 >> v1:172.21.15.164:6789/0 conn(0x7f40a006bda0 legacy=0x7f40a006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.426+0000 7f409de00640 1 -- 172.21.15.164:0/1498876922 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00b3190 con 0x7f40a015e000 2024-05-08T04:38:52.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f409de00640 1 --2- 172.21.15.164:0/1498876922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 0x7f40a00b4010 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4090009dc0 tx=0x7f409001c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78809beb24f8586d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900b2070 con 0x7f40a015e000 2024-05-08T04:38:52.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40900a5630 con 0x7f40a015e000 2024-05-08T04:38:52.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f409d400640 1 -- 172.21.15.164:0/1498876922 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088001680 con 0x7f40a015e000 2024-05-08T04:38:52.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 -- 172.21.15.164:0/1498876922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 msgr2=0x7f40a00b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 --2- 172.21.15.164:0/1498876922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 0x7f40a00b4010 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4090009dc0 tx=0x7f409001c7f0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 -- 172.21.15.164:0/1498876922 shutdown_connections 2024-05-08T04:38:52.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 --2- 172.21.15.164:0/1498876922 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a015e000 0x7f40a00b4010 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 -- 172.21.15.164:0/1498876922 >> 172.21.15.164:0/1498876922 conn(0x7f40a00b34f0 msgr2=0x7f40a00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 -- 172.21.15.164:0/1498876922 shutdown_connections 2024-05-08T04:38:52.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.430+0000 7f40a5000640 1 -- 172.21.15.164:0/1498876922 wait complete. 2024-05-08T04:38:52.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 Processor -- start 2024-05-08T04:38:52.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- start start 2024-05-08T04:38:52.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 0x7f40a0157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a015e000 0x7f40a015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cd110 con 0x7f40a015e000 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00ccf90 con 0x7f40a006c910 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cd290 con 0x7f40a006bda0 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f4095400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 0x7f40a0157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54646/0 (socket says 172.21.15.164:54646) 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 -- 172.21.15.164:0/2195708545 learned_addr learned my addr 172.21.15.164:0/2195708545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409e800640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a015e000 0x7f40a015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 -- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 msgr2=0x7f40a0157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 0x7f40a0157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 -- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a015e000 msgr2=0x7f40a015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a015e000 0x7f40a015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 -- 172.21.15.164:0/2195708545 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0068de0 con 0x7f40a006bda0 2024-05-08T04:38:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f4095400640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 0x7f40a0157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f409de00640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f409001aea0 tx=0x7f4090019590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900bd020 con 0x7f40a006bda0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40900b8670 con 0x7f40a006bda0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a015a080 con 0x7f40a006bda0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900b2040 con 0x7f40a006bda0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.434+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a015a640 con 0x7f40a006bda0 2024-05-08T04:38:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a009fb50 con 0x7f40a006bda0 2024-05-08T04:38:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40900c0020 con 0x7f40a006bda0 2024-05-08T04:38:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f4096a00640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 0x7f407c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4090120080 con 0x7f40a006bda0 2024-05-08T04:38:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f4095400640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 0x7f407c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.438+0000 7f4095400640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 0x7f407c069450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4088001ec0 tx=0x7f40880016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.442+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40900c7030 con 0x7f40a006bda0 2024-05-08T04:38:52.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.598+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40a01590f0 con 0x7f40a006bda0 2024-05-08T04:38:52.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.598+0000 7f4096a00640 1 -- 172.21.15.164:0/2195708545 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40900164a0 con 0x7f40a006bda0 2024-05-08T04:38:52.603 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 msgr2=0x7f407c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 0x7f407c069450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4088001ec0 tx=0x7f40880016a0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 msgr2=0x7f40a0152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f409001aea0 tx=0x7f4090019590 comp rx=0 tx=0).stop 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 shutdown_connections 2024-05-08T04:38:52.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f407c066f90 0x7f407c069450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a015e000 0x7f40a015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a006c910 0x7f40a0157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 --2- 172.21.15.164:0/2195708545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a006bda0 0x7f40a0152520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 >> 172.21.15.164:0/2195708545 conn(0x7f40a00b34f0 msgr2=0x7f40a0159610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 shutdown_connections 2024-05-08T04:38:52.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.602+0000 7f40a5000640 1 -- 172.21.15.164:0/2195708545 wait complete. 2024-05-08T04:38:52.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.758+0000 7f2ea0400640 1 Processor -- start 2024-05-08T04:38:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 -- start start 2024-05-08T04:38:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2e9815ba20 0x7f2e98160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e980aedf0 0x7f2e9806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980c6600 con 0x7f2e9806de50 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9815fbe0 con 0x7f2e9815ba20 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2ea0400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e980c6040 con 0x7f2e980aedf0 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9da00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2e9815ba20 0x7f2e98160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9d000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2e9806de50 0x7f2e980ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33050/0 (socket says 172.21.15.164:33050) 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9d000640 1 -- 172.21.15.164:0/2751258570 learned_addr learned my addr 172.21.15.164:0/2751258570 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9e400640 1 --2- 172.21.15.164:0/2751258570 >> v2:172.21.15.172:3300/0 conn(0x7f2e980aedf0 0x7f2e9806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9da00640 1 -- 172.21.15.164:0/2751258570 >> v2:172.21.15.172:3300/0 conn(0x7f2e980aedf0 msgr2=0x7f2e9806e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9da00640 1 --2- 172.21.15.164:0/2751258570 >> v2:172.21.15.172:3300/0 conn(0x7f2e980aedf0 0x7f2e9806e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9da00640 1 -- 172.21.15.164:0/2751258570 >> v1:172.21.15.164:6789/0 conn(0x7f2e9806de50 legacy=0x7f2e980ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e9da00640 1 -- 172.21.15.164:0/2751258570 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980c6320 con 0x7f2e9815ba20 2024-05-08T04:38:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.762+0000 7f2e97e00640 1 -- 172.21.15.164:0/2751258570 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352387512 0 0) 0x7f2e980c6600 con 0x7f2e9806de50 2024-05-08T04:38:52.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2e9da00640 1 --2- 172.21.15.164:0/2751258570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e9815ba20 0x7f2e98160070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e9806bd80 tx=0x7f2e8c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f5b7ba6c0e0029 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2e97e00640 1 -- 172.21.15.164:0/2751258570 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0ab5f0 con 0x7f2e9815ba20 2024-05-08T04:38:52.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2e97e00640 1 -- 172.21.15.164:0/2751258570 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0b0070 con 0x7f2e9815ba20 2024-05-08T04:38:52.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2e97e00640 1 -- 172.21.15.164:0/2751258570 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0ab9b0 con 0x7f2e9815ba20 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- 172.21.15.164:0/2751258570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e9815ba20 msgr2=0x7f2e98160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2751258570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e9815ba20 0x7f2e98160070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e9806bd80 tx=0x7f2e8c01c450 comp rx=0 tx=0).stop 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- 172.21.15.164:0/2751258570 shutdown_connections 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2751258570 >> v2:172.21.15.172:3300/0 conn(0x7f2e980aedf0 0x7f2e9806e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2751258570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e9815ba20 0x7f2e98160070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- 172.21.15.164:0/2751258570 >> 172.21.15.164:0/2751258570 conn(0x7f2e980cf770 msgr2=0x7f2e980cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- 172.21.15.164:0/2751258570 shutdown_connections 2024-05-08T04:38:52.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- 172.21.15.164:0/2751258570 wait complete. 2024-05-08T04:38:52.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 Processor -- start 2024-05-08T04:38:52.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.766+0000 7f2ea0400640 1 -- start start 2024-05-08T04:38:52.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9806de50 0x7f2e980cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e9815ba20 0x7f2e98150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980ce000 con 0x7f2e980aedf0 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980cde80 con 0x7f2e9806de50 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98168b70 con 0x7f2e9815ba20 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49820/0 (socket says 172.21.15.164:49820) 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 -- 172.21.15.164:0/2463493349 learned_addr learned my addr 172.21.15.164:0/2463493349 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9da00640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9806de50 0x7f2e980cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9e400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e9815ba20 0x7f2e98150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 -- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e9815ba20 msgr2=0x7f2e98150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e9815ba20 0x7f2e98150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 -- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9806de50 msgr2=0x7f2e980cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9806de50 0x7f2e980cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 -- 172.21.15.164:0/2463493349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980b04f0 con 0x7f2e980aedf0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e9d000640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e840011f0 tx=0x7f2e8400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84018070 con 0x7f2e980aedf0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e840074e0 con 0x7f2e980aedf0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e980b07d0 con 0x7f2e980aedf0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004a10 con 0x7f2e980aedf0 2024-05-08T04:38:52.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e980b0d90 con 0x7f2e980aedf0 2024-05-08T04:38:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e98072d30 con 0x7f2e980aedf0 2024-05-08T04:38:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.770+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e8401b070 con 0x7f2e980aedf0 2024-05-08T04:38:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.774+0000 7f2e95600640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.774+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e84084380 con 0x7f2e980aedf0 2024-05-08T04:38:52.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.774+0000 7f2e9da00640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:52.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.774+0000 7f2e9da00640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0af000 tx=0x7f2e8c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.778+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e84054ca0 con 0x7f2e980aedf0 2024-05-08T04:38:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.934+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e9804f010 con 0x7f2e980aedf0 2024-05-08T04:38:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.934+0000 7f2e95600640 1 -- 172.21.15.164:0/2463493349 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e84058d40 con 0x7f2e980aedf0 2024-05-08T04:38:52.940 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 msgr2=0x7f2e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0af000 tx=0x7f2e8c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 msgr2=0x7f2e980afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e840011f0 tx=0x7f2e8400c9b0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 shutdown_connections 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e9815ba20 0x7f2e98150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e980aedf0 0x7f2e980afd80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 --2- 172.21.15.164:0/2463493349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9806de50 0x7f2e980cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 >> 172.21.15.164:0/2463493349 conn(0x7f2e980cf770 msgr2=0x7f2e9815e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 shutdown_connections 2024-05-08T04:38:52.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:52.938+0000 7f2ea0400640 1 -- 172.21.15.164:0/2463493349 wait complete. 2024-05-08T04:38:52.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:53.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.094+0000 7ff2e9e00640 1 Processor -- start 2024-05-08T04:38:53.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.094+0000 7ff2e9e00640 1 -- start start 2024-05-08T04:38:53.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 0x7ff2e4071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e415c900 con 0x7ff2e40b1d40 2024-05-08T04:38:53.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e415cbe0 con 0x7ff2e40cec40 2024-05-08T04:38:53.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e415cec0 con 0x7ff2e40b1960 2024-05-08T04:38:53.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff2e40b1d40 0x7ff2e40ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33066/0 (socket says 172.21.15.164:33066) 2024-05-08T04:38:53.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e9400640 1 -- 172.21.15.164:0/2216269347 learned_addr learned my addr 172.21.15.164:0/2216269347 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:53.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e8a00640 1 --2- 172.21.15.164:0/2216269347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 0x7ff2e4071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284140580 0 0) 0x7ff2e415c900 con 0x7ff2e40b1d40 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003530 con 0x7ff2e40b1d40 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452785754 0 0) 0x7ff2e415cbe0 con 0x7ff2e40cec40 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e415c900 con 0x7ff2e40cec40 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e8a00640 1 -- 172.21.15.164:0/2216269347 >> v1:172.21.15.172:6789/0 conn(0x7ff2e40cec40 legacy=0x7ff2e415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e8a00640 1 -- 172.21.15.164:0/2216269347 >> v1:172.21.15.164:6789/0 conn(0x7ff2e40b1d40 legacy=0x7ff2e40ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.098+0000 7ff2e8a00640 1 -- 172.21.15.164:0/2216269347 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40712a0 con 0x7ff2e40b1960 2024-05-08T04:38:53.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e8a00640 1 --2- 172.21.15.164:0/2216269347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 0x7ff2e4071aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097f0 tx=0x7ff2d401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac91d5cb8291ad87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40b0070 con 0x7ff2e40b1960 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d40ab600 con 0x7ff2e40b1960 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e3e00640 1 -- 172.21.15.164:0/2216269347 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40ab9d0 con 0x7ff2e40b1960 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- 172.21.15.164:0/2216269347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 msgr2=0x7ff2e4071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/2216269347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 0x7ff2e4071aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097f0 tx=0x7ff2d401c3c0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- 172.21.15.164:0/2216269347 shutdown_connections 2024-05-08T04:38:53.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/2216269347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40b1960 0x7ff2e4071aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- 172.21.15.164:0/2216269347 >> 172.21.15.164:0/2216269347 conn(0x7ff2e406ac90 msgr2=0x7ff2e406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- 172.21.15.164:0/2216269347 shutdown_connections 2024-05-08T04:38:53.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- 172.21.15.164:0/2216269347 wait complete. 2024-05-08T04:38:53.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 Processor -- start 2024-05-08T04:38:53.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- start start 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 0x7ff2e415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40cec40 0x7ff2e4138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4169cc0 con 0x7ff2e40b1d40 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4169b40 con 0x7ff2e40b1960 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.102+0000 7ff2e9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e4169e40 con 0x7ff2e40cec40 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:33430/0 (socket says 172.21.15.164:33430) 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 -- 172.21.15.164:0/3745415345 learned_addr learned my addr 172.21.15.164:0/3745415345 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 0x7ff2e415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e9400640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40cec40 0x7ff2e4138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 -- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40cec40 msgr2=0x7ff2e4138ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40cec40 0x7ff2e4138ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 -- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 msgr2=0x7ff2e415a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 0x7ff2e415a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 -- 172.21.15.164:0/3745415345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e415ab60 con 0x7ff2e40b1960 2024-05-08T04:38:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2dbe00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 0x7ff2e415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e8a00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097f0 tx=0x7ff2d401ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40b0050 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d40bc070 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e414f320 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4006e20 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e414f8e0 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e404ef30 con 0x7ff2e40b1960 2024-05-08T04:38:53.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.106+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2d40b5830 con 0x7ff2e40b1960 2024-05-08T04:38:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.110+0000 7ff2e1600640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.110+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2d411f7b0 con 0x7ff2e40b1960 2024-05-08T04:38:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.110+0000 7ff2dbe00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.110+0000 7ff2dbe00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2e41fa900 tx=0x7ff2cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.110+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2d40160d0 con 0x7ff2e40b1960 2024-05-08T04:38:53.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.266+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2e4150c60 con 0x7ff2e40b1960 2024-05-08T04:38:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.266+0000 7ff2e1600640 1 -- 172.21.15.164:0/3745415345 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2d40f0250 con 0x7ff2e40b1960 2024-05-08T04:38:53.271 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 msgr2=0x7ff2c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2e41fa900 tx=0x7ff2cc009040 comp rx=0 tx=0).stop 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 msgr2=0x7ff2e414ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097f0 tx=0x7ff2d401ac00 comp rx=0 tx=0).stop 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 shutdown_connections 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e40cec40 0x7ff2e4138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2e40b1d40 0x7ff2e415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 --2- 172.21.15.164:0/3745415345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e40b1960 0x7ff2e414ec30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 >> 172.21.15.164:0/3745415345 conn(0x7ff2e406ac90 msgr2=0x7ff2e40ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 shutdown_connections 2024-05-08T04:38:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.270+0000 7ff2e9e00640 1 -- 172.21.15.164:0/3745415345 wait complete. 2024-05-08T04:38:53.293 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 Processor -- start 2024-05-08T04:38:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 -- start start 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb54415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb54415d430 0x7fb544155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54406e510 con 0x7fb54406aef0 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54415c0b0 con 0x7fb54415d430 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb54ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54415c370 con 0x7fb54406b2d0 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb54415d430 0x7fb544155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb54415d430 0x7fb544155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49838/0 (socket says 172.21.15.164:49838) 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 -- 172.21.15.164:0/1335090472 learned_addr learned my addr 172.21.15.164:0/1335090472 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb543400640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb54415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 -- 172.21.15.164:0/1335090472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 msgr2=0x7fb54415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb54415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 -- 172.21.15.164:0/1335090472 >> v1:172.21.15.164:6789/0 conn(0x7fb54406aef0 legacy=0x7fb54406de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.426+0000 7fb548c00640 1 -- 172.21.15.164:0/1335090472 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54415c650 con 0x7fb54415d430 2024-05-08T04:38:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb548c00640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544155ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009e50 tx=0x7fb53c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717601bf2f02fad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb542a00640 1 -- 172.21.15.164:0/1335090472 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0a58d0 con 0x7fb54415d430 2024-05-08T04:38:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb542a00640 1 -- 172.21.15.164:0/1335090472 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c0a5a70 con 0x7fb54415d430 2024-05-08T04:38:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb542a00640 1 -- 172.21.15.164:0/1335090472 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0a5e40 con 0x7fb54415d430 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 -- 172.21.15.164:0/1335090472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 msgr2=0x7fb544155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544155ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009e50 tx=0x7fb53c01ca00 comp rx=0 tx=0).stop 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 -- 172.21.15.164:0/1335090472 shutdown_connections 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544155ad0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 --2- 172.21.15.164:0/1335090472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb54415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 -- 172.21.15.164:0/1335090472 >> 172.21.15.164:0/1335090472 conn(0x7fb544066fa0 msgr2=0x7fb54406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 -- 172.21.15.164:0/1335090472 shutdown_connections 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.430+0000 7fb54ac00640 1 -- 172.21.15.164:0/1335090472 wait complete. 2024-05-08T04:38:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 Processor -- start 2024-05-08T04:38:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- start start 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 0x7fb5441607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb544144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544160190 con 0x7fb54415d430 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544156420 con 0x7fb54406aef0 2024-05-08T04:38:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb544160310 con 0x7fb54406b2d0 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49842/0 (socket says 172.21.15.164:49842) 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 -- 172.21.15.164:0/3171945839 learned_addr learned my addr 172.21.15.164:0/3171945839 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb543e00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 0x7fb5441607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb543400640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb544144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 -- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 msgr2=0x7fb544144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb544144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 -- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 msgr2=0x7fb5441607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 0x7fb5441607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 -- 172.21.15.164:0/3171945839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544164850 con 0x7fb54415d430 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb543e00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 0x7fb5441607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb548c00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0af040 tx=0x7fb53c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0a58d0 con 0x7fb54415d430 2024-05-08T04:38:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c0be070 con 0x7fb54415d430 2024-05-08T04:38:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb544145db0 con 0x7fb54415d430 2024-05-08T04:38:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c009510 con 0x7fb54415d430 2024-05-08T04:38:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.434+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb544146370 con 0x7fb54415d430 2024-05-08T04:38:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb544069d60 con 0x7fb54415d430 2024-05-08T04:38:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb53c0b86b0 con 0x7fb54415d430 2024-05-08T04:38:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb53be00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 0x7fb520069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb53c1203d0 con 0x7fb54415d430 2024-05-08T04:38:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb543e00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 0x7fb520069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.438+0000 7fb543e00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 0x7fb520069450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb534000f70 tx=0x7fb534009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.442+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb53c016620 con 0x7fb54415d430 2024-05-08T04:38:53.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.598+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5440cde60 con 0x7fb54415d430 2024-05-08T04:38:53.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.598+0000 7fb53be00640 1 -- 172.21.15.164:0/3171945839 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fb53c0f1460 con 0x7fb54415d430 2024-05-08T04:38:53.603 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:53.603 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 msgr2=0x7fb520069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 0x7fb520069450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb534000f70 tx=0x7fb534009040 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 msgr2=0x7fb544163db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0af040 tx=0x7fb53c0197a0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 shutdown_connections 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb520066f90 0x7fb520069450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54406b2d0 0x7fb544144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb54415d430 0x7fb544163db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 --2- 172.21.15.164:0/3171945839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54406aef0 0x7fb5441607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 >> 172.21.15.164:0/3171945839 conn(0x7fb544066fa0 msgr2=0x7fb54415e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 shutdown_connections 2024-05-08T04:38:53.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.602+0000 7fb54ac00640 1 -- 172.21.15.164:0/3171945839 wait complete. 2024-05-08T04:38:53.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:53.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 Processor -- start 2024-05-08T04:38:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 -- start start 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31640af920 0x7f3164071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415b020 con 0x7f31640716b0 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415b300 con 0x7f31640cc0f0 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640cf460 con 0x7f31640af920 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f315be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31640af920 0x7f3164071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f31640716b0 0x7f316415e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33084/0 (socket says 172.21.15.164:33084) 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3162a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f31640cc0f0 0x7f31640b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:35752/0 (socket says 172.21.15.164:35752) 2024-05-08T04:38:53.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3163400640 1 -- 172.21.15.164:0/3177677183 learned_addr learned my addr 172.21.15.164:0/3177677183 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:53.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f315be00640 1 -- 172.21.15.164:0/3177677183 >> v1:172.21.15.172:6789/0 conn(0x7f31640cc0f0 legacy=0x7f31640b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f315be00640 1 -- 172.21.15.164:0/3177677183 >> v1:172.21.15.164:6789/0 conn(0x7f31640716b0 legacy=0x7f316415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f315be00640 1 -- 172.21.15.164:0/3177677183 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316415b5e0 con 0x7f31640af920 2024-05-08T04:38:53.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.762+0000 7f3162000640 1 -- 172.21.15.164:0/3177677183 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856945834 0 0) 0x7f316415b300 con 0x7f31640cc0f0 2024-05-08T04:38:53.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f315be00640 1 --2- 172.21.15.164:0/3177677183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f3164071170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f314c0097c0 tx=0x7f314c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5544bd7ad2d79e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3162000640 1 -- 172.21.15.164:0/3177677183 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0b0070 con 0x7f31640af920 2024-05-08T04:38:53.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3162000640 1 -- 172.21.15.164:0/3177677183 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0ab600 con 0x7f31640af920 2024-05-08T04:38:53.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3162000640 1 -- 172.21.15.164:0/3177677183 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0ab9d0 con 0x7f31640af920 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 -- 172.21.15.164:0/3177677183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 msgr2=0x7f3164071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 --2- 172.21.15.164:0/3177677183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f3164071170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f314c0097c0 tx=0x7f314c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 -- 172.21.15.164:0/3177677183 shutdown_connections 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 --2- 172.21.15.164:0/3177677183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f3164071170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 -- 172.21.15.164:0/3177677183 >> 172.21.15.164:0/3177677183 conn(0x7f316406ac90 msgr2=0x7f316406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 -- 172.21.15.164:0/3177677183 shutdown_connections 2024-05-08T04:38:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.766+0000 7f3163e00640 1 -- 172.21.15.164:0/3177677183 wait complete. 2024-05-08T04:38:53.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 Processor -- start 2024-05-08T04:38:53.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- start start 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 0x7f31641541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 0x7f3164155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415bba0 con 0x7f31640af920 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415f150 con 0x7f31640cc0f0 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316415ac60 con 0x7f31640716b0 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49864/0 (socket says 172.21.15.164:49864) 2024-05-08T04:38:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 -- 172.21.15.164:0/3103593709 learned_addr learned my addr 172.21.15.164:0/3103593709 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163400640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 0x7f3164155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3162a00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 0x7f31641541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 -- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 msgr2=0x7f31641541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 0x7f31641541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 -- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 msgr2=0x7f3164155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 0x7f3164155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 -- 172.21.15.164:0/3103593709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164157d40 con 0x7f31640af920 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3162a00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 0x7f31641541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163400640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 0x7f3164155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315be00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f314c0190b0 tx=0x7f314c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0b0040 con 0x7f31640af920 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0ba070 con 0x7f31640af920 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0b4870 con 0x7f31640af920 2024-05-08T04:38:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164157fe0 con 0x7f31640af920 2024-05-08T04:38:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31641585a0 con 0x7f31640af920 2024-05-08T04:38:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.770+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316404ef30 con 0x7f31640af920 2024-05-08T04:38:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.774+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f314c0ab480 con 0x7f31640af920 2024-05-08T04:38:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.774+0000 7f315aa00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 0x7f3140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.774+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f314c11eee0 con 0x7f31640af920 2024-05-08T04:38:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.774+0000 7f3162a00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 0x7f3140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.774+0000 7f3162a00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 0x7f3140069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3154000f70 tx=0x7f3154009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.778+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f314c016090 con 0x7f31640af920 2024-05-08T04:38:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3164165880 con 0x7f31640af920 2024-05-08T04:38:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f315aa00640 1 -- 172.21.15.164:0/3103593709 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f314c0efe50 con 0x7f31640af920 2024-05-08T04:38:53.938 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 msgr2=0x7f3140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 0x7f3140069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3154000f70 tx=0x7f3154009040 comp rx=0 tx=0).stop 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 msgr2=0x7f31641f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f314c0190b0 tx=0x7f314c019240 comp rx=0 tx=0).stop 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 shutdown_connections 2024-05-08T04:38:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3140066f60 0x7f3140069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31640cc0f0 0x7f3164155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31640af920 0x7f31641f79a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.934+0000 7f3163e00640 1 --2- 172.21.15.164:0/3103593709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31640716b0 0x7f31641541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.938+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 >> 172.21.15.164:0/3103593709 conn(0x7f316406ac90 msgr2=0x7f31640b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.938+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 shutdown_connections 2024-05-08T04:38:53.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:53.938+0000 7f3163e00640 1 -- 172.21.15.164:0/3103593709 wait complete. 2024-05-08T04:38:53.960 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:54.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 Processor -- start 2024-05-08T04:38:54.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 -- start start 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f00adfa0 0x7fe5f00b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f015e300 con 0x7fe5f00b4dc0 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f015e570 con 0x7fe5f00af6d0 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.094+0000 7fe5f7a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f015e850 con 0x7fe5f00adfa0 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5efe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f00adfa0 0x7fe5f00b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5f5a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe5f00b4dc0 0x7fe5f006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33096/0 (socket says 172.21.15.164:33096) 2024-05-08T04:38:54.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5f5a00640 1 -- 172.21.15.164:0/1102665168 learned_addr learned my addr 172.21.15.164:0/1102665168 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:54.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5efe00640 1 -- 172.21.15.164:0/1102665168 >> v1:172.21.15.172:6789/0 conn(0x7fe5f00af6d0 legacy=0x7fe5f00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5efe00640 1 -- 172.21.15.164:0/1102665168 >> v1:172.21.15.164:6789/0 conn(0x7fe5f00b4dc0 legacy=0x7fe5f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5efe00640 1 -- 172.21.15.164:0/1102665168 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00b5250 con 0x7fe5f00adfa0 2024-05-08T04:38:54.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5ef400640 1 -- 172.21.15.164:0/1102665168 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111430163 0 0) 0x7fe5f015e300 con 0x7fe5f00b4dc0 2024-05-08T04:38:54.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5efe00640 1 --2- 172.21.15.164:0/1102665168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00adfa0 0x7fe5f00b2ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097c0 tx=0x7fe5dc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303235fda0250883 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5ef400640 1 -- 172.21.15.164:0/1102665168 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0b0070 con 0x7fe5f00adfa0 2024-05-08T04:38:54.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5ef400640 1 -- 172.21.15.164:0/1102665168 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc0ab600 con 0x7fe5f00adfa0 2024-05-08T04:38:54.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.098+0000 7fe5ef400640 1 -- 172.21.15.164:0/1102665168 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0ab9d0 con 0x7fe5f00adfa0 2024-05-08T04:38:54.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- 172.21.15.164:0/1102665168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00adfa0 msgr2=0x7fe5f00b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/1102665168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00adfa0 0x7fe5f00b2ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097c0 tx=0x7fe5dc01c410 comp rx=0 tx=0).stop 2024-05-08T04:38:54.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- 172.21.15.164:0/1102665168 shutdown_connections 2024-05-08T04:38:54.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/1102665168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00adfa0 0x7fe5f00b2ff0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- 172.21.15.164:0/1102665168 >> 172.21.15.164:0/1102665168 conn(0x7fe5f006cca0 msgr2=0x7fe5f006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- 172.21.15.164:0/1102665168 shutdown_connections 2024-05-08T04:38:54.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- 172.21.15.164:0/1102665168 wait complete. 2024-05-08T04:38:54.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 Processor -- start 2024-05-08T04:38:54.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- start start 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f00adfa0 0x7fe5f014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f00b4dc0 0x7fe5f00cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f016bcf0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f016bb70 con 0x7fe5f00b4dc0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f016be70 con 0x7fe5f00adfa0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49876/0 (socket says 172.21.15.164:49876) 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 -- 172.21.15.164:0/13418576 learned_addr learned my addr 172.21.15.164:0/13418576 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f5a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f00b4dc0 0x7fe5f00cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5f5000640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f00adfa0 0x7fe5f014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 -- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f00adfa0 msgr2=0x7fe5f014f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f00adfa0 0x7fe5f014f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 -- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f00b4dc0 msgr2=0x7fe5f00cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f00b4dc0 0x7fe5f00cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 -- 172.21.15.164:0/13418576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f015aef0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.102+0000 7fe5efe00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097c0 tx=0x7fe5dc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0b0040 con 0x7fe5f00af6d0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc0b4410 con 0x7fe5f00af6d0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f015b1d0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0b47e0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f014b370 con 0x7fe5f00af6d0 2024-05-08T04:38:54.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f006ddf0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5dc0bc020 con 0x7fe5f00af6d0 2024-05-08T04:38:54.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 0x7fe5d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5dc11e7b0 con 0x7fe5f00af6d0 2024-05-08T04:38:54.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5f5000640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 0x7fe5d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.106+0000 7fe5f5000640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 0x7fe5d0069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000f70 tx=0x7fe5e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.110+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5dc016090 con 0x7fe5f00af6d0 2024-05-08T04:38:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.262+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5f0162230 con 0x7fe5f00af6d0 2024-05-08T04:38:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5ecc00640 1 -- 172.21.15.164:0/13418576 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5dc0ef690 con 0x7fe5f00af6d0 2024-05-08T04:38:54.268 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 msgr2=0x7fe5d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 0x7fe5d0069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000f70 tx=0x7fe5e0009040 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 msgr2=0x7fe5f0160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097c0 tx=0x7fe5dc0191b0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 shutdown_connections 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d0066f60 0x7fe5d0069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f00b4dc0 0x7fe5f00cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f00af6d0 0x7fe5f0160510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 --2- 172.21.15.164:0/13418576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f00adfa0 0x7fe5f014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 >> 172.21.15.164:0/13418576 conn(0x7fe5f006cca0 msgr2=0x7fe5f015c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 shutdown_connections 2024-05-08T04:38:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.266+0000 7fe5f7a00640 1 -- 172.21.15.164:0/13418576 wait complete. 2024-05-08T04:38:54.290 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:54.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.422+0000 7f65f0e00640 1 Processor -- start 2024-05-08T04:38:54.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 -- start start 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 0x7f65ec0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f65ec0b4dc0 0x7f65ec0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ec15e300 con 0x7f65ec0addd0 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ec06bca0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65f0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec15e570 con 0x7f65ec0af6d0 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f65ec0b4dc0 0x7f65ec0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f65ec0b4dc0 0x7f65ec0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49882/0 (socket says 172.21.15.164:49882) 2024-05-08T04:38:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 -- 172.21.15.164:0/2776790458 learned_addr learned my addr 172.21.15.164:0/2776790458 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9e00640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 0x7f65ec0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 -- 172.21.15.164:0/2776790458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 msgr2=0x7f65ec0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 0x7f65ec0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 -- 172.21.15.164:0/2776790458 >> v1:172.21.15.172:6789/0 conn(0x7f65ec0addd0 legacy=0x7f65ec1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9400640 1 -- 172.21.15.164:0/2776790458 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec15e850 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.426+0000 7f65e9e00640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 0x7f65ec0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:54.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65e9400640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec0ad890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427704a19dc745f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65e8a00640 1 -- 172.21.15.164:0/2776790458 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80b0070 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65e8a00640 1 -- 172.21.15.164:0/2776790458 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80ab400 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65e8a00640 1 -- 172.21.15.164:0/2776790458 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80ab7d0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- 172.21.15.164:0/2776790458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 msgr2=0x7f65ec0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec0ad890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d801c490 comp rx=0 tx=0).stop 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- 172.21.15.164:0/2776790458 shutdown_connections 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec0ad890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 --2- 172.21.15.164:0/2776790458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0af6d0 0x7f65ec0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- 172.21.15.164:0/2776790458 >> 172.21.15.164:0/2776790458 conn(0x7f65ec06cca0 msgr2=0x7f65ec06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- 172.21.15.164:0/2776790458 shutdown_connections 2024-05-08T04:38:54.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- 172.21.15.164:0/2776790458 wait complete. 2024-05-08T04:38:54.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 Processor -- start 2024-05-08T04:38:54.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.430+0000 7f65f0e00640 1 -- start start 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0addd0 0x7f65ec134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65ec0af6d0 0x7f65ec0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec16aad0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec16a950 con 0x7f65ec0af6d0 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec16ac50 con 0x7f65ec0addd0 2024-05-08T04:38:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49884/0 (socket says 172.21.15.164:49884) 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 -- 172.21.15.164:0/3570755485 learned_addr learned my addr 172.21.15.164:0/3570755485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e9400640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65ec0af6d0 0x7f65ec0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e9e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0addd0 0x7f65ec134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 -- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0addd0 msgr2=0x7f65ec134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0addd0 0x7f65ec134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 -- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65ec0af6d0 msgr2=0x7f65ec0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65ec0af6d0 0x7f65ec0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 -- 172.21.15.164:0/3570755485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec15d9a0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65ea800640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f65e400c7b0 tx=0x7f65e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004b70 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4004d10 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec15d100 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400aa90 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec15d6c0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec04f010 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.434+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65e400ac30 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.438+0000 7f65e2000640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 0x7f65cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.438+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65e4085090 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.438+0000 7f65e9e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 0x7f65cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.438+0000 7f65e9e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 0x7f65cc069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ab60 tx=0x7f65dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.442+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65e4020070 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.594+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65ec0b1890 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.594+0000 7f65e2000640 1 -- 172.21.15.164:0/3570755485 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65e4055ae0 con 0x7f65ec0b4dc0 2024-05-08T04:38:54.599 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 msgr2=0x7f65cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 0x7f65cc069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ab60 tx=0x7f65dc008040 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 msgr2=0x7f65ec15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f65e400c7b0 tx=0x7f65e400cc70 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 shutdown_connections 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65cc066f60 0x7f65cc069420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65ec0b4dc0 0x7f65ec15ca20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65ec0af6d0 0x7f65ec0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 --2- 172.21.15.164:0/3570755485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65ec0addd0 0x7f65ec134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 >> 172.21.15.164:0/3570755485 conn(0x7f65ec06cca0 msgr2=0x7f65ec072330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 shutdown_connections 2024-05-08T04:38:54.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.598+0000 7f65f0e00640 1 -- 172.21.15.164:0/3570755485 wait complete. 2024-05-08T04:38:54.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:54.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 Processor -- start 2024-05-08T04:38:54.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 -- start start 2024-05-08T04:38:54.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0cc15ba70 0x7fb0cc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 0x7fb0cc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc0cd9f0 con 0x7fb0cc0aeda0 2024-05-08T04:38:54.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc15fe70 con 0x7fb0cc15ba70 2024-05-08T04:38:54.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0cdcd0 con 0x7fb0cc15be50 2024-05-08T04:38:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 0x7fb0cc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d0c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0cc15ba70 0x7fb0cc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d1600640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb0cc0aeda0 0x7fb0cc0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:35782/0 (socket says 172.21.15.164:35782) 2024-05-08T04:38:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0d1600640 1 -- 172.21.15.164:0/3243292641 learned_addr learned my addr 172.21.15.164:0/3243292641 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0c3e00640 1 -- 172.21.15.164:0/3243292641 >> v2:172.21.15.172:3300/0 conn(0x7fb0cc15ba70 msgr2=0x7fb0cc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0c3e00640 1 --2- 172.21.15.164:0/3243292641 >> v2:172.21.15.172:3300/0 conn(0x7fb0cc15ba70 0x7fb0cc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0c3e00640 1 -- 172.21.15.164:0/3243292641 >> v1:172.21.15.172:6789/0 conn(0x7fb0cc0aeda0 legacy=0x7fb0cc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0c3e00640 1 -- 172.21.15.164:0/3243292641 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc15f820 con 0x7fb0cc15be50 2024-05-08T04:38:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.754+0000 7fb0cbe00640 1 -- 172.21.15.164:0/3243292641 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928761582 0 0) 0x7fb0cc0cd9f0 con 0x7fb0cc0aeda0 2024-05-08T04:38:54.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0c3e00640 1 --2- 172.21.15.164:0/3243292641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 0x7fb0cc0cf690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40097c0 tx=0x7fb0b401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa9d5e53d8ebf71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0cbe00640 1 -- 172.21.15.164:0/3243292641 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40b2070 con 0x7fb0cc15be50 2024-05-08T04:38:54.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0cbe00640 1 -- 172.21.15.164:0/3243292641 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b40ad680 con 0x7fb0cc15be50 2024-05-08T04:38:54.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0cbe00640 1 -- 172.21.15.164:0/3243292641 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40ada50 con 0x7fb0cc15be50 2024-05-08T04:38:54.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 -- 172.21.15.164:0/3243292641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 msgr2=0x7fb0cc0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 --2- 172.21.15.164:0/3243292641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 0x7fb0cc0cf690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40097c0 tx=0x7fb0b401c470 comp rx=0 tx=0).stop 2024-05-08T04:38:54.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 -- 172.21.15.164:0/3243292641 shutdown_connections 2024-05-08T04:38:54.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 --2- 172.21.15.164:0/3243292641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc15be50 0x7fb0cc0cf690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 --2- 172.21.15.164:0/3243292641 >> v2:172.21.15.172:3300/0 conn(0x7fb0cc15ba70 0x7fb0cc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 -- 172.21.15.164:0/3243292641 >> 172.21.15.164:0/3243292641 conn(0x7fb0cc06ac90 msgr2=0x7fb0cc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 -- 172.21.15.164:0/3243292641 shutdown_connections 2024-05-08T04:38:54.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.758+0000 7fb0d3600640 1 -- 172.21.15.164:0/3243292641 wait complete. 2024-05-08T04:38:54.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 Processor -- start 2024-05-08T04:38:54.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- start start 2024-05-08T04:38:54.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc0aeda0 0x7fb0cc13fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0cc15be50 0x7fb0cc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc168d40 con 0x7fb0cc15ba70 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc168bc0 con 0x7fb0cc15be50 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc168ec0 con 0x7fb0cc0aeda0 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49892/0 (socket says 172.21.15.164:49892) 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 -- 172.21.15.164:0/1522070414 learned_addr learned my addr 172.21.15.164:0/1522070414 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d0c00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc0aeda0 0x7fb0cc13fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d1600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0cc15be50 0x7fb0cc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 -- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc0aeda0 msgr2=0x7fb0cc13fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc0aeda0 0x7fb0cc13fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 -- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0cc15be50 msgr2=0x7fb0cc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0cc15be50 0x7fb0cc0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 -- 172.21.15.164:0/1522070414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc1fa870 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c3e00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b401a0a0 tx=0x7fb0b401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40b2040 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b40bd070 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc152e10 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40add50 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.762+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc1fab10 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc069d60 con 0x7fb0cc15ba70 2024-05-08T04:38:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0b40c2460 con 0x7fb0cc15ba70 2024-05-08T04:38:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0c9600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0b411f1c0 con 0x7fb0cc15ba70 2024-05-08T04:38:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0d0c00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:54.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.766+0000 7fb0d0c00640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 0x7fb0a8069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc140b30 tx=0x7fb0bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:54.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.770+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0b4016090 con 0x7fb0cc15ba70 2024-05-08T04:38:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.926+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0cc0cc0c0 con 0x7fb0cc15ba70 2024-05-08T04:38:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.926+0000 7fb0c9600640 1 -- 172.21.15.164:0/1522070414 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fb0b40f01c0 con 0x7fb0cc15ba70 2024-05-08T04:38:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:54.932 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:54.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 msgr2=0x7fb0a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 0x7fb0a8069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc140b30 tx=0x7fb0bc008040 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 msgr2=0x7fb0cc1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b401a0a0 tx=0x7fb0b401acb0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 shutdown_connections 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb0a8066f60 0x7fb0a8069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0cc15be50 0x7fb0cc0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0cc15ba70 0x7fb0cc1542b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 --2- 172.21.15.164:0/1522070414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0cc0aeda0 0x7fb0cc13fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 >> 172.21.15.164:0/1522070414 conn(0x7fb0cc06ac90 msgr2=0x7fb0cc0cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 shutdown_connections 2024-05-08T04:38:54.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:54.930+0000 7fb0d3600640 1 -- 172.21.15.164:0/1522070414 wait complete. 2024-05-08T04:38:54.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:54.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:55.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.086+0000 7f9f4dc00640 1 Processor -- start 2024-05-08T04:38:55.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.086+0000 7f9f4dc00640 1 -- start start 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.086+0000 7f9f4dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f480b1960 0x7f9f48071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.086+0000 7f9f4dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f4815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f4dc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4815c900 con 0x7f9f480b1d40 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f4dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4815cec0 con 0x7f9f480b1960 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f4dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815cbe0 con 0x7f9f480cec40 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f4d200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9f480b1d40 0x7f9f480ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33104/0 (socket says 172.21.15.164:33104) 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f4d200640 1 -- 172.21.15.164:0/342454675 learned_addr learned my addr 172.21.15.164:0/342454675 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:55.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 --2- 172.21.15.164:0/342454675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f4815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47e00640 1 --2- 172.21.15.164:0/342454675 >> v2:172.21.15.172:3300/0 conn(0x7f9f480b1960 0x7f9f48071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231183499 0 0) 0x7f9f4815c900 con 0x7f9f480b1d40 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f2c003530 con 0x7f9f480b1d40 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184359786 0 0) 0x7f9f2c003530 con 0x7f9f480b1d40 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 -- 172.21.15.164:0/342454675 >> v2:172.21.15.172:3300/0 conn(0x7f9f480b1960 msgr2=0x7f9f48071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 --2- 172.21.15.164:0/342454675 >> v2:172.21.15.172:3300/0 conn(0x7f9f480b1960 0x7f9f48071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 -- 172.21.15.164:0/342454675 >> v1:172.21.15.164:6789/0 conn(0x7f9f480b1d40 legacy=0x7f9f480ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 -- 172.21.15.164:0/342454675 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480712a0 con 0x7f9f480cec40 2024-05-08T04:38:55.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.090+0000 7f9f47400640 1 --2- 172.21.15.164:0/342454675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f4815bf90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f3401c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d464a80ce71db8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b0070 con 0x7f9f480cec40 2024-05-08T04:38:55.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340ab400 con 0x7f9f480cec40 2024-05-08T04:38:55.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f46a00640 1 -- 172.21.15.164:0/342454675 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340a4e60 con 0x7f9f480cec40 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- 172.21.15.164:0/342454675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 msgr2=0x7f9f4815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/342454675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f4815bf90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f3401c270 comp rx=0 tx=0).stop 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- 172.21.15.164:0/342454675 shutdown_connections 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/342454675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f4815bf90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/342454675 >> v2:172.21.15.172:3300/0 conn(0x7f9f480b1960 0x7f9f48071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- 172.21.15.164:0/342454675 >> 172.21.15.164:0/342454675 conn(0x7f9f4806ac90 msgr2=0x7f9f4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- 172.21.15.164:0/342454675 shutdown_connections 2024-05-08T04:38:55.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- 172.21.15.164:0/342454675 wait complete. 2024-05-08T04:38:55.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 Processor -- start 2024-05-08T04:38:55.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- start start 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 0x7f9f481533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f480ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480b3bf0 con 0x7f9f480b1d40 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815c6e0 con 0x7f9f480b1960 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f4dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480b3d70 con 0x7f9f480cec40 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f47400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f47400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49908/0 (socket says 172.21.15.164:49908) 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.094+0000 7f9f47400640 1 -- 172.21.15.164:0/1088642062 learned_addr learned my addr 172.21.15.164:0/1088642062 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47e00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 0x7f9f481533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f4d200640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f480ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 -- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 msgr2=0x7f9f480ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f480ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 -- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 msgr2=0x7f9f481533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 0x7f9f481533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 -- 172.21.15.164:0/1088642062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48153930 con 0x7f9f480b1d40 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f4d200640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f480ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47e00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 0x7f9f481533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47400640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f34019010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340ba070 con 0x7f9f480b1d40 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340a4e60 con 0x7f9f480b1d40 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f48159ba0 con 0x7f9f480b1d40 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b0040 con 0x7f9f480b1d40 2024-05-08T04:38:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4815a160 con 0x7f9f480b1d40 2024-05-08T04:38:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4804ef30 con 0x7f9f480b1d40 2024-05-08T04:38:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f340bd070 con 0x7f9f480b1d40 2024-05-08T04:38:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 0x7f9f28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f3411ee80 con 0x7f9f480b1d40 2024-05-08T04:38:55.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.098+0000 7f9f47e00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 0x7f9f28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.102+0000 7f9f47e00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 0x7f9f28069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f380074c0 tx=0x7f9f3800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.102+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f340c4070 con 0x7f9f480b1d40 2024-05-08T04:38:55.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f48143760 con 0x7f9f480b1d40 2024-05-08T04:38:55.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f3fe00640 1 -- 172.21.15.164:0/1088642062 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f34015fb0 con 0x7f9f480b1d40 2024-05-08T04:38:55.262 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:55.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 msgr2=0x7f9f28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 0x7f9f28069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f380074c0 tx=0x7f9f3800b040 comp rx=0 tx=0).stop 2024-05-08T04:38:55.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 msgr2=0x7f9f48143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f34019010 comp rx=0 tx=0).stop 2024-05-08T04:38:55.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 shutdown_connections 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f28066f90 0x7f9f28069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f480cec40 0x7f9f480ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f480b1d40 0x7f9f48143220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.258+0000 7f9f4dc00640 1 --2- 172.21.15.164:0/1088642062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f480b1960 0x7f9f481533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.262+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 >> 172.21.15.164:0/1088642062 conn(0x7f9f4806ac90 msgr2=0x7f9f4806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.262+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 shutdown_connections 2024-05-08T04:38:55.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.262+0000 7f9f4dc00640 1 -- 172.21.15.164:0/1088642062 wait complete. 2024-05-08T04:38:55.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:55.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 Processor -- start 2024-05-08T04:38:55.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 -- start start 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 0x7f383806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f38380cee90 0x7f38380aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383815e870 con 0x7f38380ae6e0 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383806bb80 con 0x7f38380cee90 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f383e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383815eb50 con 0x7f383815ab10 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f38380cee90 0x7f38380aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f38380cee90 0x7f38380aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49914/0 (socket says 172.21.15.164:49914) 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 -- 172.21.15.164:0/1449336745 learned_addr learned my addr 172.21.15.164:0/1449336745 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:55.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837400640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 0x7f383806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 -- 172.21.15.164:0/1449336745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 msgr2=0x7f383806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:55.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 0x7f383806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 -- 172.21.15.164:0/1449336745 >> v1:172.21.15.172:6789/0 conn(0x7f38380ae6e0 legacy=0x7f38380ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837e00640 1 -- 172.21.15.164:0/1449336745 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383815ee30 con 0x7f38380cee90 2024-05-08T04:38:55.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.418+0000 7f3837400640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 0x7f383806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:55.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f3837e00640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380cee90 0x7f38380aeb40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5ddba38e883346 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f3836a00640 1 -- 172.21.15.164:0/1449336745 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300a57f0 con 0x7f38380cee90 2024-05-08T04:38:55.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f3836a00640 1 -- 172.21.15.164:0/1449336745 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300a5990 con 0x7f38380cee90 2024-05-08T04:38:55.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f3836a00640 1 -- 172.21.15.164:0/1449336745 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300a5d60 con 0x7f38380cee90 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 -- 172.21.15.164:0/1449336745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380cee90 msgr2=0x7f38380aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380cee90 0x7f38380aeb40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383001c970 comp rx=0 tx=0).stop 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 -- 172.21.15.164:0/1449336745 shutdown_connections 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380cee90 0x7f38380aeb40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 --2- 172.21.15.164:0/1449336745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f383815ab10 0x7f383806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 -- 172.21.15.164:0/1449336745 >> 172.21.15.164:0/1449336745 conn(0x7f38380cf770 msgr2=0x7f38380cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 -- 172.21.15.164:0/1449336745 shutdown_connections 2024-05-08T04:38:55.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.422+0000 7f383e600640 1 -- 172.21.15.164:0/1449336745 wait complete. 2024-05-08T04:38:55.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 Processor -- start 2024-05-08T04:38:55.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- start start 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38380cee90 0x7f38380ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f383815ab10 0x7f38381466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cb070 con 0x7f38380ae6e0 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380caef0 con 0x7f383815ab10 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cb1f0 con 0x7f38380cee90 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49924/0 (socket says 172.21.15.164:49924) 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 -- 172.21.15.164:0/2951453889 learned_addr learned my addr 172.21.15.164:0/2951453889 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:55.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837e00640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f383815ab10 0x7f38381466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f382ea00640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38380cee90 0x7f38380ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 -- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38380cee90 msgr2=0x7f38380ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38380cee90 0x7f38380ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 -- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f383815ab10 msgr2=0x7f38381466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f383815ab10 0x7f38381466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 -- 172.21.15.164:0/2951453889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383814d030 con 0x7f38380ae6e0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f3837400640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800b6c0 con 0x7f38380ae6e0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828019070 con 0x7f38380ae6e0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800baa0 con 0x7f38380ae6e0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383814d2d0 con 0x7f38380ae6e0 2024-05-08T04:38:55.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383815de60 con 0x7f38380ae6e0 2024-05-08T04:38:55.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.426+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3838072d30 con 0x7f38380ae6e0 2024-05-08T04:38:55.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.430+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f382800bc40 con 0x7f38380ae6e0 2024-05-08T04:38:55.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.430+0000 7f382fe00640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 0x7f3814069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.430+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3828085310 con 0x7f38380ae6e0 2024-05-08T04:38:55.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.430+0000 7f382ea00640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 0x7f3814069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.430+0000 7f382ea00640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 0x7f3814069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f383814f620 tx=0x7f3820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.434+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38280563b0 con 0x7f38380ae6e0 2024-05-08T04:38:55.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f383804f010 con 0x7f38380ae6e0 2024-05-08T04:38:55.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f382fe00640 1 -- 172.21.15.164:0/2951453889 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3828055cb0 con 0x7f38380ae6e0 2024-05-08T04:38:55.590 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 msgr2=0x7f3814069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 0x7f3814069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f383814f620 tx=0x7f3820009040 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 msgr2=0x7f38380cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 shutdown_connections 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3814066f60 0x7f3814069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38380cee90 0x7f38380ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f383815ab10 0x7f38381466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 --2- 172.21.15.164:0/2951453889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38380ae6e0 0x7f38380cde50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 >> 172.21.15.164:0/2951453889 conn(0x7f38380cf770 msgr2=0x7f38380cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 shutdown_connections 2024-05-08T04:38:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.586+0000 7f383e600640 1 -- 172.21.15.164:0/2951453889 wait complete. 2024-05-08T04:38:55.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:55.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 Processor -- start 2024-05-08T04:38:55.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 -- start start 2024-05-08T04:38:55.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31415ab10 0x7fa31406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 0x7fa3140aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3140ce780 0x7fa3140cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3140cf130 con 0x7fa3140ce780 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31406bb80 con 0x7fa3140ae6e0 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa318e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31415e870 con 0x7fa31415ab10 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3140ce780 0x7fa3140cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3140ce780 0x7fa3140cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49936/0 (socket says 172.21.15.164:49936) 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 -- 172.21.15.164:0/3870398421 learned_addr learned my addr 172.21.15.164:0/3870398421 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa311400640 1 --2- 172.21.15.164:0/3870398421 >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 0x7fa3140aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa311e00640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31415ab10 0x7fa31406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 -- 172.21.15.164:0/3870398421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31415ab10 msgr2=0x7fa31406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31415ab10 0x7fa31406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 -- 172.21.15.164:0/3870398421 >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 msgr2=0x7fa3140aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 --2- 172.21.15.164:0/3870398421 >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 0x7fa3140aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa312800640 1 -- 172.21.15.164:0/3870398421 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31415eb50 con 0x7fa3140ce780 2024-05-08T04:38:55.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.746+0000 7fa311400640 1 --2- 172.21.15.164:0/3870398421 >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 0x7fa3140aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:55.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa312800640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3140ce780 0x7fa3140cebf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3140cd2f0 tx=0x7fa30c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca89782f64406c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa310a00640 1 -- 172.21.15.164:0/3870398421 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a57b0 con 0x7fa3140ce780 2024-05-08T04:38:55.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa310a00640 1 -- 172.21.15.164:0/3870398421 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0a5950 con 0x7fa3140ce780 2024-05-08T04:38:55.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa310a00640 1 -- 172.21.15.164:0/3870398421 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a5d20 con 0x7fa3140ce780 2024-05-08T04:38:55.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 -- 172.21.15.164:0/3870398421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3140ce780 msgr2=0x7fa3140cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3140ce780 0x7fa3140cebf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3140cd2f0 tx=0x7fa30c01c930 comp rx=0 tx=0).stop 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 -- 172.21.15.164:0/3870398421 shutdown_connections 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3140ce780 0x7fa3140cebf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 --2- 172.21.15.164:0/3870398421 >> v2:172.21.15.172:3300/0 conn(0x7fa3140ae6e0 0x7fa3140aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 --2- 172.21.15.164:0/3870398421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31415ab10 0x7fa31406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 -- 172.21.15.164:0/3870398421 >> 172.21.15.164:0/3870398421 conn(0x7fa3140cf770 msgr2=0x7fa3140cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 -- 172.21.15.164:0/3870398421 shutdown_connections 2024-05-08T04:38:55.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.750+0000 7fa318e00640 1 -- 172.21.15.164:0/3870398421 wait complete. 2024-05-08T04:38:55.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 Processor -- start 2024-05-08T04:38:55.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- start start 2024-05-08T04:38:55.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3140ae6e0 0x7fa3140b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3140ce780 0x7fa3140b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140c5b70 con 0x7fa31415ab10 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31415aef0 con 0x7fa3140ae6e0 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140c5cf0 con 0x7fa3140ce780 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49950/0 (socket says 172.21.15.164:49950) 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 -- 172.21.15.164:0/3956288019 learned_addr learned my addr 172.21.15.164:0/3956288019 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa311400640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3140ce780 0x7fa3140b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa311e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3140ae6e0 0x7fa3140b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 -- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3140ce780 msgr2=0x7fa3140b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3140ce780 0x7fa3140b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 -- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3140ae6e0 msgr2=0x7fa3140b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3140ae6e0 0x7fa3140b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 -- 172.21.15.164:0/3956288019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140c6ee0 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa312800640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0af040 tx=0x7fa30c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0bc020 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0b8670 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3140c6c00 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0b8a60 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314158f80 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.754+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa314158ad0 con 0x7fa31415ab10 2024-05-08T04:38:55.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.758+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa30c0bf020 con 0x7fa31415ab10 2024-05-08T04:38:55.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.758+0000 7fa30a000640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 0x7fa2f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:55.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.758+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa30c1201c0 con 0x7fa31415ab10 2024-05-08T04:38:55.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.758+0000 7fa311e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 0x7fa2f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:55.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.758+0000 7fa311e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 0x7fa2f4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa30400d8d0 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:55.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.762+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa30c0c6030 con 0x7fa31415ab10 2024-05-08T04:38:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa31404f010 con 0x7fa31415ab10 2024-05-08T04:38:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa30a000640 1 -- 172.21.15.164:0/3956288019 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa30c0165e0 con 0x7fa31415ab10 2024-05-08T04:38:55.922 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 msgr2=0x7fa2f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 0x7fa2f4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa30400d8d0 tx=0x7fa304008040 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 msgr2=0x7fa3140b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0af040 tx=0x7fa30c0196d0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 shutdown_connections 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2f4066f60 0x7fa2f4069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3140ce780 0x7fa3140b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31415ab10 0x7fa3140b2b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 --2- 172.21.15.164:0/3956288019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3140ae6e0 0x7fa3140b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 >> 172.21.15.164:0/3956288019 conn(0x7fa3140cf770 msgr2=0x7fa3140cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 shutdown_connections 2024-05-08T04:38:55.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:55.918+0000 7fa318e00640 1 -- 172.21.15.164:0/3956288019 wait complete. 2024-05-08T04:38:55.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:56.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 Processor -- start 2024-05-08T04:38:56.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 -- start start 2024-05-08T04:38:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 0x7fa0a00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a006c770 con 0x7fa0a015ab60 2024-05-08T04:38:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a00d0060 con 0x7fa0a00cfbd0 2024-05-08T04:38:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a8000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a006ca50 con 0x7fa0a00b26c0 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a6000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 0x7fa0a00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54788/0 (socket says 172.21.15.164:54788) 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 -- 172.21.15.164:0/2389454241 learned_addr learned my addr 172.21.15.164:0/2389454241 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a5600640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa0a015ab60 0x7fa0a0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:35796/0 (socket says 172.21.15.164:35796) 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 -- 172.21.15.164:0/2389454241 >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 msgr2=0x7fa0a00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 --2- 172.21.15.164:0/2389454241 >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 0x7fa0a00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 -- 172.21.15.164:0/2389454241 >> v1:172.21.15.172:6789/0 conn(0x7fa0a015ab60 legacy=0x7fa0a0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a4c00640 1 -- 172.21.15.164:0/2389454241 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a006cd30 con 0x7fa0a00b26c0 2024-05-08T04:38:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.074+0000 7fa0a6000640 1 --2- 172.21.15.164:0/2389454241 >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 0x7fa0a00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:38:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a4c00640 1 --2- 172.21.15.164:0/2389454241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1676f0d3db9e71fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa09fe00640 1 -- 172.21.15.164:0/2389454241 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0b2070 con 0x7fa0a00b26c0 2024-05-08T04:38:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa09fe00640 1 -- 172.21.15.164:0/2389454241 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c0a9070 con 0x7fa0a00b26c0 2024-05-08T04:38:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa09fe00640 1 -- 172.21.15.164:0/2389454241 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0ad880 con 0x7fa0a00b26c0 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 -- 172.21.15.164:0/2389454241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 msgr2=0x7fa0a00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 --2- 172.21.15.164:0/2389454241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c01c560 comp rx=0 tx=0).stop 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 -- 172.21.15.164:0/2389454241 shutdown_connections 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 --2- 172.21.15.164:0/2389454241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a00b26c0 0x7fa0a00cf690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 --2- 172.21.15.164:0/2389454241 >> v2:172.21.15.172:3300/0 conn(0x7fa0a00cfbd0 0x7fa0a00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.078+0000 7fa0a8000640 1 -- 172.21.15.164:0/2389454241 >> 172.21.15.164:0/2389454241 conn(0x7fa0a006ac90 msgr2=0x7fa0a006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- 172.21.15.164:0/2389454241 shutdown_connections 2024-05-08T04:38:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- 172.21.15.164:0/2389454241 wait complete. 2024-05-08T04:38:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 Processor -- start 2024-05-08T04:38:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- start start 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a00cfbd0 0x7fa0a00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a015ab60 0x7fa0a00adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00aeeb0 con 0x7fa0a00b26c0 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00aed30 con 0x7fa0a00cfbd0 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00cc1b0 con 0x7fa0a015ab60 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49960/0 (socket says 172.21.15.164:49960) 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 -- 172.21.15.164:0/659306994 learned_addr learned my addr 172.21.15.164:0/659306994 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a4c00640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a00cfbd0 0x7fa0a00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a6000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a015ab60 0x7fa0a00adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 -- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a015ab60 msgr2=0x7fa0a00adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a015ab60 0x7fa0a00adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 -- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a00cfbd0 msgr2=0x7fa0a00b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a00cfbd0 0x7fa0a00b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 -- 172.21.15.164:0/659306994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0167bc0 con 0x7fa0a00b26c0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a5600640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c3a0 tx=0x7fa09400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0940074e0 con 0x7fa0a00b26c0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094007680 con 0x7fa0a00b26c0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094004a30 con 0x7fa0a00b26c0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.082+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a0167ea0 con 0x7fa0a00b26c0 2024-05-08T04:38:56.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a0168460 con 0x7fa0a00b26c0 2024-05-08T04:38:56.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0069d60 con 0x7fa0a00b26c0 2024-05-08T04:38:56.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa09401a020 con 0x7fa0a00b26c0 2024-05-08T04:38:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa09d600640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 0x7fa080069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa094084730 con 0x7fa0a00b26c0 2024-05-08T04:38:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa0a4c00640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 0x7fa080069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.086+0000 7fa0a4c00640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 0x7fa080069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0af000 tx=0x7fa08c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.090+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa094055740 con 0x7fa0a00b26c0 2024-05-08T04:38:56.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0a006d9a0 con 0x7fa0a00b26c0 2024-05-08T04:38:56.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa09d600640 1 -- 172.21.15.164:0/659306994 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fa094055040 con 0x7fa0a00b26c0 2024-05-08T04:38:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:56.254 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 msgr2=0x7fa080069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 0x7fa080069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0af000 tx=0x7fa08c008040 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 msgr2=0x7fa0a015f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c3a0 tx=0x7fa09400c860 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 shutdown_connections 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa080066f60 0x7fa080069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0a015ab60 0x7fa0a00adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a00cfbd0 0x7fa0a00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 --2- 172.21.15.164:0/659306994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0a00b26c0 0x7fa0a015f140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 >> 172.21.15.164:0/659306994 conn(0x7fa0a006ac90 msgr2=0x7fa0a00b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 shutdown_connections 2024-05-08T04:38:56.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.250+0000 7fa0a8000640 1 -- 172.21.15.164:0/659306994 wait complete. 2024-05-08T04:38:56.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:56.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:56.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 Processor -- start 2024-05-08T04:38:56.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 -- start start 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 0x7f87cc0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87cc0b4dc0 0x7f87cc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc15e300 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc15e570 con 0x7f87cc0af6d0 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87d1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc15e850 con 0x7f87cc0addd0 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87cc0b4dc0 0x7f87cc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87cc0b4dc0 0x7f87cc0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49968/0 (socket says 172.21.15.164:49968) 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 -- 172.21.15.164:0/2133930730 learned_addr learned my addr 172.21.15.164:0/2133930730 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:56.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87cb400640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87caa00640 1 --2- 172.21.15.164:0/2133930730 >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 0x7f87cc0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 -- 172.21.15.164:0/2133930730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 msgr2=0x7f87cc1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 -- 172.21.15.164:0/2133930730 >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 msgr2=0x7f87cc0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 --2- 172.21.15.164:0/2133930730 >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 0x7f87cc0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87ca000640 1 -- 172.21.15.164:0/2133930730 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0b5220 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87caa00640 1 --2- 172.21.15.164:0/2133930730 >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 0x7f87cc0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:56.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.414+0000 7f87cb400640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:56.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87ca000640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc0ad890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87cc070580 tx=0x7f87b801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eaa0fe3a389581 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87c9600640 1 -- 172.21.15.164:0/2133930730 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80b0070 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87c9600640 1 -- 172.21.15.164:0/2133930730 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80ab600 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87c9600640 1 -- 172.21.15.164:0/2133930730 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80ab9d0 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 -- 172.21.15.164:0/2133930730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0b4dc0 msgr2=0x7f87cc0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc0ad890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87cc070580 tx=0x7f87b801c1d0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 -- 172.21.15.164:0/2133930730 shutdown_connections 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 --2- 172.21.15.164:0/2133930730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc0ad890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 --2- 172.21.15.164:0/2133930730 >> v2:172.21.15.172:3300/0 conn(0x7f87cc0af6d0 0x7f87cc0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 -- 172.21.15.164:0/2133930730 >> 172.21.15.164:0/2133930730 conn(0x7f87cc06cca0 msgr2=0x7f87cc06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 -- 172.21.15.164:0/2133930730 shutdown_connections 2024-05-08T04:38:56.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.418+0000 7f87d1a00640 1 -- 172.21.15.164:0/2133930730 wait complete. 2024-05-08T04:38:56.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 Processor -- start 2024-05-08T04:38:56.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- start start 2024-05-08T04:38:56.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc06b060 con 0x7f87cc0af6d0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc06aee0 con 0x7f87cc0b4dc0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc06b1e0 con 0x7f87cc0addd0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49976/0 (socket says 172.21.15.164:49976) 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 -- 172.21.15.164:0/1109640214 learned_addr learned my addr 172.21.15.164:0/1109640214 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87caa00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87cb400640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 -- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 msgr2=0x7f87cc15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc15fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 -- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 msgr2=0x7f87cc14e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc14e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 -- 172.21.15.164:0/1109640214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc1603e0 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87cb400640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc14e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87ca000640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87b801ca70 tx=0x7f87b8018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80b0040 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80b4640 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc1606c0 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80b4a30 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.422+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc151710 con 0x7f87cc0af6d0 2024-05-08T04:38:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc06ddf0 con 0x7f87cc0af6d0 2024-05-08T04:38:56.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b80ab4c0 con 0x7f87cc0af6d0 2024-05-08T04:38:56.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87c2a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87b811ebf0 con 0x7f87cc0af6d0 2024-05-08T04:38:56.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87caa00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.426+0000 7f87caa00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 0x7f87ac069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87cc157eb0 tx=0x7f87bc007810 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.430+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87b8016110 con 0x7f87cc0af6d0 2024-05-08T04:38:56.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.582+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87cc04f010 con 0x7f87cc0af6d0 2024-05-08T04:38:56.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.582+0000 7f87c2a00640 1 -- 172.21.15.164:0/1109640214 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b80efc90 con 0x7f87cc0af6d0 2024-05-08T04:38:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 msgr2=0x7f87ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 0x7f87ac069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87cc157eb0 tx=0x7f87bc007810 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 msgr2=0x7f87cc14de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87b801ca70 tx=0x7f87b8018f70 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 shutdown_connections 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87cc0b4dc0 0x7f87cc14e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87cc0af6d0 0x7f87cc14de70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 --2- 172.21.15.164:0/1109640214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87cc0addd0 0x7f87cc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 >> 172.21.15.164:0/1109640214 conn(0x7f87cc06cca0 msgr2=0x7f87cc154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 shutdown_connections 2024-05-08T04:38:56.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.586+0000 7f87d1a00640 1 -- 172.21.15.164:0/1109640214 wait complete. 2024-05-08T04:38:56.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:56.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.742+0000 7f72ffe00640 1 Processor -- start 2024-05-08T04:38:56.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.742+0000 7f72ffe00640 1 -- start start 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 0x7f730006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ffe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73000cd050 0x7f730015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ffe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73000cfdc0 con 0x7f730006fe10 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ffe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7300070270 con 0x7f73000cd050 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000cc4b0 con 0x7f730015efd0 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ff400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73000cd050 0x7f730015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72f6000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f730006fe10 0x7f73000cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33120/0 (socket says 172.21.15.164:33120) 2024-05-08T04:38:56.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72f6000640 1 -- 172.21.15.164:0/3453092255 learned_addr learned my addr 172.21.15.164:0/3453092255 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ff400640 1 -- 172.21.15.164:0/3453092255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 msgr2=0x7f730006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72fea00640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 0x7f730006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72fe000640 1 -- 172.21.15.164:0/3453092255 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776159304 0 0) 0x7f73000cfdc0 con 0x7f730006fe10 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ff400640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 0x7f730006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ff400640 1 -- 172.21.15.164:0/3453092255 >> v1:172.21.15.164:6789/0 conn(0x7f730006fe10 legacy=0x7f73000cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72ff400640 1 -- 172.21.15.164:0/3453092255 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000d00a0 con 0x7f73000cd050 2024-05-08T04:38:56.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.746+0000 7f72fea00640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 0x7f730006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:56.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ff400640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73000cd050 0x7f730015f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097c0 tx=0x7f72f801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e790637dec119 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72fe000640 1 -- 172.21.15.164:0/3453092255 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f80b5070 con 0x7f73000cd050 2024-05-08T04:38:56.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72fe000640 1 -- 172.21.15.164:0/3453092255 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f80a8990 con 0x7f73000cd050 2024-05-08T04:38:56.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72fe000640 1 -- 172.21.15.164:0/3453092255 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f80a8d60 con 0x7f73000cd050 2024-05-08T04:38:56.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- 172.21.15.164:0/3453092255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73000cd050 msgr2=0x7f730015f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73000cd050 0x7f730015f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097c0 tx=0x7f72f801ec90 comp rx=0 tx=0).stop 2024-05-08T04:38:56.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- 172.21.15.164:0/3453092255 shutdown_connections 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73000cd050 0x7f730015f6f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- 172.21.15.164:0/3453092255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730015efd0 0x7f730006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- 172.21.15.164:0/3453092255 >> 172.21.15.164:0/3453092255 conn(0x7f73000b3370 msgr2=0x7f73000b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- 172.21.15.164:0/3453092255 shutdown_connections 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- 172.21.15.164:0/3453092255 wait complete. 2024-05-08T04:38:56.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 Processor -- start 2024-05-08T04:38:56.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- start start 2024-05-08T04:38:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f730006fe10 0x7f73000c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73000cd050 0x7f73000c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73000d0660 con 0x7f730015efd0 2024-05-08T04:38:56.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730015f3b0 con 0x7f730006fe10 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000cd4e0 con 0x7f73000cd050 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.750+0000 7f72ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49994/0 (socket says 172.21.15.164:49994) 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 -- 172.21.15.164:0/1787782143 learned_addr learned my addr 172.21.15.164:0/1787782143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72fea00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f730006fe10 0x7f73000c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f6000640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73000cd050 0x7f73000c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 -- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73000cd050 msgr2=0x7f73000c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73000cd050 0x7f73000c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 -- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f730006fe10 msgr2=0x7f73000c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f730006fe10 0x7f73000c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 -- 172.21.15.164:0/1787782143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73001f37c0 con 0x7f730015efd0 2024-05-08T04:38:56.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ff400640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72f801c8c0 tx=0x7f72f801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f80b5040 con 0x7f730015efd0 2024-05-08T04:38:56.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f80c0070 con 0x7f730015efd0 2024-05-08T04:38:56.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73001f3a80 con 0x7f730015efd0 2024-05-08T04:38:56.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f80b09e0 con 0x7f730015efd0 2024-05-08T04:38:56.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73001f4040 con 0x7f730015efd0 2024-05-08T04:38:56.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730004ef30 con 0x7f730015efd0 2024-05-08T04:38:56.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72f80b0b80 con 0x7f730015efd0 2024-05-08T04:38:56.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 0x7f72dc0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:56.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72f8122480 con 0x7f730015efd0 2024-05-08T04:38:56.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.754+0000 7f72fea00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 0x7f72dc0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:56.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.758+0000 7f72fea00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 0x7f72dc0694a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72f00075b0 tx=0x7f72f0007540 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:56.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.758+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72f80187f0 con 0x7f730015efd0 2024-05-08T04:38:56.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7300067720 con 0x7f730015efd0 2024-05-08T04:38:56.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72f7400640 1 -- 172.21.15.164:0/1787782143 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72f80f3570 con 0x7f730015efd0 2024-05-08T04:38:56.922 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 msgr2=0x7f72dc0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 0x7f72dc0694a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72f00075b0 tx=0x7f72f0007540 comp rx=0 tx=0).stop 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 msgr2=0x7f73000c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72f801c8c0 tx=0x7f72f801ba30 comp rx=0 tx=0).stop 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 shutdown_connections 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72dc066fe0 0x7f72dc0694a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73000cd050 0x7f73000c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730015efd0 0x7f73000c7f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 --2- 172.21.15.164:0/1787782143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f730006fe10 0x7f73000c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.918+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 >> 172.21.15.164:0/1787782143 conn(0x7f73000b3370 msgr2=0x7f73000ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.922+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 shutdown_connections 2024-05-08T04:38:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:56.922+0000 7f72ffe00640 1 -- 172.21.15.164:0/1787782143 wait complete. 2024-05-08T04:38:56.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 Processor -- start 2024-05-08T04:38:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 -- start start 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d01560d0 0x7f29d015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f29d0072310 0x7f29d006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d00cff80 con 0x7f29d006b1d0 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d006b620 con 0x7f29d0072310 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29d6800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d015d050 con 0x7f29d01560d0 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29cea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f29d0072310 0x7f29d006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29cfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f29d006b1d0 0x7f29d015f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33132/0 (socket says 172.21.15.164:33132) 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.078+0000 7f29cfe00640 1 -- 172.21.15.164:0/1408071241 learned_addr learned my addr 172.21.15.164:0/1408071241 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cf400640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d01560d0 0x7f29d015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cea00640 1 -- 172.21.15.164:0/1408071241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d01560d0 msgr2=0x7f29d015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cea00640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d01560d0 0x7f29d015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cea00640 1 -- 172.21.15.164:0/1408071241 >> v1:172.21.15.164:6789/0 conn(0x7f29d006b1d0 legacy=0x7f29d015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cea00640 1 -- 172.21.15.164:0/1408071241 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00ceee0 con 0x7f29d0072310 2024-05-08T04:38:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29ce000640 1 -- 172.21.15.164:0/1408071241 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87278576 0 0) 0x7f29c8001120 con 0x7f29d006b1d0 2024-05-08T04:38:57.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29cea00640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d0072310 0x7f29d006ac90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0097c0 tx=0x7f29bc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f55ae39a727054 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29ce000640 1 -- 172.21.15.164:0/1408071241 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0b2070 con 0x7f29d0072310 2024-05-08T04:38:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29ce000640 1 -- 172.21.15.164:0/1408071241 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0ad640 con 0x7f29d0072310 2024-05-08T04:38:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.082+0000 7f29ce000640 1 -- 172.21.15.164:0/1408071241 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0ada10 con 0x7f29d0072310 2024-05-08T04:38:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- 172.21.15.164:0/1408071241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d0072310 msgr2=0x7f29d006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d0072310 0x7f29d006ac90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0097c0 tx=0x7f29bc01c470 comp rx=0 tx=0).stop 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- 172.21.15.164:0/1408071241 shutdown_connections 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d0072310 0x7f29d006ac90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- 172.21.15.164:0/1408071241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d01560d0 0x7f29d015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- 172.21.15.164:0/1408071241 >> 172.21.15.164:0/1408071241 conn(0x7f29d0067050 msgr2=0x7f29d006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- 172.21.15.164:0/1408071241 shutdown_connections 2024-05-08T04:38:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- 172.21.15.164:0/1408071241 wait complete. 2024-05-08T04:38:57.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 Processor -- start 2024-05-08T04:38:57.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- start start 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d006b1d0 0x7f29d00b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29d0072310 0x7f29d0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01564b0 con 0x7f29d01560d0 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00cfa60 con 0x7f29d0072310 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29d6800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00ceb90 con 0x7f29d006b1d0 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50008/0 (socket says 172.21.15.164:50008) 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 -- 172.21.15.164:0/2042984238 learned_addr learned my addr 172.21.15.164:0/2042984238 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cf400640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d006b1d0 0x7f29d00b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cea00640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29d0072310 0x7f29d0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 -- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d006b1d0 msgr2=0x7f29d00b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d006b1d0 0x7f29d00b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 -- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29d0072310 msgr2=0x7f29d0154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29d0072310 0x7f29d0154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 -- 172.21.15.164:0/2042984238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00b4460 con 0x7f29d01560d0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.086+0000 7f29cfe00640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c780 tx=0x7f29c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8018070 con 0x7f29d01560d0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c80070d0 con 0x7f29d01560d0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007500 con 0x7f29d01560d0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d0155360 con 0x7f29d01560d0 2024-05-08T04:38:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d00b49e0 con 0x7f29d01560d0 2024-05-08T04:38:57.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d0072e80 con 0x7f29d01560d0 2024-05-08T04:38:57.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29c80047f0 con 0x7f29d01560d0 2024-05-08T04:38:57.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29c8084320 con 0x7f29d01560d0 2024-05-08T04:38:57.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29cf400640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.090+0000 7f29cf400640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 0x7f29b0069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29d00b0560 tx=0x7f29c000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.094+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29d00cff80 con 0x7f29d01560d0 2024-05-08T04:38:57.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.246+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29d0071fd0 con 0x7f29d01560d0 2024-05-08T04:38:57.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.246+0000 7f29c7400640 1 -- 172.21.15.164:0/2042984238 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29c8054c40 con 0x7f29d01560d0 2024-05-08T04:38:57.252 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 msgr2=0x7f29b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 0x7f29b0069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29d00b0560 tx=0x7f29c000b040 comp rx=0 tx=0).stop 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 msgr2=0x7f29d013d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c780 tx=0x7f29c800cc40 comp rx=0 tx=0).stop 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 shutdown_connections 2024-05-08T04:38:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29d01560d0 0x7f29d013d1e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29d0072310 0x7f29d0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 --2- 172.21.15.164:0/2042984238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29d006b1d0 0x7f29d00b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 >> 172.21.15.164:0/2042984238 conn(0x7f29d0067050 msgr2=0x7f29d014be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 shutdown_connections 2024-05-08T04:38:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.250+0000 7f29d6800640 1 -- 172.21.15.164:0/2042984238 wait complete. 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.344 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-05-08T04:38:57.345 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-05-08T04:38:57.345 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:38:57.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.478+0000 7fd915600640 1 Processor -- start 2024-05-08T04:38:57.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.478+0000 7fd915600640 1 -- start start 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.478+0000 7fd915600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd915600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9100d05b0 0x7fd91015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd915600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9100b42a0 con 0x7fd91015bc50 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd915600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100cd4e0 con 0x7fd9100d05b0 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd915600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100cd7c0 con 0x7fd9100ccfe0 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9100d05b0 0x7fd91015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90ea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd91015bc50 0x7fd9100b4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33144/0 (socket says 172.21.15.164:33144) 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90ea00640 1 -- 172.21.15.164:0/2601830125 learned_addr learned my addr 172.21.15.164:0/2601830125 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd906000640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 -- 172.21.15.164:0/2601830125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 msgr2=0x7fd9100d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 -- 172.21.15.164:0/2601830125 >> v1:172.21.15.164:6789/0 conn(0x7fd91015bc50 legacy=0x7fd9100b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 -- 172.21.15.164:0/2601830125 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100cf590 con 0x7fd9100d05b0 2024-05-08T04:38:57.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90e000640 1 -- 172.21.15.164:0/2601830125 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452529494 0 0) 0x7fd9100b42a0 con 0x7fd91015bc50 2024-05-08T04:38:57.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90f400640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9100d05b0 0x7fd91015f7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f66f5ec839d2870 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90e000640 1 -- 172.21.15.164:0/2601830125 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080b2070 con 0x7fd9100d05b0 2024-05-08T04:38:57.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90e000640 1 -- 172.21.15.164:0/2601830125 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080a5890 con 0x7fd9100d05b0 2024-05-08T04:38:57.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.482+0000 7fd90e000640 1 -- 172.21.15.164:0/2601830125 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080a5c60 con 0x7fd9100d05b0 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- 172.21.15.164:0/2601830125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9100d05b0 msgr2=0x7fd91015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9100d05b0 0x7fd91015f7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90801c8e0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- 172.21.15.164:0/2601830125 shutdown_connections 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9100d05b0 0x7fd91015f7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- 172.21.15.164:0/2601830125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- 172.21.15.164:0/2601830125 >> 172.21.15.164:0/2601830125 conn(0x7fd9100b34f0 msgr2=0x7fd9100b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- 172.21.15.164:0/2601830125 shutdown_connections 2024-05-08T04:38:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- 172.21.15.164:0/2601830125 wait complete. 2024-05-08T04:38:57.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 Processor -- start 2024-05-08T04:38:57.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- start start 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 0x7fd9100c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100b50a0 con 0x7fd91015bc50 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100b4f20 con 0x7fd9100d05b0 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd915600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd910168ae0 con 0x7fd9100ccfe0 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50022/0 (socket says 172.21.15.164:50022) 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 -- 172.21.15.164:0/1068877777 learned_addr learned my addr 172.21.15.164:0/1068877777 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90ea00640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd906000640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 0x7fd9100c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 -- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 msgr2=0x7fd9100c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 -- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 msgr2=0x7fd9100c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 0x7fd9100c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90f400640 1 -- 172.21.15.164:0/1068877777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100c5c60 con 0x7fd91015bc50 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd90ea00640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.486+0000 7fd906000640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 0x7fd9100c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd90f400640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd90801af90 tx=0x7fd908019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080b2050 con 0x7fd91015bc50 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080be070 con 0x7fd91015bc50 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080ada50 con 0x7fd91015bc50 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9100c5f40 con 0x7fd91015bc50 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9100c6500 con 0x7fd91015bc50 2024-05-08T04:38:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910069d60 con 0x7fd91015bc50 2024-05-08T04:38:57.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9080adbf0 con 0x7fd91015bc50 2024-05-08T04:38:57.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 0x7fd8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9081202e0 con 0x7fd91015bc50 2024-05-08T04:38:57.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.490+0000 7fd90ea00640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 0x7fd8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.494+0000 7fd90ea00640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 0x7fd8ec069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd900000f70 tx=0x7fd900009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.494+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd908016560 con 0x7fd91015bc50 2024-05-08T04:38:57.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.650+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91015b2b0 con 0x7fd91015bc50 2024-05-08T04:38:57.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.650+0000 7fd907400640 1 -- 172.21.15.164:0/1068877777 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fd9080f1260 con 0x7fd91015bc50 2024-05-08T04:38:57.655 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:38:57.655 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 msgr2=0x7fd8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 0x7fd8ec069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd900000f70 tx=0x7fd900009040 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 msgr2=0x7fd9101558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd90801af90 tx=0x7fd908019680 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 shutdown_connections 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd8ec066f60 0x7fd8ec069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd91015bc50 0x7fd9101558b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100d05b0 0x7fd9100c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 --2- 172.21.15.164:0/1068877777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100ccfe0 0x7fd9100c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 >> 172.21.15.164:0/1068877777 conn(0x7fd9100b34f0 msgr2=0x7fd9100b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 shutdown_connections 2024-05-08T04:38:57.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.654+0000 7fd915600640 1 -- 172.21.15.164:0/1068877777 wait complete. 2024-05-08T04:38:57.676 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:38:57.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:38:57.719 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:38:57.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:38:57.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 Processor -- start 2024-05-08T04:38:57.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 -- start start 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f552015ab10 0x7f552006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ae6e0 0x7f55200aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552015e870 con 0x7f55200ce780 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552006bb80 con 0x7f552015ab10 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f5524a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552015eb50 con 0x7f55200ae6e0 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551e800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f55200ce780 0x7f55200c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33148/0 (socket says 172.21.15.164:33148) 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551e800640 1 -- 172.21.15.164:0/372530216 learned_addr learned my addr 172.21.15.164:0/372530216 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:57.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551de00640 1 --2- 172.21.15.164:0/372530216 >> v2:172.21.15.164:3300/0 conn(0x7f552015ab10 0x7f552006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551d400640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ae6e0 0x7f55200aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551ca00640 1 -- 172.21.15.164:0/372530216 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195640744 0 0) 0x7f552015e870 con 0x7f55200ce780 2024-05-08T04:38:57.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551ca00640 1 -- 172.21.15.164:0/372530216 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5504003530 con 0x7f55200ce780 2024-05-08T04:38:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551de00640 1 -- 172.21.15.164:0/372530216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ae6e0 msgr2=0x7f55200aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551de00640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ae6e0 0x7f55200aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551de00640 1 -- 172.21.15.164:0/372530216 >> v1:172.21.15.164:6789/0 conn(0x7f55200ce780 legacy=0x7f55200c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.814+0000 7f551de00640 1 -- 172.21.15.164:0/372530216 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552015ee30 con 0x7f552015ab10 2024-05-08T04:38:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f551de00640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552006cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5510009dc0 tx=0x7f551001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab74bda24d03de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f551ca00640 1 -- 172.21.15.164:0/372530216 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100b2070 con 0x7f552015ab10 2024-05-08T04:38:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f551ca00640 1 -- 172.21.15.164:0/372530216 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55100a5950 con 0x7f552015ab10 2024-05-08T04:38:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f551ca00640 1 -- 172.21.15.164:0/372530216 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100a5d20 con 0x7f552015ab10 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 -- 172.21.15.164:0/372530216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 msgr2=0x7f552006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552006cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5510009dc0 tx=0x7f551001c890 comp rx=0 tx=0).stop 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 -- 172.21.15.164:0/372530216 shutdown_connections 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ae6e0 0x7f55200aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 --2- 172.21.15.164:0/372530216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552006cce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 -- 172.21.15.164:0/372530216 >> 172.21.15.164:0/372530216 conn(0x7f55200cf770 msgr2=0x7f55200cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 -- 172.21.15.164:0/372530216 shutdown_connections 2024-05-08T04:38:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.818+0000 7f5524a00640 1 -- 172.21.15.164:0/372530216 wait complete. 2024-05-08T04:38:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 Processor -- start 2024-05-08T04:38:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- start start 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55200ae6e0 0x7f55200c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ce780 0x7f55200c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200cefa0 con 0x7f552015ab10 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5520165ad0 con 0x7f55200ae6e0 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200c8aa0 con 0x7f55200ce780 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55200ae6e0 0x7f55200c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50042/0 (socket says 172.21.15.164:50042) 2024-05-08T04:38:57.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 -- 172.21.15.164:0/284455691 learned_addr learned my addr 172.21.15.164:0/284455691 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551d400640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ce780 0x7f55200c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 -- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ce780 msgr2=0x7f55200c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ce780 0x7f55200c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 -- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55200ae6e0 msgr2=0x7f55200c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55200ae6e0 0x7f55200c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 -- 172.21.15.164:0/284455691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200c7e90 con 0x7f552015ab10 2024-05-08T04:38:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f551e800640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f551800c450 tx=0x7f551800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518017070 con 0x7f552015ab10 2024-05-08T04:38:57.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518005cf0 con 0x7f552015ab10 2024-05-08T04:38:57.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180072b0 con 0x7f552015ab10 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55200c8170 con 0x7f552015ab10 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55200c8730 con 0x7f552015ab10 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.822+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520072d30 con 0x7f552015ab10 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.826+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f551801a070 con 0x7f552015ab10 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.826+0000 7f5516000640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 0x7f5500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:57.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.826+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55180842d0 con 0x7f552015ab10 2024-05-08T04:38:57.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.826+0000 7f551de00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 0x7f5500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:57.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.826+0000 7f551de00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 0x7f5500069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55100af000 tx=0x7f55100a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:57.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.830+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5518054bf0 con 0x7f552015ab10 2024-05-08T04:38:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f552004f010 con 0x7f552015ab10 2024-05-08T04:38:57.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5516000640 1 -- 172.21.15.164:0/284455691 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5518058c90 con 0x7f552015ab10 2024-05-08T04:38:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 msgr2=0x7f5500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 0x7f5500069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55100af000 tx=0x7f55100a7070 comp rx=0 tx=0).stop 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 msgr2=0x7f552014d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f551800c450 tx=0x7f551800c910 comp rx=0 tx=0).stop 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 shutdown_connections 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5500066f60 0x7f5500069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55200ce780 0x7f55200c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f552015ab10 0x7f552014d0f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 --2- 172.21.15.164:0/284455691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55200ae6e0 0x7f55200c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 >> 172.21.15.164:0/284455691 conn(0x7f55200cf770 msgr2=0x7f55200ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 shutdown_connections 2024-05-08T04:38:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:57.986+0000 7f5524a00640 1 -- 172.21.15.164:0/284455691 wait complete. 2024-05-08T04:38:58.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:38:58.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 Processor -- start 2024-05-08T04:38:58.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 -- start start 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c0affe0 0x7f461c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f461c06ac90 0x7f461c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c06da30 con 0x7f461c06e7b0 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c06e320 con 0x7f461c0affe0 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4624000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c06b610 con 0x7f461c06ac90 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c0affe0 0x7f461c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c0affe0 0x7f461c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50044/0 (socket says 172.21.15.164:50044) 2024-05-08T04:38:58.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 -- 172.21.15.164:0/1875650655 learned_addr learned my addr 172.21.15.164:0/1875650655 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:58.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 -- 172.21.15.164:0/1875650655 >> v2:172.21.15.172:3300/0 conn(0x7f461c06ac90 msgr2=0x7f461c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:58.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 --2- 172.21.15.164:0/1875650655 >> v2:172.21.15.172:3300/0 conn(0x7f461c06ac90 0x7f461c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 -- 172.21.15.164:0/1875650655 >> v1:172.21.15.172:6789/0 conn(0x7f461c06e7b0 legacy=0x7f461c06f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:58.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.186+0000 7f4621600640 1 -- 172.21.15.164:0/1875650655 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c06dcd0 con 0x7f461c0affe0 2024-05-08T04:38:58.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4621600640 1 --2- 172.21.15.164:0/1875650655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c0affe0 0x7f461c15cbd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b5b70 tx=0x7f461001cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c841fba526db8a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f461be00640 1 -- 172.21.15.164:0/1875650655 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100aa430 con 0x7f461c0affe0 2024-05-08T04:38:58.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f461be00640 1 -- 172.21.15.164:0/1875650655 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100af070 con 0x7f461c0affe0 2024-05-08T04:38:58.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f461be00640 1 -- 172.21.15.164:0/1875650655 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100aa7f0 con 0x7f461c0affe0 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 -- 172.21.15.164:0/1875650655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c0affe0 msgr2=0x7f461c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 --2- 172.21.15.164:0/1875650655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c0affe0 0x7f461c15cbd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b5b70 tx=0x7f461001cf10 comp rx=0 tx=0).stop 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 -- 172.21.15.164:0/1875650655 shutdown_connections 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 --2- 172.21.15.164:0/1875650655 >> v2:172.21.15.172:3300/0 conn(0x7f461c06ac90 0x7f461c06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 --2- 172.21.15.164:0/1875650655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c0affe0 0x7f461c15cbd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 -- 172.21.15.164:0/1875650655 >> 172.21.15.164:0/1875650655 conn(0x7f461c0b2d00 msgr2=0x7f461c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:58.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 -- 172.21.15.164:0/1875650655 shutdown_connections 2024-05-08T04:38:58.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.190+0000 7f4624000640 1 -- 172.21.15.164:0/1875650655 wait complete. 2024-05-08T04:38:58.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 Processor -- start 2024-05-08T04:38:58.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- start start 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 0x7f461c0c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c0affe0 0x7f461c0c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cf5b0 con 0x7f461c06ac90 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cf430 con 0x7f461c06e7b0 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cf730 con 0x7f461c0affe0 2024-05-08T04:38:58.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50052/0 (socket says 172.21.15.164:50052) 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 -- 172.21.15.164:0/4215442064 learned_addr learned my addr 172.21.15.164:0/4215442064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4620c00640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 0x7f461c0c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4622000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c0affe0 0x7f461c0c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 -- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c0affe0 msgr2=0x7f461c0c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c0affe0 0x7f461c0c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 -- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 msgr2=0x7f461c0c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 0x7f461c0c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 -- 172.21.15.164:0/4215442064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0cd7c0 con 0x7f461c06ac90 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4620c00640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 0x7f461c0c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4621600640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46100ac040 tx=0x7f461001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100b3600 con 0x7f461c06ac90 2024-05-08T04:38:58.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100b37c0 con 0x7f461c06ac90 2024-05-08T04:38:58.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100b3bb0 con 0x7f461c06ac90 2024-05-08T04:38:58.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0c6f20 con 0x7f461c06ac90 2024-05-08T04:38:58.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c155460 con 0x7f461c06ac90 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.194+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c072e80 con 0x7f461c06ac90 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.198+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46100c6390 con 0x7f461c06ac90 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.198+0000 7f4619600640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 0x7f45fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.198+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f461011f5a0 con 0x7f461c06ac90 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.198+0000 7f4620c00640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 0x7f45fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.198+0000 7f4620c00640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 0x7f45fc069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f461c0ccd70 tx=0x7f4608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.202+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46100c3060 con 0x7f461c06ac90 2024-05-08T04:38:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f461c1592a0 con 0x7f461c06ac90 2024-05-08T04:38:58.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4619600640 1 -- 172.21.15.164:0/4215442064 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46100160d0 con 0x7f461c06ac90 2024-05-08T04:38:58.358 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 msgr2=0x7f45fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 0x7f45fc069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f461c0ccd70 tx=0x7f4608009040 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 msgr2=0x7f461c14da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46100ac040 tx=0x7f461001acf0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 shutdown_connections 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066f60 0x7f45fc069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c0affe0 0x7f461c0c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c06e7b0 0x7f461c0c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 --2- 172.21.15.164:0/4215442064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c06ac90 0x7f461c14da40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 >> 172.21.15.164:0/4215442064 conn(0x7f461c0b2d00 msgr2=0x7f461c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:58.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 shutdown_connections 2024-05-08T04:38:58.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.354+0000 7f4624000640 1 -- 172.21.15.164:0/4215442064 wait complete. 2024-05-08T04:38:58.379 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:38:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 Processor -- start 2024-05-08T04:38:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 -- start start 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 0x7fbcbc168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0b2be0 con 0x7fbcbc0b4010 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc06cbc0 con 0x7fbcbc15e000 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcc2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc0d0c40 con 0x7fbcbc0cd080 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcb2a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbcbc0b4010 0x7fbcbc06c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:33152/0 (socket says 172.21.15.164:33152) 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcb2a00640 1 -- 172.21.15.164:0/1299787479 learned_addr learned my addr 172.21.15.164:0/1299787479 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:38:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbbe00640 1 --2- 172.21.15.164:0/1299787479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 0x7fbcbc168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774790797 0 0) 0x7fbcbc0b2be0 con 0x7fbcbc0b4010 2024-05-08T04:38:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca0003530 con 0x7fbcbc0b4010 2024-05-08T04:38:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894276671 0 0) 0x7fbcbc06cbc0 con 0x7fbcbc15e000 2024-05-08T04:38:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcbc0b2be0 con 0x7fbcbc15e000 2024-05-08T04:38:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbbe00640 1 -- 172.21.15.164:0/1299787479 >> v1:172.21.15.172:6789/0 conn(0x7fbcbc15e000 legacy=0x7fbcbc06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbbe00640 1 -- 172.21.15.164:0/1299787479 >> v1:172.21.15.164:6789/0 conn(0x7fbcbc0b4010 legacy=0x7fbcbc06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945747861 0 0) 0x7fbca0003530 con 0x7fbcbc0b4010 2024-05-08T04:38:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.514+0000 7fbcbbe00640 1 -- 172.21.15.164:0/1299787479 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0b2eb0 con 0x7fbcbc0cd080 2024-05-08T04:38:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcbbe00640 1 --2- 172.21.15.164:0/1299787479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 0x7fbcbc168cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009dc0 tx=0x7fbcb401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b690d90ce4f8d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40b2070 con 0x7fbcbc0cd080 2024-05-08T04:38:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb40a55e0 con 0x7fbcbc0cd080 2024-05-08T04:38:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcbaa00640 1 -- 172.21.15.164:0/1299787479 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4001680 con 0x7fbcbc0cd080 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 -- 172.21.15.164:0/1299787479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 msgr2=0x7fbcbc168cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 --2- 172.21.15.164:0/1299787479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 0x7fbcbc168cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009dc0 tx=0x7fbcb401c8b0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 -- 172.21.15.164:0/1299787479 shutdown_connections 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 --2- 172.21.15.164:0/1299787479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0cd080 0x7fbcbc168cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 -- 172.21.15.164:0/1299787479 >> 172.21.15.164:0/1299787479 conn(0x7fbcbc0b34f0 msgr2=0x7fbcbc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 -- 172.21.15.164:0/1299787479 shutdown_connections 2024-05-08T04:38:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.518+0000 7fbcc2400640 1 -- 172.21.15.164:0/1299787479 wait complete. 2024-05-08T04:38:58.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 Processor -- start 2024-05-08T04:38:58.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- start start 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0b4010 0x7fbcbc14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcbc15e000 0x7fbcbc0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc15e560 con 0x7fbcbc0cd080 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc15e3e0 con 0x7fbcbc15e000 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16ba20 con 0x7fbcbc0b4010 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50054/0 (socket says 172.21.15.164:50054) 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 -- 172.21.15.164:0/2837326387 learned_addr learned my addr 172.21.15.164:0/2837326387 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcbbe00640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcbc15e000 0x7fbcbc0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcbb400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0b4010 0x7fbcbc14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 -- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0b4010 msgr2=0x7fbcbc14fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0b4010 0x7fbcbc14fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 -- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcbc15e000 msgr2=0x7fbcbc0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcbc15e000 0x7fbcbc0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 -- 172.21.15.164:0/2837326387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0c89f0 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb2a00640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c750 tx=0x7fbca400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4005ae0 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4005c80 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40072b0 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc0cac70 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.522+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc0c8f90 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc09fb50 con 0x7fbcbc0cd080 2024-05-08T04:38:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbca4007450 con 0x7fbcbc0cd080 2024-05-08T04:38:58.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcb3e00640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 0x7fbc98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbca4084c70 con 0x7fbcbc0cd080 2024-05-08T04:38:58.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcbb400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 0x7fbc98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.526+0000 7fbcbb400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 0x7fbc98069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbcac001720 tx=0x7fbcac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.530+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbca4055590 con 0x7fbcbc0cd080 2024-05-08T04:38:58.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.682+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcbc04f010 con 0x7fbcbc0cd080 2024-05-08T04:38:58.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcb3e00640 1 -- 172.21.15.164:0/2837326387 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbca4059630 con 0x7fbcbc0cd080 2024-05-08T04:38:58.688 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:38:58.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 msgr2=0x7fbc98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 0x7fbc98069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbcac001720 tx=0x7fbcac009040 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 msgr2=0x7fbcbc0c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c750 tx=0x7fbca400cc10 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 shutdown_connections 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbc98066f90 0x7fbc98069450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcbc15e000 0x7fbcbc0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcbc0cd080 0x7fbcbc0c9790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 --2- 172.21.15.164:0/2837326387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcbc0b4010 0x7fbcbc14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 >> 172.21.15.164:0/2837326387 conn(0x7fbcbc0b34f0 msgr2=0x7fbcbc06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 shutdown_connections 2024-05-08T04:38:58.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.686+0000 7fbcc2400640 1 -- 172.21.15.164:0/2837326387 wait complete. 2024-05-08T04:38:58.710 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2024-05-08T04:38:58.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.842+0000 7f7f64000640 1 Processor -- start 2024-05-08T04:38:58.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.842+0000 7f7f64000640 1 -- start start 2024-05-08T04:38:58.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f64000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c15b390 0x7f7f5c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f64000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7f5c0aee30 0x7f7f5c0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f64000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c06e0e0 con 0x7f7f5c0cef80 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f64000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c15e8f0 con 0x7f7f5c0aee30 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f64000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c15a8c0 con 0x7f7f5c15b390 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7f5c0aee30 0x7f7f5c0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7f5c0aee30 0x7f7f5c0cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50070/0 (socket says 172.21.15.164:50070) 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 -- 172.21.15.164:0/942015533 learned_addr learned my addr 172.21.15.164:0/942015533 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:58.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f61600640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c15b390 0x7f7f5c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 -- 172.21.15.164:0/942015533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c15b390 msgr2=0x7f7f5c06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:38:58.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c15b390 0x7f7f5c06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 -- 172.21.15.164:0/942015533 >> v1:172.21.15.172:6789/0 conn(0x7f7f5c0cef80 legacy=0x7f7f5c0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 -- 172.21.15.164:0/942015533 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c15ed80 con 0x7f7f5c0aee30 2024-05-08T04:38:58.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.846+0000 7f7f60c00640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c0cea40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f48006fd0 tx=0x7f7f4801e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a80ff18ebd54e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f5be00640 1 -- 172.21.15.164:0/942015533 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480ad070 con 0x7f7f5c0aee30 2024-05-08T04:38:58.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f5be00640 1 -- 172.21.15.164:0/942015533 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48004230 con 0x7f7f5c0aee30 2024-05-08T04:38:58.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f5be00640 1 -- 172.21.15.164:0/942015533 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480a7750 con 0x7f7f5c0aee30 2024-05-08T04:38:58.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- 172.21.15.164:0/942015533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c0aee30 msgr2=0x7f7f5c0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c0cea40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f48006fd0 tx=0x7f7f4801e9f0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- 172.21.15.164:0/942015533 shutdown_connections 2024-05-08T04:38:58.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c0cea40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- 172.21.15.164:0/942015533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c15b390 0x7f7f5c06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- 172.21.15.164:0/942015533 >> 172.21.15.164:0/942015533 conn(0x7f7f5c06ac90 msgr2=0x7f7f5c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:58.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- 172.21.15.164:0/942015533 shutdown_connections 2024-05-08T04:38:58.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- 172.21.15.164:0/942015533 wait complete. 2024-05-08T04:38:58.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 Processor -- start 2024-05-08T04:38:58.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- start start 2024-05-08T04:38:58.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c0cef80 0x7f7f5c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c06ddf0 con 0x7f7f5c15b390 2024-05-08T04:38:58.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0cf390 con 0x7f7f5c0aee30 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f64000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c15b770 con 0x7f7f5c0cef80 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.850+0000 7f7f62000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50074/0 (socket says 172.21.15.164:50074) 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 -- 172.21.15.164:0/1651710712 learned_addr learned my addr 172.21.15.164:0/1651710712 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f61600640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f60c00640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c0cef80 0x7f7f5c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 -- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c0cef80 msgr2=0x7f7f5c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c0cef80 0x7f7f5c0a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 -- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 msgr2=0x7f7f5c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 -- 172.21.15.164:0/1651710712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0a9bf0 con 0x7f7f5c15b390 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f61600640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f62000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c7a0 tx=0x7f7f5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003b80 con 0x7f7f5c15b390 2024-05-08T04:38:58.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54003d20 con 0x7f7f5c15b390 2024-05-08T04:38:58.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400aa90 con 0x7f7f5c15b390 2024-05-08T04:38:58.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c0a9ed0 con 0x7f7f5c15b390 2024-05-08T04:38:58.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c0aa490 con 0x7f7f5c15b390 2024-05-08T04:38:58.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.854+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c069d60 con 0x7f7f5c15b390 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.858+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f5400ac30 con 0x7f7f5c15b390 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.858+0000 7f7f59600640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 0x7f7f3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.858+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f54085a00 con 0x7f7f5c15b390 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.858+0000 7f7f61600640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 0x7f7f3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.858+0000 7f7f61600640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 0x7f7f3c069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009ae0 tx=0x7f7f50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:38:58.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:58.862+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f54056a80 con 0x7f7f5c15b390 2024-05-08T04:38:59.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.066+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f7f5c0cca70 con 0x7f7f5c15b390 2024-05-08T04:38:59.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.862+0000 7f7f59600640 1 -- 172.21.15.164:0/1651710712 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v51) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7f54056380 con 0x7f7f5c15b390 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 msgr2=0x7f7f3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 0x7f7f3c069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009ae0 tx=0x7f7f50008040 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 msgr2=0x7f7f5c0ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c7a0 tx=0x7f7f5400cc60 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 shutdown_connections 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f3c066f60 0x7f7f3c069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f5c0cef80 0x7f7f5c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f5c15b390 0x7f7f5c0ad2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 --2- 172.21.15.164:0/1651710712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f5c0aee30 0x7f7f5c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:38:59.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 >> 172.21.15.164:0/1651710712 conn(0x7f7f5c06ac90 msgr2=0x7f7f5c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:38:59.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 shutdown_connections 2024-05-08T04:38:59.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:38:59.866+0000 7f7f64000640 1 -- 172.21.15.164:0/1651710712 wait complete. 2024-05-08T04:38:59.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:39:00.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 Processor -- start 2024-05-08T04:39:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 -- start start 2024-05-08T04:39:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f164415ab10 0x7f164406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16440ae6e0 0x7f16440aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164415e870 con 0x7f16440ce780 2024-05-08T04:39:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164406bb80 con 0x7f16440ae6e0 2024-05-08T04:39:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.030+0000 7f164c200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164415eb50 con 0x7f164415ab10 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16440ae6e0 0x7f16440aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16440ae6e0 0x7f16440aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56330/0 (socket says 172.21.15.164:56330) 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 -- 172.21.15.164:0/1511323899 learned_addr learned my addr 172.21.15.164:0/1511323899 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1649800640 1 --2- 172.21.15.164:0/1511323899 >> v2:172.21.15.172:3300/0 conn(0x7f164415ab10 0x7f164406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 -- 172.21.15.164:0/1511323899 >> v2:172.21.15.172:3300/0 conn(0x7f164415ab10 msgr2=0x7f164406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 --2- 172.21.15.164:0/1511323899 >> v2:172.21.15.172:3300/0 conn(0x7f164415ab10 0x7f164406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 -- 172.21.15.164:0/1511323899 >> v1:172.21.15.172:6789/0 conn(0x7f16440ce780 legacy=0x7f16440c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 -- 172.21.15.164:0/1511323899 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164415ee30 con 0x7f16440ae6e0 2024-05-08T04:39:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1648e00640 1 --2- 172.21.15.164:0/1511323899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440aeb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f16440c4920 tx=0x7f163001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcf9e252ff1ac18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1643e00640 1 -- 172.21.15.164:0/1511323899 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300ab660 con 0x7f16440ae6e0 2024-05-08T04:39:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1643e00640 1 -- 172.21.15.164:0/1511323899 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16300b0070 con 0x7f16440ae6e0 2024-05-08T04:39:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.034+0000 7f1643e00640 1 -- 172.21.15.164:0/1511323899 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300aba20 con 0x7f16440ae6e0 2024-05-08T04:39:00.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- 172.21.15.164:0/1511323899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 msgr2=0x7f16440aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- 172.21.15.164:0/1511323899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440aeb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f16440c4920 tx=0x7f163001c460 comp rx=0 tx=0).stop 2024-05-08T04:39:00.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- 172.21.15.164:0/1511323899 shutdown_connections 2024-05-08T04:39:00.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- 172.21.15.164:0/1511323899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440aeb70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- 172.21.15.164:0/1511323899 >> v2:172.21.15.172:3300/0 conn(0x7f164415ab10 0x7f164406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- 172.21.15.164:0/1511323899 >> 172.21.15.164:0/1511323899 conn(0x7f16440cf770 msgr2=0x7f16440cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:00.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- 172.21.15.164:0/1511323899 shutdown_connections 2024-05-08T04:39:00.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- 172.21.15.164:0/1511323899 wait complete. 2024-05-08T04:39:00.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 Processor -- start 2024-05-08T04:39:00.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- start start 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16440ce780 0x7f1644152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164415ab10 0x7f16441519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440cefa0 con 0x7f16440ae6e0 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644167ae0 con 0x7f16440ce780 2024-05-08T04:39:00.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16440b2fc0 con 0x7f164415ab10 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56344/0 (socket says 172.21.15.164:56344) 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 -- 172.21.15.164:0/2863219150 learned_addr learned my addr 172.21.15.164:0/2863219150 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1648e00640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16440ce780 0x7f1644152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f164a200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164415ab10 0x7f16441519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 -- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164415ab10 msgr2=0x7f16441519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164415ab10 0x7f16441519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 -- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16440ce780 msgr2=0x7f1644152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16440ce780 0x7f1644152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 -- 172.21.15.164:0/2863219150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440b32a0 con 0x7f16440ae6e0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.038+0000 7f1649800640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f163800d5f0 tx=0x7f163800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380125f0 con 0x7f16440ae6e0 2024-05-08T04:39:00.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638019070 con 0x7f16440ae6e0 2024-05-08T04:39:00.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380129d0 con 0x7f16440ae6e0 2024-05-08T04:39:00.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16440b3580 con 0x7f16440ae6e0 2024-05-08T04:39:00.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1644153420 con 0x7f16440ae6e0 2024-05-08T04:39:00.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644072d30 con 0x7f16440ae6e0 2024-05-08T04:39:00.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1638012b70 con 0x7f16440ae6e0 2024-05-08T04:39:00.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 0x7f1624069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1638085410 con 0x7f16440ae6e0 2024-05-08T04:39:00.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.042+0000 7f1648e00640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 0x7f1624069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.046+0000 7f1648e00640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 0x7f1624069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16300af000 tx=0x7f16300a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.046+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1638056410 con 0x7f16440ae6e0 2024-05-08T04:39:00.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.202+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f164404f010 con 0x7f16440ae6e0 2024-05-08T04:39:00.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.206+0000 7f1641600640 1 -- 172.21.15.164:0/2863219150 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1638055d10 con 0x7f16440ae6e0 2024-05-08T04:39:00.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 msgr2=0x7f1624069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 0x7f1624069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16300af000 tx=0x7f16300a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 msgr2=0x7f16440b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f163800d5f0 tx=0x7f163800dab0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 shutdown_connections 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1624066f60 0x7f1624069420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164415ab10 0x7f16441519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16440ce780 0x7f1644152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 --2- 172.21.15.164:0/2863219150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16440ae6e0 0x7f16440b29e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 >> 172.21.15.164:0/2863219150 conn(0x7f16440cf770 msgr2=0x7f16441574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 shutdown_connections 2024-05-08T04:39:00.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.210+0000 7f164c200640 1 -- 172.21.15.164:0/2863219150 wait complete. 2024-05-08T04:39:00.232 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:39:00.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:00.275 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:00.365 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:39:00.365 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:00.365 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:00.374 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:00.374 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:00.424 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:00.424 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:00.475 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:00.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 Processor -- start 2024-05-08T04:39:00.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 -- start start 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c06bf00 0x7f020c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f020c0cb690 0x7f020c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0cf550 con 0x7f020c066d70 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0af5d0 con 0x7f020c0cb690 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f0211c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c072380 con 0x7f020c06bf00 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f020c0cb690 0x7f020c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f020c0cb690 0x7f020c0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56346/0 (socket says 172.21.15.164:56346) 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 -- 172.21.15.164:0/528799247 learned_addr learned my addr 172.21.15.164:0/528799247 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020aa00640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c06bf00 0x7f020c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 -- 172.21.15.164:0/528799247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c06bf00 msgr2=0x7f020c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c06bf00 0x7f020c0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 -- 172.21.15.164:0/528799247 >> v1:172.21.15.172:6789/0 conn(0x7f020c066d70 legacy=0x7f020c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.646+0000 7f020b400640 1 -- 172.21.15.164:0/528799247 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0725d0 con 0x7f020c0cb690 2024-05-08T04:39:00.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f020b400640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c0cb690 0x7f020c0ce970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0204009870 tx=0x7f020401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a7959079ff010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0209600640 1 -- 172.21.15.164:0/528799247 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040b0070 con 0x7f020c0cb690 2024-05-08T04:39:00.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0209600640 1 -- 172.21.15.164:0/528799247 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040ab600 con 0x7f020c0cb690 2024-05-08T04:39:00.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0209600640 1 -- 172.21.15.164:0/528799247 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040ab9d0 con 0x7f020c0cb690 2024-05-08T04:39:00.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 -- 172.21.15.164:0/528799247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c0cb690 msgr2=0x7f020c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c0cb690 0x7f020c0ce970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0204009870 tx=0x7f020401c440 comp rx=0 tx=0).stop 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 -- 172.21.15.164:0/528799247 shutdown_connections 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c0cb690 0x7f020c0ce970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 --2- 172.21.15.164:0/528799247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c06bf00 0x7f020c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 -- 172.21.15.164:0/528799247 >> 172.21.15.164:0/528799247 conn(0x7f020c0ccab0 msgr2=0x7f020c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 -- 172.21.15.164:0/528799247 shutdown_connections 2024-05-08T04:39:00.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.650+0000 7f0211c00640 1 -- 172.21.15.164:0/528799247 wait complete. 2024-05-08T04:39:00.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 Processor -- start 2024-05-08T04:39:00.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- start start 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c06bf00 0x7f020c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0cb690 0x7f020c09f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c16b0f0 con 0x7f020c066d70 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0cf2c0 con 0x7f020c06bf00 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c16b270 con 0x7f020c0cb690 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56354/0 (socket says 172.21.15.164:56354) 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 -- 172.21.15.164:0/1873237159 learned_addr learned my addr 172.21.15.164:0/1873237159 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020a000640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c06bf00 0x7f020c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020b400640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0cb690 0x7f020c09f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 -- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0cb690 msgr2=0x7f020c09f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0cb690 0x7f020c09f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 -- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c06bf00 msgr2=0x7f020c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c06bf00 0x7f020c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 -- 172.21.15.164:0/1873237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c09b2b0 con 0x7f020c066d70 2024-05-08T04:39:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f020aa00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00efc0 tx=0x7f01fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc018020 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc014480 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c09b590 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc014850 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.654+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c09bb50 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c069d60 con 0x7f020c066d70 2024-05-08T04:39:00.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01fc01b020 con 0x7f020c066d70 2024-05-08T04:39:00.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f0202a00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01fc085a60 con 0x7f020c066d70 2024-05-08T04:39:00.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f020a000640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.658+0000 7f020a000640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f020c0aef50 tx=0x7f01f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.662+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01fc022070 con 0x7f020c066d70 2024-05-08T04:39:00.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020c066b50 con 0x7f020c066d70 2024-05-08T04:39:00.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0202a00640 1 -- 172.21.15.164:0/1873237159 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f01fc010040 con 0x7f020c066d70 2024-05-08T04:39:00.826 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:00.826 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 msgr2=0x7f01ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f020c0aef50 tx=0x7f01f8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 msgr2=0x7f020c09aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00efc0 tx=0x7f01fc00c490 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 shutdown_connections 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0cb690 0x7f020c09f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c06bf00 0x7f020c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 --2- 172.21.15.164:0/1873237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020c066d70 0x7f020c09aa20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 >> 172.21.15.164:0/1873237159 conn(0x7f020c0ccab0 msgr2=0x7f020c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:00.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 shutdown_connections 2024-05-08T04:39:00.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.822+0000 7f0211c00640 1 -- 172.21.15.164:0/1873237159 wait complete. 2024-05-08T04:39:00.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:00.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 Processor -- start 2024-05-08T04:39:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 -- start start 2024-05-08T04:39:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe5d80ca850 0x7fe5d80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d806d150 con 0x7fe5d806dfa0 2024-05-08T04:39:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d806bca0 con 0x7fe5d80ca850 2024-05-08T04:39:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80af150 con 0x7fe5d815a200 2024-05-08T04:39:00.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5d7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5d7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:51302/0 (socket says 172.21.15.164:51302) 2024-05-08T04:39:00.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5d7e00640 1 -- 172.21.15.164:0/4149013353 learned_addr learned my addr 172.21.15.164:0/4149013353 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5dcc00640 1 --2- 172.21.15.164:0/4149013353 >> v2:172.21.15.172:3300/0 conn(0x7fe5d80ca850 0x7fe5d80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.982+0000 7fe5cf400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe5d806dfa0 0x7fe5d80ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:57770/0 (socket says 172.21.15.164:57770) 2024-05-08T04:39:00.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7e00640 1 -- 172.21.15.164:0/4149013353 >> v2:172.21.15.172:3300/0 conn(0x7fe5d80ca850 msgr2=0x7fe5d80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:00.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/4149013353 >> v2:172.21.15.172:3300/0 conn(0x7fe5d80ca850 0x7fe5d80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7e00640 1 -- 172.21.15.164:0/4149013353 >> v1:172.21.15.172:6789/0 conn(0x7fe5d806dfa0 legacy=0x7fe5d80ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7e00640 1 -- 172.21.15.164:0/4149013353 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d806d430 con 0x7fe5d815a200 2024-05-08T04:39:00.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/4149013353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c801e930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5fbfd87cdaa475 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7400640 1 -- 172.21.15.164:0/4149013353 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80ad070 con 0x7fe5d815a200 2024-05-08T04:39:00.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7400640 1 -- 172.21.15.164:0/4149013353 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8004230 con 0x7fe5d815a200 2024-05-08T04:39:00.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5d7400640 1 -- 172.21.15.164:0/4149013353 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80a7780 con 0x7fe5d815a200 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 -- 172.21.15.164:0/4149013353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 msgr2=0x7fe5d80aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 --2- 172.21.15.164:0/4149013353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006fd0 tx=0x7fe5c801e930 comp rx=0 tx=0).stop 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 -- 172.21.15.164:0/4149013353 shutdown_connections 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 --2- 172.21.15.164:0/4149013353 >> v2:172.21.15.172:3300/0 conn(0x7fe5d80ca850 0x7fe5d80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 --2- 172.21.15.164:0/4149013353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d815a200 0x7fe5d80aeac0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.986+0000 7fe5dec00640 1 -- 172.21.15.164:0/4149013353 >> 172.21.15.164:0/4149013353 conn(0x7fe5d80cf770 msgr2=0x7fe5d80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:00.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- 172.21.15.164:0/4149013353 shutdown_connections 2024-05-08T04:39:00.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- 172.21.15.164:0/4149013353 wait complete. 2024-05-08T04:39:00.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 Processor -- start 2024-05-08T04:39:00.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- start start 2024-05-08T04:39:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 0x7fe5d81f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d80ca850 0x7fe5d80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 0x7fe5d80b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80cdd30 con 0x7fe5d815a200 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80cdbb0 con 0x7fe5d80ca850 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80cdeb0 con 0x7fe5d806dfa0 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 0x7fe5d80b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 0x7fe5d80b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56362/0 (socket says 172.21.15.164:56362) 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5dcc00640 1 -- 172.21.15.164:0/2256963007 learned_addr learned my addr 172.21.15.164:0/2256963007 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 0x7fe5d81f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5cf400640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d80ca850 0x7fe5d80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 -- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d80ca850 msgr2=0x7fe5d80c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d80ca850 0x7fe5d80c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 -- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 msgr2=0x7fe5d80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 0x7fe5d80b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 -- 172.21.15.164:0/2256963007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80c91c0 con 0x7fe5d806dfa0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.990+0000 7fe5d7e00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 0x7fe5d81f6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8004000 tx=0x7fe5c801b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80be070 con 0x7fe5d806dfa0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8003620 con 0x7fe5d806dfa0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80ad040 con 0x7fe5d806dfa0 2024-05-08T04:39:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d80c94a0 con 0x7fe5d806dfa0 2024-05-08T04:39:00.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d81460c0 con 0x7fe5d806dfa0 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d8072d30 con 0x7fe5d806dfa0 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5c8003a30 con 0x7fe5d806dfa0 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 0x7fe5b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5c811ef90 con 0x7fe5d806dfa0 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5cf400640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 0x7fe5b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.994+0000 7fe5cf400640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 0x7fe5b4069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8148af0 tx=0x7fe5c0007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:00.998+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5c80185e0 con 0x7fe5d806dfa0 2024-05-08T04:39:01.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.150+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5d804f010 con 0x7fe5d806dfa0 2024-05-08T04:39:01.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5d4c00640 1 -- 172.21.15.164:0/2256963007 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5c80ef8b0 con 0x7fe5d806dfa0 2024-05-08T04:39:01.157 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 msgr2=0x7fe5b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 0x7fe5b4069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8148af0 tx=0x7fe5c0007e90 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 msgr2=0x7fe5d81f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 0x7fe5d81f6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8004000 tx=0x7fe5c801b6d0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 shutdown_connections 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5b4066f60 0x7fe5b4069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5d815a200 0x7fe5d80b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d80ca850 0x7fe5d80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 --2- 172.21.15.164:0/2256963007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d806dfa0 0x7fe5d81f6430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 >> 172.21.15.164:0/2256963007 conn(0x7fe5d80cf770 msgr2=0x7fe5d8158560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 shutdown_connections 2024-05-08T04:39:01.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.154+0000 7fe5dec00640 1 -- 172.21.15.164:0/2256963007 wait complete. 2024-05-08T04:39:01.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:01.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.310+0000 7f8dcf800640 1 Processor -- start 2024-05-08T04:39:01.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.310+0000 7f8dcf800640 1 -- start start 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcf800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8dc815ba20 0x7f8dc8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcf800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 0x7f8dc806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcf800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc80c6600 con 0x7f8dc806de50 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcf800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc815fbe0 con 0x7f8dc815ba20 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcf800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80c6040 con 0x7f8dc80aedf0 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dbfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8dc806de50 0x7f8dc80ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55652/0 (socket says 172.21.15.164:55652) 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dbfe00640 1 -- 172.21.15.164:0/530917027 learned_addr learned my addr 172.21.15.164:0/530917027 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcce00640 1 --2- 172.21.15.164:0/530917027 >> v2:172.21.15.172:3300/0 conn(0x7f8dc815ba20 0x7f8dc8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcd800640 1 --2- 172.21.15.164:0/530917027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 0x7f8dc806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dc7e00640 1 -- 172.21.15.164:0/530917027 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227274441 0 0) 0x7f8dc80c6600 con 0x7f8dc806de50 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dc7e00640 1 -- 172.21.15.164:0/530917027 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dac003530 con 0x7f8dc806de50 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcd800640 1 -- 172.21.15.164:0/530917027 >> v2:172.21.15.172:3300/0 conn(0x7f8dc815ba20 msgr2=0x7f8dc8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcd800640 1 --2- 172.21.15.164:0/530917027 >> v2:172.21.15.172:3300/0 conn(0x7f8dc815ba20 0x7f8dc8160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcd800640 1 -- 172.21.15.164:0/530917027 >> v1:172.21.15.164:6789/0 conn(0x7f8dc806de50 legacy=0x7f8dc80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.314+0000 7f8dcd800640 1 -- 172.21.15.164:0/530917027 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80c6320 con 0x7f8dc80aedf0 2024-05-08T04:39:01.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcd800640 1 --2- 172.21.15.164:0/530917027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 0x7f8dc806e2c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009d90 tx=0x7f8dc001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdb71b740a08a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dc7e00640 1 -- 172.21.15.164:0/530917027 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00b2070 con 0x7f8dc80aedf0 2024-05-08T04:39:01.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dc7e00640 1 -- 172.21.15.164:0/530917027 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc00a59a0 con 0x7f8dc80aedf0 2024-05-08T04:39:01.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dc7e00640 1 -- 172.21.15.164:0/530917027 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00a5d70 con 0x7f8dc80aedf0 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- 172.21.15.164:0/530917027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 msgr2=0x7f8dc806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 --2- 172.21.15.164:0/530917027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 0x7f8dc806e2c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009d90 tx=0x7f8dc001c940 comp rx=0 tx=0).stop 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- 172.21.15.164:0/530917027 shutdown_connections 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 --2- 172.21.15.164:0/530917027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc80aedf0 0x7f8dc806e2c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 --2- 172.21.15.164:0/530917027 >> v2:172.21.15.172:3300/0 conn(0x7f8dc815ba20 0x7f8dc8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- 172.21.15.164:0/530917027 >> 172.21.15.164:0/530917027 conn(0x7f8dc80cf770 msgr2=0x7f8dc80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- 172.21.15.164:0/530917027 shutdown_connections 2024-05-08T04:39:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- 172.21.15.164:0/530917027 wait complete. 2024-05-08T04:39:01.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 Processor -- start 2024-05-08T04:39:01.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 -- start start 2024-05-08T04:39:01.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.318+0000 7f8dcf800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 0x7f8dc814e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dc815ba20 0x7f8dc80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80ce000 con 0x7f8dc80aedf0 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80cde80 con 0x7f8dc815ba20 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8168b70 con 0x7f8dc806de50 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56368/0 (socket says 172.21.15.164:56368) 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 -- 172.21.15.164:0/906871545 learned_addr learned my addr 172.21.15.164:0/906871545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcd800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dc815ba20 0x7f8dc80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcce00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 0x7f8dc814e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 -- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 msgr2=0x7f8dc814e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 0x7f8dc814e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 -- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dc815ba20 msgr2=0x7f8dc80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dc815ba20 0x7f8dc80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 -- 172.21.15.164:0/906871545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80afb40 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcce00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 0x7f8dc814e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dbfe00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c450 tx=0x7f8db000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0017070 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0005cf0 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007290 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc80afe20 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc80c9dc0 con 0x7f8dc80aedf0 2024-05-08T04:39:01.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc8072d30 con 0x7f8dc80aedf0 2024-05-08T04:39:01.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.322+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8db001a070 con 0x7f8dc80aedf0 2024-05-08T04:39:01.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.326+0000 7f8dc5600640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 0x7f8da4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.326+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8db0084480 con 0x7f8dc80aedf0 2024-05-08T04:39:01.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.326+0000 7f8dcce00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 0x7f8da4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.326+0000 7f8dcce00640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 0x7f8da4069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009a40 tx=0x7f8db8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.330+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8db00553f0 con 0x7f8dc80aedf0 2024-05-08T04:39:01.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.482+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8dc8072170 con 0x7f8dc80aedf0 2024-05-08T04:39:01.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.482+0000 7f8dc5600640 1 -- 172.21.15.164:0/906871545 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8db0054cf0 con 0x7f8dc80aedf0 2024-05-08T04:39:01.488 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:01.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 msgr2=0x7f8da4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 0x7f8da4069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009a40 tx=0x7f8db8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 msgr2=0x7f8dc80af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c450 tx=0x7f8db000c910 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 shutdown_connections 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8da4066f60 0x7f8da4069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dc815ba20 0x7f8dc80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc80aedf0 0x7f8dc80af1d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 --2- 172.21.15.164:0/906871545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dc806de50 0x7f8dc814e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 >> 172.21.15.164:0/906871545 conn(0x7f8dc80cf770 msgr2=0x7f8dc815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 shutdown_connections 2024-05-08T04:39:01.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.486+0000 7f8dcf800640 1 -- 172.21.15.164:0/906871545 wait complete. 2024-05-08T04:39:01.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:01.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.642+0000 7f4ae4600640 1 Processor -- start 2024-05-08T04:39:01.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.642+0000 7f4ae4600640 1 -- start start 2024-05-08T04:39:01.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae4600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4adc15ab10 0x7f4adc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 0x7f4adc0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae4600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc15e870 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae4600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc06bb80 con 0x7f4adc15ab10 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae4600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15eb50 con 0x7f4adc0cee90 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4adc15ab10 0x7f4adc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4adc15ab10 0x7f4adc06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56380/0 (socket says 172.21.15.164:56380) 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 -- 172.21.15.164:0/2937959660 learned_addr learned my addr 172.21.15.164:0/2937959660 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae2600640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 0x7f4adc0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 -- 172.21.15.164:0/2937959660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 msgr2=0x7f4adc0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 0x7f4adc0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 -- 172.21.15.164:0/2937959660 >> v1:172.21.15.172:6789/0 conn(0x7f4adc0ae6e0 legacy=0x7f4adc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae1c00640 1 -- 172.21.15.164:0/2937959660 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc15ee30 con 0x7f4adc15ab10 2024-05-08T04:39:01.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.646+0000 7f4ae2600640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 0x7f4adc0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:01.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae1c00640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc15ab10 0x7f4adc06cce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e50 tx=0x7f4ad001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0d636621c001c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4adbe00640 1 -- 172.21.15.164:0/2937959660 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00a57f0 con 0x7f4adc15ab10 2024-05-08T04:39:01.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4adbe00640 1 -- 172.21.15.164:0/2937959660 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00a5990 con 0x7f4adc15ab10 2024-05-08T04:39:01.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4adbe00640 1 -- 172.21.15.164:0/2937959660 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00a5d60 con 0x7f4adc15ab10 2024-05-08T04:39:01.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- 172.21.15.164:0/2937959660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc15ab10 msgr2=0x7f4adc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc15ab10 0x7f4adc06cce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e50 tx=0x7f4ad001c970 comp rx=0 tx=0).stop 2024-05-08T04:39:01.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- 172.21.15.164:0/2937959660 shutdown_connections 2024-05-08T04:39:01.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc0cee90 0x7f4adc0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- 172.21.15.164:0/2937959660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc15ab10 0x7f4adc06cce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- 172.21.15.164:0/2937959660 >> 172.21.15.164:0/2937959660 conn(0x7f4adc0cf770 msgr2=0x7f4adc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- 172.21.15.164:0/2937959660 shutdown_connections 2024-05-08T04:39:01.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- 172.21.15.164:0/2937959660 wait complete. 2024-05-08T04:39:01.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 Processor -- start 2024-05-08T04:39:01.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- start start 2024-05-08T04:39:01.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4adc0cee90 0x7f4adc0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc15ab10 0x7f4adc13e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cb070 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0caef0 con 0x7f4adc0cee90 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.650+0000 7f4ae4600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cb1f0 con 0x7f4adc15ab10 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56394/0 (socket says 172.21.15.164:56394) 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 -- 172.21.15.164:0/479950380 learned_addr learned my addr 172.21.15.164:0/479950380 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1200640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4adc0cee90 0x7f4adc0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae2600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc15ab10 0x7f4adc13e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 -- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc15ab10 msgr2=0x7f4adc13e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc15ab10 0x7f4adc13e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 -- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4adc0cee90 msgr2=0x7f4adc0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4adc0cee90 0x7f4adc0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 -- 172.21.15.164:0/479950380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc15cd70 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae1c00640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00af040 tx=0x7f4ad0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00b8670 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00be070 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc13e580 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00b8a90 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc13eaf0 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc072d30 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ad00b8c30 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.654+0000 7f4ad9600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.658+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ad01201e0 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.658+0000 7f4ae1200640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.658+0000 7f4ae1200640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 0x7f4abc069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0ce230 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.658+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4adc15cd70 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.818+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4adc04f010 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.818+0000 7f4ad9600640 1 -- 172.21.15.164:0/479950380 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ad0016620 con 0x7f4adc0ae6e0 2024-05-08T04:39:01.823 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:01.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 msgr2=0x7f4abc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 0x7f4abc069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0ce230 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:01.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 msgr2=0x7f4adc15b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00af040 tx=0x7f4ad0019710 comp rx=0 tx=0).stop 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 shutdown_connections 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4abc066f60 0x7f4abc069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4adc15ab10 0x7f4adc13e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4adc0cee90 0x7f4adc0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 --2- 172.21.15.164:0/479950380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4adc0ae6e0 0x7f4adc15b420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 >> 172.21.15.164:0/479950380 conn(0x7f4adc0cf770 msgr2=0x7f4adc0b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 shutdown_connections 2024-05-08T04:39:01.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.822+0000 7f4ae4600640 1 -- 172.21.15.164:0/479950380 wait complete. 2024-05-08T04:39:01.845 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:01.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.978+0000 7f3e46800640 1 Processor -- start 2024-05-08T04:39:01.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.978+0000 7f3e46800640 1 -- start start 2024-05-08T04:39:01.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e46800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e400b26c0 0x7f3e400cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e46800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e400cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e46800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4006c770 con 0x7f3e4015ab60 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e46800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e400d0060 con 0x7f3e400b26c0 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e46800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4006ca50 con 0x7f3e400cfbd0 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e400b26c0 0x7f3e400cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e400b26c0 0x7f3e400cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56398/0 (socket says 172.21.15.164:56398) 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 -- 172.21.15.164:0/1726924098 learned_addr learned my addr 172.21.15.164:0/1726924098 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e3fe00640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e400cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 -- 172.21.15.164:0/1726924098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 msgr2=0x7f3e400cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:01.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e400cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 -- 172.21.15.164:0/1726924098 >> v1:172.21.15.172:6789/0 conn(0x7f3e4015ab60 legacy=0x7f3e40072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 -- 172.21.15.164:0/1726924098 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4006cd30 con 0x7f3e400b26c0 2024-05-08T04:39:01.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e36a00640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e400cf690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e50 tx=0x7f3e2801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5ddaf8042e9376 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.982+0000 7f3e3ea00640 1 -- 172.21.15.164:0/1726924098 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280a5700 con 0x7f3e400b26c0 2024-05-08T04:39:01.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3ea00640 1 -- 172.21.15.164:0/1726924098 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e280a58a0 con 0x7f3e400b26c0 2024-05-08T04:39:01.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3ea00640 1 -- 172.21.15.164:0/1726924098 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280a5c70 con 0x7f3e400b26c0 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- 172.21.15.164:0/1726924098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 msgr2=0x7f3e400cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e400cf690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e50 tx=0x7f3e2801ca30 comp rx=0 tx=0).stop 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- 172.21.15.164:0/1726924098 shutdown_connections 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e400cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- 172.21.15.164:0/1726924098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e400cf690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- 172.21.15.164:0/1726924098 >> 172.21.15.164:0/1726924098 conn(0x7f3e4006ac90 msgr2=0x7f3e4006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- 172.21.15.164:0/1726924098 shutdown_connections 2024-05-08T04:39:01.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- 172.21.15.164:0/1726924098 wait complete. 2024-05-08T04:39:01.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 Processor -- start 2024-05-08T04:39:01.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- start start 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e4015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e4015ab60 0x7f3e40166970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400aeeb0 con 0x7f3e400b26c0 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400aed30 con 0x7f3e4015ab60 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e46800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40165960 con 0x7f3e400cfbd0 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56414/0 (socket says 172.21.15.164:56414) 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 -- 172.21.15.164:0/2547817294 learned_addr learned my addr 172.21.15.164:0/2547817294 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3fe00640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e4015ab60 0x7f3e40166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e36a00640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e4015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 -- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 msgr2=0x7f3e4015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e4015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 -- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e4015ab60 msgr2=0x7f3e40166970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e4015ab60 0x7f3e40166970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.986+0000 7f3e3f400640 1 -- 172.21.15.164:0/2547817294 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4015bb10 con 0x7f3e400b26c0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e3f400640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c370 tx=0x7f3e3000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300052d0 con 0x7f3e400b26c0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30005470 con 0x7f3e400b26c0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e40158270 con 0x7f3e400b26c0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30007260 con 0x7f3e400b26c0 2024-05-08T04:39:01.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4015bdf0 con 0x7f3e400b26c0 2024-05-08T04:39:01.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e40069d60 con 0x7f3e400b26c0 2024-05-08T04:39:01.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e30007400 con 0x7f3e400b26c0 2024-05-08T04:39:01.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:01.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e30084560 con 0x7f3e400b26c0 2024-05-08T04:39:01.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.990+0000 7f3e36a00640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:01.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.994+0000 7f3e36a00640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 0x7f3e1c069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e280ae000 tx=0x7f3e280a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:01.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:01.994+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e300554d0 con 0x7f3e400b26c0 2024-05-08T04:39:02.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.162+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e4014c810 con 0x7f3e400b26c0 2024-05-08T04:39:02.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.162+0000 7f3e37e00640 1 -- 172.21.15.164:0/2547817294 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3e30054dd0 con 0x7f3e400b26c0 2024-05-08T04:39:02.166 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:02.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.162+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 msgr2=0x7f3e1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.162+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 0x7f3e1c069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e280ae000 tx=0x7f3e280a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:02.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.162+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 msgr2=0x7f3e40159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c370 tx=0x7f3e3000c830 comp rx=0 tx=0).stop 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 shutdown_connections 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e400cfbd0 0x7f3e4015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e4015ab60 0x7f3e40166970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 --2- 172.21.15.164:0/2547817294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e400b26c0 0x7f3e40159ca0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 >> 172.21.15.164:0/2547817294 conn(0x7f3e4006ac90 msgr2=0x7f3e400cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 shutdown_connections 2024-05-08T04:39:02.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.166+0000 7f3e46800640 1 -- 172.21.15.164:0/2547817294 wait complete. 2024-05-08T04:39:02.188 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:39:02.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:02.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-08T04:39:02.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:39:02.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:02.223 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:39:02.285 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:02.285 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:02.295 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:02.295 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:02.346 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:02.346 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:02.403 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:02.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 Processor -- start 2024-05-08T04:39:02.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 -- start start 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff40415ba70 0x7ff404160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff4040cfda0 0x7ff4040b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040cd9f0 con 0x7ff40415be50 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40415fe70 con 0x7ff40415ba70 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff40c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040cdcd0 con 0x7ff4040cfda0 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff409600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff40415ba70 0x7ff404160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff409600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff40415ba70 0x7ff404160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56420/0 (socket says 172.21.15.164:56420) 2024-05-08T04:39:02.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.534+0000 7ff409600640 1 -- 172.21.15.164:0/3263247961 learned_addr learned my addr 172.21.15.164:0/3263247961 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:02.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff409600640 1 -- 172.21.15.164:0/3263247961 >> v2:172.21.15.172:3300/0 conn(0x7ff4040cfda0 msgr2=0x7ff4040b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:02.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff409600640 1 --2- 172.21.15.164:0/3263247961 >> v2:172.21.15.172:3300/0 conn(0x7ff4040cfda0 0x7ff4040b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff409600640 1 -- 172.21.15.164:0/3263247961 >> v1:172.21.15.172:6789/0 conn(0x7ff40415be50 legacy=0x7ff4040cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:02.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff409600640 1 -- 172.21.15.164:0/3263247961 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40415f820 con 0x7ff40415ba70 2024-05-08T04:39:02.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff409600640 1 --2- 172.21.15.164:0/3263247961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415ba70 0x7ff404160300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40415fde0 tx=0x7ff3f801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2407460122ed678b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff403e00640 1 -- 172.21.15.164:0/3263247961 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80b1070 con 0x7ff40415ba70 2024-05-08T04:39:02.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff403e00640 1 -- 172.21.15.164:0/3263247961 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f80ac620 con 0x7ff40415ba70 2024-05-08T04:39:02.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.538+0000 7ff403e00640 1 -- 172.21.15.164:0/3263247961 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80ac9f0 con 0x7ff40415ba70 2024-05-08T04:39:02.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- 172.21.15.164:0/3263247961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415ba70 msgr2=0x7ff404160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- 172.21.15.164:0/3263247961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415ba70 0x7ff404160300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40415fde0 tx=0x7ff3f801c4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- 172.21.15.164:0/3263247961 shutdown_connections 2024-05-08T04:39:02.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- 172.21.15.164:0/3263247961 >> v2:172.21.15.172:3300/0 conn(0x7ff4040cfda0 0x7ff4040b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- 172.21.15.164:0/3263247961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415ba70 0x7ff404160300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- 172.21.15.164:0/3263247961 >> 172.21.15.164:0/3263247961 conn(0x7ff40406ac90 msgr2=0x7ff40406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:02.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- 172.21.15.164:0/3263247961 shutdown_connections 2024-05-08T04:39:02.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- 172.21.15.164:0/3263247961 wait complete. 2024-05-08T04:39:02.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 Processor -- start 2024-05-08T04:39:02.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- start start 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040cfda0 0x7ff404156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40415ba70 0x7ff404158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168d40 con 0x7ff40415be50 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168bc0 con 0x7ff4040cfda0 2024-05-08T04:39:02.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168ec0 con 0x7ff40415ba70 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56426/0 (socket says 172.21.15.164:56426) 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 -- 172.21.15.164:0/1994685143 learned_addr learned my addr 172.21.15.164:0/1994685143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 -- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40415ba70 msgr2=0x7ff404158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40415ba70 0x7ff404158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 -- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040cfda0 msgr2=0x7ff404156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040cfda0 0x7ff404156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 -- 172.21.15.164:0/1994685143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404155e30 con 0x7ff40415be50 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.542+0000 7ff40a000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b9a0 tx=0x7ff3fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc019070 con 0x7ff40415be50 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc00cbc0 con 0x7ff40415be50 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404156110 con 0x7ff40415be50 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc014440 con 0x7ff40415be50 2024-05-08T04:39:02.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041566d0 con 0x7ff40415be50 2024-05-08T04:39:02.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404069d60 con 0x7ff40415be50 2024-05-08T04:39:02.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3fc01c070 con 0x7ff40415be50 2024-05-08T04:39:02.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 0x7ff3e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3fc084dd0 con 0x7ff40415be50 2024-05-08T04:39:02.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff409600640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 0x7ff3e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.546+0000 7ff409600640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 0x7ff3e4069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ae000 tx=0x7ff3f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.550+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3fc055de0 con 0x7ff40415be50 2024-05-08T04:39:02.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.702+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff40406e210 con 0x7ff40415be50 2024-05-08T04:39:02.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff401600640 1 -- 172.21.15.164:0/1994685143 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7ff3fc086090 con 0x7ff40415be50 2024-05-08T04:39:02.709 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:02.709 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 msgr2=0x7ff3e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 0x7ff3e4069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ae000 tx=0x7ff3f8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 msgr2=0x7ff4040b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b9a0 tx=0x7ff3fc00be60 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 shutdown_connections 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4066f60 0x7ff3e4069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40415ba70 0x7ff404158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40415be50 0x7ff4040b0260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 --2- 172.21.15.164:0/1994685143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040cfda0 0x7ff404156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 >> 172.21.15.164:0/1994685143 conn(0x7ff40406ac90 msgr2=0x7ff40406d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:02.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 shutdown_connections 2024-05-08T04:39:02.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.706+0000 7ff40c000640 1 -- 172.21.15.164:0/1994685143 wait complete. 2024-05-08T04:39:02.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:02.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:02.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 Processor -- start 2024-05-08T04:39:02.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 -- start start 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6880cee90 0x7fb688165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68815e870 con 0x7fb68806cce0 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68815eb50 con 0x7fb68815ab10 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb68d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68806bb80 con 0x7fb6880cee90 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb67e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb68806cce0 0x7fb6880ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55654/0 (socket says 172.21.15.164:55654) 2024-05-08T04:39:02.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb67e000640 1 -- 172.21.15.164:0/356731188 learned_addr learned my addr 172.21.15.164:0/356731188 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 --2- 172.21.15.164:0/356731188 >> v2:172.21.15.172:3300/0 conn(0x7fb6880cee90 0x7fb688165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045217041 0 0) 0x7fb68815e870 con 0x7fb68806cce0 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb66c003530 con 0x7fb68806cce0 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286855688 0 0) 0x7fb68815eb50 con 0x7fb68815ab10 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb68815e870 con 0x7fb68815ab10 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189636109 0 0) 0x7fb66c003530 con 0x7fb68806cce0 2024-05-08T04:39:02.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 >> v2:172.21.15.172:3300/0 conn(0x7fb6880cee90 msgr2=0x7fb688165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 --2- 172.21.15.164:0/356731188 >> v2:172.21.15.172:3300/0 conn(0x7fb6880cee90 0x7fb688165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 >> v1:172.21.15.172:6789/0 conn(0x7fb68815ab10 legacy=0x7fb6880ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68815ee30 con 0x7fb68806cce0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fb670001600 con 0x7fb68806cce0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 >> v1:172.21.15.164:6789/0 conn(0x7fb68806cce0 legacy=0x7fb6880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb66c003ba0 0x7fb66c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66c004810 0x7fb66c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb66c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68806bb80 con 0x7fb66c005220 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c00e360 con 0x7fb66c004810 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c00e560 con 0x7fb66c003ba0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb67e000640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66c004810 0x7fb66c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb66c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb686a00640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb66c003ba0 0x7fb66c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 -- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb66c003ba0 msgr2=0x7fb66c003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb66c003ba0 0x7fb66c003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 -- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66c004810 msgr2=0x7fb66c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66c004810 0x7fb66c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.866+0000 7fb687400640 1 -- 172.21.15.164:0/356731188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c003530 con 0x7fb66c005220 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.870+0000 7fb687400640 1 -- 172.21.15.164:0/356731188 reap_dead start 2024-05-08T04:39:02.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.870+0000 7fb687400640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb66c00dbe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb680009f90 tx=0x7fb68001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d619ee472b5641 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800b2070 con 0x7fb66c005220 2024-05-08T04:39:02.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800a5c90 con 0x7fb66c005220 2024-05-08T04:39:02.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb686000640 1 -- 172.21.15.164:0/356731188 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680009c10 con 0x7fb66c005220 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 msgr2=0x7fb66c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb66c00dbe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb680009f90 tx=0x7fb68001cc80 comp rx=0 tx=0).stop 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- 172.21.15.164:0/356731188 shutdown_connections 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 --2- 172.21.15.164:0/356731188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb66c00dbe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- 172.21.15.164:0/356731188 >> 172.21.15.164:0/356731188 conn(0x7fb6880cf770 msgr2=0x7fb6880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- 172.21.15.164:0/356731188 shutdown_connections 2024-05-08T04:39:02.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- 172.21.15.164:0/356731188 wait complete. 2024-05-08T04:39:02.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 Processor -- start 2024-05-08T04:39:02.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.874+0000 7fb68d600640 1 -- start start 2024-05-08T04:39:02.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb688153a50 0x7fb6880addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688147420 0x7fb6880b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815aef0 con 0x7fb66c005220 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815f3f0 con 0x7fb688147420 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880ae6e0 con 0x7fb688153a50 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56442/0 (socket says 172.21.15.164:56442) 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 -- 172.21.15.164:0/1577426238 learned_addr learned my addr 172.21.15.164:0/1577426238 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb67e000640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb688153a50 0x7fb6880addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb687400640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688147420 0x7fb6880b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 -- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb688153a50 msgr2=0x7fb6880addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb688153a50 0x7fb6880addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 -- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688147420 msgr2=0x7fb6880b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688147420 0x7fb6880b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 -- 172.21.15.164:0/1577426238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6881529b0 con 0x7fb66c005220 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb686a00640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb678012830 tx=0x7fb678012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678010bd0 con 0x7fb66c005220 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67801b070 con 0x7fb66c005220 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678016480 con 0x7fb66c005220 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb688152c90 con 0x7fb66c005220 2024-05-08T04:39:02.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb688153250 con 0x7fb66c005220 2024-05-08T04:39:02.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb688072d30 con 0x7fb66c005220 2024-05-08T04:39:02.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.878+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb678016620 con 0x7fb66c005220 2024-05-08T04:39:02.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.882+0000 7fb67f400640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:02.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.882+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb678088620 con 0x7fb66c005220 2024-05-08T04:39:02.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.882+0000 7fb67e000640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:02.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.882+0000 7fb67e000640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 0x7fb664069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb670000c10 tx=0x7fb670001ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:02.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:02.886+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6780596c0 con 0x7fb66c005220 2024-05-08T04:39:03.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6880ce090 con 0x7fb66c005220 2024-05-08T04:39:03.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb67f400640 1 -- 172.21.15.164:0/1577426238 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb678058fc0 con 0x7fb66c005220 2024-05-08T04:39:03.045 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 msgr2=0x7fb664069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 0x7fb664069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb670000c10 tx=0x7fb670001ab0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 msgr2=0x7fb6880ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb678012830 tx=0x7fb678012cf0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 shutdown_connections 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb664066f60 0x7fb664069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb688153a50 0x7fb6880addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688147420 0x7fb6880b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 --2- 172.21.15.164:0/1577426238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c005220 0x7fb6880ad890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 >> 172.21.15.164:0/1577426238 conn(0x7fb6880cf770 msgr2=0x7fb68806db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 shutdown_connections 2024-05-08T04:39:03.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.042+0000 7fb68d600640 1 -- 172.21.15.164:0/1577426238 wait complete. 2024-05-08T04:39:03.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 Processor -- start 2024-05-08T04:39:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 -- start start 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 0x7f4fb80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb815e300 con 0x7f4fb80af6d0 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb815e570 con 0x7f4fb80adfa0 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fc0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80b5300 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbde00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4fb80af6d0 0x7f4fb80ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55670/0 (socket says 172.21.15.164:55670) 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbde00640 1 -- 172.21.15.164:0/2934857183 learned_addr learned my addr 172.21.15.164:0/2934857183 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbe800640 1 --2- 172.21.15.164:0/2934857183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 0x7f4fb80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974406564 0 0) 0x7f4fb815e300 con 0x7f4fb80af6d0 2024-05-08T04:39:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa0003530 con 0x7f4fb80af6d0 2024-05-08T04:39:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451552875 0 0) 0x7f4fb815e570 con 0x7f4fb80adfa0 2024-05-08T04:39:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb815e300 con 0x7f4fb80adfa0 2024-05-08T04:39:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbe800640 1 -- 172.21.15.164:0/2934857183 >> v1:172.21.15.172:6789/0 conn(0x7f4fb80adfa0 legacy=0x7f4fb806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbe800640 1 -- 172.21.15.164:0/2934857183 >> v1:172.21.15.164:6789/0 conn(0x7f4fb80af6d0 legacy=0x7f4fb80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.202+0000 7f4fbe800640 1 -- 172.21.15.164:0/2934857183 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb815e850 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fbe800640 1 --2- 172.21.15.164:0/2934857183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 0x7f4fb80b4dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009dc0 tx=0x7f4fb401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908f65bd2a857f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2070 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40a57c0 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fbca00640 1 -- 172.21.15.164:0/2934857183 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40a5b90 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 -- 172.21.15.164:0/2934857183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 msgr2=0x7f4fb80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2934857183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 0x7f4fb80b4dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009dc0 tx=0x7f4fb401c900 comp rx=0 tx=0).stop 2024-05-08T04:39:03.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 -- 172.21.15.164:0/2934857183 shutdown_connections 2024-05-08T04:39:03.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2934857183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80b2ff0 0x7f4fb80b4dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 -- 172.21.15.164:0/2934857183 >> 172.21.15.164:0/2934857183 conn(0x7f4fb806cca0 msgr2=0x7f4fb806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 -- 172.21.15.164:0/2934857183 shutdown_connections 2024-05-08T04:39:03.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.206+0000 7f4fc0800640 1 -- 172.21.15.164:0/2934857183 wait complete. 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 Processor -- start 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- start start 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80adfa0 0x7f4fb80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb80af6d0 0x7f4fb81497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816bcf0 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816bb70 con 0x7f4fb80af6d0 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816be70 con 0x7f4fb80adfa0 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56452/0 (socket says 172.21.15.164:56452) 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 -- 172.21.15.164:0/2818099883 learned_addr learned my addr 172.21.15.164:0/2818099883 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbd400640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb80af6d0 0x7f4fb81497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbde00640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80adfa0 0x7f4fb80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 -- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80adfa0 msgr2=0x7f4fb80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80adfa0 0x7f4fb80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 -- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb80af6d0 msgr2=0x7f4fb81497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb80af6d0 0x7f4fb81497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 -- 172.21.15.164:0/2818099883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80b0b30 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fbe800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4fb401a530 tx=0x7f4fb40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2050 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40be070 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4009450 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb80b0e10 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb80b13d0 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.210+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb806ddf0 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.214+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fb40ad8b0 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.214+0000 7f4fb2000640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.214+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fb411ff10 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.214+0000 7f4fbde00640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.214+0000 7f4fbde00640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 0x7f4f9c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f70 tx=0x7f4fac007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.218+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fb4016580 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.370+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fb80aef80 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.370+0000 7f4fb2000640 1 -- 172.21.15.164:0/2818099883 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fb40f0a10 con 0x7f4fb80b2ff0 2024-05-08T04:39:03.375 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 msgr2=0x7f4f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 0x7f4f9c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f70 tx=0x7f4fac007d40 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 msgr2=0x7f4fb8141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4fb401a530 tx=0x7f4fb40196a0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 shutdown_connections 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb80b2ff0 0x7f4fb8141130 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb80af6d0 0x7f4fb81497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 --2- 172.21.15.164:0/2818099883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fb80adfa0 0x7f4fb80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 >> 172.21.15.164:0/2818099883 conn(0x7f4fb806cca0 msgr2=0x7f4fb81510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 shutdown_connections 2024-05-08T04:39:03.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.374+0000 7f4fc0800640 1 -- 172.21.15.164:0/2818099883 wait complete. 2024-05-08T04:39:03.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.530+0000 7f6e58000640 1 Processor -- start 2024-05-08T04:39:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e58000640 1 -- start start 2024-05-08T04:39:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e58000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e5015ab10 0x7f6e5006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e58000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5015e870 con 0x7f6e500ae6e0 2024-05-08T04:39:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e58000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5015eb50 con 0x7f6e500cee90 2024-05-08T04:39:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e58000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5006bb80 con 0x7f6e5015ab10 2024-05-08T04:39:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6e500ae6e0 0x7f6e500ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55684/0 (socket says 172.21.15.164:55684) 2024-05-08T04:39:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 -- 172.21.15.164:0/3455610324 learned_addr learned my addr 172.21.15.164:0/3455610324 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:03.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237009600 0 0) 0x7f6e5015e870 con 0x7f6e500ae6e0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e38003530 con 0x7f6e500ae6e0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e55600640 1 --2- 172.21.15.164:0/3455610324 >> v2:172.21.15.172:3300/0 conn(0x7f6e5015ab10 0x7f6e5006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272511539 0 0) 0x7f6e38003530 con 0x7f6e500ae6e0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 >> v2:172.21.15.172:3300/0 conn(0x7f6e5015ab10 msgr2=0x7f6e5006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 --2- 172.21.15.164:0/3455610324 >> v2:172.21.15.172:3300/0 conn(0x7f6e5015ab10 0x7f6e5006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 >> v1:172.21.15.172:6789/0 conn(0x7f6e500cee90 legacy=0x7f6e500ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5015ee30 con 0x7f6e500ae6e0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f6e3c0016a0 con 0x7f6e500ae6e0 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 >> v1:172.21.15.164:6789/0 conn(0x7f6e500ae6e0 legacy=0x7f6e500ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e380039c0 0x7f6e38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 0x7f6e38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e380051d0 0x7f6e3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5006bb80 con 0x7f6e38004810 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3800c170 con 0x7f6e380051d0 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3800e580 con 0x7f6e380039c0 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 0x7f6e38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e55600640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e380039c0 0x7f6e38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e56000640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e380051d0 0x7f6e3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 -- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e380039c0 msgr2=0x7f6e38003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e380039c0 0x7f6e38003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 -- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e380051d0 msgr2=0x7f6e3800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e380051d0 0x7f6e3800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e54c00640 1 -- 172.21.15.164:0/3455610324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38003530 con 0x7f6e38004810 2024-05-08T04:39:03.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.534+0000 7f6e56000640 1 -- 172.21.15.164:0/3455610324 reap_dead start 2024-05-08T04:39:03.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e54c00640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 0x7f6e38004c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c290 tx=0x7f6e3c01ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aac816a9c3ab94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c007d60 con 0x7f6e38004810 2024-05-08T04:39:03.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0a9070 con 0x7f6e38004810 2024-05-08T04:39:03.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e4fe00640 1 -- 172.21.15.164:0/3455610324 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c005d40 con 0x7f6e38004810 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 -- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 msgr2=0x7f6e38004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 0x7f6e38004c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c290 tx=0x7f6e3c01ccd0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 -- 172.21.15.164:0/3455610324 shutdown_connections 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 --2- 172.21.15.164:0/3455610324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e38004810 0x7f6e38004c90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 -- 172.21.15.164:0/3455610324 >> 172.21.15.164:0/3455610324 conn(0x7f6e500cf770 msgr2=0x7f6e500cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 -- 172.21.15.164:0/3455610324 shutdown_connections 2024-05-08T04:39:03.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.538+0000 7f6e58000640 1 -- 172.21.15.164:0/3455610324 wait complete. 2024-05-08T04:39:03.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 Processor -- start 2024-05-08T04:39:03.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- start start 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e38004810 0x7f6e5014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5014de40 0x7f6e500c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500c5b70 con 0x7f6e501fb650 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5015aef0 con 0x7f6e38004810 2024-05-08T04:39:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500c5cf0 con 0x7f6e5014de40 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e55600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e38004810 0x7f6e5014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56472/0 (socket says 172.21.15.164:56472) 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 -- 172.21.15.164:0/3215307337 learned_addr learned my addr 172.21.15.164:0/3215307337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e54c00640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5014de40 0x7f6e500c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 -- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5014de40 msgr2=0x7f6e500c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5014de40 0x7f6e500c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 -- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e38004810 msgr2=0x7f6e5014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e38004810 0x7f6e5014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 -- 172.21.15.164:0/3215307337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500c9110 con 0x7f6e501fb650 2024-05-08T04:39:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e56000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c9d70 tx=0x7f6e4800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801a070 con 0x7f6e501fb650 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48006d70 con 0x7f6e501fb650 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e500c93f0 con 0x7f6e501fb650 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48003bc0 con 0x7f6e501fb650 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e501fb130 con 0x7f6e501fb650 2024-05-08T04:39:03.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.542+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e501fb330 con 0x7f6e501fb650 2024-05-08T04:39:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.546+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e4801d070 con 0x7f6e501fb650 2024-05-08T04:39:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.546+0000 7f6e4d600640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 0x7f6e30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.546+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e480844c0 con 0x7f6e501fb650 2024-05-08T04:39:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.546+0000 7f6e55600640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 0x7f6e30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.546+0000 7f6e55600640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 0x7f6e30069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e500736e0 tx=0x7f6e44009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.550+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e501fb330 con 0x7f6e501fb650 2024-05-08T04:39:03.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.702+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e50072d30 con 0x7f6e501fb650 2024-05-08T04:39:03.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.702+0000 7f6e4d600640 1 -- 172.21.15.164:0/3215307337 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e48054e70 con 0x7f6e501fb650 2024-05-08T04:39:03.707 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:03.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 msgr2=0x7f6e30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 0x7f6e30069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e500736e0 tx=0x7f6e44009190 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 msgr2=0x7f6e500ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c9d70 tx=0x7f6e4800fbb0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 shutdown_connections 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e30066f60 0x7f6e30069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5014de40 0x7f6e500c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e501fb650 0x7f6e500ad890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 --2- 172.21.15.164:0/3215307337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e38004810 0x7f6e5014d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 >> 172.21.15.164:0/3215307337 conn(0x7f6e500cf770 msgr2=0x7f6e500cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 shutdown_connections 2024-05-08T04:39:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.706+0000 7f6e58000640 1 -- 172.21.15.164:0/3215307337 wait complete. 2024-05-08T04:39:03.729 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 Processor -- start 2024-05-08T04:39:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 -- start start 2024-05-08T04:39:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b1815bc50 0x7f7b180b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b180b42a0 con 0x7f7b180ccfe0 2024-05-08T04:39:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b180cd4e0 con 0x7f7b1815bc50 2024-05-08T04:39:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b1ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180cd7c0 con 0x7f7b180d0780 2024-05-08T04:39:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b17e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b1815bc50 0x7f7b180b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b17400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7b180ccfe0 0x7f7b180d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55692/0 (socket says 172.21.15.164:55692) 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.862+0000 7f7b17400640 1 -- 172.21.15.164:0/2672050547 learned_addr learned my addr 172.21.15.164:0/2672050547 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b1ce00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b17e00640 1 -- 172.21.15.164:0/2672050547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 msgr2=0x7f7b1815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b17e00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b17e00640 1 -- 172.21.15.164:0/2672050547 >> v1:172.21.15.164:6789/0 conn(0x7f7b180ccfe0 legacy=0x7f7b180d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b17e00640 1 -- 172.21.15.164:0/2672050547 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180cf590 con 0x7f7b1815bc50 2024-05-08T04:39:03.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b17e00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b1815bc50 0x7f7b180b4810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097c0 tx=0x7f7b0801c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5fa90dbd2a1a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b16a00640 1 -- 172.21.15.164:0/2672050547 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b0070 con 0x7f7b1815bc50 2024-05-08T04:39:03.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b16a00640 1 -- 172.21.15.164:0/2672050547 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b080ab600 con 0x7f7b1815bc50 2024-05-08T04:39:03.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.866+0000 7f7b16a00640 1 -- 172.21.15.164:0/2672050547 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080ab9d0 con 0x7f7b1815bc50 2024-05-08T04:39:03.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- 172.21.15.164:0/2672050547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b1815bc50 msgr2=0x7f7b180b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b1815bc50 0x7f7b180b4810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097c0 tx=0x7f7b0801c270 comp rx=0 tx=0).stop 2024-05-08T04:39:03.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- 172.21.15.164:0/2672050547 shutdown_connections 2024-05-08T04:39:03.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/2672050547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b1815bc50 0x7f7b180b4810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- 172.21.15.164:0/2672050547 >> 172.21.15.164:0/2672050547 conn(0x7f7b180b34f0 msgr2=0x7f7b180b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:03.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- 172.21.15.164:0/2672050547 shutdown_connections 2024-05-08T04:39:03.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- 172.21.15.164:0/2672050547 wait complete. 2024-05-08T04:39:03.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 Processor -- start 2024-05-08T04:39:03.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- start start 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 0x7f7b1815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18168c60 con 0x7f7b180ccfe0 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18168ae0 con 0x7f7b1815bc50 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18168de0 con 0x7f7b180d0780 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56494/0 (socket says 172.21.15.164:56494) 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 -- 172.21.15.164:0/377294060 learned_addr learned my addr 172.21.15.164:0/377294060 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ce00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 0x7f7b1815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17400640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 -- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 msgr2=0x7f7b1806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 -- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 msgr2=0x7f7b1815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 0x7f7b1815a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17e00640 1 -- 172.21.15.164:0/377294060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18073210 con 0x7f7b180ccfe0 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b17400640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.870+0000 7f7b1ce00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 0x7f7b1815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:03.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b17e00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b080ae040 tx=0x7f7b0801aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b0050 con 0x7f7b180ccfe0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b080bc070 con 0x7f7b180ccfe0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1806d9a0 con 0x7f7b180ccfe0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b5850 con 0x7f7b180ccfe0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b18164ca0 con 0x7f7b180ccfe0 2024-05-08T04:39:03.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.874+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b18069d60 con 0x7f7b180ccfe0 2024-05-08T04:39:03.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.878+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b080b59f0 con 0x7f7b180ccfe0 2024-05-08T04:39:03.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.878+0000 7f7b0fe00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 0x7f7af8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:03.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.878+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b0811f9e0 con 0x7f7b180ccfe0 2024-05-08T04:39:03.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.878+0000 7f7b17400640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 0x7f7af8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:03.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.878+0000 7f7b17400640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 0x7f7af8069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806e670 tx=0x7f7b04007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:03.882+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b08015fb0 con 0x7f7b180ccfe0 2024-05-08T04:39:04.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.046+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b18158750 con 0x7f7b180ccfe0 2024-05-08T04:39:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.046+0000 7f7b0fe00640 1 -- 172.21.15.164:0/377294060 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7b080f0950 con 0x7f7b180ccfe0 2024-05-08T04:39:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 msgr2=0x7f7af8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 0x7f7af8069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b1806e670 tx=0x7f7b04007d30 comp rx=0 tx=0).stop 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 msgr2=0x7f7b181520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b080ae040 tx=0x7f7b0801aab0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 shutdown_connections 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7af8066f60 0x7f7af8069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b180d0780 0x7f7b1806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1815bc50 0x7f7b1815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 --2- 172.21.15.164:0/377294060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b180ccfe0 0x7f7b181520e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 >> 172.21.15.164:0/377294060 conn(0x7f7b180b34f0 msgr2=0x7f7b18153b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:04.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 shutdown_connections 2024-05-08T04:39:04.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.050+0000 7f7b1ee00640 1 -- 172.21.15.164:0/377294060 wait complete. 2024-05-08T04:39:04.074 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:39:04.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:04.074 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-08T04:39:04.117 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:39:04.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:04.117 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:39:04.173 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:04.173 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:04.182 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:04.182 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:04.231 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:04.231 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:04.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:04.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.454+0000 7fd109c00640 1 Processor -- start 2024-05-08T04:39:04.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.454+0000 7fd109c00640 1 -- start start 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd109c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd10415dd70 0x7fd1041619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd109c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10406cc70 con 0x7fd10406c7b0 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd109c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10406cf50 con 0x7fd1041625f0 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd109c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040b1570 con 0x7fd10415dd70 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd10406c7b0 0x7fd104161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55696/0 (socket says 172.21.15.164:55696) 2024-05-08T04:39:04.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 -- 172.21.15.164:0/2421464248 learned_addr learned my addr 172.21.15.164:0/2421464248 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:04.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102a00640 1 --2- 172.21.15.164:0/2421464248 >> v2:172.21.15.172:3300/0 conn(0x7fd10415dd70 0x7fd1041619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078207011 0 0) 0x7fd10406cc70 con 0x7fd10406c7b0 2024-05-08T04:39:04.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e8003530 con 0x7fd10406c7b0 2024-05-08T04:39:04.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021074503 0 0) 0x7fd0e8003530 con 0x7fd10406c7b0 2024-05-08T04:39:04.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 >> v2:172.21.15.172:3300/0 conn(0x7fd10415dd70 msgr2=0x7fd1041619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 --2- 172.21.15.164:0/2421464248 >> v2:172.21.15.172:3300/0 conn(0x7fd10415dd70 0x7fd1041619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 >> v1:172.21.15.172:6789/0 conn(0x7fd1041625f0 legacy=0x7fd1040b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040b1850 con 0x7fd10406c7b0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd0ec001680 con 0x7fd10406c7b0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 >> v1:172.21.15.164:6789/0 conn(0x7fd10406c7b0 legacy=0x7fd104161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0e80039c0 0x7fd0e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd0e8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0e8005090 0x7fd0e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040b1570 con 0x7fd0e80043c0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e800bff0 con 0x7fd0e80039c0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e800c1f0 con 0x7fd0e8005090 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd0e8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd102a00640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0e80039c0 0x7fd0e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd103400640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0e8005090 0x7fd0e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 -- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0e8005090 msgr2=0x7fd0e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0e8005090 0x7fd0e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 -- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0e80039c0 msgr2=0x7fd0e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0e80039c0 0x7fd0e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd0fa000640 1 -- 172.21.15.164:0/2421464248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8003530 con 0x7fd0e80043c0 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.458+0000 7fd103400640 1 -- 172.21.15.164:0/2421464248 reap_dead start 2024-05-08T04:39:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd0fa000640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd0e8004890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c2c0 tx=0x7fd0ec01cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2bd2505b4c6988 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec007d60 con 0x7fd0e80043c0 2024-05-08T04:39:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec0a9070 con 0x7fd0e80043c0 2024-05-08T04:39:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd102000640 1 -- 172.21.15.164:0/2421464248 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0a5e30 con 0x7fd0e80043c0 2024-05-08T04:39:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 -- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 msgr2=0x7fd0e8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd0e8004890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c2c0 tx=0x7fd0ec01cd20 comp rx=0 tx=0).stop 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 -- 172.21.15.164:0/2421464248 shutdown_connections 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 --2- 172.21.15.164:0/2421464248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd0e8004890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 -- 172.21.15.164:0/2421464248 >> 172.21.15.164:0/2421464248 conn(0x7fd1040b34f0 msgr2=0x7fd1040b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 -- 172.21.15.164:0/2421464248 shutdown_connections 2024-05-08T04:39:04.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 -- 172.21.15.164:0/2421464248 wait complete. 2024-05-08T04:39:04.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.462+0000 7fd109c00640 1 Processor -- start 2024-05-08T04:39:04.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- start start 2024-05-08T04:39:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd104151e50 0x7fd1040b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 0x7fd10415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10415e2d0 con 0x7fd0e80043c0 2024-05-08T04:39:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10415e150 con 0x7fd104151e50 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041619a0 con 0x7fd1040b6f20 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56506/0 (socket says 172.21.15.164:56506) 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 -- 172.21.15.164:0/3779918921 learned_addr learned my addr 172.21.15.164:0/3779918921 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd0fa000640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd104151e50 0x7fd1040b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd103400640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 0x7fd10415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 -- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 msgr2=0x7fd10415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 0x7fd10415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 -- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd104151e50 msgr2=0x7fd1040b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd104151e50 0x7fd1040b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 -- 172.21.15.164:0/3779918921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040c54c0 con 0x7fd0e80043c0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd103400640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 0x7fd10415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd102a00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4011a20 tx=0x7fd0f4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400fbb0 con 0x7fd0e80043c0 2024-05-08T04:39:04.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400fd50 con 0x7fd0e80043c0 2024-05-08T04:39:04.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4014640 con 0x7fd0e80043c0 2024-05-08T04:39:04.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1040c57a0 con 0x7fd0e80043c0 2024-05-08T04:39:04.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1040c5d60 con 0x7fd0e80043c0 2024-05-08T04:39:04.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.466+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104069d60 con 0x7fd0e80043c0 2024-05-08T04:39:04.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.470+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0f4012040 con 0x7fd0e80043c0 2024-05-08T04:39:04.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.470+0000 7fd0fb400640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 0x7fd0e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.470+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f4085840 con 0x7fd0e80043c0 2024-05-08T04:39:04.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.470+0000 7fd0fa000640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 0x7fd0e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.470+0000 7fd0fa000640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 0x7fd0e0069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0ab000 tx=0x7fd0ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.474+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0f4019030 con 0x7fd0e80043c0 2024-05-08T04:39:04.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.630+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1040b4010 con 0x7fd0e80043c0 2024-05-08T04:39:04.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.630+0000 7fd0fb400640 1 -- 172.21.15.164:0/3779918921 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7fd0f40567c0 con 0x7fd0e80043c0 2024-05-08T04:39:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:04.636 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:04.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 msgr2=0x7fd0e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 0x7fd0e0069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0ab000 tx=0x7fd0ec0a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:04.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 msgr2=0x7fd1041f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4011a20 tx=0x7fd0f4011ee0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 shutdown_connections 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f60 0x7fd0e0069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1040b6f20 0x7fd10415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd104151e50 0x7fd1040b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 --2- 172.21.15.164:0/3779918921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0e80043c0 0x7fd1041f4c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 >> 172.21.15.164:0/3779918921 conn(0x7fd1040b34f0 msgr2=0x7fd10406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 shutdown_connections 2024-05-08T04:39:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.634+0000 7fd109c00640 1 -- 172.21.15.164:0/3779918921 wait complete. 2024-05-08T04:39:04.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:04.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:04.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 Processor -- start 2024-05-08T04:39:04.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 -- start start 2024-05-08T04:39:04.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12d8144360 0x7f12d80c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d80adc50 con 0x7f12d806ac90 2024-05-08T04:39:04.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80ae860 con 0x7f12d8144360 2024-05-08T04:39:04.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12dda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8144740 con 0x7f12d80cb290 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:51444/0 (socket says 172.21.15.164:51444) 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d7400640 1 -- 172.21.15.164:0/2700832241 learned_addr learned my addr 172.21.15.164:0/2700832241 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6000640 1 --1- 172.21.15.164:0/2700832241 >> v1:172.21.15.172:6789/0 conn(0x7f12d806ac90 0x7f12d81619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:57860/0 (socket says 172.21.15.164:57860) 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6a00640 1 --2- 172.21.15.164:0/2700832241 >> v2:172.21.15.172:3300/0 conn(0x7f12d8144360 0x7f12d80c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6a00640 1 -- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 msgr2=0x7f12d806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:04.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6a00640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6a00640 1 -- 172.21.15.164:0/2700832241 >> v1:172.21.15.172:6789/0 conn(0x7f12d806ac90 legacy=0x7f12d81619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d6a00640 1 -- 172.21.15.164:0/2700832241 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80adf30 con 0x7f12d8144360 2024-05-08T04:39:04.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.794+0000 7f12d7400640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12d6a00640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d8144360 0x7f12d80c5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097c0 tx=0x7f12c801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e8e4f96a81af1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12d5600640 1 -- 172.21.15.164:0/2700832241 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80b0070 con 0x7f12d8144360 2024-05-08T04:39:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12d5600640 1 -- 172.21.15.164:0/2700832241 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c80a4e50 con 0x7f12d8144360 2024-05-08T04:39:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12d5600640 1 -- 172.21.15.164:0/2700832241 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80ab820 con 0x7f12d8144360 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 -- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d8144360 msgr2=0x7f12d80c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d8144360 0x7f12d80c5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097c0 tx=0x7f12c801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 -- 172.21.15.164:0/2700832241 shutdown_connections 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d80cb290 0x7f12d806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 --2- 172.21.15.164:0/2700832241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d8144360 0x7f12d80c5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 -- 172.21.15.164:0/2700832241 >> 172.21.15.164:0/2700832241 conn(0x7f12d80cf770 msgr2=0x7f12d80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 -- 172.21.15.164:0/2700832241 shutdown_connections 2024-05-08T04:39:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.798+0000 7f12dda00640 1 -- 172.21.15.164:0/2700832241 wait complete. 2024-05-08T04:39:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 Processor -- start 2024-05-08T04:39:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- start start 2024-05-08T04:39:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 0x7f12d81fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d8144360 0x7f12d80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80c4be0 con 0x7f12d80cb290 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80c4a60 con 0x7f12d806ac90 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d816ad70 con 0x7f12d8144360 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 0x7f12d81fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56510/0 (socket says 172.21.15.164:56510) 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 0x7f12d81fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:57850/0 (socket says 172.21.15.164:57850) 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 -- 172.21.15.164:0/2885960524 learned_addr learned my addr 172.21.15.164:0/2885960524 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d7400640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d8144360 0x7f12d80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 -- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d8144360 msgr2=0x7f12d80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d8144360 0x7f12d80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 -- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 msgr2=0x7f12d81fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 0x7f12d81fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 -- 172.21.15.164:0/2885960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d814e850 con 0x7f12d80cb290 2024-05-08T04:39:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12d6000640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c370 tx=0x7f12c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40074e0 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4007680 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d80c7420 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4004a30 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d80c83e0 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.802+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d8072d30 con 0x7f12d80cb290 2024-05-08T04:39:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.806+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12c401a020 con 0x7f12d80cb290 2024-05-08T04:39:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.806+0000 7f12cea00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 0x7f12b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.806+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c40846c0 con 0x7f12d80cb290 2024-05-08T04:39:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.806+0000 7f12d6a00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 0x7f12b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.806+0000 7f12d6a00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 0x7f12b8069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12c80ad000 tx=0x7f12c80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.810+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12d814e850 con 0x7f12d80cb290 2024-05-08T04:39:04.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12d80aee20 con 0x7f12d80cb290 2024-05-08T04:39:04.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12cea00640 1 -- 172.21.15.164:0/2885960524 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12c4055070 con 0x7f12d80cb290 2024-05-08T04:39:04.970 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 msgr2=0x7f12b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 0x7f12b8069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12c80ad000 tx=0x7f12c80a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 msgr2=0x7f12d80c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c370 tx=0x7f12c400c830 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 shutdown_connections 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f12b8066f60 0x7f12b8069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12d8144360 0x7f12d80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d80cb290 0x7f12d80c8be0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 --2- 172.21.15.164:0/2885960524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12d806ac90 0x7f12d81fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:04.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 >> 172.21.15.164:0/2885960524 conn(0x7f12d80cf770 msgr2=0x7f12d80b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:04.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 shutdown_connections 2024-05-08T04:39:04.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:04.966+0000 7f12dda00640 1 -- 172.21.15.164:0/2885960524 wait complete. 2024-05-08T04:39:04.991 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:05.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 Processor -- start 2024-05-08T04:39:05.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 -- start start 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37f015ab10 0x7f37f006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f015e870 con 0x7f37f00ce780 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f006bb80 con 0x7f37f015ab10 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37f7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f015eb50 con 0x7f37f00ae6e0 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37f015ab10 0x7f37f006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37f015ab10 0x7f37f006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56512/0 (socket says 172.21.15.164:56512) 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 -- 172.21.15.164:0/1511129202 learned_addr learned my addr 172.21.15.164:0/1511129202 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37e7400640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 -- 172.21.15.164:0/1511129202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 msgr2=0x7f37f00aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f00aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 -- 172.21.15.164:0/1511129202 >> v1:172.21.15.172:6789/0 conn(0x7f37f00ce780 legacy=0x7f37f00c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:05.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.126+0000 7f37efe00640 1 -- 172.21.15.164:0/1511129202 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f015ee30 con 0x7f37f015ab10 2024-05-08T04:39:05.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37efe00640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f015ab10 0x7f37f006cce0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fb2ef3f9a3bbe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37ef400640 1 -- 172.21.15.164:0/1511129202 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00aa600 con 0x7f37f015ab10 2024-05-08T04:39:05.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37ef400640 1 -- 172.21.15.164:0/1511129202 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e00af070 con 0x7f37f015ab10 2024-05-08T04:39:05.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37ef400640 1 -- 172.21.15.164:0/1511129202 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00aa9c0 con 0x7f37f015ab10 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 -- 172.21.15.164:0/1511129202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f015ab10 msgr2=0x7f37f006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f015ab10 0x7f37f006cce0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009870 tx=0x7f37e001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 -- 172.21.15.164:0/1511129202 shutdown_connections 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f00aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 --2- 172.21.15.164:0/1511129202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f015ab10 0x7f37f006cce0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 -- 172.21.15.164:0/1511129202 >> 172.21.15.164:0/1511129202 conn(0x7f37f00cf770 msgr2=0x7f37f00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 -- 172.21.15.164:0/1511129202 shutdown_connections 2024-05-08T04:39:05.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.130+0000 7f37f7000640 1 -- 172.21.15.164:0/1511129202 wait complete. 2024-05-08T04:39:05.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 Processor -- start 2024-05-08T04:39:05.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- start start 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37f015ab10 0x7f37f00c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00cefa0 con 0x7f37f00ce780 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0167ae0 con 0x7f37f015ab10 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01521b0 con 0x7f37f00ae6e0 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56526/0 (socket says 172.21.15.164:56526) 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 -- 172.21.15.164:0/1669683227 learned_addr learned my addr 172.21.15.164:0/1669683227 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f5000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37f015ab10 0x7f37f00c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37efe00640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 -- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 msgr2=0x7f37f014c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f014c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 -- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37f015ab10 msgr2=0x7f37f00c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37f015ab10 0x7f37f00c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 -- 172.21.15.164:0/1669683227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f01515e0 con 0x7f37f00ce780 2024-05-08T04:39:05.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37e7400640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37d800d5f0 tx=0x7f37d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d80125f0 con 0x7f37f00ce780 2024-05-08T04:39:05.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8019070 con 0x7f37f00ce780 2024-05-08T04:39:05.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d80129b0 con 0x7f37f00ce780 2024-05-08T04:39:05.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f014ca70 con 0x7f37f00ce780 2024-05-08T04:39:05.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f0151b60 con 0x7f37f00ce780 2024-05-08T04:39:05.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.134+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f0072d30 con 0x7f37f00ce780 2024-05-08T04:39:05.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.138+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37d8012b50 con 0x7f37f00ce780 2024-05-08T04:39:05.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.138+0000 7f37ecc00640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 0x7f37cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.138+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37d8085280 con 0x7f37f00ce780 2024-05-08T04:39:05.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.138+0000 7f37efe00640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 0x7f37cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.138+0000 7f37efe00640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 0x7f37cc069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ae000 tx=0x7f37e00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.142+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37d8055c30 con 0x7f37f00ce780 2024-05-08T04:39:05.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.294+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37f004f010 con 0x7f37f00ce780 2024-05-08T04:39:05.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.294+0000 7f37ecc00640 1 -- 172.21.15.164:0/1669683227 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37d8059cd0 con 0x7f37f00ce780 2024-05-08T04:39:05.298 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:05.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.294+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 msgr2=0x7f37cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.294+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 0x7f37cc069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ae000 tx=0x7f37e00a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.294+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 msgr2=0x7f37f0152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37d800d5f0 tx=0x7f37d800dab0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 shutdown_connections 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f37cc066f90 0x7f37cc069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37f015ab10 0x7f37f00c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f00ce780 0x7f37f0152640 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 --2- 172.21.15.164:0/1669683227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37f00ae6e0 0x7f37f014c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 >> 172.21.15.164:0/1669683227 conn(0x7f37f00cf770 msgr2=0x7f37f00cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 shutdown_connections 2024-05-08T04:39:05.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.298+0000 7f37f7000640 1 -- 172.21.15.164:0/1669683227 wait complete. 2024-05-08T04:39:05.319 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:05.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.454+0000 7fae07200640 1 Processor -- start 2024-05-08T04:39:05.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.454+0000 7fae07200640 1 -- start start 2024-05-08T04:39:05.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae07200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fae080b1960 0x7fae08071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae07200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 0x7fae080ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae07200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0815c900 con 0x7fae080cec40 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae07200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0815cec0 con 0x7fae080b1960 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae07200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0815c6a0 con 0x7fae080b1d40 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fae080b1960 0x7fae08071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fae080b1960 0x7fae08071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56528/0 (socket says 172.21.15.164:56528) 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 -- 172.21.15.164:0/3614445468 learned_addr learned my addr 172.21.15.164:0/3614445468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae06800640 1 --2- 172.21.15.164:0/3614445468 >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 0x7fae080ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 -- 172.21.15.164:0/3614445468 >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 msgr2=0x7fae080ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 --2- 172.21.15.164:0/3614445468 >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 0x7fae080ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 -- 172.21.15.164:0/3614445468 >> v1:172.21.15.172:6789/0 conn(0x7fae080cec40 legacy=0x7fae0815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 -- 172.21.15.164:0/3614445468 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0815cbe0 con 0x7fae080b1960 2024-05-08T04:39:05.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae06800640 1 --2- 172.21.15.164:0/3614445468 >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 0x7fae080ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.458+0000 7fae05e00640 1 --2- 172.21.15.164:0/3614445468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080b1960 0x7fae08071aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae08072460 tx=0x7fadf801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7428563a37fd624 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae04a00640 1 -- 172.21.15.164:0/3614445468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80a5630 con 0x7fae080b1960 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae04a00640 1 -- 172.21.15.164:0/3614445468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80a57d0 con 0x7fae080b1960 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae04a00640 1 -- 172.21.15.164:0/3614445468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80a5ba0 con 0x7fae080b1960 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- 172.21.15.164:0/3614445468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080b1960 msgr2=0x7fae08071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- 172.21.15.164:0/3614445468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080b1960 0x7fae08071aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae08072460 tx=0x7fadf801c860 comp rx=0 tx=0).stop 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- 172.21.15.164:0/3614445468 shutdown_connections 2024-05-08T04:39:05.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- 172.21.15.164:0/3614445468 >> v2:172.21.15.172:3300/0 conn(0x7fae080b1d40 0x7fae080ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- 172.21.15.164:0/3614445468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080b1960 0x7fae08071aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- 172.21.15.164:0/3614445468 >> 172.21.15.164:0/3614445468 conn(0x7fae0806ac90 msgr2=0x7fae0806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- 172.21.15.164:0/3614445468 shutdown_connections 2024-05-08T04:39:05.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- 172.21.15.164:0/3614445468 wait complete. 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 Processor -- start 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- start start 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 0x7fae0813ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae080b1d40 0x7fae08156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080b3bf0 con 0x7fae080cec40 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08071fe0 con 0x7fae080b1960 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae07200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae080b3d70 con 0x7fae080b1d40 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae06800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae06800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56538/0 (socket says 172.21.15.164:56538) 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae06800640 1 -- 172.21.15.164:0/247562793 learned_addr learned my addr 172.21.15.164:0/247562793 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae05e00640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 0x7fae0813ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.462+0000 7fae05400640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae080b1d40 0x7fae08156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 -- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae080b1d40 msgr2=0x7fae08156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae080b1d40 0x7fae08156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 -- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 msgr2=0x7fae0813ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 0x7fae0813ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 -- 172.21.15.164:0/247562793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08160470 con 0x7fae080cec40 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae05e00640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 0x7fae0813ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae06800640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c7f0 tx=0x7fae0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00015070 con 0x7fae080cec40 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae00007860 con 0x7fae080cec40 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00007c50 con 0x7fae080cec40 2024-05-08T04:39:05.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae08157a20 con 0x7fae080cec40 2024-05-08T04:39:05.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae08157fe0 con 0x7fae080cec40 2024-05-08T04:39:05.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0804ef30 con 0x7fae080cec40 2024-05-08T04:39:05.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae00018070 con 0x7fae080cec40 2024-05-08T04:39:05.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 0x7fade8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae000851e0 con 0x7fae080cec40 2024-05-08T04:39:05.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.466+0000 7fae05e00640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 0x7fade8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.470+0000 7fae05e00640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 0x7fade8069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadf80ae000 tx=0x7fadf80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.470+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae000561e0 con 0x7fae080cec40 2024-05-08T04:39:05.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae080b2ef0 con 0x7fae080cec40 2024-05-08T04:39:05.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fadfe000640 1 -- 172.21.15.164:0/247562793 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae00055ae0 con 0x7fae080cec40 2024-05-08T04:39:05.629 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 msgr2=0x7fade8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 0x7fade8069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadf80ae000 tx=0x7fadf80a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 msgr2=0x7fae08150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c7f0 tx=0x7fae0000ccb0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 shutdown_connections 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fade8066f60 0x7fade8069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae080b1d40 0x7fae08156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae080cec40 0x7fae08150cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 --2- 172.21.15.164:0/247562793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080b1960 0x7fae0813ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 >> 172.21.15.164:0/247562793 conn(0x7fae0806ac90 msgr2=0x7fae080ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 shutdown_connections 2024-05-08T04:39:05.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.626+0000 7fae07200640 1 -- 172.21.15.164:0/247562793 wait complete. 2024-05-08T04:39:05.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:05.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 Processor -- start 2024-05-08T04:39:05.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 -- start start 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdb6815cd80 0x7fdb680b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb680cc350 con 0x7fdb6809ae30 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb680d2110 con 0x7fdb680ce110 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6e200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb680d1ec0 con 0x7fdb6815cd80 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdb6815cd80 0x7fdb680b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb67e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdb6809ae30 0x7fdb6815d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55706/0 (socket says 172.21.15.164:55706) 2024-05-08T04:39:05.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb67e00640 1 -- 172.21.15.164:0/3561084996 learned_addr learned my addr 172.21.15.164:0/3561084996 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:05.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6ce00640 1 -- 172.21.15.164:0/3561084996 >> v1:172.21.15.172:6789/0 conn(0x7fdb680ce110 legacy=0x7fdb680ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:05.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6ce00640 1 -- 172.21.15.164:0/3561084996 >> v1:172.21.15.164:6789/0 conn(0x7fdb6809ae30 legacy=0x7fdb6815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb6ce00640 1 -- 172.21.15.164:0/3561084996 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb680d23f0 con 0x7fdb6815cd80 2024-05-08T04:39:05.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.782+0000 7fdb67400640 1 -- 172.21.15.164:0/3561084996 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271887650 0 0) 0x7fdb680cc350 con 0x7fdb6809ae30 2024-05-08T04:39:05.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6ce00640 1 --2- 172.21.15.164:0/3561084996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb6815cd80 0x7fdb680b06b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb580097c0 tx=0x7fdb5801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94643b622f9138e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb67400640 1 -- 172.21.15.164:0/3561084996 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580b0070 con 0x7fdb6815cd80 2024-05-08T04:39:05.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb67400640 1 -- 172.21.15.164:0/3561084996 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb580ab600 con 0x7fdb6815cd80 2024-05-08T04:39:05.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb67400640 1 -- 172.21.15.164:0/3561084996 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580ab9d0 con 0x7fdb6815cd80 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 -- 172.21.15.164:0/3561084996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb6815cd80 msgr2=0x7fdb680b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 --2- 172.21.15.164:0/3561084996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb6815cd80 0x7fdb680b06b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb580097c0 tx=0x7fdb5801c460 comp rx=0 tx=0).stop 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 -- 172.21.15.164:0/3561084996 shutdown_connections 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 --2- 172.21.15.164:0/3561084996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb6815cd80 0x7fdb680b06b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 -- 172.21.15.164:0/3561084996 >> 172.21.15.164:0/3561084996 conn(0x7fdb6809aa20 msgr2=0x7fdb680d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 -- 172.21.15.164:0/3561084996 shutdown_connections 2024-05-08T04:39:05.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.786+0000 7fdb6e200640 1 -- 172.21.15.164:0/3561084996 wait complete. 2024-05-08T04:39:05.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 Processor -- start 2024-05-08T04:39:05.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- start start 2024-05-08T04:39:05.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb6809ae30 0x7fdb680b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb6815cd80 0x7fdb68159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680cc960 con 0x7fdb680ce110 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6809b2a0 con 0x7fdb6809ae30 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6815d160 con 0x7fdb6815cd80 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56558/0 (socket says 172.21.15.164:56558) 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 -- 172.21.15.164:0/500034434 learned_addr learned my addr 172.21.15.164:0/500034434 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6ce00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb6809ae30 0x7fdb680b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6d800640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb6815cd80 0x7fdb68159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 -- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb6815cd80 msgr2=0x7fdb68159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb6815cd80 0x7fdb68159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 -- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb6809ae30 msgr2=0x7fdb680b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb6809ae30 0x7fdb680b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 -- 172.21.15.164:0/500034434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6806c2a0 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb67e00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c520 tx=0x7fdb5400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54018070 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb540074e0 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540049f0 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6806c580 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.790+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6806cb40 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6804ef30 con 0x7fdb680ce110 2024-05-08T04:39:05.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb54004b90 con 0x7fdb680ce110 2024-05-08T04:39:05.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb64c00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 0x7fdb48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:05.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb54084300 con 0x7fdb680ce110 2024-05-08T04:39:05.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb6ce00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 0x7fdb48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:05.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.794+0000 7fdb6ce00640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 0x7fdb48069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb580ad000 tx=0x7fdb580a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:05.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.798+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb54054cb0 con 0x7fdb680ce110 2024-05-08T04:39:05.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb680cf000 con 0x7fdb680ce110 2024-05-08T04:39:05.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb64c00640 1 -- 172.21.15.164:0/500034434 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdb54058d50 con 0x7fdb680ce110 2024-05-08T04:39:05.974 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:05.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 msgr2=0x7fdb48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 0x7fdb48069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb580ad000 tx=0x7fdb580a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:05.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 msgr2=0x7fdb68153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c520 tx=0x7fdb5400c9e0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 shutdown_connections 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb48066f60 0x7fdb48069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb6815cd80 0x7fdb68159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb680ce110 0x7fdb68153170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 --2- 172.21.15.164:0/500034434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb6809ae30 0x7fdb680b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:05.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 >> 172.21.15.164:0/500034434 conn(0x7fdb6809aa20 msgr2=0x7fdb680c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:05.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 shutdown_connections 2024-05-08T04:39:05.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:05.970+0000 7fdb6e200640 1 -- 172.21.15.164:0/500034434 wait complete. 2024-05-08T04:39:05.994 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-08T04:39:05.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:05.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-08T04:39:06.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:39:06.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:06.017 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T04:39:06.068 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:06.068 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:06.079 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:06.079 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:06.130 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:06.130 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:06.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:06.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 Processor -- start 2024-05-08T04:39:06.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 -- start start 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1cb00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1cb00cfbd0 0x7f1cb00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb006c770 con 0x7f1cb015ab60 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00d0060 con 0x7f1cb00cfbd0 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006ca50 con 0x7f1cb00b26c0 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cafe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1cb015ab60 0x7f1cb0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55710/0 (socket says 172.21.15.164:55710) 2024-05-08T04:39:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cafe00640 1 -- 172.21.15.164:0/1659966489 learned_addr learned my addr 172.21.15.164:0/1659966489 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:06.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1ca7400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1cb00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1cb4c00640 1 --2- 172.21.15.164:0/1659966489 >> v2:172.21.15.172:3300/0 conn(0x7f1cb00cfbd0 0x7f1cb00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328671714 0 0) 0x7f1cb006c770 con 0x7f1cb015ab60 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c94005740 con 0x7f1cb015ab60 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3395094173 0 0) 0x7f1c94005740 con 0x7f1cb015ab60 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 msgr2=0x7f1cb00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1cb00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 >> v2:172.21.15.172:3300/0 conn(0x7f1cb00cfbd0 msgr2=0x7f1cb00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 --2- 172.21.15.164:0/1659966489 >> v2:172.21.15.172:3300/0 conn(0x7f1cb00cfbd0 0x7f1cb00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb006cd30 con 0x7f1cb015ab60 2024-05-08T04:39:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1ca0001680 con 0x7f1cb015ab60 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 >> v1:172.21.15.164:6789/0 conn(0x7f1cb015ab60 legacy=0x7f1cb0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1c94005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c94006150 0x7f1c94006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.314+0000 7f1caf400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c94006f60 0x7f1c940073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006ca50 con 0x7f1c94006150 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400dbe0 con 0x7f1c94006f60 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400dd60 con 0x7f1cb00b26c0 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1ca7400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c94006150 0x7f1c94006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1c94005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cb4c00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c94006f60 0x7f1c940073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 -- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c94006f60 msgr2=0x7f1c940073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c94006f60 0x7f1c940073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 -- 172.21.15.164:0/1659966489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c94006150 msgr2=0x7f1c94006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c94006150 0x7f1c94006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 -- 172.21.15.164:0/1659966489 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94005740 con 0x7f1cb00b26c0 2024-05-08T04:39:06.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1ca7400640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c94006150 0x7f1c94006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1cafe00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1c94005bd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c240 tx=0x7f1ca001f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929d91f6033f750b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0007d40 con 0x7f1cb00b26c0 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0005b00 con 0x7f1cb00b26c0 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.318+0000 7f1caf400640 1 -- 172.21.15.164:0/1659966489 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0007070 con 0x7f1cb00b26c0 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 msgr2=0x7f1c94005bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/1659966489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb00b26c0 0x7f1c94005bd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c240 tx=0x7f1ca001f410 comp rx=0 tx=0).stop 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb4c00640 1 -- 172.21.15.164:0/1659966489 reap_dead start 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- 172.21.15.164:0/1659966489 shutdown_connections 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- 172.21.15.164:0/1659966489 >> 172.21.15.164:0/1659966489 conn(0x7f1cb006ac90 msgr2=0x7f1cb006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:06.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- 172.21.15.164:0/1659966489 shutdown_connections 2024-05-08T04:39:06.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- 172.21.15.164:0/1659966489 wait complete. 2024-05-08T04:39:06.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 Processor -- start 2024-05-08T04:39:06.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- start start 2024-05-08T04:39:06.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cb00cdc80 0x7f1cb01f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb01f73d0 0x7f1cb00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006c3d0 con 0x7f1cb00b26c0 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006d2f0 con 0x7f1cb00cdc80 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb006c550 con 0x7f1cb01f73d0 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56582/0 (socket says 172.21.15.164:56582) 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 -- 172.21.15.164:0/2887731073 learned_addr learned my addr 172.21.15.164:0/2887731073 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1ca7400640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cb00cdc80 0x7f1cb01f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cb4c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb01f73d0 0x7f1cb00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 -- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb01f73d0 msgr2=0x7f1cb00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb01f73d0 0x7f1cb00b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 -- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cb00cdc80 msgr2=0x7f1cb01f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cb00cdc80 0x7f1cb01f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.322+0000 7f1cafe00640 1 -- 172.21.15.164:0/2887731073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb01f3e50 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cafe00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ca001f240 tx=0x7f1c94007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0001c80 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0005cb0 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb01f4130 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0005e60 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01f6bd0 con 0x7f1cb00b26c0 2024-05-08T04:39:06.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb0069d60 con 0x7f1cb00b26c0 2024-05-08T04:39:06.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ca00ba030 con 0x7f1cb00b26c0 2024-05-08T04:39:06.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.326+0000 7f1cacc00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 0x7f1c8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.330+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ca01219b0 con 0x7f1cb00b26c0 2024-05-08T04:39:06.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.330+0000 7f1ca7400640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 0x7f1c8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.330+0000 7f1ca7400640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 0x7f1c8c069450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00ce4c0 tx=0x7f1c98009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.334+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ca0019120 con 0x7f1cb00b26c0 2024-05-08T04:39:06.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.486+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cb00ce4f0 con 0x7f1cb00b26c0 2024-05-08T04:39:06.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cacc00640 1 -- 172.21.15.164:0/2887731073 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f1ca00f2a30 con 0x7f1cb00b26c0 2024-05-08T04:39:06.493 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:06.493 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 msgr2=0x7f1c8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 0x7f1c8c069450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00ce4c0 tx=0x7f1c98009190 comp rx=0 tx=0).stop 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 msgr2=0x7f1cb0151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ca001f240 tx=0x7f1c94007a50 comp rx=0 tx=0).stop 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 shutdown_connections 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c8c066f90 0x7f1c8c069450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cb01f73d0 0x7f1cb00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cb00cdc80 0x7f1cb01f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 --2- 172.21.15.164:0/2887731073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cb00b26c0 0x7f1cb0151580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 >> 172.21.15.164:0/2887731073 conn(0x7f1cb006ac90 msgr2=0x7f1cb0156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:06.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 shutdown_connections 2024-05-08T04:39:06.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.490+0000 7f1cb6c00640 1 -- 172.21.15.164:0/2887731073 wait complete. 2024-05-08T04:39:06.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:06.515 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:06.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.646+0000 7f331e000640 1 Processor -- start 2024-05-08T04:39:06.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.646+0000 7f331e000640 1 -- start start 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f331e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f331806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f331e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f331806e390 0x7f33180ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f331e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33180c6600 con 0x7f331815ba20 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f331e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331815fbe0 con 0x7f331806e390 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f331e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180c6040 con 0x7f331815ce10 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f331815ba20 0x7f3318160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55722/0 (socket says 172.21.15.164:55722) 2024-05-08T04:39:06.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 -- 172.21.15.164:0/531574972 learned_addr learned my addr 172.21.15.164:0/531574972 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:06.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831674056 0 0) 0x7f33180c6600 con 0x7f331815ba20 2024-05-08T04:39:06.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc003530 con 0x7f331815ba20 2024-05-08T04:39:06.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317e00640 1 --2- 172.21.15.164:0/531574972 >> v2:172.21.15.172:3300/0 conn(0x7f331806e390 0x7f33180ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f330ea00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f331806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970547428 0 0) 0x7f32fc003530 con 0x7f331815ba20 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 msgr2=0x7f331806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f331806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 >> v2:172.21.15.172:3300/0 conn(0x7f331806e390 msgr2=0x7f33180ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 --2- 172.21.15.164:0/531574972 >> v2:172.21.15.172:3300/0 conn(0x7f331806e390 0x7f33180ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180c6320 con 0x7f331815ba20 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f3308001680 con 0x7f331815ba20 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 >> v1:172.21.15.164:6789/0 conn(0x7f331815ba20 legacy=0x7f3318160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32fc0039c0 0x7f32fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc004810 0x7f32fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f32fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331815fbe0 con 0x7f32fc0039c0 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180c6040 con 0x7f32fc004810 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc00b9d0 con 0x7f331815ce10 2024-05-08T04:39:06.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32fc0039c0 0x7f32fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f330ea00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc004810 0x7f32fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317e00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f32fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 -- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 msgr2=0x7f32fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f32fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 -- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc004810 msgr2=0x7f32fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc004810 0x7f32fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317400640 1 -- 172.21.15.164:0/531574972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc003530 con 0x7f32fc0039c0 2024-05-08T04:39:06.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.650+0000 7f3317e00640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f331815ce10 0x7f32fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f3317400640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32fc0039c0 0x7f32fc003de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f330800c2f0 tx=0x7f330801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719360823efe69f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308007d60 con 0x7f32fc0039c0 2024-05-08T04:39:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33080a9020 con 0x7f32fc0039c0 2024-05-08T04:39:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f3316a00640 1 -- 172.21.15.164:0/531574972 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308005ae0 con 0x7f32fc0039c0 2024-05-08T04:39:06.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 -- 172.21.15.164:0/531574972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32fc0039c0 msgr2=0x7f32fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 --2- 172.21.15.164:0/531574972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32fc0039c0 0x7f32fc003de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f330800c2f0 tx=0x7f330801cd40 comp rx=0 tx=0).stop 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f3317e00640 1 -- 172.21.15.164:0/531574972 reap_dead start 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 -- 172.21.15.164:0/531574972 shutdown_connections 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 -- 172.21.15.164:0/531574972 >> 172.21.15.164:0/531574972 conn(0x7f33180cf770 msgr2=0x7f33180cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 -- 172.21.15.164:0/531574972 shutdown_connections 2024-05-08T04:39:06.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.654+0000 7f331e000640 1 -- 172.21.15.164:0/531574972 wait complete. 2024-05-08T04:39:06.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 Processor -- start 2024-05-08T04:39:06.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- start start 2024-05-08T04:39:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0039c0 0x7f3318071500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 0x7f33180c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180af030 con 0x7f33180b0dc0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180ce090 con 0x7f32fc0039c0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3318166b60 con 0x7f3318070ad0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56596/0 (socket says 172.21.15.164:56596) 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 -- 172.21.15.164:0/3125112337 learned_addr learned my addr 172.21.15.164:0/3125112337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f3317400640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0039c0 0x7f3318071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f3317e00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 0x7f33180c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 -- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 msgr2=0x7f33180c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 0x7f33180c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 -- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0039c0 msgr2=0x7f3318071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0039c0 0x7f3318071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 -- 172.21.15.164:0/3125112337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318070310 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f3317e00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 0x7f33180c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330ea00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33000118b0 tx=0x7f3300011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000fa30 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000fbd0 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300014440 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33180cba70 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331814d540 con 0x7f33180b0dc0 2024-05-08T04:39:06.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.658+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318072d30 con 0x7f33180b0dc0 2024-05-08T04:39:06.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.662+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3300012040 con 0x7f33180b0dc0 2024-05-08T04:39:06.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.662+0000 7f330fe00640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 0x7f32f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.662+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3300085670 con 0x7f33180b0dc0 2024-05-08T04:39:06.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.662+0000 7f3317400640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 0x7f32f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.662+0000 7f3317400640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 0x7f32f4069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f33080aa000 tx=0x7f33080b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.666+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3300019030 con 0x7f33180b0dc0 2024-05-08T04:39:06.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.818+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f331804f010 con 0x7f33180b0dc0 2024-05-08T04:39:06.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.818+0000 7f330fe00640 1 -- 172.21.15.164:0/3125112337 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3300056680 con 0x7f33180b0dc0 2024-05-08T04:39:06.822 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 msgr2=0x7f32f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 0x7f32f4069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f33080aa000 tx=0x7f33080b0040 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 msgr2=0x7f33181f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33000118b0 tx=0x7f3300011d70 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 shutdown_connections 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32f4066f60 0x7f32f4069420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3318070ad0 0x7f33180c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33180b0dc0 0x7f33181f6e00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 --2- 172.21.15.164:0/3125112337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0039c0 0x7f3318071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 >> 172.21.15.164:0/3125112337 conn(0x7f33180cf770 msgr2=0x7f331806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 shutdown_connections 2024-05-08T04:39:06.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.822+0000 7f331e000640 1 -- 172.21.15.164:0/3125112337 wait complete. 2024-05-08T04:39:06.845 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:06.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 Processor -- start 2024-05-08T04:39:06.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 -- start start 2024-05-08T04:39:06.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fab6c0aee90 0x7fab6c0d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0af9a0 con 0x7fab6c15e900 2024-05-08T04:39:06.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0afc80 con 0x7fab6c0d0a90 2024-05-08T04:39:06.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab72e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0af750 con 0x7fab6c0aee90 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab70e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fab6c15e900 0x7fab6c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55734/0 (socket says 172.21.15.164:55734) 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab70e00640 1 -- 172.21.15.164:0/3160229835 learned_addr learned my addr 172.21.15.164:0/3160229835 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6be00640 1 --2- 172.21.15.164:0/3160229835 >> v2:172.21.15.172:3300/0 conn(0x7fab6c0aee90 0x7fab6c0d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6b400640 1 -- 172.21.15.164:0/3160229835 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082827073 0 0) 0x7fab6c0af9a0 con 0x7fab6c15e900 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6b400640 1 -- 172.21.15.164:0/3160229835 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab50003530 con 0x7fab6c15e900 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6be00640 1 -- 172.21.15.164:0/3160229835 >> v1:172.21.15.172:6789/0 conn(0x7fab6c0d0a90 legacy=0x7fab6c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6be00640 1 -- 172.21.15.164:0/3160229835 >> v1:172.21.15.164:6789/0 conn(0x7fab6c15e900 legacy=0x7fab6c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.982+0000 7fab6be00640 1 -- 172.21.15.164:0/3160229835 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0aff60 con 0x7fab6c0aee90 2024-05-08T04:39:06.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab6be00640 1 --2- 172.21.15.164:0/3160229835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c0d0550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097c0 tx=0x7fab5c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867788da8997b422 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab6b400640 1 -- 172.21.15.164:0/3160229835 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0a7070 con 0x7fab6c0aee90 2024-05-08T04:39:06.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab6b400640 1 -- 172.21.15.164:0/3160229835 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c0b0070 con 0x7fab6c0aee90 2024-05-08T04:39:06.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab6b400640 1 -- 172.21.15.164:0/3160229835 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0ab890 con 0x7fab6c0aee90 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 -- 172.21.15.164:0/3160229835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 msgr2=0x7fab6c0d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 --2- 172.21.15.164:0/3160229835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c0d0550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097c0 tx=0x7fab5c01c470 comp rx=0 tx=0).stop 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 -- 172.21.15.164:0/3160229835 shutdown_connections 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 --2- 172.21.15.164:0/3160229835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c0d0550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 -- 172.21.15.164:0/3160229835 >> 172.21.15.164:0/3160229835 conn(0x7fab6c067050 msgr2=0x7fab6c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 -- 172.21.15.164:0/3160229835 shutdown_connections 2024-05-08T04:39:06.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.986+0000 7fab72e00640 1 -- 172.21.15.164:0/3160229835 wait complete. 2024-05-08T04:39:06.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 Processor -- start 2024-05-08T04:39:06.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- start start 2024-05-08T04:39:06.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab6c0d0a90 0x7fab6c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0720f0 con 0x7fab6c15e900 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0af270 con 0x7fab6c0aee90 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c072270 con 0x7fab6c0d0a90 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56598/0 (socket says 172.21.15.164:56598) 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 -- 172.21.15.164:0/326202256 learned_addr learned my addr 172.21.15.164:0/326202256 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab6be00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab63e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab6c0d0a90 0x7fab6c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 -- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab6c0d0a90 msgr2=0x7fab6c150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab6c0d0a90 0x7fab6c150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 -- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 msgr2=0x7fab6c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 -- 172.21.15.164:0/326202256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c1515b0 con 0x7fab6c15e900 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab70e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cb40 tx=0x7fab6400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64017070 con 0x7fab6c15e900 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400aca0 con 0x7fab6c15e900 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64012620 con 0x7fab6c15e900 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c151890 con 0x7fab6c15e900 2024-05-08T04:39:06.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.990+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c152350 con 0x7fab6c15e900 2024-05-08T04:39:06.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c152e10 con 0x7fab6c15e900 2024-05-08T04:39:06.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab6401a070 con 0x7fab6c15e900 2024-05-08T04:39:06.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab68c00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:06.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab64084da0 con 0x7fab6c15e900 2024-05-08T04:39:06.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab6be00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:06.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.994+0000 7fab6be00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0af000 tx=0x7fab5c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:06.998+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab64055750 con 0x7fab6c15e900 2024-05-08T04:39:07.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.150+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab6c072e80 con 0x7fab6c15e900 2024-05-08T04:39:07.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.150+0000 7fab68c00640 1 -- 172.21.15.164:0/326202256 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab640597f0 con 0x7fab6c15e900 2024-05-08T04:39:07.156 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 msgr2=0x7fab48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0af000 tx=0x7fab5c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 msgr2=0x7fab6c15d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cb40 tx=0x7fab6400a060 comp rx=0 tx=0).stop 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 shutdown_connections 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab6c0d0a90 0x7fab6c150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab6c15e900 0x7fab6c15d070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 --2- 172.21.15.164:0/326202256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab6c0aee90 0x7fab6c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 >> 172.21.15.164:0/326202256 conn(0x7fab6c067050 msgr2=0x7fab6c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:07.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 shutdown_connections 2024-05-08T04:39:07.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.154+0000 7fab72e00640 1 -- 172.21.15.164:0/326202256 wait complete. 2024-05-08T04:39:07.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:07.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.310+0000 7f5ba3e00640 1 Processor -- start 2024-05-08T04:39:07.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.310+0000 7f5ba3e00640 1 -- start start 2024-05-08T04:39:07.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba406c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ba40ce900 0x7f5ba40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba406eea0 con 0x7f5ba406ce60 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba406c490 con 0x7f5ba40ce900 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba406f120 con 0x7f5ba40b0b90 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5b9a000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5ba406ce60 0x7f5ba40ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55740/0 (socket says 172.21.15.164:55740) 2024-05-08T04:39:07.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5b9a000640 1 -- 172.21.15.164:0/3466423353 learned_addr learned my addr 172.21.15.164:0/3466423353 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3400640 1 --2- 172.21.15.164:0/3466423353 >> v2:172.21.15.172:3300/0 conn(0x7f5ba40ce900 0x7f5ba40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba2000640 1 -- 172.21.15.164:0/3466423353 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172928406 0 0) 0x7f5b8c001120 con 0x7f5ba406ce60 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba2000640 1 -- 172.21.15.164:0/3466423353 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b88003530 con 0x7f5ba406ce60 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3400640 1 -- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 msgr2=0x7f5ba406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3400640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba406c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3400640 1 -- 172.21.15.164:0/3466423353 >> v1:172.21.15.164:6789/0 conn(0x7f5ba406ce60 legacy=0x7f5ba40ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.314+0000 7f5ba3400640 1 -- 172.21.15.164:0/3466423353 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba406f400 con 0x7f5ba40ce900 2024-05-08T04:39:07.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3400640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba40ce900 0x7f5ba40cf970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622e749ca71b77b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba2000640 1 -- 172.21.15.164:0/3466423353 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0b2070 con 0x7f5ba40ce900 2024-05-08T04:39:07.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba2000640 1 -- 172.21.15.164:0/3466423353 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c0ad640 con 0x7f5ba40ce900 2024-05-08T04:39:07.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba2000640 1 -- 172.21.15.164:0/3466423353 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0ada10 con 0x7f5ba40ce900 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba40ce900 msgr2=0x7f5ba40cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba40ce900 0x7f5ba40cf970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- 172.21.15.164:0/3466423353 shutdown_connections 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba40ce900 0x7f5ba40cf970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/3466423353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- 172.21.15.164:0/3466423353 >> 172.21.15.164:0/3466423353 conn(0x7f5ba40b3370 msgr2=0x7f5ba40b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- 172.21.15.164:0/3466423353 shutdown_connections 2024-05-08T04:39:07.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- 172.21.15.164:0/3466423353 wait complete. 2024-05-08T04:39:07.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 Processor -- start 2024-05-08T04:39:07.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 -- start start 2024-05-08T04:39:07.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba406ce60 0x7f5ba40b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.318+0000 7f5ba3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba4147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40d0030 con 0x7f5ba40ce900 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40cfeb0 con 0x7f5ba406ce60 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40d01b0 con 0x7f5ba40b0b90 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56606/0 (socket says 172.21.15.164:56606) 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 -- 172.21.15.164:0/421537858 learned_addr learned my addr 172.21.15.164:0/421537858 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba406ce60 0x7f5ba40b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5b9a000640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba4147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 -- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 msgr2=0x7f5ba4147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba4147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 -- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba406ce60 msgr2=0x7f5ba40b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba406ce60 0x7f5ba40b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 -- 172.21.15.164:0/421537858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4148110 con 0x7f5ba40ce900 2024-05-08T04:39:07.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3400640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c01a0d0 tx=0x7f5b9c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0b2040 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c0bb070 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba40b5870 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0b6670 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba40b5e30 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba404ef30 con 0x7f5ba40ce900 2024-05-08T04:39:07.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.322+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b9c0b6810 con 0x7f5ba40ce900 2024-05-08T04:39:07.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.326+0000 7f5b9be00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.326+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b9c11efb0 con 0x7f5ba40ce900 2024-05-08T04:39:07.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.326+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.326+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 0x7f5b80069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5b940113e0 tx=0x7f5b9400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.330+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b9c016090 con 0x7f5ba40ce900 2024-05-08T04:39:07.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ba40b38f0 con 0x7f5ba40ce900 2024-05-08T04:39:07.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5b9be00640 1 -- 172.21.15.164:0/421537858 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b9c0ef850 con 0x7f5ba40ce900 2024-05-08T04:39:07.489 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 msgr2=0x7f5b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 0x7f5b80069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5b940113e0 tx=0x7f5b9400b040 comp rx=0 tx=0).stop 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 msgr2=0x7f5ba4157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c01a0d0 tx=0x7f5b9c019240 comp rx=0 tx=0).stop 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 shutdown_connections 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba40b0b90 0x7f5ba4147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba40ce900 0x7f5ba4157310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 --2- 172.21.15.164:0/421537858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba406ce60 0x7f5ba40b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 >> 172.21.15.164:0/421537858 conn(0x7f5ba40b3370 msgr2=0x7f5ba40af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:07.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 shutdown_connections 2024-05-08T04:39:07.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.486+0000 7f5ba3e00640 1 -- 172.21.15.164:0/421537858 wait complete. 2024-05-08T04:39:07.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 Processor -- start 2024-05-08T04:39:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 -- start start 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f207815be50 0x7f2078160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780cd9f0 con 0x7f20780cfda0 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780cdcd0 con 0x7f207815ba70 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.642+0000 7f207d200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207815fe70 con 0x7f207815be50 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2075400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f20780cfda0 0x7f20780b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55752/0 (socket says 172.21.15.164:55752) 2024-05-08T04:39:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2075400640 1 -- 172.21.15.164:0/2834923106 learned_addr learned my addr 172.21.15.164:0/2834923106 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519618795 0 0) 0x7f20780cd9f0 con 0x7f20780cfda0 2024-05-08T04:39:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c003530 con 0x7f20780cfda0 2024-05-08T04:39:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> v2:172.21.15.172:3300/0 conn(0x7f207815be50 0x7f2078160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 655674327 0 0) 0x7f205c003530 con 0x7f20780cfda0 2024-05-08T04:39:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 >> v2:172.21.15.172:3300/0 conn(0x7f207815be50 msgr2=0x7f2078160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 --2- 172.21.15.164:0/2834923106 >> v2:172.21.15.172:3300/0 conn(0x7f207815be50 0x7f2078160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 >> v1:172.21.15.172:6789/0 conn(0x7f207815ba70 legacy=0x7f20780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207815f820 con 0x7f20780cfda0 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2064001680 con 0x7f20780cfda0 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 >> v1:172.21.15.164:6789/0 conn(0x7f20780cfda0 legacy=0x7f20780b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205c003a10 0x7f205c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c004810 0x7f205c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 0x7f205c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207815fe70 con 0x7f205c005220 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00c170 con 0x7f205c004810 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00c370 con 0x7f205c003a10 2024-05-08T04:39:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> v2:172.21.15.172:3300/0 conn(0x7f207815be50 0x7f2078160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 0x7f205c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2075400640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c004810 0x7f205c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2075e00640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205c003a10 0x7f205c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 -- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205c003a10 msgr2=0x7f205c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205c003a10 0x7f205c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 -- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c004810 msgr2=0x7f205c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c004810 0x7f205c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.646+0000 7f2076800640 1 -- 172.21.15.164:0/2834923106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c003530 con 0x7f205c005220 2024-05-08T04:39:07.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f2076800640 1 -- 172.21.15.164:0/2834923106 reap_dead start 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f2076800640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 0x7f205c00b9d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20700098b0 tx=0x7f207001c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36a9ce1eb1ccb26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000f040 con 0x7f205c005220 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2070004220 con 0x7f205c005220 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f2074a00640 1 -- 172.21.15.164:0/2834923106 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070006e40 con 0x7f205c005220 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 msgr2=0x7f205c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 0x7f205c00b9d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20700098b0 tx=0x7f207001c630 comp rx=0 tx=0).stop 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- 172.21.15.164:0/2834923106 shutdown_connections 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 --2- 172.21.15.164:0/2834923106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c005220 0x7f205c00b9d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- 172.21.15.164:0/2834923106 >> 172.21.15.164:0/2834923106 conn(0x7f207806ac90 msgr2=0x7f207806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- 172.21.15.164:0/2834923106 shutdown_connections 2024-05-08T04:39:07.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- 172.21.15.164:0/2834923106 wait complete. 2024-05-08T04:39:07.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 Processor -- start 2024-05-08T04:39:07.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.650+0000 7f207d200640 1 -- start start 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c005220 0x7f2078149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 0x7f20780a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780cd7b0 con 0x7f2078070d10 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780cd630 con 0x7f205c005220 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780aeda0 con 0x7f20780a9480 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56624/0 (socket says 172.21.15.164:56624) 2024-05-08T04:39:07.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 -- 172.21.15.164:0/2149212054 learned_addr learned my addr 172.21.15.164:0/2149212054 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2075e00640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c005220 0x7f2078149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2075400640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 0x7f20780a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 -- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 msgr2=0x7f20780a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 0x7f20780a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 -- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c005220 msgr2=0x7f2078149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c005220 0x7f2078149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 -- 172.21.15.164:0/2149212054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078143950 con 0x7f2078070d10 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2075400640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 0x7f20780a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:07.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f2076800640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f207001c460 tx=0x7f205c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700b7070 con 0x7f2078070d10 2024-05-08T04:39:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700a6850 con 0x7f2078070d10 2024-05-08T04:39:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700a6c20 con 0x7f2078070d10 2024-05-08T04:39:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2078143bc0 con 0x7f2078070d10 2024-05-08T04:39:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2078144180 con 0x7f2078070d10 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078069d60 con 0x7f2078070d10 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.654+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20700ba070 con 0x7f2078070d10 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.658+0000 7f206e000640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 0x7f2058069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.658+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f207011de50 con 0x7f2078070d10 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.658+0000 7f2075e00640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 0x7f2058069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.658+0000 7f2075e00640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 0x7f2058069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2068001e80 tx=0x7f2068009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:07.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.662+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2070016340 con 0x7f2078070d10 2024-05-08T04:39:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f207815b270 con 0x7f2078070d10 2024-05-08T04:39:07.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f206e000640 1 -- 172.21.15.164:0/2149212054 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f207000f000 con 0x7f2078070d10 2024-05-08T04:39:07.829 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 msgr2=0x7f2058069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 0x7f2058069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2068001e80 tx=0x7f2068009040 comp rx=0 tx=0).stop 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 msgr2=0x7f2078071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f207001c460 tx=0x7f205c005840 comp rx=0 tx=0).stop 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 shutdown_connections 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2058066f90 0x7f2058069450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20780a9480 0x7f20780a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2078070d10 0x7f2078071190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 --2- 172.21.15.164:0/2149212054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c005220 0x7f2078149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:07.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 >> 172.21.15.164:0/2149212054 conn(0x7f207806ac90 msgr2=0x7f20780b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:07.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 shutdown_connections 2024-05-08T04:39:07.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:07.826+0000 7f207d200640 1 -- 172.21.15.164:0/2149212054 wait complete. 2024-05-08T04:39:07.851 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-08T04:39:07.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:07.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-08T04:39:07.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:39:07.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:07.878 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T04:39:07.932 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:39:07.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 Processor -- start 2024-05-08T04:39:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 -- start start 2024-05-08T04:39:08.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f84040af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840406d580 con 0x7f84041561f0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840406d820 con 0x7f84040ce210 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f840a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415d040 con 0x7f840406b3a0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f84041561f0 0x7f840415c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55768/0 (socket says 172.21.15.164:55768) 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403400640 1 -- 172.21.15.164:0/1949255945 learned_addr learned my addr 172.21.15.164:0/1949255945 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403e00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f84040af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830265339 0 0) 0x7f840406d580 con 0x7f84041561f0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e8003530 con 0x7f84041561f0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846114046 0 0) 0x7f840406d820 con 0x7f84040ce210 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840406d580 con 0x7f84040ce210 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750665503 0 0) 0x7f83e8003530 con 0x7f84041561f0 2024-05-08T04:39:08.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 msgr2=0x7f84040af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f84040af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 >> v1:172.21.15.172:6789/0 conn(0x7f84040ce210 legacy=0x7f840406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840406db00 con 0x7f84041561f0 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f83f40016a0 con 0x7f84041561f0 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403e00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f84040af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 >> v1:172.21.15.164:6789/0 conn(0x7f84041561f0 legacy=0x7f840415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8003b80 0x7f83e8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83e8004810 0x7f83e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f83e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415d040 con 0x7f83e8004810 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800ba80 con 0x7f83e8003b80 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800bc80 con 0x7f840406b3a0 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83e8004810 0x7f83e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403400640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8003b80 0x7f83e8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8403e00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f83e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 -- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 msgr2=0x7f83e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840406b3a0 0x7f83e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 -- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8003b80 msgr2=0x7f83e8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8003b80 0x7f83e8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.066+0000 7f8402a00640 1 -- 172.21.15.164:0/1949255945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8003530 con 0x7f83e8004810 2024-05-08T04:39:08.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f8402a00640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83e8004810 0x7f83e8004c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83f0007fd0 tx=0x7f83f001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dec9990b90db64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0001520 con 0x7f83e8004810 2024-05-08T04:39:08.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f00052f0 con 0x7f83e8004810 2024-05-08T04:39:08.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f8402000640 1 -- 172.21.15.164:0/1949255945 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a88d0 con 0x7f83e8004810 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 -- 172.21.15.164:0/1949255945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83e8004810 msgr2=0x7f83e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 --2- 172.21.15.164:0/1949255945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83e8004810 0x7f83e8004c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83f0007fd0 tx=0x7f83f001ec70 comp rx=0 tx=0).stop 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f8403e00640 1 -- 172.21.15.164:0/1949255945 reap_dead start 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 -- 172.21.15.164:0/1949255945 shutdown_connections 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 -- 172.21.15.164:0/1949255945 >> 172.21.15.164:0/1949255945 conn(0x7f8404066fa0 msgr2=0x7f840406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 -- 172.21.15.164:0/1949255945 shutdown_connections 2024-05-08T04:39:08.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.070+0000 7f840a800640 1 -- 172.21.15.164:0/1949255945 wait complete. 2024-05-08T04:39:08.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 Processor -- start 2024-05-08T04:39:08.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- start start 2024-05-08T04:39:08.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8004810 0x7f8404159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84041e4420 0x7f84040cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415e3b0 con 0x7f8404157ad0 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415e230 con 0x7f83e8004810 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415e530 con 0x7f84041e4420 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56640/0 (socket says 172.21.15.164:56640) 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 -- 172.21.15.164:0/156920309 learned_addr learned my addr 172.21.15.164:0/156920309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8403400640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8004810 0x7f8404159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8403e00640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84041e4420 0x7f84040cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 -- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84041e4420 msgr2=0x7f84040cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84041e4420 0x7f84040cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 -- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8004810 msgr2=0x7f8404159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8004810 0x7f8404159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 -- 172.21.15.164:0/156920309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84041558c0 con 0x7f8404157ad0 2024-05-08T04:39:08.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f8402a00640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83f0000c10 tx=0x7f83e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00054a0 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0001500 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a9da0 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8404157f60 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.074+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8404158520 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8404069d60 con 0x7f8404157ad0 2024-05-08T04:39:08.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83f00067e0 con 0x7f8404157ad0 2024-05-08T04:39:08.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f83fb400640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83f0121ff0 con 0x7f8404157ad0 2024-05-08T04:39:08.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f8403400640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.078+0000 7f8403400640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 0x7f83e4069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f83f4001060 tx=0x7f83f4001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.082+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83f0018980 con 0x7f8404157ad0 2024-05-08T04:39:08.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.270+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f840415bb00 con 0x7f8404157ad0 2024-05-08T04:39:08.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f83fb400640 1 -- 172.21.15.164:0/156920309 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f83f00f3050 con 0x7f8404157ad0 2024-05-08T04:39:08.276 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:08.276 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:39:08.277 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:39:08.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 msgr2=0x7f83e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 0x7f83e4069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f83f4001060 tx=0x7f83f4001d80 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 msgr2=0x7f8404155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83f0000c10 tx=0x7f83e8005840 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 shutdown_connections 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84041e4420 0x7f84040cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404157ad0 0x7f8404155380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 --2- 172.21.15.164:0/156920309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83e8004810 0x7f8404159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 >> 172.21.15.164:0/156920309 conn(0x7f8404066fa0 msgr2=0x7f840406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 shutdown_connections 2024-05-08T04:39:08.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.274+0000 7f840a800640 1 -- 172.21.15.164:0/156920309 wait complete. 2024-05-08T04:39:08.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:08.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 Processor -- start 2024-05-08T04:39:08.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 -- start start 2024-05-08T04:39:08.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 0x7f323c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f323c0cfda0 0x7f323c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c0cd9f0 con 0x7f323c15be50 2024-05-08T04:39:08.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c15fe70 con 0x7f323c0cfda0 2024-05-08T04:39:08.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.430+0000 7f3243600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0cdcd0 con 0x7f323c15ba70 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f323c0cfda0 0x7f323c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3240c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 0x7f323c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f323c0cfda0 0x7f323c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:58028/0 (socket says 172.21.15.164:58028) 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3240c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 0x7f323c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:51632/0 (socket says 172.21.15.164:51632) 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 -- 172.21.15.164:0/1924186101 learned_addr learned my addr 172.21.15.164:0/1924186101 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 -- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 msgr2=0x7f323c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 --2- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 0x7f323c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 -- 172.21.15.164:0/1924186101 >> v1:172.21.15.172:6789/0 conn(0x7f323c15be50 legacy=0x7f323c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 -- 172.21.15.164:0/1924186101 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c15f820 con 0x7f323c0cfda0 2024-05-08T04:39:08.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f3241600640 1 --2- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c0cfda0 0x7f323c0b31a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1682ec5f8104eef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f323be00640 1 -- 172.21.15.164:0/1924186101 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a57e0 con 0x7f323c0cfda0 2024-05-08T04:39:08.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f323be00640 1 -- 172.21.15.164:0/1924186101 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340a5980 con 0x7f323c0cfda0 2024-05-08T04:39:08.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.434+0000 7f323be00640 1 -- 172.21.15.164:0/1924186101 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a5d50 con 0x7f323c0cfda0 2024-05-08T04:39:08.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c0cfda0 msgr2=0x7f323c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c0cfda0 0x7f323c0b31a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323401ca00 comp rx=0 tx=0).stop 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- 172.21.15.164:0/1924186101 shutdown_connections 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c0cfda0 0x7f323c0b31a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- 172.21.15.164:0/1924186101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15ba70 0x7f323c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- 172.21.15.164:0/1924186101 >> 172.21.15.164:0/1924186101 conn(0x7f323c06ac90 msgr2=0x7f323c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- 172.21.15.164:0/1924186101 shutdown_connections 2024-05-08T04:39:08.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- 172.21.15.164:0/1924186101 wait complete. 2024-05-08T04:39:08.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 Processor -- start 2024-05-08T04:39:08.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- start start 2024-05-08T04:39:08.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 0x7f323c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 0x7f323c146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c168d40 con 0x7f323c0cfda0 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c168bc0 con 0x7f323c15ba70 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3243600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c168ec0 con 0x7f323c15be50 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56650/0 (socket says 172.21.15.164:56650) 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 -- 172.21.15.164:0/592013607 learned_addr learned my addr 172.21.15.164:0/592013607 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3233e00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 0x7f323c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3241600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 0x7f323c146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 -- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 msgr2=0x7f323c146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 0x7f323c146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 -- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 msgr2=0x7f323c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 0x7f323c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3240c00640 1 -- 172.21.15.164:0/592013607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c147f10 con 0x7f323c0cfda0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3233e00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 0x7f323c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.438+0000 7f3241600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 0x7f323c146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3240c00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f322c00e960 tx=0x7f322c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c017070 con 0x7f323c0cfda0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c00ca00 con 0x7f323c0cfda0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c00cdb0 con 0x7f323c0cfda0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c1481f0 con 0x7f323c0cfda0 2024-05-08T04:39:08.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c14b730 con 0x7f323c0cfda0 2024-05-08T04:39:08.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c069d60 con 0x7f323c0cfda0 2024-05-08T04:39:08.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f322c01a070 con 0x7f323c0cfda0 2024-05-08T04:39:08.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f322c0853f0 con 0x7f323c0cfda0 2024-05-08T04:39:08.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.442+0000 7f3233e00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.446+0000 7f3233e00640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f323c15e640 tx=0x7f3224009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.446+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f322c056400 con 0x7f323c0cfda0 2024-05-08T04:39:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f323c1483c0 con 0x7f323c0cfda0 2024-05-08T04:39:08.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3239600640 1 -- 172.21.15.164:0/592013607 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16665 (secure 0 0 0) 0x7f322c055d00 con 0x7f323c0cfda0 2024-05-08T04:39:08.606 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:08.606 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:38:30.726451+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 msgr2=0x7f3218069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f323c15e640 tx=0x7f3224009040 comp rx=0 tx=0).stop 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 msgr2=0x7f323c150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f322c00e960 tx=0x7f322c00ee20 comp rx=0 tx=0).stop 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 shutdown_connections 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f323c15be50 0x7f323c146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f323c15ba70 0x7f323c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 --2- 172.21.15.164:0/592013607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f323c0cfda0 0x7f323c150ac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 >> 172.21.15.164:0/592013607 conn(0x7f323c06ac90 msgr2=0x7f323c159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 shutdown_connections 2024-05-08T04:39:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.602+0000 7f3243600640 1 -- 172.21.15.164:0/592013607 wait complete. 2024-05-08T04:39:08.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:38:27.601306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:38:28.134395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:08.627 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:08.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.762+0000 7f347be00640 1 Processor -- start 2024-05-08T04:39:08.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.762+0000 7f347be00640 1 -- start start 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f347be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f347415ab10 0x7f347406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f347be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34740cee90 0x7f34740aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f347be00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347415e870 con 0x7f34740ae6e0 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f347be00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347406bb80 con 0x7f347415ab10 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f347be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347415eb50 con 0x7f34740cee90 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f347415ab10 0x7f347406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3478a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f34740ae6e0 0x7f34740ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55774/0 (socket says 172.21.15.164:55774) 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3478a00640 1 -- 172.21.15.164:0/1186814446 learned_addr learned my addr 172.21.15.164:0/1186814446 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479e00640 1 --2- 172.21.15.164:0/1186814446 >> v2:172.21.15.172:3300/0 conn(0x7f34740cee90 0x7f34740aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3473e00640 1 -- 172.21.15.164:0/1186814446 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767092954 0 0) 0x7f347415e870 con 0x7f34740ae6e0 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3473e00640 1 -- 172.21.15.164:0/1186814446 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f345c005740 con 0x7f34740ae6e0 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479400640 1 -- 172.21.15.164:0/1186814446 >> v2:172.21.15.172:3300/0 conn(0x7f34740cee90 msgr2=0x7f34740aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479400640 1 --2- 172.21.15.164:0/1186814446 >> v2:172.21.15.172:3300/0 conn(0x7f34740cee90 0x7f34740aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479400640 1 -- 172.21.15.164:0/1186814446 >> v1:172.21.15.164:6789/0 conn(0x7f34740ae6e0 legacy=0x7f34740ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.766+0000 7f3479400640 1 -- 172.21.15.164:0/1186814446 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347415ee30 con 0x7f347415ab10 2024-05-08T04:39:08.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f3479400640 1 --2- 172.21.15.164:0/1186814446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f347415ab10 0x7f347406cce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34740ca1f0 tx=0x7f346801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347f4d8c517d519 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f3473e00640 1 -- 172.21.15.164:0/1186814446 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680b2070 con 0x7f347415ab10 2024-05-08T04:39:08.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f3473e00640 1 -- 172.21.15.164:0/1186814446 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680a5930 con 0x7f347415ab10 2024-05-08T04:39:08.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f3473e00640 1 -- 172.21.15.164:0/1186814446 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680a5d00 con 0x7f347415ab10 2024-05-08T04:39:08.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- 172.21.15.164:0/1186814446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f347415ab10 msgr2=0x7f347406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 --2- 172.21.15.164:0/1186814446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f347415ab10 0x7f347406cce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34740ca1f0 tx=0x7f346801c870 comp rx=0 tx=0).stop 2024-05-08T04:39:08.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- 172.21.15.164:0/1186814446 shutdown_connections 2024-05-08T04:39:08.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 --2- 172.21.15.164:0/1186814446 >> v2:172.21.15.172:3300/0 conn(0x7f34740cee90 0x7f34740aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 --2- 172.21.15.164:0/1186814446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f347415ab10 0x7f347406cce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- 172.21.15.164:0/1186814446 >> 172.21.15.164:0/1186814446 conn(0x7f34740cf770 msgr2=0x7f34740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- 172.21.15.164:0/1186814446 shutdown_connections 2024-05-08T04:39:08.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- 172.21.15.164:0/1186814446 wait complete. 2024-05-08T04:39:08.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 Processor -- start 2024-05-08T04:39:08.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.770+0000 7f347be00640 1 -- start start 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 0x7f347414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415ab10 0x7f347414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34740cb070 con 0x7f34740cee90 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34740caef0 con 0x7f34740ae6e0 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34740cb1f0 con 0x7f347415ab10 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56658/0 (socket says 172.21.15.164:56658) 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 -- 172.21.15.164:0/1264296429 learned_addr learned my addr 172.21.15.164:0/1264296429 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:08.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3479400640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 0x7f347414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 -- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415ab10 msgr2=0x7f347414ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415ab10 0x7f347414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 -- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 msgr2=0x7f347414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 0x7f347414db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 -- 172.21.15.164:0/1264296429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474139df0 con 0x7f34740cee90 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3479400640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 0x7f347414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3478a00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34600010f0 tx=0x7f346000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460017070 con 0x7f34740cee90 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460005d10 con 0x7f34740cee90 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460007310 con 0x7f34740cee90 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347413a0d0 con 0x7f34740cee90 2024-05-08T04:39:08.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347413a690 con 0x7f34740cee90 2024-05-08T04:39:08.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474072d30 con 0x7f34740cee90 2024-05-08T04:39:08.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.774+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f346001a070 con 0x7f34740cee90 2024-05-08T04:39:08.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.778+0000 7f3471600640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 0x7f3454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:08.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.778+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3460084410 con 0x7f34740cee90 2024-05-08T04:39:08.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.778+0000 7f3479400640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 0x7f3454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:08.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.778+0000 7f3479400640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 0x7f3454069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34680af000 tx=0x7f34680a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:08.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.782+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34600554b0 con 0x7f34740cee90 2024-05-08T04:39:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.938+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34740c4bf0 con 0x7f34740cee90 2024-05-08T04:39:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.938+0000 7f3471600640 1 -- 172.21.15.164:0/1264296429 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3460059e20 con 0x7f34740cee90 2024-05-08T04:39:08.942 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 msgr2=0x7f3454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 0x7f3454069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34680af000 tx=0x7f34680a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 msgr2=0x7f34740b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34600010f0 tx=0x7f346000c8f0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 shutdown_connections 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3454066f60 0x7f3454069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f347415ab10 0x7f347414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34740cee90 0x7f34740b23a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 --2- 172.21.15.164:0/1264296429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34740ae6e0 0x7f347414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 >> 172.21.15.164:0/1264296429 conn(0x7f34740cf770 msgr2=0x7f3474159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 shutdown_connections 2024-05-08T04:39:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:08.942+0000 7f347be00640 1 -- 172.21.15.164:0/1264296429 wait complete. 2024-05-08T04:39:08.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:09.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.098+0000 7f05a6e00640 1 Processor -- start 2024-05-08T04:39:09.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.098+0000 7f05a6e00640 1 -- start start 2024-05-08T04:39:09.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a6e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05a006cce0 0x7f05a00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a6e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a015e870 con 0x7f05a00cecc0 2024-05-08T04:39:09.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a6e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a015eb50 con 0x7f05a015ab10 2024-05-08T04:39:09.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a6e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a006bb80 con 0x7f05a006cce0 2024-05-08T04:39:09.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a4e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f05a00cecc0 0x7f05a00ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55786/0 (socket says 172.21.15.164:55786) 2024-05-08T04:39:09.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f05a4e00640 1 -- 172.21.15.164:0/4129760699 learned_addr learned my addr 172.21.15.164:0/4129760699 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:09.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f0597400640 1 --2- 172.21.15.164:0/4129760699 >> v2:172.21.15.172:3300/0 conn(0x7f05a006cce0 0x7f05a00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308820484 0 0) 0x7f05a015e870 con 0x7f05a00cecc0 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0584003530 con 0x7f05a00cecc0 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f0597400640 1 -- 172.21.15.164:0/4129760699 >> v1:172.21.15.172:6789/0 conn(0x7f05a015ab10 legacy=0x7f05a00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f0597400640 1 -- 172.21.15.164:0/4129760699 >> v1:172.21.15.164:6789/0 conn(0x7f05a00cecc0 legacy=0x7f05a00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1535620192 0 0) 0x7f0584003530 con 0x7f05a00cecc0 2024-05-08T04:39:09.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.102+0000 7f0597400640 1 -- 172.21.15.164:0/4129760699 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a015ee30 con 0x7f05a006cce0 2024-05-08T04:39:09.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f0597400640 1 --2- 172.21.15.164:0/4129760699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a006cce0 0x7f05a00ce780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05880097c0 tx=0x7f058801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71259cd5f422942a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880b0070 con 0x7f05a006cce0 2024-05-08T04:39:09.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05880ab400 con 0x7f05a006cce0 2024-05-08T04:39:09.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f059f400640 1 -- 172.21.15.164:0/4129760699 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598001680 con 0x7f05a006cce0 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- 172.21.15.164:0/4129760699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a006cce0 msgr2=0x7f05a00ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 --2- 172.21.15.164:0/4129760699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a006cce0 0x7f05a00ce780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05880097c0 tx=0x7f058801c330 comp rx=0 tx=0).stop 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- 172.21.15.164:0/4129760699 shutdown_connections 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 --2- 172.21.15.164:0/4129760699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a006cce0 0x7f05a00ce780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- 172.21.15.164:0/4129760699 >> 172.21.15.164:0/4129760699 conn(0x7f05a00cf770 msgr2=0x7f05a00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- 172.21.15.164:0/4129760699 shutdown_connections 2024-05-08T04:39:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- 172.21.15.164:0/4129760699 wait complete. 2024-05-08T04:39:09.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 Processor -- start 2024-05-08T04:39:09.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- start start 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a00cecc0 0x7f05a00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a015ab10 0x7f05a014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00aef70 con 0x7f05a006cce0 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00aedf0 con 0x7f05a015ab10 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.106+0000 7f05a6e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0168af0 con 0x7f05a00cecc0 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56664/0 (socket says 172.21.15.164:56664) 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 -- 172.21.15.164:0/477172597 learned_addr learned my addr 172.21.15.164:0/477172597 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f05a4e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a015ab10 0x7f05a014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f0597400640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a00cecc0 0x7f05a00c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 -- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a00cecc0 msgr2=0x7f05a00c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a00cecc0 0x7f05a00c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 -- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a015ab10 msgr2=0x7f05a014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a015ab10 0x7f05a014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 -- 172.21.15.164:0/477172597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a00c7b70 con 0x7f05a006cce0 2024-05-08T04:39:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059fe00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f059000c7f0 tx=0x7f059000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590003b80 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590003d20 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000aa90 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a00b3e90 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a00b4450 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a0072d30 con 0x7f05a006cce0 2024-05-08T04:39:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.110+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f059000ac30 con 0x7f05a006cce0 2024-05-08T04:39:09.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.114+0000 7f059cc00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 0x7f057c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.114+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0590084e80 con 0x7f05a006cce0 2024-05-08T04:39:09.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.114+0000 7f0597400640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 0x7f057c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.114+0000 7f0597400640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 0x7f057c069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05880ad000 tx=0x7f05880a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.118+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0590055830 con 0x7f05a006cce0 2024-05-08T04:39:09.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05a004f010 con 0x7f05a006cce0 2024-05-08T04:39:09.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f059cc00640 1 -- 172.21.15.164:0/477172597 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05900598d0 con 0x7f05a006cce0 2024-05-08T04:39:09.274 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 msgr2=0x7f057c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 0x7f057c069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05880ad000 tx=0x7f05880a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 msgr2=0x7f05a00b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f059000c7f0 tx=0x7f059000ccb0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 shutdown_connections 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f057c066f60 0x7f057c069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05a00cecc0 0x7f05a00c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05a015ab10 0x7f05a014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 --2- 172.21.15.164:0/477172597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05a006cce0 0x7f05a00b38b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 >> 172.21.15.164:0/477172597 conn(0x7f05a00cf770 msgr2=0x7f05a00c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 shutdown_connections 2024-05-08T04:39:09.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.270+0000 7f05a6e00640 1 -- 172.21.15.164:0/477172597 wait complete. 2024-05-08T04:39:09.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:09.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 Processor -- start 2024-05-08T04:39:09.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 -- start start 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42a415ce10 0x7f42a406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f42a40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40c6600 con 0x7f42a415ba20 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a415fbe0 con 0x7f42a415ce10 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42aa800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c6040 con 0x7f42a406e390 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f42a415ba20 0x7f42a4160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54200/0 (socket says 172.21.15.164:54200) 2024-05-08T04:39:09.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 -- 172.21.15.164:0/735271071 learned_addr learned my addr 172.21.15.164:0/735271071 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:09.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995317866 0 0) 0x7f42a40c6600 con 0x7f42a415ba20 2024-05-08T04:39:09.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4288003530 con 0x7f42a415ba20 2024-05-08T04:39:09.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2a00640 1 --2- 172.21.15.164:0/735271071 >> v2:172.21.15.172:3300/0 conn(0x7f42a415ce10 0x7f42a406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3e00640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f42a40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084617374 0 0) 0x7f4288003530 con 0x7f42a415ba20 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 msgr2=0x7f42a40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f42a40ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 >> v2:172.21.15.172:3300/0 conn(0x7f42a415ce10 msgr2=0x7f42a406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 --2- 172.21.15.164:0/735271071 >> v2:172.21.15.172:3300/0 conn(0x7f42a415ce10 0x7f42a406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40c6320 con 0x7f42a415ba20 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f4294001680 con 0x7f42a415ba20 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 >> v1:172.21.15.164:6789/0 conn(0x7f42a415ba20 legacy=0x7f42a4160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f4288003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f4288004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4288004d50 0x7f42880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a415fbe0 con 0x7f42880039c0 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c6040 con 0x7f4288004d50 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428800b9d0 con 0x7f42a406e390 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f4288003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3e00640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4288004d50 0x7f42880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a2a00640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f4288004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 -- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 msgr2=0x7f4288004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a406e390 0x7f4288004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 -- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4288004d50 msgr2=0x7f42880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4288004d50 0x7f42880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.430+0000 7f42a3400640 1 -- 172.21.15.164:0/735271071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288003530 con 0x7f42880039c0 2024-05-08T04:39:09.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42a3400640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f4288003de0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f429400c270 tx=0x7f429401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb549d88f32dce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294007d60 con 0x7f42880039c0 2024-05-08T04:39:09.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940a9070 con 0x7f42880039c0 2024-05-08T04:39:09.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42a2000640 1 -- 172.21.15.164:0/735271071 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294005d20 con 0x7f42880039c0 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 -- 172.21.15.164:0/735271071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 msgr2=0x7f4288003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 --2- 172.21.15.164:0/735271071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f4288003de0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f429400c270 tx=0x7f429401ccc0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42a3e00640 1 -- 172.21.15.164:0/735271071 reap_dead start 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 -- 172.21.15.164:0/735271071 shutdown_connections 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 -- 172.21.15.164:0/735271071 >> 172.21.15.164:0/735271071 conn(0x7f42a40cf770 msgr2=0x7f42a40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 -- 172.21.15.164:0/735271071 shutdown_connections 2024-05-08T04:39:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.434+0000 7f42aa800640 1 -- 172.21.15.164:0/735271071 wait complete. 2024-05-08T04:39:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 Processor -- start 2024-05-08T04:39:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 -- start start 2024-05-08T04:39:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a41ff5d0 0x7f42a41fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a40b0390 0x7f42a40b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cda00 con 0x7f42880039c0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cd880 con 0x7f42a41ff5d0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42aa800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cdb80 con 0x7f42a40b0390 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46574/0 (socket says 172.21.15.164:46574) 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 -- 172.21.15.164:0/4134266039 learned_addr learned my addr 172.21.15.164:0/4134266039 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a2a00640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a41ff5d0 0x7f42a41fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3e00640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a40b0390 0x7f42a40b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 -- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a40b0390 msgr2=0x7f42a40b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a40b0390 0x7f42a40b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 -- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a41ff5d0 msgr2=0x7f42a41fadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a41ff5d0 0x7f42a41fadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 -- 172.21.15.164:0/4134266039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4151820 con 0x7f42880039c0 2024-05-08T04:39:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f42a3400640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f429400c270 tx=0x7f4288005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294005d00 con 0x7f42880039c0 2024-05-08T04:39:09.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940047f0 con 0x7f42880039c0 2024-05-08T04:39:09.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.438+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294004ba0 con 0x7f42880039c0 2024-05-08T04:39:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a4151b00 con 0x7f42880039c0 2024-05-08T04:39:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a41520c0 con 0x7f42880039c0 2024-05-08T04:39:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a4072d30 con 0x7f42880039c0 2024-05-08T04:39:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42940a57a0 con 0x7f42880039c0 2024-05-08T04:39:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f429b400640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 0x7f4284069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f429411e770 con 0x7f42880039c0 2024-05-08T04:39:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.442+0000 7f42a2a00640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 0x7f4284069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.446+0000 7f42a2a00640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 0x7f4284069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f42a40b0f30 tx=0x7f4290009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.446+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42940169d0 con 0x7f42880039c0 2024-05-08T04:39:09.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.602+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42a406e130 con 0x7f42880039c0 2024-05-08T04:39:09.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.602+0000 7f429b400640 1 -- 172.21.15.164:0/4134266039 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42940ef110 con 0x7f42880039c0 2024-05-08T04:39:09.606 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:09.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.602+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 msgr2=0x7f4284069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 0x7f4284069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f42a40b0f30 tx=0x7f4290009190 comp rx=0 tx=0).stop 2024-05-08T04:39:09.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 msgr2=0x7f42a4151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f429400c270 tx=0x7f4288005840 comp rx=0 tx=0).stop 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 shutdown_connections 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4284066f90 0x7f4284069450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a40b0390 0x7f42a40b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a41ff5d0 0x7f42a41fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 --2- 172.21.15.164:0/4134266039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42880039c0 0x7f42a4151240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 >> 172.21.15.164:0/4134266039 conn(0x7f42a40cf770 msgr2=0x7f42a4153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 shutdown_connections 2024-05-08T04:39:09.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.606+0000 7f42aa800640 1 -- 172.21.15.164:0/4134266039 wait complete. 2024-05-08T04:39:09.627 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:09.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 Processor -- start 2024-05-08T04:39:09.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 -- start start 2024-05-08T04:39:09.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b440addd0 0x7f8b4406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b44072320 con 0x7f8b4406b1d0 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4406d4e0 con 0x7f8b440addd0 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b48a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44072600 con 0x7f8b440ccdf0 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b42800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8b4406b1d0 0x7f8b4406df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54212/0 (socket says 172.21.15.164:54212) 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b42800640 1 -- 172.21.15.164:0/3593709282 learned_addr learned my addr 172.21.15.164:0/3593709282 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b39400640 1 --2- 172.21.15.164:0/3593709282 >> v2:172.21.15.172:3300/0 conn(0x7f8b440addd0 0x7f8b4406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b41e00640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b41400640 1 -- 172.21.15.164:0/3593709282 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394587935 0 0) 0x7f8b44072320 con 0x7f8b4406b1d0 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b41400640 1 -- 172.21.15.164:0/3593709282 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b28003530 con 0x7f8b4406b1d0 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b39400640 1 -- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 msgr2=0x7f8b440ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b39400640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b39400640 1 -- 172.21.15.164:0/3593709282 >> v1:172.21.15.164:6789/0 conn(0x7f8b4406b1d0 legacy=0x7f8b4406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b39400640 1 -- 172.21.15.164:0/3593709282 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4406d730 con 0x7f8b440addd0 2024-05-08T04:39:09.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.762+0000 7f8b41e00640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:09.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b39400640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b440addd0 0x7f8b4406ac90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0097c0 tx=0x7f8b2c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e25affffff74dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b41400640 1 -- 172.21.15.164:0/3593709282 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0b0070 con 0x7f8b440addd0 2024-05-08T04:39:09.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b41400640 1 -- 172.21.15.164:0/3593709282 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0ab600 con 0x7f8b440addd0 2024-05-08T04:39:09.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b41400640 1 -- 172.21.15.164:0/3593709282 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0ab9b0 con 0x7f8b440addd0 2024-05-08T04:39:09.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 -- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b440addd0 msgr2=0x7f8b4406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b440addd0 0x7f8b4406ac90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0097c0 tx=0x7f8b2c01c410 comp rx=0 tx=0).stop 2024-05-08T04:39:09.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 -- 172.21.15.164:0/3593709282 shutdown_connections 2024-05-08T04:39:09.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b440addd0 0x7f8b4406ac90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 --2- 172.21.15.164:0/3593709282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 -- 172.21.15.164:0/3593709282 >> 172.21.15.164:0/3593709282 conn(0x7f8b44066fa0 msgr2=0x7f8b4406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 -- 172.21.15.164:0/3593709282 shutdown_connections 2024-05-08T04:39:09.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.766+0000 7f8b48a00640 1 -- 172.21.15.164:0/3593709282 wait complete. 2024-05-08T04:39:09.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 Processor -- start 2024-05-08T04:39:09.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- start start 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b4406b1d0 0x7f8b44154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44157af0 con 0x7f8b440addd0 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44071ff0 con 0x7f8b4406b1d0 2024-05-08T04:39:09.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44157c70 con 0x7f8b440ccdf0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46582/0 (socket says 172.21.15.164:46582) 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 -- 172.21.15.164:0/801377724 learned_addr learned my addr 172.21.15.164:0/801377724 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b42800640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b41e00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b4406b1d0 0x7f8b44154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 -- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 msgr2=0x7f8b440cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 -- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b4406b1d0 msgr2=0x7f8b44154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b4406b1d0 0x7f8b44154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 -- 172.21.15.164:0/801377724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440b5530 con 0x7f8b440addd0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b39400640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c01a040 tx=0x7f8b2c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0b8020 con 0x7f8b440addd0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0acbf0 con 0x7f8b440addd0 2024-05-08T04:39:09.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b440b28d0 con 0x7f8b440addd0 2024-05-08T04:39:09.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0b0040 con 0x7f8b440addd0 2024-05-08T04:39:09.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.770+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b440b5010 con 0x7f8b440addd0 2024-05-08T04:39:09.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44069d60 con 0x7f8b440addd0 2024-05-08T04:39:09.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b2c0bb020 con 0x7f8b440addd0 2024-05-08T04:39:09.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b3aa00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 0x7f8b20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:09.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c11ed60 con 0x7f8b440addd0 2024-05-08T04:39:09.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b41e00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 0x7f8b20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:09.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.774+0000 7f8b41e00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 0x7f8b20069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009a50 tx=0x7f8b34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:09.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.778+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b2c0c2030 con 0x7f8b440addd0 2024-05-08T04:39:09.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b440cdd10 con 0x7f8b440addd0 2024-05-08T04:39:09.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b3aa00640 1 -- 172.21.15.164:0/801377724 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f8b2c016090 con 0x7f8b440addd0 2024-05-08T04:39:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:09.974 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:39:09.974 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 msgr2=0x7f8b20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 0x7f8b20069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009a50 tx=0x7f8b34008040 comp rx=0 tx=0).stop 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 msgr2=0x7f8b4414c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c01a040 tx=0x7f8b2c0191b0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 shutdown_connections 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8b20066f60 0x7f8b20069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b440ccdf0 0x7f8b440cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b440addd0 0x7f8b4414c630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 --2- 172.21.15.164:0/801377724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b4406b1d0 0x7f8b44154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:09.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 >> 172.21.15.164:0/801377724 conn(0x7f8b44066fa0 msgr2=0x7f8b4415f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:09.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 shutdown_connections 2024-05-08T04:39:09.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:09.970+0000 7f8b48a00640 1 -- 172.21.15.164:0/801377724 wait complete. 2024-05-08T04:39:09.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:10.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.126+0000 7fbe4f000640 1 Processor -- start 2024-05-08T04:39:10.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.126+0000 7fbe4f000640 1 -- start start 2024-05-08T04:39:10.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4f000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4f000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe480cd9f0 con 0x7fbe4815ba70 2024-05-08T04:39:10.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4f000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4815fe70 con 0x7fbe4815be50 2024-05-08T04:39:10.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cdcd0 con 0x7fbe480cfda0 2024-05-08T04:39:10.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4d000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4d000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:41000/0 (socket says 172.21.15.164:41000) 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe3f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4d000640 1 -- 172.21.15.164:0/1717234314 learned_addr learned my addr 172.21.15.164:0/1717234314 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe3f400640 1 -- 172.21.15.164:0/1717234314 >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 msgr2=0x7fbe480b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe3f400640 1 --2- 172.21.15.164:0/1717234314 >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe3f400640 1 -- 172.21.15.164:0/1717234314 >> v1:172.21.15.172:6789/0 conn(0x7fbe4815ba70 legacy=0x7fbe480cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe3f400640 1 -- 172.21.15.164:0/1717234314 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4815f820 con 0x7fbe480cfda0 2024-05-08T04:39:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.130+0000 7fbe4d000640 1 --2- 172.21.15.164:0/1717234314 >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe3f400640 1 --2- 172.21.15.164:0/1717234314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48160300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097c0 tx=0x7fbe3001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619748aa1febab7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe47400640 1 -- 172.21.15.164:0/1717234314 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300b2070 con 0x7fbe480cfda0 2024-05-08T04:39:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe47400640 1 -- 172.21.15.164:0/1717234314 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe300ad680 con 0x7fbe480cfda0 2024-05-08T04:39:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe47400640 1 -- 172.21.15.164:0/1717234314 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300ada50 con 0x7fbe480cfda0 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- 172.21.15.164:0/1717234314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 msgr2=0x7fbe48160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 --2- 172.21.15.164:0/1717234314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48160300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097c0 tx=0x7fbe3001c470 comp rx=0 tx=0).stop 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- 172.21.15.164:0/1717234314 shutdown_connections 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 --2- 172.21.15.164:0/1717234314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48160300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 --2- 172.21.15.164:0/1717234314 >> v2:172.21.15.172:3300/0 conn(0x7fbe4815be50 0x7fbe480b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- 172.21.15.164:0/1717234314 >> 172.21.15.164:0/1717234314 conn(0x7fbe4806ac90 msgr2=0x7fbe4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- 172.21.15.164:0/1717234314 shutdown_connections 2024-05-08T04:39:10.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- 172.21.15.164:0/1717234314 wait complete. 2024-05-08T04:39:10.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 Processor -- start 2024-05-08T04:39:10.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.134+0000 7fbe4f000640 1 -- start start 2024-05-08T04:39:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe4815ba70 0x7fbe4814a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48168d40 con 0x7fbe4815be50 2024-05-08T04:39:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48168bc0 con 0x7fbe4815ba70 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48168ec0 con 0x7fbe480cfda0 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46586/0 (socket says 172.21.15.164:46586) 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 -- 172.21.15.164:0/3394537988 learned_addr learned my addr 172.21.15.164:0/3394537988 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe3f400640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe4815ba70 0x7fbe4814a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe47e00640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 -- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 msgr2=0x7fbe48149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 -- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe4815ba70 msgr2=0x7fbe4814a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe4815ba70 0x7fbe4814a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 -- 172.21.15.164:0/3394537988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48152320 con 0x7fbe4815be50 2024-05-08T04:39:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4d000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e960 tx=0x7fbe4000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40017070 con 0x7fbe4815be50 2024-05-08T04:39:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4000ca00 con 0x7fbe4815be50 2024-05-08T04:39:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe48152600 con 0x7fbe4815be50 2024-05-08T04:39:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000cdd0 con 0x7fbe4815be50 2024-05-08T04:39:10.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe48152bc0 con 0x7fbe4815be50 2024-05-08T04:39:10.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe48069d60 con 0x7fbe4815be50 2024-05-08T04:39:10.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.138+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe40013470 con 0x7fbe4815be50 2024-05-08T04:39:10.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.142+0000 7fbe44c00640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 0x7fbe24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.142+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbe400851b0 con 0x7fbe4815be50 2024-05-08T04:39:10.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.142+0000 7fbe47e00640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 0x7fbe24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.142+0000 7fbe47e00640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 0x7fbe24069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe38001d50 tx=0x7fbe38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.146+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe40055af0 con 0x7fbe4815be50 2024-05-08T04:39:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.298+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe4806c580 con 0x7fbe4815be50 2024-05-08T04:39:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe44c00640 1 -- 172.21.15.164:0/3394537988 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fbe40059b90 con 0x7fbe4815be50 2024-05-08T04:39:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:10.305 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":21,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:09.901855+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 msgr2=0x7fbe24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 0x7fbe24069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe38001d50 tx=0x7fbe38009040 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 msgr2=0x7fbe481562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e960 tx=0x7fbe4000ee20 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 shutdown_connections 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe24066f60 0x7fbe24069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe4815be50 0x7fbe481562c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe4815ba70 0x7fbe4814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 --2- 172.21.15.164:0/3394537988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe480cfda0 0x7fbe48149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 >> 172.21.15.164:0/3394537988 conn(0x7fbe4806ac90 msgr2=0x7fbe480cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 shutdown_connections 2024-05-08T04:39:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.302+0000 7fbe4f000640 1 -- 172.21.15.164:0/3394537988 wait complete. 2024-05-08T04:39:10.326 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:39:10.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:10.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 Processor -- start 2024-05-08T04:39:10.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 -- start start 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e600b2f00 0x7f0e60168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6006bfc0 con 0x7f0e600d1fb0 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600b1130 con 0x7f0e600d1540 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e65200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600d1a40 con 0x7f0e600b2f00 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5f400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0e600d1fb0 0x7f0e6006b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54216/0 (socket says 172.21.15.164:54216) 2024-05-08T04:39:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5f400640 1 -- 172.21.15.164:0/1784876138 learned_addr learned my addr 172.21.15.164:0/1784876138 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5fe00640 1 --2- 172.21.15.164:0/1784876138 >> v2:172.21.15.172:3300/0 conn(0x7f0e600b2f00 0x7f0e60168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708921229 0 0) 0x7f0e6006bfc0 con 0x7f0e600d1fb0 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e44003530 con 0x7f0e600d1fb0 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5fe00640 1 -- 172.21.15.164:0/1784876138 >> v1:172.21.15.172:6789/0 conn(0x7f0e600d1540 legacy=0x7f0e6006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5fe00640 1 -- 172.21.15.164:0/1784876138 >> v1:172.21.15.164:6789/0 conn(0x7f0e600d1fb0 legacy=0x7f0e6006b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191181889 0 0) 0x7f0e600b1130 con 0x7f0e600d1540 2024-05-08T04:39:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.462+0000 7f0e5fe00640 1 -- 172.21.15.164:0/1784876138 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600b1410 con 0x7f0e600b2f00 2024-05-08T04:39:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e5fe00640 1 --2- 172.21.15.164:0/1784876138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600b2f00 0x7f0e60168f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009d90 tx=0x7f0e5801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5063bf0047bea40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580b2070 con 0x7f0e600b2f00 2024-05-08T04:39:10.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e580a5540 con 0x7f0e600b2f00 2024-05-08T04:39:10.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e5ea00640 1 -- 172.21.15.164:0/1784876138 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50001600 con 0x7f0e600b2f00 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 -- 172.21.15.164:0/1784876138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600b2f00 msgr2=0x7f0e60168f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 --2- 172.21.15.164:0/1784876138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600b2f00 0x7f0e60168f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009d90 tx=0x7f0e5801c7b0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 -- 172.21.15.164:0/1784876138 shutdown_connections 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 --2- 172.21.15.164:0/1784876138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600b2f00 0x7f0e60168f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 -- 172.21.15.164:0/1784876138 >> 172.21.15.164:0/1784876138 conn(0x7f0e6009cd00 msgr2=0x7f0e600d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 -- 172.21.15.164:0/1784876138 shutdown_connections 2024-05-08T04:39:10.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.466+0000 7f0e65200640 1 -- 172.21.15.164:0/1784876138 wait complete. 2024-05-08T04:39:10.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 Processor -- start 2024-05-08T04:39:10.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- start start 2024-05-08T04:39:10.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 0x7f0e600b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 0x7f0e60152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016c060 con 0x7f0e600b2f00 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016bee0 con 0x7f0e600d1540 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016c1e0 con 0x7f0e600d1fb0 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46590/0 (socket says 172.21.15.164:46590) 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 -- 172.21.15.164:0/3861357402 learned_addr learned my addr 172.21.15.164:0/3861357402 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e57e00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 0x7f0e600b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5fe00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 0x7f0e60152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 -- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 msgr2=0x7f0e60152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 0x7f0e60152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 -- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 msgr2=0x7f0e600b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 0x7f0e600b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 -- 172.21.15.164:0/3861357402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600cbfa0 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e57e00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 0x7f0e600b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5fe00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 0x7f0e60152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e5f400640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c730 tx=0x7f0e5000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50018070 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500072f0 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600cc280 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500047f0 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600cc840 con 0x7f0e600b2f00 2024-05-08T04:39:10.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.470+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6004ef30 con 0x7f0e600b2f00 2024-05-08T04:39:10.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.474+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e5001b070 con 0x7f0e600b2f00 2024-05-08T04:39:10.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.474+0000 7f0e57400640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.474+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0e50084650 con 0x7f0e600b2f00 2024-05-08T04:39:10.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.474+0000 7f0e57e00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.474+0000 7f0e57e00640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 0x7f0e3c069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b9a20 tx=0x7f0e48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.478+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e500555e0 con 0x7f0e600b2f00 2024-05-08T04:39:10.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e6006bf70 con 0x7f0e600b2f00 2024-05-08T04:39:10.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e57400640 1 -- 172.21.15.164:0/3861357402 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f0e50054ee0 con 0x7f0e600b2f00 2024-05-08T04:39:10.670 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:10.670 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:39:10.670 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 msgr2=0x7f0e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 0x7f0e3c069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b9a20 tx=0x7f0e48009040 comp rx=0 tx=0).stop 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 msgr2=0x7f0e60152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c730 tx=0x7f0e5000cb30 comp rx=0 tx=0).stop 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 shutdown_connections 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e600d1fb0 0x7f0e60152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e600d1540 0x7f0e600b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 --2- 172.21.15.164:0/3861357402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600b2f00 0x7f0e60152070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 >> 172.21.15.164:0/3861357402 conn(0x7f0e6009cd00 msgr2=0x7f0e600b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.666+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 shutdown_connections 2024-05-08T04:39:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.670+0000 7f0e65200640 1 -- 172.21.15.164:0/3861357402 wait complete. 2024-05-08T04:39:10.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:10.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 Processor -- start 2024-05-08T04:39:10.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 -- start start 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa8c815ab60 0x7fa8c8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c80cfda0 0x7fa8c80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c8072640 con 0x7fa8c80b26c0 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c806c770 con 0x7fa8c815ab60 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8cec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c806ca50 con 0x7fa8c80cfda0 2024-05-08T04:39:10.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa8c815ab60 0x7fa8c8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa8c80b26c0 0x7fa8c80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54232/0 (socket says 172.21.15.164:54232) 2024-05-08T04:39:10.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7400640 1 -- 172.21.15.164:0/1369303052 learned_addr learned my addr 172.21.15.164:0/1369303052 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:10.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8ccc00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c80cfda0 0x7fa8c80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7e00640 1 -- 172.21.15.164:0/1369303052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c80cfda0 msgr2=0x7fa8c80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:10.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7e00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c80cfda0 0x7fa8c80cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7e00640 1 -- 172.21.15.164:0/1369303052 >> v1:172.21.15.164:6789/0 conn(0x7fa8c80b26c0 legacy=0x7fa8c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.826+0000 7fa8c7e00640 1 -- 172.21.15.164:0/1369303052 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c806cd30 con 0x7fa8c815ab60 2024-05-08T04:39:10.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8c7e00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c815ab60 0x7fa8c8072100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009d90 tx=0x7fa8b801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5819ccd8ab49a2b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8c6a00640 1 -- 172.21.15.164:0/1369303052 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80b2070 con 0x7fa8c815ab60 2024-05-08T04:39:10.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8c6a00640 1 -- 172.21.15.164:0/1369303052 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80a5830 con 0x7fa8c815ab60 2024-05-08T04:39:10.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8c6a00640 1 -- 172.21.15.164:0/1369303052 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80a5c00 con 0x7fa8c815ab60 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 -- 172.21.15.164:0/1369303052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c815ab60 msgr2=0x7fa8c8072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c815ab60 0x7fa8c8072100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009d90 tx=0x7fa8b801c970 comp rx=0 tx=0).stop 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 -- 172.21.15.164:0/1369303052 shutdown_connections 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c80cfda0 0x7fa8c80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 --2- 172.21.15.164:0/1369303052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c815ab60 0x7fa8c8072100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 -- 172.21.15.164:0/1369303052 >> 172.21.15.164:0/1369303052 conn(0x7fa8c806ac90 msgr2=0x7fa8c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 -- 172.21.15.164:0/1369303052 shutdown_connections 2024-05-08T04:39:10.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.830+0000 7fa8cec00640 1 -- 172.21.15.164:0/1369303052 wait complete. 2024-05-08T04:39:10.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 Processor -- start 2024-05-08T04:39:10.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- start start 2024-05-08T04:39:10.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c80b26c0 0x7fa8c8152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c815ab60 0x7fa8c814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80aeeb0 con 0x7fa8c80cfda0 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80aed30 con 0x7fa8c80b26c0 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8167970 con 0x7fa8c815ab60 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46618/0 (socket says 172.21.15.164:46618) 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 -- 172.21.15.164:0/2423558614 learned_addr learned my addr 172.21.15.164:0/2423558614 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8ccc00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c815ab60 0x7fa8c814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7e00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c80b26c0 0x7fa8c8152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 -- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c815ab60 msgr2=0x7fa8c814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c815ab60 0x7fa8c814c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 -- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c80b26c0 msgr2=0x7fa8c8152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c80b26c0 0x7fa8c8152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 -- 172.21.15.164:0/2423558614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c81533b0 con 0x7fa8c80cfda0 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8c7400640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c420 tx=0x7fa8b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4017070 con 0x7fa8c80cfda0 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4005cf0 con 0x7fa8c80cfda0 2024-05-08T04:39:10.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40072f0 con 0x7fa8c80cfda0 2024-05-08T04:39:10.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c814eea0 con 0x7fa8c80cfda0 2024-05-08T04:39:10.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.834+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81fad20 con 0x7fa8c80cfda0 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c8069d60 con 0x7fa8c80cfda0 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8b401a070 con 0x7fa8c80cfda0 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8bfe00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 0x7fa8a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa8b4084480 con 0x7fa8c80cfda0 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8c7e00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 0x7fa8a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.838+0000 7fa8c7e00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 0x7fa8a8069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80af000 tx=0x7fa8b80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:10.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:10.842+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8b4055400 con 0x7fa8c80cfda0 2024-05-08T04:39:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.034+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8c806d9a0 con 0x7fa8c80cfda0 2024-05-08T04:39:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8bfe00640 1 -- 172.21.15.164:0/2423558614 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4941 (secure 0 0 0) 0x7fa8b4054d00 con 0x7fa8c80cfda0 2024-05-08T04:39:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:11.041 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3084083078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3084083078},{"type":"v1","addr":"172.21.15.164:6837","nonce":3084083078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/2816560269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2816560269},{"type":"v1","addr":"172.21.15.164:6836","nonce":2816560269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/710216123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":710216123},{"type":"v1","addr":"172.21.15.172:6834","nonce":710216123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:38:29.101740+0000","modified":"2024-05-08T04:38:32.110093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3487014303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3487014303},{"type":"v1","addr":"172.21.15.172:6835","nonce":3487014303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T04:39:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-05-08T04:39:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 msgr2=0x7fa8a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 0x7fa8a8069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80af000 tx=0x7fa8b80a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 msgr2=0x7fa8c814e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c420 tx=0x7fa8b400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 shutdown_connections 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8a8066f60 0x7fa8a8069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c815ab60 0x7fa8c814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8c80cfda0 0x7fa8c814e960 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 --2- 172.21.15.164:0/2423558614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c80b26c0 0x7fa8c8152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 >> 172.21.15.164:0/2423558614 conn(0x7fa8c806ac90 msgr2=0x7fa8c8153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 shutdown_connections 2024-05-08T04:39:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.038+0000 7fa8cec00640 1 -- 172.21.15.164:0/2423558614 wait complete. 2024-05-08T04:39:11.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.190+0000 7f4027e00640 1 Processor -- start 2024-05-08T04:39:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.190+0000 7f4027e00640 1 -- start start 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4027e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f402015ab60 0x7f4020072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4027e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f40200cfda0 0x7f40200cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4027e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4020072640 con 0x7f40200b26c0 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4027e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402006c770 con 0x7f40200cfda0 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4027e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402006ca50 con 0x7f402015ab60 2024-05-08T04:39:11.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f40200cfda0 0x7f40200cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4024a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f40200b26c0 0x7f40200cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54238/0 (socket says 172.21.15.164:54238) 2024-05-08T04:39:11.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4024a00640 1 -- 172.21.15.164:0/3693853816 learned_addr learned my addr 172.21.15.164:0/3693853816 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:11.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025400640 1 --2- 172.21.15.164:0/3693853816 >> v2:172.21.15.172:3300/0 conn(0x7f402015ab60 0x7f4020072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025e00640 1 -- 172.21.15.164:0/3693853816 >> v2:172.21.15.172:3300/0 conn(0x7f402015ab60 msgr2=0x7f4020072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025e00640 1 --2- 172.21.15.164:0/3693853816 >> v2:172.21.15.172:3300/0 conn(0x7f402015ab60 0x7f4020072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025e00640 1 -- 172.21.15.164:0/3693853816 >> v1:172.21.15.164:6789/0 conn(0x7f40200b26c0 legacy=0x7f40200cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.194+0000 7f4025e00640 1 -- 172.21.15.164:0/3693853816 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402006cd30 con 0x7f40200cfda0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4025e00640 1 --2- 172.21.15.164:0/3693853816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40200cfda0 0x7f40200cd910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40200cd420 tx=0x7f401801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37ebf509e8c33aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f401fe00640 1 -- 172.21.15.164:0/3693853816 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180b4070 con 0x7f40200cfda0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f401fe00640 1 -- 172.21.15.164:0/3693853816 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180a7a10 con 0x7f40200cfda0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f401fe00640 1 -- 172.21.15.164:0/3693853816 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180a7de0 con 0x7f40200cfda0 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- 172.21.15.164:0/3693853816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40200cfda0 msgr2=0x7f40200cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 --2- 172.21.15.164:0/3693853816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40200cfda0 0x7f40200cd910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40200cd420 tx=0x7f401801eb40 comp rx=0 tx=0).stop 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- 172.21.15.164:0/3693853816 shutdown_connections 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 --2- 172.21.15.164:0/3693853816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40200cfda0 0x7f40200cd910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 --2- 172.21.15.164:0/3693853816 >> v2:172.21.15.172:3300/0 conn(0x7f402015ab60 0x7f4020072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- 172.21.15.164:0/3693853816 >> 172.21.15.164:0/3693853816 conn(0x7f402006ac90 msgr2=0x7f402006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- 172.21.15.164:0/3693853816 shutdown_connections 2024-05-08T04:39:11.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- 172.21.15.164:0/3693853816 wait complete. 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 Processor -- start 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.198+0000 7f4027e00640 1 -- start start 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40200b26c0 0x7f40201492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 0x7f40200ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200aeeb0 con 0x7f402015ab60 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200aed30 con 0x7f40200b26c0 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cc1b0 con 0x7f40200cfda0 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46630/0 (socket says 172.21.15.164:46630) 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 -- 172.21.15.164:0/1482510681 learned_addr learned my addr 172.21.15.164:0/1482510681 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:11.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025400640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40200b26c0 0x7f40201492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4024a00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 0x7f40200ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 -- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 msgr2=0x7f40200ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 0x7f40200ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 -- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40200b26c0 msgr2=0x7f40201492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40200b26c0 0x7f40201492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 -- 172.21.15.164:0/1482510681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40201560e0 con 0x7f402015ab60 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4024a00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 0x7f40200ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4025e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f401801b750 tx=0x7f401801b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180b4040 con 0x7f402015ab60 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180bd070 con 0x7f402015ab60 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180b88a0 con 0x7f402015ab60 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40201563c0 con 0x7f402015ab60 2024-05-08T04:39:11.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4020156980 con 0x7f402015ab60 2024-05-08T04:39:11.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4020069d60 con 0x7f402015ab60 2024-05-08T04:39:11.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.202+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40180b8a40 con 0x7f402015ab60 2024-05-08T04:39:11.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.206+0000 7f401d600640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.206+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f401811f8d0 con 0x7f402015ab60 2024-05-08T04:39:11.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.206+0000 7f4025400640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.206+0000 7f4025400640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 0x7f4000069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f401400b930 tx=0x7f4014008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.210+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4018018730 con 0x7f402015ab60 2024-05-08T04:39:11.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.366+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f402015a210 con 0x7f402015ab60 2024-05-08T04:39:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.366+0000 7f401d600640 1 -- 172.21.15.164:0/1482510681 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f40180f0970 con 0x7f402015ab60 2024-05-08T04:39:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:11.372 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":22,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:10.910621+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:38:27.601306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T04:38:28.134395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 msgr2=0x7f4000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 0x7f4000069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f401400b930 tx=0x7f4014008040 comp rx=0 tx=0).stop 2024-05-08T04:39:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 msgr2=0x7f40200addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f401801b750 tx=0x7f401801b8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 shutdown_connections 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000066f60 0x7f4000069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40200cfda0 0x7f40200ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402015ab60 0x7f40200addd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 --2- 172.21.15.164:0/1482510681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40200b26c0 0x7f40201492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 >> 172.21.15.164:0/1482510681 conn(0x7f402006ac90 msgr2=0x7f40200af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 shutdown_connections 2024-05-08T04:39:11.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.370+0000 7f4027e00640 1 -- 172.21.15.164:0/1482510681 wait complete. 2024-05-08T04:39:11.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:39:11.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.526+0000 7f0613000640 1 Processor -- start 2024-05-08T04:39:11.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.526+0000 7f0613000640 1 -- start start 2024-05-08T04:39:11.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f060c0af480 0x7f060c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 0x7f060c06cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f060c06bca0 0x7f060c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c161810 con 0x7f060c06bca0 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c06eed0 con 0x7f060c0af480 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0613000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c071f50 con 0x7f060c0cdfa0 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f060c06bca0 0x7f060c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f060c06bca0 0x7f060c06c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46646/0 (socket says 172.21.15.164:46646) 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 -- 172.21.15.164:0/3700114264 learned_addr learned my addr 172.21.15.164:0/3700114264 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f060be00640 1 --2- 172.21.15.164:0/3700114264 >> v2:172.21.15.172:3300/0 conn(0x7f060c0af480 0x7f060c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0603400640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 0x7f060c06cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 -- 172.21.15.164:0/3700114264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 msgr2=0x7f060c06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 0x7f060c06cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 -- 172.21.15.164:0/3700114264 >> v2:172.21.15.172:3300/0 conn(0x7f060c0af480 msgr2=0x7f060c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:11.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 --2- 172.21.15.164:0/3700114264 >> v2:172.21.15.172:3300/0 conn(0x7f060c0af480 0x7f060c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0611000640 1 -- 172.21.15.164:0/3700114264 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c071980 con 0x7f060c06bca0 2024-05-08T04:39:11.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.530+0000 7f0603400640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 0x7f060c06cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0611000640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c06c130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f060c070780 tx=0x7f060401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d44aa04f8dc10e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f060b400640 1 -- 172.21.15.164:0/3700114264 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040a5890 con 0x7f060c06bca0 2024-05-08T04:39:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f060b400640 1 -- 172.21.15.164:0/3700114264 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06040a5a30 con 0x7f060c06bca0 2024-05-08T04:39:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f060b400640 1 -- 172.21.15.164:0/3700114264 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040a5e00 con 0x7f060c06bca0 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- 172.21.15.164:0/3700114264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 msgr2=0x7f060c06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c06c130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f060c070780 tx=0x7f060401c9f0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- 172.21.15.164:0/3700114264 shutdown_connections 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c06c130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 --2- 172.21.15.164:0/3700114264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0cdfa0 0x7f060c06cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 --2- 172.21.15.164:0/3700114264 >> v2:172.21.15.172:3300/0 conn(0x7f060c0af480 0x7f060c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- 172.21.15.164:0/3700114264 >> 172.21.15.164:0/3700114264 conn(0x7f060c0ccab0 msgr2=0x7f060c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- 172.21.15.164:0/3700114264 shutdown_connections 2024-05-08T04:39:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- 172.21.15.164:0/3700114264 wait complete. 2024-05-08T04:39:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 Processor -- start 2024-05-08T04:39:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.534+0000 7f0613000640 1 -- start start 2024-05-08T04:39:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0af480 0x7f060c0c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f060c0cdfa0 0x7f060c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0af860 con 0x7f060c06bca0 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c06d1e0 con 0x7f060c0cdfa0 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c072bd0 con 0x7f060c0af480 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46652/0 (socket says 172.21.15.164:46652) 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 -- 172.21.15.164:0/3309045463 learned_addr learned my addr 172.21.15.164:0/3309045463 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 -- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0af480 msgr2=0x7f060c0c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0af480 0x7f060c0c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 -- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f060c0cdfa0 msgr2=0x7f060c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f060c0cdfa0 0x7f060c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 -- 172.21.15.164:0/3309045463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c15c6c0 con 0x7f060c06bca0 2024-05-08T04:39:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f060be00640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e8b0 tx=0x7f05fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc017070 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00c940 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00cd10 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c15c9a0 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c15cf60 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c069d60 con 0x7f060c06bca0 2024-05-08T04:39:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05fc01a070 con 0x7f060c06bca0 2024-05-08T04:39:11.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.538+0000 7f0608c00640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 0x7f05e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.542+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f05fc085500 con 0x7f060c06bca0 2024-05-08T04:39:11.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.542+0000 7f0603400640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 0x7f05e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.542+0000 7f0603400640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 0x7f05e8069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006f10 tx=0x7f05f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.542+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05fc0563f0 con 0x7f060c06bca0 2024-05-08T04:39:11.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.730+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f060c066df0 con 0x7f060c06bca0 2024-05-08T04:39:11.733 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:39:11.730+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:39:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.730+0000 7f0608c00640 1 -- 172.21.15.164:0/3309045463 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f05fc055cf0 con 0x7f060c06bca0 2024-05-08T04:39:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:39:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 msgr2=0x7f05e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 0x7f05e8069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006f10 tx=0x7f05f4008040 comp rx=0 tx=0).stop 2024-05-08T04:39:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 msgr2=0x7f060c14cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e8b0 tx=0x7f05fc00ed70 comp rx=0 tx=0).stop 2024-05-08T04:39:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 shutdown_connections 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05e8066f60 0x7f05e8069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f060c0af480 0x7f060c0c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f060c0cdfa0 0x7f060c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 --2- 172.21.15.164:0/3309045463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f060c06bca0 0x7f060c14cbf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 >> 172.21.15.164:0/3309045463 conn(0x7f060c0ccab0 msgr2=0x7f060c0ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 shutdown_connections 2024-05-08T04:39:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.734+0000 7f0613000640 1 -- 172.21.15.164:0/3309045463 wait complete. 2024-05-08T04:39:11.752 INFO:tasks.ceph.mds.d.smithi172.stderr: -6929> 2024-05-08T04:38:33.966+0000 7f1867800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:39:11.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:39:11.804 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:39:11.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 Processor -- start 2024-05-08T04:39:11.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 -- start start 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb5540c5040 0x7fb5540ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540c54b0 con 0x7fb554144360 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540adc50 con 0x7fb55406b3a0 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55c800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540ae860 con 0x7fb5540c5040 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb559e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb554144360 0x7fb55406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54254/0 (socket says 172.21.15.164:54254) 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb559e00640 1 -- 172.21.15.164:0/2229925314 learned_addr learned my addr 172.21.15.164:0/2229925314 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb55a800640 1 --2- 172.21.15.164:0/2229925314 >> v2:172.21.15.164:3300/0 conn(0x7fb5540c5040 0x7fb5540ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb558a00640 1 -- 172.21.15.164:0/2229925314 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189642147 0 0) 0x7fb5540c54b0 con 0x7fb554144360 2024-05-08T04:39:11.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.894+0000 7fb558a00640 1 -- 172.21.15.164:0/2229925314 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb53c003530 con 0x7fb554144360 2024-05-08T04:39:11.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55a800640 1 -- 172.21.15.164:0/2229925314 >> v1:172.21.15.172:6789/0 conn(0x7fb55406b3a0 legacy=0x7fb5540ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55a800640 1 -- 172.21.15.164:0/2229925314 >> v1:172.21.15.164:6789/0 conn(0x7fb554144360 legacy=0x7fb55406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55a800640 1 -- 172.21.15.164:0/2229925314 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540adf30 con 0x7fb5540c5040 2024-05-08T04:39:11.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55a800640 1 --2- 172.21.15.164:0/2229925314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5540c5040 0x7fb5540ca990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb550009d90 tx=0x7fb55001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0151fd4d5684671 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb558a00640 1 -- 172.21.15.164:0/2229925314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500b2070 con 0x7fb5540c5040 2024-05-08T04:39:11.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb558a00640 1 -- 172.21.15.164:0/2229925314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5500a5920 con 0x7fb5540c5040 2024-05-08T04:39:11.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb558a00640 1 -- 172.21.15.164:0/2229925314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500a5cd0 con 0x7fb5540c5040 2024-05-08T04:39:11.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 -- 172.21.15.164:0/2229925314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5540c5040 msgr2=0x7fb5540ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 --2- 172.21.15.164:0/2229925314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5540c5040 0x7fb5540ca990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb550009d90 tx=0x7fb55001c970 comp rx=0 tx=0).stop 2024-05-08T04:39:11.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 -- 172.21.15.164:0/2229925314 shutdown_connections 2024-05-08T04:39:11.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 --2- 172.21.15.164:0/2229925314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5540c5040 0x7fb5540ca990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 -- 172.21.15.164:0/2229925314 >> 172.21.15.164:0/2229925314 conn(0x7fb5540cf770 msgr2=0x7fb5540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:11.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 -- 172.21.15.164:0/2229925314 shutdown_connections 2024-05-08T04:39:11.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.898+0000 7fb55c800640 1 -- 172.21.15.164:0/2229925314 wait complete. 2024-05-08T04:39:11.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 Processor -- start 2024-05-08T04:39:11.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 -- start start 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 0x7fb55406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 0x7fb55413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540c4bc0 con 0x7fb554144360 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540c4a40 con 0x7fb55406b3a0 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55416bd80 con 0x7fb5540c5040 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:41080/0 (socket says 172.21.15.164:41080) 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559e00640 1 -- 172.21.15.164:0/191157558 learned_addr learned my addr 172.21.15.164:0/191157558 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 0x7fb55406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55a800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 0x7fb55413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 -- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 msgr2=0x7fb55415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 -- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 msgr2=0x7fb55413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 0x7fb55413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 -- 172.21.15.164:0/191157558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55406e2f0 con 0x7fb5540c5040 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55a800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 0x7fb55413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559e00640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb559400640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 0x7fb55406d8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c3c0 tx=0x7fb54400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440052d0 con 0x7fb5540c5040 2024-05-08T04:39:11.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544005470 con 0x7fb5540c5040 2024-05-08T04:39:11.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55406e010 con 0x7fb5540c5040 2024-05-08T04:39:11.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.902+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544007260 con 0x7fb5540c5040 2024-05-08T04:39:11.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541fa780 con 0x7fb5540c5040 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554072d30 con 0x7fb5540c5040 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb544007400 con 0x7fb5540c5040 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb54e000640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fb544084460 con 0x7fb5540c5040 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb559e00640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.906+0000 7fb559e00640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 0x7fb538069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5480011f0 tx=0x7fb548009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:11.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:11.910+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb544054d40 con 0x7fb5540c5040 2024-05-08T04:39:12.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.098+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb55404f010 con 0x7fb5540c5040 2024-05-08T04:39:12.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb54e000640 1 -- 172.21.15.164:0/191157558 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb544058de0 con 0x7fb5540c5040 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 msgr2=0x7fb538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 0x7fb538069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5480011f0 tx=0x7fb548009040 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 msgr2=0x7fb55406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 0x7fb55406d8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c3c0 tx=0x7fb54400c880 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 shutdown_connections 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5540c5040 0x7fb55406d8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb554144360 0x7fb55413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 --2- 172.21.15.164:0/191157558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb55406b3a0 0x7fb55415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 >> 172.21.15.164:0/191157558 conn(0x7fb5540cf770 msgr2=0x7fb5541435b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 shutdown_connections 2024-05-08T04:39:12.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.738+0000 7fb55c800640 1 -- 172.21.15.164:0/191157558 wait complete. 2024-05-08T04:39:12.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:39:12.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 Processor -- start 2024-05-08T04:39:12.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 -- start start 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe754165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75415e870 con 0x7fe75415ab10 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75415eb50 con 0x7fe75406cce0 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe75aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75406bb80 con 0x7fe7540cee90 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe753e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe75415ab10 0x7fe7540ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54268/0 (socket says 172.21.15.164:54268) 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe753e00640 1 -- 172.21.15.164:0/2905524018 learned_addr learned my addr 172.21.15.164:0/2905524018 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:12.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe754165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760907826 0 0) 0x7fe75415e870 con 0x7fe75415ab10 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe738005740 con 0x7fe75415ab10 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675556501 0 0) 0x7fe738005740 con 0x7fe75415ab10 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 msgr2=0x7fe754165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe754165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 >> v1:172.21.15.172:6789/0 conn(0x7fe75406cce0 legacy=0x7fe7540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75415ee30 con 0x7fe75415ab10 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fe744001680 con 0x7fe75415ab10 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 >> v1:172.21.15.164:6789/0 conn(0x7fe75415ab10 legacy=0x7fe7540ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe738005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738006180 0x7fe738006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe738006f60 0x7fe7380073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75406bb80 con 0x7fe738006f60 2024-05-08T04:39:12.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73800dd70 con 0x7fe738006180 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73800df70 con 0x7fe7540cee90 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe738006f60 0x7fe7380073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe753e00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe738005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe753400640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738006180 0x7fe738006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 -- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 msgr2=0x7fe738005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7540cee90 0x7fe738005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 -- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738006180 msgr2=0x7fe738006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738006180 0x7fe738006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe758a00640 1 -- 172.21.15.164:0/2905524018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe738005740 con 0x7fe738006f60 2024-05-08T04:39:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.902+0000 7fe753400640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738006180 0x7fe738006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe758a00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe738006f60 0x7fe7380073e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7540ca210 tx=0x7fe74c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fd6918bc0aa416 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0b2070 con 0x7fe738006f60 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c0a5b20 con 0x7fe738006f60 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe752a00640 1 -- 172.21.15.164:0/2905524018 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c009c10 con 0x7fe738006f60 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 -- 172.21.15.164:0/2905524018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe738006f60 msgr2=0x7fe7380073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2905524018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe738006f60 0x7fe7380073e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7540ca210 tx=0x7fe74c01cc20 comp rx=0 tx=0).stop 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe758a00640 1 -- 172.21.15.164:0/2905524018 reap_dead start 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 -- 172.21.15.164:0/2905524018 shutdown_connections 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 -- 172.21.15.164:0/2905524018 >> 172.21.15.164:0/2905524018 conn(0x7fe7540cf770 msgr2=0x7fe7540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 -- 172.21.15.164:0/2905524018 shutdown_connections 2024-05-08T04:39:12.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.906+0000 7fe75aa00640 1 -- 172.21.15.164:0/2905524018 wait complete. 2024-05-08T04:39:12.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 Processor -- start 2024-05-08T04:39:12.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- start start 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe738006f60 0x7fe7540b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 0x7fe7541540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415aef0 con 0x7fe754071ef0 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415f3f0 con 0x7fe754153c10 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540aedf0 con 0x7fe738006f60 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46686/0 (socket says 172.21.15.164:46686) 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 -- 172.21.15.164:0/2598979846 learned_addr learned my addr 172.21.15.164:0/2598979846 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753e00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe738006f60 0x7fe7540b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe758a00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 0x7fe7541540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 -- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe738006f60 msgr2=0x7fe7540b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe738006f60 0x7fe7540b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 -- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 msgr2=0x7fe7541540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 0x7fe7541540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 -- 172.21.15.164:0/2598979846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540b48c0 con 0x7fe754071ef0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe758a00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 0x7fe7541540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe753400640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe74000e4e0 tx=0x7fe74000e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740019070 con 0x7fe754071ef0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400053a0 con 0x7fe754071ef0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7540b4ba0 con 0x7fe754071ef0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740003820 con 0x7fe754071ef0 2024-05-08T04:39:12.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.910+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7540b5160 con 0x7fe754071ef0 2024-05-08T04:39:12.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754072d30 con 0x7fe754071ef0 2024-05-08T04:39:12.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe74001c070 con 0x7fe754071ef0 2024-05-08T04:39:12.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe74be00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 0x7fe734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:12.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fe7400845f0 con 0x7fe754071ef0 2024-05-08T04:39:12.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe753e00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 0x7fe734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:12.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.914+0000 7fe753e00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 0x7fe734069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe744004880 tx=0x7fe744004810 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:12.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:12.918+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe740054ed0 con 0x7fe754071ef0 2024-05-08T04:39:13.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.078+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe75404f010 con 0x7fe754071ef0 2024-05-08T04:39:13.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.930+0000 7fe74be00640 1 -- 172.21.15.164:0/2598979846 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe740058f70 con 0x7fe754071ef0 2024-05-08T04:39:13.934 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:39:13.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.930+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 msgr2=0x7fe734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:13.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.930+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 0x7fe734069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe744004880 tx=0x7fe744004810 comp rx=0 tx=0).stop 2024-05-08T04:39:13.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.930+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 msgr2=0x7fe7541553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:13.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.930+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe74000e4e0 tx=0x7fe74000e8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:13.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 shutdown_connections 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe734066f60 0x7fe734069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe754153c10 0x7fe7541540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe754071ef0 0x7fe7541553d0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe74000e4e0 tx=0x7fe74000e8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 --2- 172.21.15.164:0/2598979846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe738006f60 0x7fe7540b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 >> 172.21.15.164:0/2598979846 conn(0x7fe7540cf770 msgr2=0x7fe7540ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 shutdown_connections 2024-05-08T04:39:13.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:13.934+0000 7fe75aa00640 1 -- 172.21.15.164:0/2598979846 wait complete. 2024-05-08T04:39:13.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:39:14.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.094+0000 7f5a66600640 1 Processor -- start 2024-05-08T04:39:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.094+0000 7f5a66600640 1 -- start start 2024-05-08T04:39:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a66600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 0x7f5a60160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a66600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a600aedf0 0x7f5a6006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a66600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a600c6600 con 0x7f5a6006de50 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a66600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6015fbe0 con 0x7f5a600aedf0 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a66600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600c6040 con 0x7f5a6015ba20 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a600aedf0 0x7f5a6006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a600aedf0 0x7f5a6006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46688/0 (socket says 172.21.15.164:46688) 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2726393133 learned_addr learned my addr 172.21.15.164:0/2726393133 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 0x7f5a60160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2726393133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 msgr2=0x7f5a60160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 0x7f5a60160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2726393133 >> v1:172.21.15.172:6789/0 conn(0x7f5a6006de50 legacy=0x7f5a600ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2726393133 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600c6320 con 0x7f5a600aedf0 2024-05-08T04:39:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.098+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 0x7f5a60160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:14.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a5fe00640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a600aedf0 0x7f5a6006e290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009e50 tx=0x7f5a5801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dd5b6d99c7c7ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:14.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a5ea00640 1 -- 172.21.15.164:0/2726393133 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580a57e0 con 0x7f5a600aedf0 2024-05-08T04:39:14.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a5ea00640 1 -- 172.21.15.164:0/2726393133 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a580a5980 con 0x7f5a600aedf0 2024-05-08T04:39:14.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a5ea00640 1 -- 172.21.15.164:0/2726393133 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580a5d50 con 0x7f5a600aedf0 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- 172.21.15.164:0/2726393133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a600aedf0 msgr2=0x7f5a6006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a600aedf0 0x7f5a6006e290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009e50 tx=0x7f5a5801c9c0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- 172.21.15.164:0/2726393133 shutdown_connections 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a600aedf0 0x7f5a6006e290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 --2- 172.21.15.164:0/2726393133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6015ba20 0x7f5a60160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- 172.21.15.164:0/2726393133 >> 172.21.15.164:0/2726393133 conn(0x7f5a600cf770 msgr2=0x7f5a600cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- 172.21.15.164:0/2726393133 shutdown_connections 2024-05-08T04:39:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- 172.21.15.164:0/2726393133 wait complete. 2024-05-08T04:39:14.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 Processor -- start 2024-05-08T04:39:14.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.102+0000 7f5a66600640 1 -- start start 2024-05-08T04:39:14.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 0x7f5a60071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 0x7f5a60143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600ce000 con 0x7f5a6015ba20 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600cde80 con 0x7f5a600aedf0 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60168b70 con 0x7f5a6006de50 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46702/0 (socket says 172.21.15.164:46702) 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2388681653 learned_addr learned my addr 172.21.15.164:0/2388681653 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 0x7f5a60071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a56a00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 0x7f5a60143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 msgr2=0x7f5a60071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:14.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 0x7f5a60071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 msgr2=0x7f5a60143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 0x7f5a60143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 -- 172.21.15.164:0/2388681653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60070bb0 con 0x7f5a6015ba20 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 0x7f5a60071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a56a00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 0x7f5a60143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a5fe00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a580af040 tx=0x7f5a58019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580b8880 con 0x7f5a6015ba20 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a580be070 con 0x7f5a6015ba20 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a60070e90 con 0x7f5a6015ba20 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580b8c80 con 0x7f5a6015ba20 2024-05-08T04:39:14.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a60071450 con 0x7f5a6015ba20 2024-05-08T04:39:14.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a60072d30 con 0x7f5a6015ba20 2024-05-08T04:39:14.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.106+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a580c3460 con 0x7f5a6015ba20 2024-05-08T04:39:14.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.110+0000 7f5a57e00640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 0x7f5a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:14.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.110+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f5a58120310 con 0x7f5a6015ba20 2024-05-08T04:39:14.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.110+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 0x7f5a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:14.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.110+0000 7f5a5f400640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 0x7f5a3c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000a9b0 tx=0x7f5a50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:14.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.114+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a580165e0 con 0x7f5a6015ba20 2024-05-08T04:39:14.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.266+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a6004f010 con 0x7f5a6015ba20 2024-05-08T04:39:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.938+0000 7f5a57e00640 1 -- 172.21.15.164:0/2388681653 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a580f1360 con 0x7f5a6015ba20 2024-05-08T04:39:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 msgr2=0x7f5a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 0x7f5a3c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000a9b0 tx=0x7f5a50008040 comp rx=0 tx=0).stop 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 msgr2=0x7f5a600702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a580af040 tx=0x7f5a58019760 comp rx=0 tx=0).stop 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 shutdown_connections 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a3c066f60 0x7f5a3c069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a6015ba20 0x7f5a600702b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a600aedf0 0x7f5a60143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 --2- 172.21.15.164:0/2388681653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a6006de50 0x7f5a60071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 >> 172.21.15.164:0/2388681653 conn(0x7f5a600cf770 msgr2=0x7f5a6015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.942+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 shutdown_connections 2024-05-08T04:39:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:14.946+0000 7f5a66600640 1 -- 172.21.15.164:0/2388681653 wait complete. 2024-05-08T04:39:14.972 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:39:14.972 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:39:14.972 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:39:14.972 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:14.972 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:39:14.973 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:39:14.973 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:39:14.973 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:14.973 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:39:14.973 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:39:14.973 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:39:14.973 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:14.974 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:39:14.974 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:39:14.974 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:39:14.974 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:14.985 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:39:14.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:39:14.986 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:39:14.986 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:39:14.989 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:39:14.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:39:14.990 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:39:14.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:39:15.031 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:39:15.031 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:15.032 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:39:15.032 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:15.037 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:39:15.037 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:15.039 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:39:15.039 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:15.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:39:15.117 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:39:15.118 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:39:15.126 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:39:15.126 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:39:15.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 Processor -- start 2024-05-08T04:39:15.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 -- start start 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c1015bc50 0x7f4c100b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c100d0780 0x7f4c1015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c100b42a0 con 0x7f4c100ccfe0 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c100cd4e0 con 0x7f4c100d0780 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c17400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100cd7c0 con 0x7f4c1015bc50 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c0fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4c100ccfe0 0x7f4c100d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54280/0 (socket says 172.21.15.164:54280) 2024-05-08T04:39:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c0fe00640 1 -- 172.21.15.164:0/1748034422 learned_addr learned my addr 172.21.15.164:0/1748034422 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:15.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c15400640 1 --2- 172.21.15.164:0/1748034422 >> v2:172.21.15.172:3300/0 conn(0x7f4c100d0780 0x7f4c1015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c14a00640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c1015bc50 0x7f4c100b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c0f400640 1 -- 172.21.15.164:0/1748034422 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609512837 0 0) 0x7f4c100b42a0 con 0x7f4c100ccfe0 2024-05-08T04:39:15.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c0f400640 1 -- 172.21.15.164:0/1748034422 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf4003530 con 0x7f4c100ccfe0 2024-05-08T04:39:15.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c15400640 1 -- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c1015bc50 msgr2=0x7f4c100b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:15.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c15400640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c1015bc50 0x7f4c100b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c15400640 1 -- 172.21.15.164:0/1748034422 >> v1:172.21.15.164:6789/0 conn(0x7f4c100ccfe0 legacy=0x7f4c100d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.182+0000 7f4c15400640 1 -- 172.21.15.164:0/1748034422 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100cf590 con 0x7f4c100d0780 2024-05-08T04:39:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c15400640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c1015f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097c0 tx=0x7f4c0801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5611948d601ab849 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c0f400640 1 -- 172.21.15.164:0/1748034422 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080b1070 con 0x7f4c100d0780 2024-05-08T04:39:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c0f400640 1 -- 172.21.15.164:0/1748034422 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c080a8050 con 0x7f4c100d0780 2024-05-08T04:39:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c0f400640 1 -- 172.21.15.164:0/1748034422 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080ac860 con 0x7f4c100d0780 2024-05-08T04:39:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 -- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 msgr2=0x7f4c1015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c1015f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097c0 tx=0x7f4c0801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 -- 172.21.15.164:0/1748034422 shutdown_connections 2024-05-08T04:39:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c1015f7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 --2- 172.21.15.164:0/1748034422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c1015bc50 0x7f4c100b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 -- 172.21.15.164:0/1748034422 >> 172.21.15.164:0/1748034422 conn(0x7f4c100b34f0 msgr2=0x7f4c100b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 -- 172.21.15.164:0/1748034422 shutdown_connections 2024-05-08T04:39:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.186+0000 7f4c17400640 1 -- 172.21.15.164:0/1748034422 wait complete. 2024-05-08T04:39:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 Processor -- start 2024-05-08T04:39:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- start start 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c100ccfe0 0x7f4c1006d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c10154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c10168c60 con 0x7f4c1015bc50 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c10168ae0 con 0x7f4c100d0780 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c10168de0 con 0x7f4c100ccfe0 2024-05-08T04:39:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46728/0 (socket says 172.21.15.164:46728) 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 -- 172.21.15.164:0/808533423 learned_addr learned my addr 172.21.15.164:0/808533423 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c14a00640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c100ccfe0 0x7f4c1006d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c0fe00640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c10154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 -- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c100ccfe0 msgr2=0x7f4c1006d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c100ccfe0 0x7f4c1006d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 -- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 msgr2=0x7f4c10154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c10154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 -- 172.21.15.164:0/808533423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1006d8e0 con 0x7f4c1015bc50 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c15400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c0801aa90 tx=0x7f4c08019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080b1040 con 0x7f4c1015bc50 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c080bb070 con 0x7f4c1015bc50 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1006dbc0 con 0x7f4c1015bc50 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080b5410 con 0x7f4c1015bc50 2024-05-08T04:39:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c10165100 con 0x7f4c1015bc50 2024-05-08T04:39:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.190+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c10069d60 con 0x7f4c1015bc50 2024-05-08T04:39:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.194+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c080b55b0 con 0x7f4c1015bc50 2024-05-08T04:39:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.194+0000 7f4c0cc00640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 0x7f4bf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.194+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4c0811e920 con 0x7f4c1015bc50 2024-05-08T04:39:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.194+0000 7f4c14a00640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 0x7f4bf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.194+0000 7f4c14a00640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 0x7f4bf0069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4c100ce640 tx=0x7f4c0000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.198+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c080c5020 con 0x7f4c1015bc50 2024-05-08T04:39:15.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.374+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4c100b7940 con 0x7f4c1015bc50 2024-05-08T04:39:15.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.954+0000 7f4c0cc00640 1 -- 172.21.15.164:0/808533423 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4c08016090 con 0x7f4c1015bc50 2024-05-08T04:39:15.959 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 msgr2=0x7f4bf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 0x7f4bf0069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4c100ce640 tx=0x7f4c0000b040 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 msgr2=0x7f4c10165620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c0801aa90 tx=0x7f4c08019180 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 shutdown_connections 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4bf0066f60 0x7f4bf0069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1015bc50 0x7f4c10165620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c100d0780 0x7f4c10154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 --2- 172.21.15.164:0/808533423 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c100ccfe0 0x7f4c1006d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 >> 172.21.15.164:0/808533423 conn(0x7f4c100b34f0 msgr2=0x7f4c100b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 shutdown_connections 2024-05-08T04:39:15.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:15.958+0000 7f4c17400640 1 -- 172.21.15.164:0/808533423 wait complete. 2024-05-08T04:39:15.984 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:39:16.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.114+0000 7fa656800640 1 Processor -- start 2024-05-08T04:39:16.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.114+0000 7fa656800640 1 -- start start 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa656800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa650072c90 0x7fa65006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa656800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650161830 con 0x7fa65006c1e0 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa656800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650161b10 con 0x7fa65015da40 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa656800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa650161df0 con 0x7fa650072c90 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa65006c1e0 0x7fa6500cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54296/0 (socket says 172.21.15.164:54296) 2024-05-08T04:39:16.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 -- 172.21.15.164:0/739630100 learned_addr learned my addr 172.21.15.164:0/739630100 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925298827 0 0) 0x7fa650161830 con 0x7fa65006c1e0 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003530 con 0x7fa65006c1e0 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa654a00640 1 --2- 172.21.15.164:0/739630100 >> v2:172.21.15.172:3300/0 conn(0x7fa650072c90 0x7fa65006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 998542386 0 0) 0x7fa638003530 con 0x7fa65006c1e0 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 >> v2:172.21.15.172:3300/0 conn(0x7fa650072c90 msgr2=0x7fa65006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 --2- 172.21.15.164:0/739630100 >> v2:172.21.15.172:3300/0 conn(0x7fa650072c90 0x7fa65006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 >> v1:172.21.15.172:6789/0 conn(0x7fa65015da40 legacy=0x7fa6500cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6501620d0 con 0x7fa65006c1e0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa648001680 con 0x7fa65006c1e0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 >> v1:172.21.15.164:6789/0 conn(0x7fa65006c1e0 legacy=0x7fa6500cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6380039c0 0x7fa638003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa638004810 0x7fa638004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa63800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650161df0 con 0x7fa6380053d0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63800c260 con 0x7fa6380039c0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63800c460 con 0x7fa638004810 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa63800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655400640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6380039c0 0x7fa638003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa654a00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa638004810 0x7fa638004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 -- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa638004810 msgr2=0x7fa638004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa638004810 0x7fa638004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 -- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6380039c0 msgr2=0x7fa638003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:16.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6380039c0 0x7fa638003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.118+0000 7fa655e00640 1 -- 172.21.15.164:0/739630100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638003530 con 0x7fa6380053d0 2024-05-08T04:39:16.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa655e00640 1 -- 172.21.15.164:0/739630100 reap_dead start 2024-05-08T04:39:16.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa655e00640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa63800b9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c1c0 tx=0x7fa64801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718822f3fb3eb33b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648007d00 con 0x7fa6380053d0 2024-05-08T04:39:16.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480a9070 con 0x7fa6380053d0 2024-05-08T04:39:16.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa64fe00640 1 -- 172.21.15.164:0/739630100 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648005d40 con 0x7fa6380053d0 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 -- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 msgr2=0x7fa63800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa63800b9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c1c0 tx=0x7fa64801cc60 comp rx=0 tx=0).stop 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 -- 172.21.15.164:0/739630100 shutdown_connections 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 --2- 172.21.15.164:0/739630100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa63800b9d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 -- 172.21.15.164:0/739630100 >> 172.21.15.164:0/739630100 conn(0x7fa6500cca50 msgr2=0x7fa6500cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 -- 172.21.15.164:0/739630100 shutdown_connections 2024-05-08T04:39:16.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.122+0000 7fa656800640 1 -- 172.21.15.164:0/739630100 wait complete. 2024-05-08T04:39:16.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 Processor -- start 2024-05-08T04:39:16.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- start start 2024-05-08T04:39:16.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 0x7fa65014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa650159fd0 0x7fa65015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65006c030 con 0x7fa6380053d0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65006beb0 con 0x7fa650159fd0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500cdc70 con 0x7fa6501595f0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46742/0 (socket says 172.21.15.164:46742) 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 -- 172.21.15.164:0/3086299365 learned_addr learned my addr 172.21.15.164:0/3086299365 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa654a00640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 0x7fa65014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655e00640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa650159fd0 0x7fa65015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 -- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 msgr2=0x7fa65014e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 0x7fa65014e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 -- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa650159fd0 msgr2=0x7fa65015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa650159fd0 0x7fa65015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 -- 172.21.15.164:0/3086299365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650158570 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa654a00640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 0x7fa65014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa655400640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa64400f980 tx=0x7fa64400fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644003b90 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644018070 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa650158850 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400d7f0 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa650158e10 con 0x7fa6380053d0 2024-05-08T04:39:16.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.126+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65004ef30 con 0x7fa6380053d0 2024-05-08T04:39:16.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.130+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa64400d990 con 0x7fa6380053d0 2024-05-08T04:39:16.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.130+0000 7fa64d600640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.130+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa644084ed0 con 0x7fa6380053d0 2024-05-08T04:39:16.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.130+0000 7fa654a00640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.130+0000 7fa654a00640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 0x7fa630069450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6500cf7c0 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.134+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa64401a310 con 0x7fa6380053d0 2024-05-08T04:39:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa6500af480 con 0x7fa6380053d0 2024-05-08T04:39:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa64d600640 1 -- 172.21.15.164:0/3086299365 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa644056190 con 0x7fa6380053d0 2024-05-08T04:39:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:39:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:39:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:39:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:39:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:39:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-mds", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtmwiFPFhAAMAsKDQODjRcnJspv6s3YjQ==", 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T04:39:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtm/StPFhAAue/m9RQQoVeEobWbqp0hvA==", 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-osd", 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtmxjRPFhAAwPuUrnEAmpw6Y9Zt9S3Z+w==", 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T04:39:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtmxz1PFhAAPi3GzOtyFNQA8azK+Mz1RA==", 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtmbEZPFhAA049/MftFLOgvhtN9441P5Q==", 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T04:39:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCsATtmbk9PFhAALCocoscL+reuyZcuisi6Vw==", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:39:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:39:16.361 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:39:16.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 msgr2=0x7fa630069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 0x7fa630069450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6500cf7c0 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 msgr2=0x7fa650149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa64400f980 tx=0x7fa64400fe40 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 shutdown_connections 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6501595f0 0x7fa65014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa650159fd0 0x7fa65015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 --2- 172.21.15.164:0/3086299365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6380053d0 0x7fa650149c40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 >> 172.21.15.164:0/3086299365 conn(0x7fa6500cca50 msgr2=0x7fa65015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 shutdown_connections 2024-05-08T04:39:16.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.342+0000 7fa656800640 1 -- 172.21.15.164:0/3086299365 wait complete. 2024-05-08T04:39:16.367 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T04:39:16.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 Processor -- start 2024-05-08T04:39:16.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 -- start start 2024-05-08T04:39:16.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe04c0cdfa0 0x7fe04c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe04c06bca0 0x7fe04c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c071980 con 0x7fe04c0af480 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c161810 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.498+0000 7fe054400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c06eed0 con 0x7fe04c06bca0 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe04c0cdfa0 0x7fe04c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe04c0cdfa0 0x7fe04c0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46744/0 (socket says 172.21.15.164:46744) 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 -- 172.21.15.164:0/1433948858 learned_addr learned my addr 172.21.15.164:0/1433948858 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe052400640 1 --2- 172.21.15.164:0/1433948858 >> v2:172.21.15.172:3300/0 conn(0x7fe04c06bca0 0x7fe04c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 -- 172.21.15.164:0/1433948858 >> v2:172.21.15.172:3300/0 conn(0x7fe04c06bca0 msgr2=0x7fe04c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 --2- 172.21.15.164:0/1433948858 >> v2:172.21.15.172:3300/0 conn(0x7fe04c06bca0 0x7fe04c06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 -- 172.21.15.164:0/1433948858 >> v1:172.21.15.172:6789/0 conn(0x7fe04c0af480 legacy=0x7fe04c06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:16.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 -- 172.21.15.164:0/1433948858 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c071c00 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe051000640 1 --2- 172.21.15.164:0/1433948858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0cdfa0 0x7fe04c0cf470 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe04c066d20 tx=0x7fe03801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe97d3af4963619 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe04be00640 1 -- 172.21.15.164:0/1433948858 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380b0070 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe04be00640 1 -- 172.21.15.164:0/1433948858 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380ab600 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.502+0000 7fe04be00640 1 -- 172.21.15.164:0/1433948858 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380ab9d0 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- 172.21.15.164:0/1433948858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0cdfa0 msgr2=0x7fe04c0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- 172.21.15.164:0/1433948858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0cdfa0 0x7fe04c0cf470 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe04c066d20 tx=0x7fe03801c300 comp rx=0 tx=0).stop 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- 172.21.15.164:0/1433948858 shutdown_connections 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- 172.21.15.164:0/1433948858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0cdfa0 0x7fe04c0cf470 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- 172.21.15.164:0/1433948858 >> v2:172.21.15.172:3300/0 conn(0x7fe04c06bca0 0x7fe04c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- 172.21.15.164:0/1433948858 >> 172.21.15.164:0/1433948858 conn(0x7fe04c0ccab0 msgr2=0x7fe04c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- 172.21.15.164:0/1433948858 shutdown_connections 2024-05-08T04:39:16.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- 172.21.15.164:0/1433948858 wait complete. 2024-05-08T04:39:16.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 Processor -- start 2024-05-08T04:39:16.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- start start 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 0x7fe04c0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 0x7fe04c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c06c610 con 0x7fe04c0af480 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c071f50 con 0x7fe04c06bca0 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe054400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c072bf0 con 0x7fe04c0cdfa0 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46760/0 (socket says 172.21.15.164:46760) 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 -- 172.21.15.164:0/2897125753 learned_addr learned my addr 172.21.15.164:0/2897125753 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051a00640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 0x7fe04c0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 -- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 msgr2=0x7fe04c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe052400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 0x7fe04c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 0x7fe04c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 -- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 msgr2=0x7fe04c0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 0x7fe04c0c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051000640 1 -- 172.21.15.164:0/2897125753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c159f80 con 0x7fe04c0af480 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe051a00640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 0x7fe04c0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.506+0000 7fe052400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 0x7fe04c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:16.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe051000640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe038019f30 tx=0x7fe0380190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380b0040 con 0x7fe04c0af480 2024-05-08T04:39:16.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380b4620 con 0x7fe04c0af480 2024-05-08T04:39:16.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0c7350 con 0x7fe04c0af480 2024-05-08T04:39:16.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380b4a10 con 0x7fe04c0af480 2024-05-08T04:39:16.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c09ff50 con 0x7fe04c0af480 2024-05-08T04:39:16.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c0c70e0 con 0x7fe04c0af480 2024-05-08T04:39:16.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0380ab4c0 con 0x7fe04c0af480 2024-05-08T04:39:16.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 0x7fe02c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe03811e870 con 0x7fe04c0af480 2024-05-08T04:39:16.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.510+0000 7fe051a00640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 0x7fe02c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.514+0000 7fe051a00640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 0x7fe02c069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe040003e40 tx=0x7fe040009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.518+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe04c159f80 con 0x7fe04c0af480 2024-05-08T04:39:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.726+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe04c069d60 con 0x7fe04c0af480 2024-05-08T04:39:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.726+0000 7fe049600640 1 -- 172.21.15.164:0/2897125753 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe04c071980 con 0x7fe04c0af480 2024-05-08T04:39:16.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 msgr2=0x7fe02c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 0x7fe02c069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe040003e40 tx=0x7fe040009040 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 msgr2=0x7fe04c1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe038019f30 tx=0x7fe0380190a0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 shutdown_connections 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe02c066f60 0x7fe02c069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe04c0cdfa0 0x7fe04c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe04c0af480 0x7fe04c1595b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 --2- 172.21.15.164:0/2897125753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe04c06bca0 0x7fe04c0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 >> 172.21.15.164:0/2897125753 conn(0x7fe04c0ccab0 msgr2=0x7fe04c160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 shutdown_connections 2024-05-08T04:39:16.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.730+0000 7fe054400640 1 -- 172.21.15.164:0/2897125753 wait complete. 2024-05-08T04:39:16.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T04:39:16.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 Processor -- start 2024-05-08T04:39:16.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 -- start start 2024-05-08T04:39:16.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f657c15dd70 0x7f657c1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c06cc70 con 0x7f657c1625f0 2024-05-08T04:39:16.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c06cf50 con 0x7f657c06c7b0 2024-05-08T04:39:16.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6580c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0b1570 con 0x7f657c15dd70 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f657a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f657c1625f0 0x7f657c0b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54302/0 (socket says 172.21.15.164:54302) 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f657a800640 1 -- 172.21.15.164:0/1196824188 learned_addr learned my addr 172.21.15.164:0/1196824188 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579e00640 1 --2- 172.21.15.164:0/1196824188 >> v2:172.21.15.172:3300/0 conn(0x7f657c15dd70 0x7f657c1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256087851 0 0) 0x7f657c06cc70 con 0x7f657c1625f0 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6560003530 con 0x7f657c1625f0 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291130143 0 0) 0x7f657c06cf50 con 0x7f657c06c7b0 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c06cc70 con 0x7f657c06c7b0 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579e00640 1 -- 172.21.15.164:0/1196824188 >> v1:172.21.15.172:6789/0 conn(0x7f657c06c7b0 legacy=0x7f657c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579e00640 1 -- 172.21.15.164:0/1196824188 >> v1:172.21.15.164:6789/0 conn(0x7f657c1625f0 legacy=0x7f657c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.886+0000 7f6579e00640 1 -- 172.21.15.164:0/1196824188 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0b1850 con 0x7f657c15dd70 2024-05-08T04:39:16.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6579e00640 1 --2- 172.21.15.164:0/1196824188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c15dd70 0x7f657c1619a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097c0 tx=0x7f656c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3eb4ea40b4600dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0b0070 con 0x7f657c15dd70 2024-05-08T04:39:16.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c0ab600 con 0x7f657c15dd70 2024-05-08T04:39:16.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6579400640 1 -- 172.21.15.164:0/1196824188 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0ab9b0 con 0x7f657c15dd70 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 -- 172.21.15.164:0/1196824188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c15dd70 msgr2=0x7f657c1619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 --2- 172.21.15.164:0/1196824188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c15dd70 0x7f657c1619a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097c0 tx=0x7f656c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 -- 172.21.15.164:0/1196824188 shutdown_connections 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 --2- 172.21.15.164:0/1196824188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c15dd70 0x7f657c1619a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 -- 172.21.15.164:0/1196824188 >> 172.21.15.164:0/1196824188 conn(0x7f657c0b34f0 msgr2=0x7f657c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 -- 172.21.15.164:0/1196824188 shutdown_connections 2024-05-08T04:39:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.890+0000 7f6580c00640 1 -- 172.21.15.164:0/1196824188 wait complete. 2024-05-08T04:39:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 Processor -- start 2024-05-08T04:39:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 -- start start 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 0x7f657c0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657c15dd70 0x7f657c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0b3060 con 0x7f657c1625f0 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0b2ee0 con 0x7f657c06c7b0 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c16bcf0 con 0x7f657c15dd70 2024-05-08T04:39:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6579e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 0x7f657c0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46764/0 (socket says 172.21.15.164:46764) 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 -- 172.21.15.164:0/4170519268 learned_addr learned my addr 172.21.15.164:0/4170519268 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6571400640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657c15dd70 0x7f657c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 -- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657c15dd70 msgr2=0x7f657c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657c15dd70 0x7f657c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 -- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 msgr2=0x7f657c0c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 0x7f657c0c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 -- 172.21.15.164:0/4170519268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c14a2b0 con 0x7f657c1625f0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6579e00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 0x7f657c0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f657a800640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f657400caa0 tx=0x7f657400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574017070 con 0x7f657c1625f0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400ac00 con 0x7f657c1625f0 2024-05-08T04:39:16.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574012400 con 0x7f657c1625f0 2024-05-08T04:39:16.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.894+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c14a550 con 0x7f657c1625f0 2024-05-08T04:39:16.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c14ab10 con 0x7f657c1625f0 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c069d60 con 0x7f657c1625f0 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f657401a070 con 0x7f657c1625f0 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6572a00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 0x7f6558069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f65740849b0 con 0x7f657c1625f0 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6579e00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 0x7f6558069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:16.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.898+0000 7f6579e00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 0x7f6558069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ad000 tx=0x7f656c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:16.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:16.902+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65740556b0 con 0x7f657c1625f0 2024-05-08T04:39:17.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.114+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f657c0b7940 con 0x7f657c1625f0 2024-05-08T04:39:17.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6572a00640 1 -- 172.21.15.164:0/4170519268 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6574059750 con 0x7f657c1625f0 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 msgr2=0x7f6558069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 0x7f6558069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ad000 tx=0x7f656c008040 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 msgr2=0x7f657c0d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f657400caa0 tx=0x7f657400cf60 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 shutdown_connections 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6558066f60 0x7f6558069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657c15dd70 0x7f657c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f657c1625f0 0x7f657c0d2280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 --2- 172.21.15.164:0/4170519268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657c06c7b0 0x7f657c0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 >> 172.21.15.164:0/4170519268 conn(0x7f657c0b34f0 msgr2=0x7f657c06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:17.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 shutdown_connections 2024-05-08T04:39:17.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.118+0000 7f6580c00640 1 -- 172.21.15.164:0/4170519268 wait complete. 2024-05-08T04:39:17.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T04:39:17.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 Processor -- start 2024-05-08T04:39:17.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 -- start start 2024-05-08T04:39:17.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f5015ab60 0x7f2f50072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f50072640 con 0x7f2f500b26c0 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5006c770 con 0x7f2f500cfda0 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f58000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5006ca50 con 0x7f2f5015ab60 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f54c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2f500b26c0 0x7f2f500cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54318/0 (socket says 172.21.15.164:54318) 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f54c00640 1 -- 172.21.15.164:0/245606579 learned_addr learned my addr 172.21.15.164:0/245606579 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:17.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.274+0000 7f2f55600640 1 --2- 172.21.15.164:0/245606579 >> v2:172.21.15.172:3300/0 conn(0x7f2f5015ab60 0x7f2f50072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206339651 0 0) 0x7f2f50072640 con 0x7f2f500b26c0 2024-05-08T04:39:17.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f38005740 con 0x7f2f500b26c0 2024-05-08T04:39:17.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315805788 0 0) 0x7f2f5006c770 con 0x7f2f500cfda0 2024-05-08T04:39:17.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f50072640 con 0x7f2f500cfda0 2024-05-08T04:39:17.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 560135342 0 0) 0x7f2f38005740 con 0x7f2f500b26c0 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 >> v2:172.21.15.172:3300/0 conn(0x7f2f5015ab60 msgr2=0x7f2f50072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 --2- 172.21.15.164:0/245606579 >> v2:172.21.15.172:3300/0 conn(0x7f2f5015ab60 0x7f2f50072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 >> v1:172.21.15.172:6789/0 conn(0x7f2f500cfda0 legacy=0x7f2f500cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5006cd30 con 0x7f2f500b26c0 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2f3c001680 con 0x7f2f500b26c0 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 >> v1:172.21.15.164:6789/0 conn(0x7f2f500b26c0 legacy=0x7f2f500cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 0x7f2f38006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38006a20 0x7f2f38006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f380073e0 0x7f2f3800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5006ca50 con 0x7f2f38005d60 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3800e260 con 0x7f2f380073e0 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3800e460 con 0x7f2f38006a20 2024-05-08T04:39:17.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 0x7f2f38006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f54c00640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38006a20 0x7f2f38006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f56000640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f380073e0 0x7f2f3800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 -- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38006a20 msgr2=0x7f2f38006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38006a20 0x7f2f38006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 -- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f380073e0 msgr2=0x7f2f3800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f380073e0 0x7f2f3800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f55600640 1 -- 172.21.15.164:0/245606579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38005740 con 0x7f2f38005d60 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f56000640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f380073e0 0x7f2f3800dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.278+0000 7f2f56000640 1 -- 172.21.15.164:0/245606579 reap_dead start 2024-05-08T04:39:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f55600640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 0x7f2f38006160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009ae0 tx=0x7f2f4401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d816d8fe27c1432 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440096b0 con 0x7f2f38005d60 2024-05-08T04:39:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440a6450 con 0x7f2f38005d60 2024-05-08T04:39:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f4fe00640 1 -- 172.21.15.164:0/245606579 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440a6840 con 0x7f2f38005d60 2024-05-08T04:39:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 msgr2=0x7f2f38006160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 0x7f2f38006160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009ae0 tx=0x7f2f4401ca70 comp rx=0 tx=0).stop 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- 172.21.15.164:0/245606579 shutdown_connections 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 --2- 172.21.15.164:0/245606579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f38005d60 0x7f2f38006160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- 172.21.15.164:0/245606579 >> 172.21.15.164:0/245606579 conn(0x7f2f5006ac90 msgr2=0x7f2f5006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- 172.21.15.164:0/245606579 shutdown_connections 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- 172.21.15.164:0/245606579 wait complete. 2024-05-08T04:39:17.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 Processor -- start 2024-05-08T04:39:17.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- start start 2024-05-08T04:39:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38005d60 0x7f2f501faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f5015a0d0 0x7f2f50153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cfc20 con 0x7f2f50151ec0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cfaa0 con 0x7f2f5015a0d0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f58000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500aed30 con 0x7f2f38005d60 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.282+0000 7f2f56000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46778/0 (socket says 172.21.15.164:46778) 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 -- 172.21.15.164:0/935542004 learned_addr learned my addr 172.21.15.164:0/935542004 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f54c00640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f5015a0d0 0x7f2f50153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f55600640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38005d60 0x7f2f501faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 -- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38005d60 msgr2=0x7f2f501faaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38005d60 0x7f2f501faaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 -- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f5015a0d0 msgr2=0x7f2f50153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f5015a0d0 0x7f2f50153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 -- 172.21.15.164:0/935542004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f501f7610 con 0x7f2f50151ec0 2024-05-08T04:39:17.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f56000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f380069a0 tx=0x7f2f4800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48019070 con 0x7f2f50151ec0 2024-05-08T04:39:17.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48006b60 con 0x7f2f50151ec0 2024-05-08T04:39:17.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f501fb0d0 con 0x7f2f50151ec0 2024-05-08T04:39:17.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48003980 con 0x7f2f50151ec0 2024-05-08T04:39:17.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f501fb690 con 0x7f2f50151ec0 2024-05-08T04:39:17.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50069d60 con 0x7f2f50151ec0 2024-05-08T04:39:17.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f4801c070 con 0x7f2f50151ec0 2024-05-08T04:39:17.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 0x7f2f30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.286+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2f48083f40 con 0x7f2f50151ec0 2024-05-08T04:39:17.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.290+0000 7f2f55600640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 0x7f2f30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.290+0000 7f2f55600640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 0x7f2f30069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f440ad000 tx=0x7f2f440b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.290+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f48054bb0 con 0x7f2f50151ec0 2024-05-08T04:39:17.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.502+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f2f5006d9a0 con 0x7f2f50151ec0 2024-05-08T04:39:17.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.502+0000 7f2f4d600640 1 -- 172.21.15.164:0/935542004 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2f48058c50 con 0x7f2f50151ec0 2024-05-08T04:39:17.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 msgr2=0x7f2f30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 0x7f2f30069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f440ad000 tx=0x7f2f440b2040 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 msgr2=0x7f2f50155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f380069a0 tx=0x7f2f4800ec20 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 shutdown_connections 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f30066f90 0x7f2f30069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f50151ec0 0x7f2f50155c30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f5015a0d0 0x7f2f50153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 --2- 172.21.15.164:0/935542004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f38005d60 0x7f2f501faaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 >> 172.21.15.164:0/935542004 conn(0x7f2f5006ac90 msgr2=0x7f2f500cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 shutdown_connections 2024-05-08T04:39:17.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.506+0000 7f2f58000640 1 -- 172.21.15.164:0/935542004 wait complete. 2024-05-08T04:39:17.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T04:39:17.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 Processor -- start 2024-05-08T04:39:17.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 -- start start 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f85c815ba70 0x7f85c8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 0x7f85c80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cd9f0 con 0x7f85c815be50 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c815fe70 con 0x7f85c815ba70 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85d0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80cdcd0 con 0x7f85c80cfda0 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f85c815ba70 0x7f85c8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cd200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f85c815be50 0x7f85c80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54324/0 (socket says 172.21.15.164:54324) 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f85c815ba70 0x7f85c8160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46788/0 (socket says 172.21.15.164:46788) 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cd200640 1 -- 172.21.15.164:0/1511276955 learned_addr learned my addr 172.21.15.164:0/1511276955 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:17.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85ce600640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 0x7f85c80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85c7e00640 1 -- 172.21.15.164:0/1511276955 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140205111 0 0) 0x7f85c80cd9f0 con 0x7f85c815be50 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85c7e00640 1 -- 172.21.15.164:0/1511276955 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0003530 con 0x7f85c815be50 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 -- 172.21.15.164:0/1511276955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 msgr2=0x7f85c80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 0x7f85c80b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 -- 172.21.15.164:0/1511276955 >> v1:172.21.15.164:6789/0 conn(0x7f85c815be50 legacy=0x7f85c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85cdc00640 1 -- 172.21.15.164:0/1511276955 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c815f820 con 0x7f85c815ba70 2024-05-08T04:39:17.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.662+0000 7f85ce600640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 0x7f85c80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:17.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85cdc00640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815ba70 0x7f85c8160300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0097c0 tx=0x7f85bc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7778ec1269c377a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85c7e00640 1 -- 172.21.15.164:0/1511276955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0b2070 con 0x7f85c815ba70 2024-05-08T04:39:17.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85c7e00640 1 -- 172.21.15.164:0/1511276955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc0ad680 con 0x7f85c815ba70 2024-05-08T04:39:17.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85c7e00640 1 -- 172.21.15.164:0/1511276955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0ada50 con 0x7f85c815ba70 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 -- 172.21.15.164:0/1511276955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815ba70 msgr2=0x7f85c8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815ba70 0x7f85c8160300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0097c0 tx=0x7f85bc01c470 comp rx=0 tx=0).stop 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 -- 172.21.15.164:0/1511276955 shutdown_connections 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c80cfda0 0x7f85c80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 --2- 172.21.15.164:0/1511276955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815ba70 0x7f85c8160300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 -- 172.21.15.164:0/1511276955 >> 172.21.15.164:0/1511276955 conn(0x7f85c806ac90 msgr2=0x7f85c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 -- 172.21.15.164:0/1511276955 shutdown_connections 2024-05-08T04:39:17.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.666+0000 7f85d0600640 1 -- 172.21.15.164:0/1511276955 wait complete. 2024-05-08T04:39:17.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 Processor -- start 2024-05-08T04:39:17.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- start start 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 0x7f85c80b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 0x7f85c8070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8168d40 con 0x7f85c815be50 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8168bc0 con 0x7f85c80cfda0 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8168ec0 con 0x7f85c815ba70 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46802/0 (socket says 172.21.15.164:46802) 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 -- 172.21.15.164:0/4193285053 learned_addr learned my addr 172.21.15.164:0/4193285053 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85cd200640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 0x7f85c8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 -- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 msgr2=0x7f85c8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85cdc00640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 0x7f85c80b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 0x7f85c8070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 -- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 msgr2=0x7f85c80b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 0x7f85c80b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 -- 172.21.15.164:0/4193285053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80af5f0 con 0x7f85c815be50 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85cdc00640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 0x7f85c80b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85cd200640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 0x7f85c8070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85ce600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007e70 tx=0x7f85c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c001a070 con 0x7f85c815be50 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0015600 con 0x7f85c815be50 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80af8d0 con 0x7f85c815be50 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c00159f0 con 0x7f85c815be50 2024-05-08T04:39:17.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.670+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80afe90 con 0x7f85c815be50 2024-05-08T04:39:17.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8067820 con 0x7f85c815be50 2024-05-08T04:39:17.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85c001d070 con 0x7f85c815be50 2024-05-08T04:39:17.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85c5600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 0x7f85a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:17.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85c00850d0 con 0x7f85c815be50 2024-05-08T04:39:17.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85cdc00640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 0x7f85a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:17.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.674+0000 7f85cdc00640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 0x7f85a8069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0af000 tx=0x7f85bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:17.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.678+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85c0010070 con 0x7f85c815be50 2024-05-08T04:39:17.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.886+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f85c8069d60 con 0x7f85c815be50 2024-05-08T04:39:17.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85c5600640 1 -- 172.21.15.164:0/4193285053 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f85c0056390 con 0x7f85c815be50 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 msgr2=0x7f85a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 0x7f85a8069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0af000 tx=0x7f85bc008040 comp rx=0 tx=0).stop 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 msgr2=0x7f85c813ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007e70 tx=0x7f85c000d4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 shutdown_connections 2024-05-08T04:39:17.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f85a8066f60 0x7f85a8069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85c815ba70 0x7f85c8070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85c815be50 0x7f85c813ded0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 --2- 172.21.15.164:0/4193285053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c80cfda0 0x7f85c80b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 >> 172.21.15.164:0/4193285053 conn(0x7f85c806ac90 msgr2=0x7f85c806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 shutdown_connections 2024-05-08T04:39:17.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:17.890+0000 7f85d0600640 1 -- 172.21.15.164:0/4193285053 wait complete. 2024-05-08T04:39:17.913 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T04:39:18.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.046+0000 7f3f13800640 1 Processor -- start 2024-05-08T04:39:18.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.046+0000 7f3f13800640 1 -- start start 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f13800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f0c0cfda0 0x7f3f0c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f13800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f13800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0cd9f0 con 0x7f3f0c15ba70 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f13800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c15fe70 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f13800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0cdcd0 con 0x7f3f0c15be50 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f0c0cfda0 0x7f3f0c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f10e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3f0c15ba70 0x7f3f0c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54338/0 (socket says 172.21.15.164:54338) 2024-05-08T04:39:18.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f10e00640 1 -- 172.21.15.164:0/1692302615 learned_addr learned my addr 172.21.15.164:0/1692302615 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:18.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 --2- 172.21.15.164:0/1692302615 >> v2:172.21.15.164:3300/0 conn(0x7f3f0c0cfda0 0x7f3f0c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46808/0 (socket says 172.21.15.164:46808) 2024-05-08T04:39:18.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f11800640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 -- 172.21.15.164:0/1692302615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 msgr2=0x7f3f0c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f0be00640 1 -- 172.21.15.164:0/1692302615 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126877210 0 0) 0x7f3f0c0cd9f0 con 0x7f3f0c15ba70 2024-05-08T04:39:18.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 -- 172.21.15.164:0/1692302615 >> v1:172.21.15.164:6789/0 conn(0x7f3f0c15ba70 legacy=0x7f3f0c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.050+0000 7f3f03e00640 1 -- 172.21.15.164:0/1692302615 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c15f820 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f03e00640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c160300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40097c0 tx=0x7f3ef401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e0e32b9f43c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f0be00640 1 -- 172.21.15.164:0/1692302615 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40b2070 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f0be00640 1 -- 172.21.15.164:0/1692302615 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40ad680 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f0be00640 1 -- 172.21.15.164:0/1692302615 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40ada50 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- 172.21.15.164:0/1692302615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c0cfda0 msgr2=0x7f3f0c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c160300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40097c0 tx=0x7f3ef401c470 comp rx=0 tx=0).stop 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- 172.21.15.164:0/1692302615 shutdown_connections 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 --2- 172.21.15.164:0/1692302615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c160300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- 172.21.15.164:0/1692302615 >> 172.21.15.164:0/1692302615 conn(0x7f3f0c06ac90 msgr2=0x7f3f0c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- 172.21.15.164:0/1692302615 shutdown_connections 2024-05-08T04:39:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- 172.21.15.164:0/1692302615 wait complete. 2024-05-08T04:39:18.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 Processor -- start 2024-05-08T04:39:18.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.054+0000 7f3f13800640 1 -- start start 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c168d40 con 0x7f3f0c15ba70 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c168bc0 con 0x7f3f0c0cfda0 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c168ec0 con 0x7f3f0c15be50 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46810/0 (socket says 172.21.15.164:46810) 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 -- 172.21.15.164:0/2292291154 learned_addr learned my addr 172.21.15.164:0/2292291154 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f10e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f11800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 -- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 msgr2=0x7f3f0c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 -- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0c0cfda0 msgr2=0x7f3f0c148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 -- 172.21.15.164:0/2292291154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c159070 con 0x7f3f0c15ba70 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f03e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401a0a0 tx=0x7f3ef401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40b2040 con 0x7f3f0c15ba70 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40bd070 con 0x7f3f0c15ba70 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40addf0 con 0x7f3f0c15ba70 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c15d3b0 con 0x7f3f0c15ba70 2024-05-08T04:39:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c15d970 con 0x7f3f0c15ba70 2024-05-08T04:39:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c069d60 con 0x7f3f0c15ba70 2024-05-08T04:39:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.058+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef40c2460 con 0x7f3f0c15ba70 2024-05-08T04:39:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.062+0000 7f3f09600640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 0x7f3ee8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.062+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ef411f070 con 0x7f3f0c15ba70 2024-05-08T04:39:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.062+0000 7f3f10e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 0x7f3ee8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.062+0000 7f3f10e00640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 0x7f3ee8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3efc000f70 tx=0x7f3efc007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.066+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef4016090 con 0x7f3f0c15ba70 2024-05-08T04:39:18.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.274+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3f0c06e210 con 0x7f3f0c15ba70 2024-05-08T04:39:18.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.278+0000 7f3f09600640 1 -- 172.21.15.164:0/2292291154 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3ef40f0330 con 0x7f3f0c15ba70 2024-05-08T04:39:18.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 msgr2=0x7f3ee8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 0x7f3ee8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3efc000f70 tx=0x7f3efc007be0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 msgr2=0x7f3f0c15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ef401a0a0 tx=0x7f3ef401acb0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 shutdown_connections 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ee8066f90 0x7f3ee8069450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f0c15be50 0x7f3f0c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f0c15ba70 0x7f3f0c15cc60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 --2- 172.21.15.164:0/2292291154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0c0cfda0 0x7f3f0c148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 >> 172.21.15.164:0/2292291154 conn(0x7f3f0c06ac90 msgr2=0x7f3f0c157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 shutdown_connections 2024-05-08T04:39:18.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.282+0000 7f3f13800640 1 -- 172.21.15.164:0/2292291154 wait complete. 2024-05-08T04:39:18.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T04:39:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 Processor -- start 2024-05-08T04:39:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 -- start start 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f29640b26c0 0x7f29640cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f29640cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296406c770 con 0x7f296415ab60 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640d0060 con 0x7f29640b26c0 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296406ca50 con 0x7f29640cfbd0 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2969800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f296415ab60 0x7f2964072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54354/0 (socket says 172.21.15.164:54354) 2024-05-08T04:39:18.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2969800640 1 -- 172.21.15.164:0/3731682474 learned_addr learned my addr 172.21.15.164:0/3731682474 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794048340 0 0) 0x7f296406c770 con 0x7f296415ab60 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003530 con 0x7f296415ab60 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296a200640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f29640cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2968e00640 1 --2- 172.21.15.164:0/3731682474 >> v2:172.21.15.172:3300/0 conn(0x7f29640b26c0 0x7f29640cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882063282 0 0) 0x7f294c003530 con 0x7f296415ab60 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 msgr2=0x7f29640cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f29640cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 >> v2:172.21.15.172:3300/0 conn(0x7f29640b26c0 msgr2=0x7f29640cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 --2- 172.21.15.164:0/3731682474 >> v2:172.21.15.172:3300/0 conn(0x7f29640b26c0 0x7f29640cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296406cd30 con 0x7f296415ab60 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2958001680 con 0x7f296415ab60 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2968e00640 1 --2- 172.21.15.164:0/3731682474 >> v2:172.21.15.172:3300/0 conn(0x7f29640b26c0 0x7f29640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 >> v1:172.21.15.164:6789/0 conn(0x7f296415ab60 legacy=0x7f2964072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f294c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f294c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c004d50 0x7f294c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296406ca50 con 0x7f294c004d50 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00b9d0 con 0x7f294c0039c0 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00bb80 con 0x7f29640cfbd0 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2969800640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f294c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f2968e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f294c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.438+0000 7f296a200640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c004d50 0x7f294c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 -- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 msgr2=0x7f294c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f294c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 -- 172.21.15.164:0/3731682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c004d50 msgr2=0x7f294c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c004d50 0x7f294c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 -- 172.21.15.164:0/3731682474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c003530 con 0x7f294c0039c0 2024-05-08T04:39:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296a200640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f294c004d50 0x7f294c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2968e00640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29640cfbd0 0x7f294c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2969800640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f294c003de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f295800c270 tx=0x7f295801cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3aa7737aa4ba89e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958007d40 con 0x7f294c0039c0 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29580a9020 con 0x7f294c0039c0 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f2963e00640 1 -- 172.21.15.164:0/3731682474 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29580a5ba0 con 0x7f294c0039c0 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296c200640 1 -- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 msgr2=0x7f294c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296c200640 1 --2- 172.21.15.164:0/3731682474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f294c003de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f295800c270 tx=0x7f295801cca0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296a200640 1 -- 172.21.15.164:0/3731682474 reap_dead start 2024-05-08T04:39:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296c200640 1 -- 172.21.15.164:0/3731682474 shutdown_connections 2024-05-08T04:39:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296c200640 1 -- 172.21.15.164:0/3731682474 >> 172.21.15.164:0/3731682474 conn(0x7f296406ac90 msgr2=0x7f296406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.442+0000 7f296c200640 1 -- 172.21.15.164:0/3731682474 shutdown_connections 2024-05-08T04:39:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- 172.21.15.164:0/3731682474 wait complete. 2024-05-08T04:39:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 Processor -- start 2024-05-08T04:39:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- start start 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f29641f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 0x7f29641591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296406c3d0 con 0x7f29641520d0 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296406d2f0 con 0x7f294c0039c0 2024-05-08T04:39:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296406c550 con 0x7f29641f71a0 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46820/0 (socket says 172.21.15.164:46820) 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 -- 172.21.15.164:0/587954712 learned_addr learned my addr 172.21.15.164:0/587954712 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2969800640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f29641f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296a200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 0x7f29641591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 -- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 msgr2=0x7f29641591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 0x7f29641591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 -- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 msgr2=0x7f29641f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f29641f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 -- 172.21.15.164:0/587954712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29641f3670 con 0x7f29641520d0 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2969800640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f29641f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296a200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 0x7f29641591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2968e00640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2950011620 tx=0x7f2950011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950019070 con 0x7f29641520d0 2024-05-08T04:39:18.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000f920 con 0x7f29641520d0 2024-05-08T04:39:18.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29641f7700 con 0x7f29641520d0 2024-05-08T04:39:18.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000fd10 con 0x7f29641520d0 2024-05-08T04:39:18.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.446+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29641f38f0 con 0x7f29641520d0 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2964069d60 con 0x7f29641520d0 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f295001c070 con 0x7f29641520d0 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f2961600640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 0x7f2944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2950085150 con 0x7f29641520d0 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f2969800640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 0x7f2944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.450+0000 7f2969800640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 0x7f2944069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29580aa000 tx=0x7f296406c940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.454+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2950023030 con 0x7f29641520d0 2024-05-08T04:39:18.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.662+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f296406d9a0 con 0x7f29641520d0 2024-05-08T04:39:18.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f2961600640 1 -- 172.21.15.164:0/587954712 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f29641f3670 con 0x7f29641520d0 2024-05-08T04:39:18.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 msgr2=0x7f2944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 0x7f2944069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29580aa000 tx=0x7f296406c940 comp rx=0 tx=0).stop 2024-05-08T04:39:18.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 msgr2=0x7f29641f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2950011620 tx=0x7f2950011ae0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 shutdown_connections 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2944066f60 0x7f2944069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29641f71a0 0x7f29641591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29641520d0 0x7f29641f6c60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 --2- 172.21.15.164:0/587954712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f294c0039c0 0x7f29641f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.666+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 >> 172.21.15.164:0/587954712 conn(0x7f296406ac90 msgr2=0x7f2964156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.670+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 shutdown_connections 2024-05-08T04:39:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.670+0000 7f296c200640 1 -- 172.21.15.164:0/587954712 wait complete. 2024-05-08T04:39:18.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:18.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.822+0000 7feb34200640 1 Processor -- start 2024-05-08T04:39:18.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.822+0000 7feb34200640 1 -- start start 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.822+0000 7feb34200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb34200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb2c0cfda0 0x7feb2c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb34200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c072640 con 0x7feb2c0b26c0 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb34200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c06c770 con 0x7feb2c0cfda0 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb34200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c06ca50 con 0x7feb2c15ab60 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb2c0cfda0 0x7feb2c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb2c0cfda0 0x7feb2c0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46830/0 (socket says 172.21.15.164:46830) 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 -- 172.21.15.164:0/4220752634 learned_addr learned my addr 172.21.15.164:0/4220752634 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb31800640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 -- 172.21.15.164:0/4220752634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 msgr2=0x7feb2c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 -- 172.21.15.164:0/4220752634 >> v1:172.21.15.172:6789/0 conn(0x7feb2c0b26c0 legacy=0x7feb2c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 -- 172.21.15.164:0/4220752634 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c06cd30 con 0x7feb2c0cfda0 2024-05-08T04:39:18.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb31800640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:18.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb32200640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c0cd910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f55a8f004375fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.826+0000 7feb2be00640 1 -- 172.21.15.164:0/4220752634 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240b2070 con 0x7feb2c0cfda0 2024-05-08T04:39:18.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb2be00640 1 -- 172.21.15.164:0/4220752634 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb240a9070 con 0x7feb2c0cfda0 2024-05-08T04:39:18.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb2be00640 1 -- 172.21.15.164:0/4220752634 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240ad880 con 0x7feb2c0cfda0 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- 172.21.15.164:0/4220752634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 msgr2=0x7feb2c0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c0cd910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2401c560 comp rx=0 tx=0).stop 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- 172.21.15.164:0/4220752634 shutdown_connections 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c0cd910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- 172.21.15.164:0/4220752634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- 172.21.15.164:0/4220752634 >> 172.21.15.164:0/4220752634 conn(0x7feb2c06ac90 msgr2=0x7feb2c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- 172.21.15.164:0/4220752634 shutdown_connections 2024-05-08T04:39:18.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- 172.21.15.164:0/4220752634 wait complete. 2024-05-08T04:39:18.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 Processor -- start 2024-05-08T04:39:18.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- start start 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb2c0b26c0 0x7feb2c0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0aeeb0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0aed30 con 0x7feb2c0b26c0 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb34200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c167970 con 0x7feb2c15ab60 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb30e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb30e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46834/0 (socket says 172.21.15.164:46834) 2024-05-08T04:39:18.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb30e00640 1 -- 172.21.15.164:0/3976798893 learned_addr learned my addr 172.21.15.164:0/3976798893 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb31800640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb2c0b26c0 0x7feb2c0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.830+0000 7feb32200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 -- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 msgr2=0x7feb2c15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 -- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb2c0b26c0 msgr2=0x7feb2c0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb2c0b26c0 0x7feb2c0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 -- 172.21.15.164:0/3976798893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c1501d0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb30e00640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c420 tx=0x7feb1800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180052d0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18005470 con 0x7feb2c0cfda0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18007260 con 0x7feb2c0cfda0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c1fb2d0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c1fa7f0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c069d60 con 0x7feb2c0cfda0 2024-05-08T04:39:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb18007400 con 0x7feb2c0cfda0 2024-05-08T04:39:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb18084170 con 0x7feb2c0cfda0 2024-05-08T04:39:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.834+0000 7feb31800640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.838+0000 7feb31800640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 0x7feb0c069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb2c15d320 tx=0x7feb200097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.838+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb18054e70 con 0x7feb2c0cfda0 2024-05-08T04:39:18.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.990+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c06d9a0 con 0x7feb2c0cfda0 2024-05-08T04:39:18.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb29600640 1 -- 172.21.15.164:0/3976798893 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12951 (secure 0 0 0) 0x7feb18058f10 con 0x7feb2c0cfda0 2024-05-08T04:39:18.997 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:18.997 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:15.970283+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 msgr2=0x7feb0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 0x7feb0c069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb2c15d320 tx=0x7feb200097f0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 msgr2=0x7feb2c15ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c420 tx=0x7feb1800c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 shutdown_connections 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb2c15ab60 0x7feb2c15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb2c0cfda0 0x7feb2c15ea00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 --2- 172.21.15.164:0/3976798893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb2c0b26c0 0x7feb2c0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 >> 172.21.15.164:0/3976798893 conn(0x7feb2c06ac90 msgr2=0x7feb2c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:18.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 shutdown_connections 2024-05-08T04:39:19.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:18.994+0000 7feb34200640 1 -- 172.21.15.164:0/3976798893 wait complete. 2024-05-08T04:39:19.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:19.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:19.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.154+0000 7fc797e00640 1 Processor -- start 2024-05-08T04:39:19.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.154+0000 7fc797e00640 1 -- start start 2024-05-08T04:39:19.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc7980cec40 0x7fc79815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79815c900 con 0x7fc7980b1d40 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79815cec0 con 0x7fc7980cec40 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815cbe0 con 0x7fc7980b1960 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc7980cec40 0x7fc79815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc7980b1d40 0x7fc7980ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:54370/0 (socket says 172.21.15.164:54370) 2024-05-08T04:39:19.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc797400640 1 -- 172.21.15.164:0/698891097 learned_addr learned my addr 172.21.15.164:0/698891097 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc796a00640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 -- 172.21.15.164:0/698891097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 msgr2=0x7fc798071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 -- 172.21.15.164:0/698891097 >> v1:172.21.15.164:6789/0 conn(0x7fc7980b1d40 legacy=0x7fc7980ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 -- 172.21.15.164:0/698891097 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980712a0 con 0x7fc7980cec40 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc796000640 1 -- 172.21.15.164:0/698891097 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365401569 0 0) 0x7fc79815c900 con 0x7fc7980b1d40 2024-05-08T04:39:19.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc796a00640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:19.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.158+0000 7fc78e000640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc79815bf90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc7800097c0 tx=0x7fc78001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e952d8a6d6dcd07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc796000640 1 -- 172.21.15.164:0/698891097 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7800b0070 con 0x7fc7980cec40 2024-05-08T04:39:19.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc796000640 1 -- 172.21.15.164:0/698891097 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7800ab600 con 0x7fc7980cec40 2024-05-08T04:39:19.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc796000640 1 -- 172.21.15.164:0/698891097 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7800ab9d0 con 0x7fc7980cec40 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- 172.21.15.164:0/698891097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 msgr2=0x7fc79815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc79815bf90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc7800097c0 tx=0x7fc78001c350 comp rx=0 tx=0).stop 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- 172.21.15.164:0/698891097 shutdown_connections 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc79815bf90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- 172.21.15.164:0/698891097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- 172.21.15.164:0/698891097 >> 172.21.15.164:0/698891097 conn(0x7fc79806ac90 msgr2=0x7fc79806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- 172.21.15.164:0/698891097 shutdown_connections 2024-05-08T04:39:19.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- 172.21.15.164:0/698891097 wait complete. 2024-05-08T04:39:19.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 Processor -- start 2024-05-08T04:39:19.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- start start 2024-05-08T04:39:19.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7980b1d40 0x7fc7980ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b3bf0 con 0x7fc7980cec40 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815c6e0 con 0x7fc7980b1d40 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b3d70 con 0x7fc7980b1960 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46850/0 (socket says 172.21.15.164:46850) 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 -- 172.21.15.164:0/1514096305 learned_addr learned my addr 172.21.15.164:0/1514096305 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 -- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 msgr2=0x7fc798146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 -- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7980b1d40 msgr2=0x7fc7980ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.162+0000 7fc797400640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7980b1d40 0x7fc7980ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc797400640 1 -- 172.21.15.164:0/1514096305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798150440 con 0x7fc7980cec40 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc797400640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c7d0 tx=0x7fc79000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790018070 con 0x7fc7980cec40 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000a640 con 0x7fc7980cec40 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc798150720 con 0x7fc7980cec40 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000aa70 con 0x7fc7980cec40 2024-05-08T04:39:19.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc798150ce0 con 0x7fc7980cec40 2024-05-08T04:39:19.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79804ef30 con 0x7fc7980cec40 2024-05-08T04:39:19.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc79001b070 con 0x7fc7980cec40 2024-05-08T04:39:19.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 0x7fc774069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc790084320 con 0x7fc7980cec40 2024-05-08T04:39:19.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.166+0000 7fc796a00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 0x7fc774069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.170+0000 7fc796a00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 0x7fc774069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc798071f90 tx=0x7fc788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.170+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc790055660 con 0x7fc7980cec40 2024-05-08T04:39:19.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.326+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7980b2ef0 con 0x7fc7980cec40 2024-05-08T04:39:19.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.326+0000 7fc78f400640 1 -- 172.21.15.164:0/1514096305 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc790054f60 con 0x7fc7980cec40 2024-05-08T04:39:19.330 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.326+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 msgr2=0x7fc774069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 0x7fc774069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc798071f90 tx=0x7fc788008040 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 msgr2=0x7fc7980addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c7d0 tx=0x7fc79000cc90 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 shutdown_connections 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc774066f60 0x7fc774069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7980cec40 0x7fc7980addd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7980b1d40 0x7fc7980ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 --2- 172.21.15.164:0/1514096305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980b1960 0x7fc798146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 >> 172.21.15.164:0/1514096305 conn(0x7fc79806ac90 msgr2=0x7fc79806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 shutdown_connections 2024-05-08T04:39:19.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.330+0000 7fc797e00640 1 -- 172.21.15.164:0/1514096305 wait complete. 2024-05-08T04:39:19.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:39:19.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 Processor -- start 2024-05-08T04:39:19.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 -- start start 2024-05-08T04:39:19.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 0x7f1bb40c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40ccc40 con 0x7f1bb40c5530 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb415ae90 con 0x7f1bb40cc5e0 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40c5ed0 con 0x7f1bb415aab0 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb8e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 0x7f1bb40c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49270/0 (socket says 172.21.15.164:49270) 2024-05-08T04:39:19.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 -- 172.21.15.164:0/3448232720 learned_addr learned my addr 172.21.15.164:0/3448232720 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:19.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 -- 172.21.15.164:0/3448232720 >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 msgr2=0x7f1bb40c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:19.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 --2- 172.21.15.164:0/3448232720 >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 0x7f1bb40c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 -- 172.21.15.164:0/3448232720 >> v1:172.21.15.172:6789/0 conn(0x7f1bb40c5530 legacy=0x7f1bb40cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb3e00640 1 -- 172.21.15.164:0/3448232720 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40ce170 con 0x7f1bb415aab0 2024-05-08T04:39:19.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.486+0000 7f1bb8e00640 1 --2- 172.21.15.164:0/3448232720 >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 0x7f1bb40c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb3e00640 1 --2- 172.21.15.164:0/3448232720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009870 tx=0x7f1ba401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e4205838821313 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb3400640 1 -- 172.21.15.164:0/3448232720 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40b2070 con 0x7f1bb415aab0 2024-05-08T04:39:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb3400640 1 -- 172.21.15.164:0/3448232720 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba40a9070 con 0x7f1bb415aab0 2024-05-08T04:39:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb3400640 1 -- 172.21.15.164:0/3448232720 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40ad880 con 0x7f1bb415aab0 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 -- 172.21.15.164:0/3448232720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 msgr2=0x7f1bb40cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 --2- 172.21.15.164:0/3448232720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009870 tx=0x7f1ba401c560 comp rx=0 tx=0).stop 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 -- 172.21.15.164:0/3448232720 shutdown_connections 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 --2- 172.21.15.164:0/3448232720 >> v2:172.21.15.172:3300/0 conn(0x7f1bb40cc5e0 0x7f1bb40c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 --2- 172.21.15.164:0/3448232720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb415aab0 0x7f1bb40cec50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 -- 172.21.15.164:0/3448232720 >> 172.21.15.164:0/3448232720 conn(0x7f1bb406b670 msgr2=0x7f1bb40b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 -- 172.21.15.164:0/3448232720 shutdown_connections 2024-05-08T04:39:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.490+0000 7f1bb9800640 1 -- 172.21.15.164:0/3448232720 wait complete. 2024-05-08T04:39:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 Processor -- start 2024-05-08T04:39:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- start start 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb40c5530 0x7f1bb4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bb40cc5e0 0x7f1bb415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40724b0 con 0x7f1bb415aab0 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40713f0 con 0x7f1bb40cc5e0 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40ce7c0 con 0x7f1bb40c5530 2024-05-08T04:39:19.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51920/0 (socket says 172.21.15.164:51920) 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 -- 172.21.15.164:0/4005967254 learned_addr learned my addr 172.21.15.164:0/4005967254 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb3e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb40c5530 0x7f1bb4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1babe00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bb40cc5e0 0x7f1bb415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 -- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb40c5530 msgr2=0x7f1bb4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb40c5530 0x7f1bb4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 -- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bb40cc5e0 msgr2=0x7f1bb415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bb40cc5e0 0x7f1bb415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 -- 172.21.15.164:0/4005967254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb415cd00 con 0x7f1bb415aab0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb8e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00efc0 tx=0x7f1bac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac018030 con 0x7f1bb415aab0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac00cde0 con 0x7f1bb415aab0 2024-05-08T04:39:19.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac003bc0 con 0x7f1bb415aab0 2024-05-08T04:39:19.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb415cfe0 con 0x7f1bb415aab0 2024-05-08T04:39:19.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.494+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb415d5a0 con 0x7f1bb415aab0 2024-05-08T04:39:19.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb404ef30 con 0x7f1bb415aab0 2024-05-08T04:39:19.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bac01b070 con 0x7f1bb415aab0 2024-05-08T04:39:19.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb0c00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 0x7f1b90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1bac085750 con 0x7f1bb415aab0 2024-05-08T04:39:19.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb3e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 0x7f1b90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.498+0000 7f1bb3e00640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 0x7f1b90069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.502+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bac010040 con 0x7f1bb415aab0 2024-05-08T04:39:19.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1bb4068170 con 0x7f1bb415aab0 2024-05-08T04:39:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb0c00640 1 -- 172.21.15.164:0/4005967254 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1bac056330 con 0x7f1bb415aab0 2024-05-08T04:39:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:19.661 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:39:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:39:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:39:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:19.662 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:39:19.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 msgr2=0x7f1b90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 0x7f1b90069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba40a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 msgr2=0x7f1bb41469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00efc0 tx=0x7f1bac009f90 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 shutdown_connections 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b90066f60 0x7f1b90069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb415aab0 0x7f1bb41469d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bb40cc5e0 0x7f1bb415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 --2- 172.21.15.164:0/4005967254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1bb40c5530 0x7f1bb4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 >> 172.21.15.164:0/4005967254 conn(0x7f1bb406b670 msgr2=0x7f1bb406df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 shutdown_connections 2024-05-08T04:39:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.658+0000 7f1bb9800640 1 -- 172.21.15.164:0/4005967254 wait complete. 2024-05-08T04:39:19.682 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:39:19.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:39:19.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 Processor -- start 2024-05-08T04:39:19.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 -- start start 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c15ba20 0x7faa2c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa2c06de50 0x7faa2c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0c6600 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15fbe0 con 0x7faa2c06de50 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa33200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c6040 con 0x7faa2c15ba20 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa31200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7faa2c0ae6e0 0x7faa2c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38922/0 (socket says 172.21.15.164:38922) 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa31200640 1 -- 172.21.15.164:0/2939946719 learned_addr learned my addr 172.21.15.164:0/2939946719 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:19.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2b400640 1 --2- 172.21.15.164:0/2939946719 >> v2:172.21.15.172:3300/0 conn(0x7faa2c06de50 0x7faa2c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2be00640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c15ba20 0x7faa2c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2aa00640 1 -- 172.21.15.164:0/2939946719 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367381346 0 0) 0x7faa2c0c6600 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2aa00640 1 -- 172.21.15.164:0/2939946719 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa10003530 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2b400640 1 -- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c15ba20 msgr2=0x7faa2c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2b400640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c15ba20 0x7faa2c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2b400640 1 -- 172.21.15.164:0/2939946719 >> v1:172.21.15.164:6789/0 conn(0x7faa2c0ae6e0 legacy=0x7faa2c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.818+0000 7faa2b400640 1 -- 172.21.15.164:0/2939946719 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0c6320 con 0x7faa2c06de50 2024-05-08T04:39:19.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa2b400640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c06de50 0x7faa2c06e290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faa2c15fd60 tx=0x7faa1801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347ae4a50e873fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa2aa00640 1 -- 172.21.15.164:0/2939946719 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180ab610 con 0x7faa2c06de50 2024-05-08T04:39:19.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa2aa00640 1 -- 172.21.15.164:0/2939946719 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180b0070 con 0x7faa2c06de50 2024-05-08T04:39:19.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa2aa00640 1 -- 172.21.15.164:0/2939946719 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180ab9d0 con 0x7faa2c06de50 2024-05-08T04:39:19.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 -- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c06de50 msgr2=0x7faa2c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c06de50 0x7faa2c06e290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faa2c15fd60 tx=0x7faa1801c470 comp rx=0 tx=0).stop 2024-05-08T04:39:19.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 -- 172.21.15.164:0/2939946719 shutdown_connections 2024-05-08T04:39:19.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c06de50 0x7faa2c06e290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 --2- 172.21.15.164:0/2939946719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c15ba20 0x7faa2c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 -- 172.21.15.164:0/2939946719 >> 172.21.15.164:0/2939946719 conn(0x7faa2c0cf770 msgr2=0x7faa2c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:19.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 -- 172.21.15.164:0/2939946719 shutdown_connections 2024-05-08T04:39:19.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.822+0000 7faa33200640 1 -- 172.21.15.164:0/2939946719 wait complete. 2024-05-08T04:39:19.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 Processor -- start 2024-05-08T04:39:19.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- start start 2024-05-08T04:39:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c06de50 0x7faa2c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c15ba20 0x7faa2c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0aec40 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0aeac0 con 0x7faa2c15ba20 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0aedc0 con 0x7faa2c06de50 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51926/0 (socket says 172.21.15.164:51926) 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 -- 172.21.15.164:0/3247308774 learned_addr learned my addr 172.21.15.164:0/3247308774 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2be00640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c06de50 0x7faa2c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa31200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c15ba20 0x7faa2c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 -- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c06de50 msgr2=0x7faa2c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c06de50 0x7faa2c0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 -- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c15ba20 msgr2=0x7faa2c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c15ba20 0x7faa2c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 -- 172.21.15.164:0/3247308774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c153840 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa2b400640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa180ad040 tx=0x7faa1801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180bf410 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180bf5b0 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180bf9a0 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c153b20 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.826+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c14ebd0 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c072d30 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa180bd020 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa23e00640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faa1811ede0 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa2be00640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:19.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.830+0000 7faa2be00640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 0x7faa0c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa2c15d8d0 tx=0x7faa1c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:19.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.834+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa180c6030 con 0x7faa2c0ae6e0 2024-05-08T04:39:19.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:19.986+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faa2c14b1c0 con 0x7faa2c0ae6e0 2024-05-08T04:39:20.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.126+0000 7faa23e00640 1 -- 172.21.15.164:0/3247308774 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7faa180a8050 con 0x7faa2c0ae6e0 2024-05-08T04:39:20.131 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 msgr2=0x7faa0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 0x7faa0c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa2c15d8d0 tx=0x7faa1c009190 comp rx=0 tx=0).stop 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 msgr2=0x7faa2c0c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa180ad040 tx=0x7faa1801acb0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 shutdown_connections 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa2c15ba20 0x7faa2c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa2c0ae6e0 0x7faa2c0c8380 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 --2- 172.21.15.164:0/3247308774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2c06de50 0x7faa2c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 >> 172.21.15.164:0/3247308774 conn(0x7faa2c0cf770 msgr2=0x7faa2c151390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:20.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 shutdown_connections 2024-05-08T04:39:20.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.130+0000 7faa33200640 1 -- 172.21.15.164:0/3247308774 wait complete. 2024-05-08T04:39:20.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:39:20.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 Processor -- start 2024-05-08T04:39:20.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 -- start start 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f940415ba20 0x7f9404160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f94040aedf0 0x7f940406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94040c6600 con 0x7f940406de50 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940415fbe0 con 0x7f940415ba20 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9409600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94040c6040 con 0x7f94040aedf0 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9402a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f940415ba20 0x7f9404160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f93fa000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f940406de50 0x7f94040ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38938/0 (socket says 172.21.15.164:38938) 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f93fa000640 1 -- 172.21.15.164:0/4058615576 learned_addr learned my addr 172.21.15.164:0/4058615576 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9403400640 1 --2- 172.21.15.164:0/4058615576 >> v2:172.21.15.172:3300/0 conn(0x7f94040aedf0 0x7f940406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9402a00640 1 -- 172.21.15.164:0/4058615576 >> v2:172.21.15.172:3300/0 conn(0x7f94040aedf0 msgr2=0x7f940406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9402a00640 1 --2- 172.21.15.164:0/4058615576 >> v2:172.21.15.172:3300/0 conn(0x7f94040aedf0 0x7f940406e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9402a00640 1 -- 172.21.15.164:0/4058615576 >> v1:172.21.15.164:6789/0 conn(0x7f940406de50 legacy=0x7f94040ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.298+0000 7f9402a00640 1 -- 172.21.15.164:0/4058615576 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040c6320 con 0x7f940415ba20 2024-05-08T04:39:20.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9402a00640 1 --2- 172.21.15.164:0/4058615576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940415ba20 0x7f9404160070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f940406bd80 tx=0x7f93f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872f4f0d4922d6f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9402000640 1 -- 172.21.15.164:0/4058615576 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40ab5f0 con 0x7f940415ba20 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9402000640 1 -- 172.21.15.164:0/4058615576 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40b0070 con 0x7f940415ba20 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9402000640 1 -- 172.21.15.164:0/4058615576 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40ab9b0 con 0x7f940415ba20 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 -- 172.21.15.164:0/4058615576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940415ba20 msgr2=0x7f9404160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 --2- 172.21.15.164:0/4058615576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940415ba20 0x7f9404160070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f940406bd80 tx=0x7f93f401c450 comp rx=0 tx=0).stop 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 -- 172.21.15.164:0/4058615576 shutdown_connections 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 --2- 172.21.15.164:0/4058615576 >> v2:172.21.15.172:3300/0 conn(0x7f94040aedf0 0x7f940406e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 --2- 172.21.15.164:0/4058615576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940415ba20 0x7f9404160070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 -- 172.21.15.164:0/4058615576 >> 172.21.15.164:0/4058615576 conn(0x7f94040cf770 msgr2=0x7f94040cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 -- 172.21.15.164:0/4058615576 shutdown_connections 2024-05-08T04:39:20.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.302+0000 7f9409600640 1 -- 172.21.15.164:0/4058615576 wait complete. 2024-05-08T04:39:20.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 Processor -- start 2024-05-08T04:39:20.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- start start 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 0x7f940414ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 0x7f940414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040ce000 con 0x7f940406de50 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040cde80 con 0x7f94040aedf0 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9404168b70 con 0x7f940415ba20 2024-05-08T04:39:20.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51940/0 (socket says 172.21.15.164:51940) 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 -- 172.21.15.164:0/3131137936 learned_addr learned my addr 172.21.15.164:0/3131137936 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9403400640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 0x7f940414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f93fa000640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 0x7f940414ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 -- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 msgr2=0x7f940414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 0x7f940414f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 -- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 msgr2=0x7f940414ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 0x7f940414ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 -- 172.21.15.164:0/3131137936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040caae0 con 0x7f940406de50 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9403400640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 0x7f940414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f93fa000640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 0x7f940414ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9402a00640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93f40ad040 tx=0x7f93f401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40abe60 con 0x7f940406de50 2024-05-08T04:39:20.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40bf610 con 0x7f940406de50 2024-05-08T04:39:20.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9404152850 con 0x7f940406de50 2024-05-08T04:39:20.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40bf9e0 con 0x7f940406de50 2024-05-08T04:39:20.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.306+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9404152e10 con 0x7f940406de50 2024-05-08T04:39:20.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9404072d30 con 0x7f940406de50 2024-05-08T04:39:20.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93f40bd020 con 0x7f940406de50 2024-05-08T04:39:20.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f93fb400640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 0x7f93e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:20.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f93f411f560 con 0x7f940406de50 2024-05-08T04:39:20.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f93fa000640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 0x7f93e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:20.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.310+0000 7f93fa000640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 0x7f93e0069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007fd0 tx=0x7f93ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:20.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.314+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94040caae0 con 0x7f940406de50 2024-05-08T04:39:20.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:20.470+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f940404f010 con 0x7f940406de50 2024-05-08T04:39:21.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.130+0000 7f93fb400640 1 -- 172.21.15.164:0/3131137936 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f93f4016070 con 0x7f940406de50 2024-05-08T04:39:21.134 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:39:21.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.130+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 msgr2=0x7f93e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.130+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 0x7f93e0069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007fd0 tx=0x7f93ec009040 comp rx=0 tx=0).stop 2024-05-08T04:39:21.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.130+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 msgr2=0x7f940414d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.130+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93f40ad040 tx=0x7f93f401ac90 comp rx=0 tx=0).stop 2024-05-08T04:39:21.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 shutdown_connections 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93e0066f60 0x7f93e0069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f940415ba20 0x7f940414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040aedf0 0x7f940414ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 --2- 172.21.15.164:0/3131137936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f940406de50 0x7f940414d3f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 >> 172.21.15.164:0/3131137936 conn(0x7f94040cf770 msgr2=0x7f940415d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 shutdown_connections 2024-05-08T04:39:21.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.134+0000 7f9409600640 1 -- 172.21.15.164:0/3131137936 wait complete. 2024-05-08T04:39:21.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:39:21.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.314+0000 7fe8d3e00640 1 Processor -- start 2024-05-08T04:39:21.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.314+0000 7fe8d3e00640 1 -- start start 2024-05-08T04:39:21.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8d415dca0 0x7fe8d40b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 0x7fe8d406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d4161ad0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d4161880 con 0x7fe8d415dca0 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4161db0 con 0x7fe8d40d1df0 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8d415dca0 0x7fe8d40b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8d415dca0 0x7fe8d40b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51950/0 (socket says 172.21.15.164:51950) 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe8d40b0f00 0x7fe8d40d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38946/0 (socket says 172.21.15.164:38946) 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 -- 172.21.15.164:0/3404555243 learned_addr learned my addr 172.21.15.164:0/3404555243 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3400640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 0x7fe8d406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 -- 172.21.15.164:0/3404555243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 msgr2=0x7fe8d406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 0x7fe8d406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 -- 172.21.15.164:0/3404555243 >> v1:172.21.15.164:6789/0 conn(0x7fe8d40b0f00 legacy=0x7fe8d40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d2a00640 1 -- 172.21.15.164:0/3404555243 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4162090 con 0x7fe8d415dca0 2024-05-08T04:39:21.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d1600640 1 -- 172.21.15.164:0/3404555243 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835078300 0 0) 0x7fe8d4161ad0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.318+0000 7fe8d3400640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 0x7fe8d406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:21.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d2a00640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d415dca0 0x7fe8d40b0820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009e80 tx=0x7fe8c401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5441c0cc048c6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:21.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d1600640 1 -- 172.21.15.164:0/3404555243 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40a57f0 con 0x7fe8d415dca0 2024-05-08T04:39:21.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d1600640 1 -- 172.21.15.164:0/3404555243 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c40a5990 con 0x7fe8d415dca0 2024-05-08T04:39:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d1600640 1 -- 172.21.15.164:0/3404555243 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40a5d60 con 0x7fe8d415dca0 2024-05-08T04:39:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- 172.21.15.164:0/3404555243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d415dca0 msgr2=0x7fe8d40b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d415dca0 0x7fe8d40b0820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009e80 tx=0x7fe8c401ca10 comp rx=0 tx=0).stop 2024-05-08T04:39:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- 172.21.15.164:0/3404555243 shutdown_connections 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d40d1df0 0x7fe8d406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/3404555243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d415dca0 0x7fe8d40b0820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- 172.21.15.164:0/3404555243 >> 172.21.15.164:0/3404555243 conn(0x7fe8d40723e0 msgr2=0x7fe8d40727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- 172.21.15.164:0/3404555243 shutdown_connections 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- 172.21.15.164:0/3404555243 wait complete. 2024-05-08T04:39:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 Processor -- start 2024-05-08T04:39:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- start start 2024-05-08T04:39:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d40d1df0 0x7fe8d40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d415dca0 0x7fe8d40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d406d5c0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d406d440 con 0x7fe8d40d1df0 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d406d740 con 0x7fe8d415dca0 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.322+0000 7fe8d2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51962/0 (socket says 172.21.15.164:51962) 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 -- 172.21.15.164:0/2733665232 learned_addr learned my addr 172.21.15.164:0/2733665232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d3400640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d415dca0 0x7fe8d40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2000640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d40d1df0 0x7fe8d40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 -- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d415dca0 msgr2=0x7fe8d40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d415dca0 0x7fe8d40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 -- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d40d1df0 msgr2=0x7fe8d40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:21.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d40d1df0 0x7fe8d40cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:21.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 -- 172.21.15.164:0/2733665232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40ccde0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d2a00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af040 tx=0x7fe8c40197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40b8470 con 0x7fe8d40b0f00 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c40b8610 con 0x7fe8d40b0f00 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d40cd0c0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40b89c0 con 0x7fe8d40b0f00 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d4143740 con 0x7fe8d40b0f00 2024-05-08T04:39:21.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.326+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d40cab90 con 0x7fe8d40b0f00 2024-05-08T04:39:21.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.330+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8c40c0020 con 0x7fe8d40b0f00 2024-05-08T04:39:21.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.330+0000 7fe8caa00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 0x7fe8b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:21.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.330+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe8c4120770 con 0x7fe8d40b0f00 2024-05-08T04:39:21.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.330+0000 7fe8d2000640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 0x7fe8b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:21.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.330+0000 7fe8d2000640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 0x7fe8b4069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40cf810 tx=0x7fe8c0007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:21.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.334+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8c4016600 con 0x7fe8d40b0f00 2024-05-08T04:39:21.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:21.522+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe8d404ef30 con 0x7fe8d40b0f00 2024-05-08T04:39:22.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.154+0000 7fe8caa00640 1 -- 172.21.15.164:0/2733665232 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe8c40f17b0 con 0x7fe8d40b0f00 2024-05-08T04:39:22.161 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 4 and data pool 5 2024-05-08T04:39:22.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 msgr2=0x7fe8b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 0x7fe8b4069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40cf810 tx=0x7fe8c0007b60 comp rx=0 tx=0).stop 2024-05-08T04:39:22.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 msgr2=0x7fe8d4143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af040 tx=0x7fe8c40197b0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 shutdown_connections 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8b4066f60 0x7fe8b4069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d415dca0 0x7fe8d40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d40d1df0 0x7fe8d40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/2733665232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8d40b0f00 0x7fe8d4143200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 >> 172.21.15.164:0/2733665232 conn(0x7fe8d40723e0 msgr2=0x7fe8d40cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 shutdown_connections 2024-05-08T04:39:22.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.162+0000 7fe8d3e00640 1 -- 172.21.15.164:0/2733665232 wait complete. 2024-05-08T04:39:22.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 Processor -- start 2024-05-08T04:39:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 -- start start 2024-05-08T04:39:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 0x7f239815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2398161fe0 con 0x7f23980cfd60 2024-05-08T04:39:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2398162880 con 0x7f239815ea20 2024-05-08T04:39:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f239dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23981622c0 con 0x7f2398144200 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f2396a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f2397400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 0x7f239815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f238e000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f23980cfd60 0x7f23980cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:41502/0 (socket says 172.21.15.164:41502) 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f2396a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49336/0 (socket says 172.21.15.164:49336) 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f238e000640 1 -- 172.21.15.164:0/2285611359 learned_addr learned my addr 172.21.15.164:0/2285611359 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f2396a00640 1 -- 172.21.15.164:0/2285611359 >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 msgr2=0x7f239815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.330+0000 7f2396a00640 1 --2- 172.21.15.164:0/2285611359 >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 0x7f239815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396a00640 1 -- 172.21.15.164:0/2285611359 >> v1:172.21.15.172:6789/0 conn(0x7f23980cfd60 legacy=0x7f23980cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396a00640 1 -- 172.21.15.164:0/2285611359 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23981625a0 con 0x7f2398144200 2024-05-08T04:39:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2397400640 1 --2- 172.21.15.164:0/2285611359 >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 0x7f239815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:22.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396a00640 1 --2- 172.21.15.164:0/2285611359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2388009d90 tx=0x7f238801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5f24beb7523345 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396000640 1 -- 172.21.15.164:0/2285611359 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880b2070 con 0x7f2398144200 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396000640 1 -- 172.21.15.164:0/2285611359 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880a57a0 con 0x7f2398144200 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f2396000640 1 -- 172.21.15.164:0/2285611359 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880a5b70 con 0x7f2398144200 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f239dc00640 1 -- 172.21.15.164:0/2285611359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 msgr2=0x7f2398161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.334+0000 7f239dc00640 1 --2- 172.21.15.164:0/2285611359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2388009d90 tx=0x7f238801c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- 172.21.15.164:0/2285611359 shutdown_connections 2024-05-08T04:39:22.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 --2- 172.21.15.164:0/2285611359 >> v2:172.21.15.172:3300/0 conn(0x7f239815ea20 0x7f239815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 --2- 172.21.15.164:0/2285611359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2398144200 0x7f2398161950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- 172.21.15.164:0/2285611359 >> 172.21.15.164:0/2285611359 conn(0x7f239806ac90 msgr2=0x7f239806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:22.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- 172.21.15.164:0/2285611359 shutdown_connections 2024-05-08T04:39:22.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- 172.21.15.164:0/2285611359 wait complete. 2024-05-08T04:39:22.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 Processor -- start 2024-05-08T04:39:22.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- start start 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23980cfd60 0x7f239814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398144200 0x7f23980718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980cf1f0 con 0x7f239815ea20 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980d0160 con 0x7f2398144200 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f239dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980cf370 con 0x7f23980cfd60 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51970/0 (socket says 172.21.15.164:51970) 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 -- 172.21.15.164:0/2186491001 learned_addr learned my addr 172.21.15.164:0/2186491001 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2396a00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23980cfd60 0x7f239814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f238e000640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398144200 0x7f23980718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 -- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23980cfd60 msgr2=0x7f239814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23980cfd60 0x7f239814dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 -- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398144200 msgr2=0x7f23980718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398144200 0x7f23980718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 -- 172.21.15.164:0/2186491001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239815ceb0 con 0x7f239815ea20 2024-05-08T04:39:22.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.338+0000 7f2397400640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f239000e950 tx=0x7f239000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390017070 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000ca00 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239815d190 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000cdd0 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239813d160 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2398069d60 con 0x7f239815ea20 2024-05-08T04:39:22.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2390013470 con 0x7f239815ea20 2024-05-08T04:39:22.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 0x7f2374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2390084f10 con 0x7f239815ea20 2024-05-08T04:39:22.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f2396a00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 0x7f2374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.342+0000 7f2396a00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 0x7f2374069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23880af000 tx=0x7f23880a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.346+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2390055830 con 0x7f239815ea20 2024-05-08T04:39:22.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23980add30 con 0x7f239815ea20 2024-05-08T04:39:22.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f238f400640 1 -- 172.21.15.164:0/2186491001 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f23900598d0 con 0x7f239815ea20 2024-05-08T04:39:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:22.510 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:21.525975+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 msgr2=0x7f2374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 0x7f2374069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23880af000 tx=0x7f23880a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 msgr2=0x7f2398071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f239000e950 tx=0x7f239000ee10 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 shutdown_connections 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2374066f60 0x7f2374069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f239815ea20 0x7f2398071e00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398144200 0x7f23980718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 --2- 172.21.15.164:0/2186491001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23980cfd60 0x7f239814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 >> 172.21.15.164:0/2186491001 conn(0x7f239806ac90 msgr2=0x7f239815fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:22.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 shutdown_connections 2024-05-08T04:39:22.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.506+0000 7f239dc00640 1 -- 172.21.15.164:0/2186491001 wait complete. 2024-05-08T04:39:22.531 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:22.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 Processor -- start 2024-05-08T04:39:22.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 -- start start 2024-05-08T04:39:22.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f597806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f59780cf150 0x7f597806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597806e0e0 con 0x7f59780aee30 2024-05-08T04:39:22.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597815e8f0 con 0x7f59780cf150 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597fa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597815a8c0 con 0x7f597815b390 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597da00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f59780cf150 0x7f597806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f5977e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f59780aee30 0x7f59780cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38962/0 (socket says 172.21.15.164:38962) 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f5977e00640 1 -- 172.21.15.164:0/3867788492 learned_addr learned my addr 172.21.15.164:0/3867788492 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.662+0000 7f597d000640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f597806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597da00640 1 -- 172.21.15.164:0/3867788492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 msgr2=0x7f597806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597da00640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f597806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597da00640 1 -- 172.21.15.164:0/3867788492 >> v1:172.21.15.164:6789/0 conn(0x7f59780aee30 legacy=0x7f59780cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597da00640 1 -- 172.21.15.164:0/3867788492 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597815ed80 con 0x7f59780cf150 2024-05-08T04:39:22.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f5977400640 1 -- 172.21.15.164:0/3867788492 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097839975 0 0) 0x7f597806e0e0 con 0x7f59780aee30 2024-05-08T04:39:22.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597da00640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780cf150 0x7f597806c3d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f597815ea70 tx=0x7f597001eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd18c57856399415 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f5977400640 1 -- 172.21.15.164:0/3867788492 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700b4070 con 0x7f59780cf150 2024-05-08T04:39:22.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f5977400640 1 -- 172.21.15.164:0/3867788492 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59700a7ab0 con 0x7f59780cf150 2024-05-08T04:39:22.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f5977400640 1 -- 172.21.15.164:0/3867788492 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700af440 con 0x7f59780cf150 2024-05-08T04:39:22.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597fa00640 1 -- 172.21.15.164:0/3867788492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780cf150 msgr2=0x7f597806c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.666+0000 7f597fa00640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780cf150 0x7f597806c3d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f597815ea70 tx=0x7f597001eb70 comp rx=0 tx=0).stop 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- 172.21.15.164:0/3867788492 shutdown_connections 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780cf150 0x7f597806c3d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 --2- 172.21.15.164:0/3867788492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f597806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- 172.21.15.164:0/3867788492 >> 172.21.15.164:0/3867788492 conn(0x7f597806ac90 msgr2=0x7f597806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- 172.21.15.164:0/3867788492 shutdown_connections 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- 172.21.15.164:0/3867788492 wait complete. 2024-05-08T04:39:22.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 Processor -- start 2024-05-08T04:39:22.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- start start 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 0x7f59780ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f5978153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597815b770 con 0x7f59780aee30 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597806ddf0 con 0x7f59780cf150 2024-05-08T04:39:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597806c910 con 0x7f597815b390 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51994/0 (socket says 172.21.15.164:51994) 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 -- 172.21.15.164:0/887094861 learned_addr learned my addr 172.21.15.164:0/887094861 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f5977e00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 0x7f59780ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597da00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f5978153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 -- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 msgr2=0x7f5978153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f5978153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 -- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 msgr2=0x7f59780ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 0x7f59780ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 -- 172.21.15.164:0/887094861 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780acba0 con 0x7f59780aee30 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f5977e00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 0x7f59780ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597d000640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5968009db0 tx=0x7f596800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968018020 con 0x7f59780aee30 2024-05-08T04:39:22.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968014480 con 0x7f59780aee30 2024-05-08T04:39:22.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780ace80 con 0x7f59780aee30 2024-05-08T04:39:22.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.670+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968014850 con 0x7f59780aee30 2024-05-08T04:39:22.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59780ad440 con 0x7f59780aee30 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978069d60 con 0x7f59780aee30 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f596801b020 con 0x7f59780aee30 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f5974c00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 0x7f5958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5968085bd0 con 0x7f59780aee30 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f5977e00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 0x7f5958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:22.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.674+0000 7f5977e00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 0x7f5958069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5978165150 tx=0x7f5964007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:22.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.678+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5968022070 con 0x7f59780aee30 2024-05-08T04:39:22.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.834+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5978071d20 con 0x7f59780aee30 2024-05-08T04:39:22.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.834+0000 7f5974c00640 1 -- 172.21.15.164:0/887094861 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f5968010040 con 0x7f59780aee30 2024-05-08T04:39:22.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:22.839 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:21.525975+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 msgr2=0x7f5958069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 0x7f5958069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5978165150 tx=0x7f5964007cb0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 msgr2=0x7f5978147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5968009db0 tx=0x7f596800c490 comp rx=0 tx=0).stop 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 shutdown_connections 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5958066f60 0x7f5958069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f597815b390 0x7f5978153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59780cf150 0x7f59780ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 --2- 172.21.15.164:0/887094861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59780aee30 0x7f5978147430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:22.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 >> 172.21.15.164:0/887094861 conn(0x7f597806ac90 msgr2=0x7f597815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:22.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 shutdown_connections 2024-05-08T04:39:22.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.838+0000 7f597fa00640 1 -- 172.21.15.164:0/887094861 wait complete. 2024-05-08T04:39:22.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:39:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.994+0000 7f1d5f200640 1 Processor -- start 2024-05-08T04:39:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.994+0000 7f1d5f200640 1 -- start start 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d600b1960 0x7f1d60071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d6015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6015c900 con 0x7f1d600b1d40 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6015cec0 con 0x7f1d600b1960 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6015cbe0 con 0x7f1d600cec40 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5e800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1d600b1d40 0x7f1d600ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38964/0 (socket says 172.21.15.164:38964) 2024-05-08T04:39:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5e800640 1 -- 172.21.15.164:0/2445951017 learned_addr learned my addr 172.21.15.164:0/2445951017 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919825436 0 0) 0x7f1d6015c900 con 0x7f1d600b1d40 2024-05-08T04:39:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5de00640 1 --2- 172.21.15.164:0/2445951017 >> v2:172.21.15.172:3300/0 conn(0x7f1d600b1960 0x7f1d60071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d6015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d44003530 con 0x7f1d600b1d40 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854446336 0 0) 0x7f1d44003530 con 0x7f1d600b1d40 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 msgr2=0x7f1d6015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d6015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 >> v2:172.21.15.172:3300/0 conn(0x7f1d600b1960 msgr2=0x7f1d60071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 --2- 172.21.15.164:0/2445951017 >> v2:172.21.15.172:3300/0 conn(0x7f1d600b1960 0x7f1d60071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600712a0 con 0x7f1d600b1d40 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1d58001680 con 0x7f1d600b1d40 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 >> v1:172.21.15.164:6789/0 conn(0x7f1d600b1d40 legacy=0x7f1d600ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d440039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d44004d50 0x7f1d440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6015cec0 con 0x7f1d44003f40 2024-05-08T04:39:23.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6015cbe0 con 0x7f1d44004d50 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4400bab0 con 0x7f1d600cec40 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5e800640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d44004d50 0x7f1d440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d5de00640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d440039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 -- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 msgr2=0x7f1d440039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d600cec40 0x7f1d440039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 -- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d44004d50 msgr2=0x7f1d440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d44004d50 0x7f1d440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:22.998+0000 7f1d55400640 1 -- 172.21.15.164:0/2445951017 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44003530 con 0x7f1d44003f40 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d55400640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d44004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009ab0 tx=0x7f1d4801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1ac4e71c81ce1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480b2070 con 0x7f1d44003f40 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d480a5550 con 0x7f1d44003f40 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5d400640 1 -- 172.21.15.164:0/2445951017 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480096b0 con 0x7f1d44003f40 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 -- 172.21.15.164:0/2445951017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 msgr2=0x7f1d44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 --2- 172.21.15.164:0/2445951017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d44004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009ab0 tx=0x7f1d4801ca20 comp rx=0 tx=0).stop 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5e800640 1 -- 172.21.15.164:0/2445951017 reap_dead start 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 -- 172.21.15.164:0/2445951017 shutdown_connections 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 -- 172.21.15.164:0/2445951017 >> 172.21.15.164:0/2445951017 conn(0x7f1d6006ac90 msgr2=0x7f1d6006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 -- 172.21.15.164:0/2445951017 shutdown_connections 2024-05-08T04:39:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.002+0000 7f1d5f200640 1 -- 172.21.15.164:0/2445951017 wait complete. 2024-05-08T04:39:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 Processor -- start 2024-05-08T04:39:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- start start 2024-05-08T04:39:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 0x7f1d60160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d60147f30 0x7f1d60156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b37b0 con 0x7f1d44003f40 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b3630 con 0x7f1d601522a0 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b3930 con 0x7f1d60147f30 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52010/0 (socket says 172.21.15.164:52010) 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 -- 172.21.15.164:0/1637120314 learned_addr learned my addr 172.21.15.164:0/1637120314 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 -- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d60147f30 msgr2=0x7f1d60156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d60147f30 0x7f1d60156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 -- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 msgr2=0x7f1d60160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d55400640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 0x7f1d60160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 0x7f1d60160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 -- 172.21.15.164:0/1637120314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d601567c0 con 0x7f1d44003f40 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d55400640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 0x7f1d60160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5de00640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d50011e60 tx=0x7f1d5000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50015630 con 0x7f1d44003f40 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d500157d0 con 0x7f1d44003f40 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50015ba0 con 0x7f1d44003f40 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d60160bd0 con 0x7f1d44003f40 2024-05-08T04:39:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601534e0 con 0x7f1d44003f40 2024-05-08T04:39:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.006+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6004ef30 con 0x7f1d44003f40 2024-05-08T04:39:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.010+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d5001d400 con 0x7f1d44003f40 2024-05-08T04:39:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.010+0000 7f1d56a00640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 0x7f1d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.010+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d50086330 con 0x7f1d44003f40 2024-05-08T04:39:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.010+0000 7f1d55400640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 0x7f1d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:23.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.010+0000 7f1d55400640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 0x7f1d3c069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d480af000 tx=0x7f1d480046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.014+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d50019030 con 0x7f1d44003f40 2024-05-08T04:39:23.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:23.202+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d60156220 con 0x7f1d44003f40 2024-05-08T04:39:24.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.166+0000 7f1d56a00640 1 -- 172.21.15.164:0/1637120314 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1d50010040 con 0x7f1d44003f40 2024-05-08T04:39:24.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 msgr2=0x7f1d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 0x7f1d3c069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d480af000 tx=0x7f1d480046d0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 msgr2=0x7f1d60159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d50011e60 tx=0x7f1d5000f580 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 shutdown_connections 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d3c066f60 0x7f1d3c069420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d60147f30 0x7f1d60156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d601522a0 0x7f1d60160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 --2- 172.21.15.164:0/1637120314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d44003f40 0x7f1d60159030 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 >> 172.21.15.164:0/1637120314 conn(0x7f1d6006ac90 msgr2=0x7f1d6015fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 shutdown_connections 2024-05-08T04:39:24.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.170+0000 7f1d5f200640 1 -- 172.21.15.164:0/1637120314 wait complete. 2024-05-08T04:39:24.196 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:39:24.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:39:24.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.334+0000 7fa1e5800640 1 Processor -- start 2024-05-08T04:39:24.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.334+0000 7fa1e5800640 1 -- start start 2024-05-08T04:39:24.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1e5800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1e015ab10 0x7fa1e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1e5800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e015e870 con 0x7fa1e00cee90 2024-05-08T04:39:24.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1e5800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e015eb50 con 0x7fa1e00ae6e0 2024-05-08T04:39:24.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1e5800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e006bb80 con 0x7fa1e015ab10 2024-05-08T04:39:24.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa1e00cee90 0x7fa1e00ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38968/0 (socket says 172.21.15.164:38968) 2024-05-08T04:39:24.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 -- 172.21.15.164:0/1098800044 learned_addr learned my addr 172.21.15.164:0/1098800044 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1dea00640 1 --2- 172.21.15.164:0/1098800044 >> v2:172.21.15.172:3300/0 conn(0x7fa1e015ab10 0x7fa1e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307008063 0 0) 0x7fa1e015e870 con 0x7fa1e00cee90 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c4003530 con 0x7fa1e00cee90 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756346363 0 0) 0x7fa1e015eb50 con 0x7fa1e00ae6e0 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e015e870 con 0x7fa1e00ae6e0 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1708995834 0 0) 0x7fa1c4003530 con 0x7fa1e00cee90 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 >> v2:172.21.15.172:3300/0 conn(0x7fa1e015ab10 msgr2=0x7fa1e006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 --2- 172.21.15.164:0/1098800044 >> v2:172.21.15.172:3300/0 conn(0x7fa1e015ab10 0x7fa1e006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 >> v1:172.21.15.172:6789/0 conn(0x7fa1e00ae6e0 legacy=0x7fa1e00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e015ee30 con 0x7fa1e00cee90 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa1d8001a20 con 0x7fa1e00cee90 2024-05-08T04:39:24.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 >> v1:172.21.15.164:6789/0 conn(0x7fa1e00cee90 legacy=0x7fa1e00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1c4003b50 0x7fa1c4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c4004810 0x7fa1c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e006bb80 con 0x7fa1c40051d0 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c400c100 con 0x7fa1c4004810 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c400e510 con 0x7fa1c4003b50 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1dea00640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1c4003b50 0x7fa1c4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c4004810 0x7fa1c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 -- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1c4003b50 msgr2=0x7fa1c4003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1c4003b50 0x7fa1c4003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 -- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c4004810 msgr2=0x7fa1c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c4004810 0x7fa1c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1df400640 1 -- 172.21.15.164:0/1098800044 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4003530 con 0x7fa1c40051d0 2024-05-08T04:39:24.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.338+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c4004810 0x7fa1c4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1df400640 1 -- 172.21.15.164:0/1098800044 reap_dead start 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1df400640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b9d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cc1c0 tx=0x7fa1d801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4bb468b534410e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800a460 con 0x7fa1c40051d0 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d80ac620 con 0x7fa1c40051d0 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1de000640 1 -- 172.21.15.164:0/1098800044 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80aca10 con 0x7fa1c40051d0 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 -- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 msgr2=0x7fa1c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b9d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00cc1c0 tx=0x7fa1d801d2a0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 -- 172.21.15.164:0/1098800044 shutdown_connections 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1098800044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b9d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 -- 172.21.15.164:0/1098800044 >> 172.21.15.164:0/1098800044 conn(0x7fa1e00cf770 msgr2=0x7fa1e00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 -- 172.21.15.164:0/1098800044 shutdown_connections 2024-05-08T04:39:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.342+0000 7fa1e5800640 1 -- 172.21.15.164:0/1098800044 wait complete. 2024-05-08T04:39:24.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 Processor -- start 2024-05-08T04:39:24.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- start start 2024-05-08T04:39:24.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 0x7fa1e00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 0x7fa1e0158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e015aef0 con 0x7fa1c40051d0 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e015f3f0 con 0x7fa1e01f8320 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e006cce0 con 0x7fa1e00b4f70 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52038/0 (socket says 172.21.15.164:52038) 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 -- 172.21.15.164:0/1494782150 learned_addr learned my addr 172.21.15.164:0/1494782150 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 0x7fa1e00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1df400640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 0x7fa1e0158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 -- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 msgr2=0x7fa1e0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 0x7fa1e0158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 -- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 msgr2=0x7fa1e00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 0x7fa1e00b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 -- 172.21.15.164:0/1494782150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0148490 con 0x7fa1c40051d0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 0x7fa1e00b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1df400640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 0x7fa1e0158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1dea00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0011960 tx=0x7fa1d0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000faf0 con 0x7fa1c40051d0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000fc90 con 0x7fa1c40051d0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e00b5450 con 0x7fa1c40051d0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0014440 con 0x7fa1c40051d0 2024-05-08T04:39:24.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01584c0 con 0x7fa1c40051d0 2024-05-08T04:39:24.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.346+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0072d30 con 0x7fa1c40051d0 2024-05-08T04:39:24.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.350+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1d0012040 con 0x7fa1c40051d0 2024-05-08T04:39:24.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.350+0000 7fa1d6a00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 0x7fa1bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:24.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.350+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d00856b0 con 0x7fa1c40051d0 2024-05-08T04:39:24.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.350+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 0x7fa1bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:24.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.350+0000 7fa1d7e00640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 0x7fa1bc069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1e014f510 tx=0x7fa1c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:24.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.354+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1d0019030 con 0x7fa1c40051d0 2024-05-08T04:39:24.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:24.546+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1e004f010 con 0x7fa1c40051d0 2024-05-08T04:39:25.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1d6a00640 1 -- 172.21.15.164:0/1494782150 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1d0056060 con 0x7fa1c40051d0 2024-05-08T04:39:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 msgr2=0x7fa1bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 0x7fa1bc069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1e014f510 tx=0x7fa1c8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 msgr2=0x7fa1e0147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0011960 tx=0x7fa1d0011e20 comp rx=0 tx=0).stop 2024-05-08T04:39:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 shutdown_connections 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa1bc066f60 0x7fa1bc069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e00b4f70 0x7fa1e0158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01f8320 0x7fa1e00b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 --2- 172.21.15.164:0/1494782150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1c40051d0 0x7fa1e0147f50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 >> 172.21.15.164:0/1494782150 conn(0x7fa1e00cf770 msgr2=0x7fa1e00c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 shutdown_connections 2024-05-08T04:39:25.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.174+0000 7fa1e5800640 1 -- 172.21.15.164:0/1494782150 wait complete. 2024-05-08T04:39:25.201 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:39:25.201 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:25.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 Processor -- start 2024-05-08T04:39:25.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 -- start start 2024-05-08T04:39:25.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f66600ccfe0 0x7f66600d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66600b42a0 con 0x7f66600b4d50 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600cd4e0 con 0x7f66600ccfe0 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6668200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600cd7c0 con 0x7f666015bc50 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6664e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f66600ccfe0 0x7f66600d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6666200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f66600b4d50 0x7f66600ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38980/0 (socket says 172.21.15.164:38980) 2024-05-08T04:39:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6666200640 1 -- 172.21.15.164:0/3070447040 learned_addr learned my addr 172.21.15.164:0/3070447040 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6665800640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6664e00640 1 -- 172.21.15.164:0/3070447040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 msgr2=0x7f66600b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6664e00640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6664e00640 1 -- 172.21.15.164:0/3070447040 >> v1:172.21.15.164:6789/0 conn(0x7f66600b4d50 legacy=0x7f66600ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f6664e00640 1 -- 172.21.15.164:0/3070447040 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600cf590 con 0x7f66600ccfe0 2024-05-08T04:39:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.338+0000 7f665fe00640 1 -- 172.21.15.164:0/3070447040 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060157261 0 0) 0x7f66600b42a0 con 0x7f66600b4d50 2024-05-08T04:39:25.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6664e00640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f66600d0070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097c0 tx=0x7f664c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab444b13742b6b01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f665fe00640 1 -- 172.21.15.164:0/3070447040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0b0070 con 0x7f66600ccfe0 2024-05-08T04:39:25.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f665fe00640 1 -- 172.21.15.164:0/3070447040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c0ab600 con 0x7f66600ccfe0 2024-05-08T04:39:25.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f665fe00640 1 -- 172.21.15.164:0/3070447040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0ab9d0 con 0x7f66600ccfe0 2024-05-08T04:39:25.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- 172.21.15.164:0/3070447040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 msgr2=0x7f66600d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f66600d0070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097c0 tx=0x7f664c01c3e0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- 172.21.15.164:0/3070447040 shutdown_connections 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f66600d0070 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 --2- 172.21.15.164:0/3070447040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- 172.21.15.164:0/3070447040 >> 172.21.15.164:0/3070447040 conn(0x7f66600b34f0 msgr2=0x7f66600b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- 172.21.15.164:0/3070447040 shutdown_connections 2024-05-08T04:39:25.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- 172.21.15.164:0/3070447040 wait complete. 2024-05-08T04:39:25.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 Processor -- start 2024-05-08T04:39:25.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.342+0000 7f6668200640 1 -- start start 2024-05-08T04:39:25.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 0x7f66600c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660168c60 con 0x7f66600ccfe0 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660168ae0 con 0x7f66600b4d50 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6660168de0 con 0x7f666015bc50 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52066/0 (socket says 172.21.15.164:52066) 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 -- 172.21.15.164:0/1401767421 learned_addr learned my addr 172.21.15.164:0/1401767421 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6665800640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 0x7f66600c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6666200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 -- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 msgr2=0x7f66600b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 -- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 msgr2=0x7f66600c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 0x7f66600c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 -- 172.21.15.164:0/1401767421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660152600 con 0x7f66600ccfe0 2024-05-08T04:39:25.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6665800640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 0x7f66600c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6664e00640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f664c01a010 tx=0x7f664c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0b0040 con 0x7f66600ccfe0 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c0b43f0 con 0x7f66600ccfe0 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0b47c0 con 0x7f66600ccfe0 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66601528e0 con 0x7f66600ccfe0 2024-05-08T04:39:25.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6660152ea0 con 0x7f66600ccfe0 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.346+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660069d60 con 0x7f66600ccfe0 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.350+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f664c0b4960 con 0x7f66600ccfe0 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.350+0000 7f665d600640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 0x7f6640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.350+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f664c11eb90 con 0x7f66600ccfe0 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.350+0000 7f6665800640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 0x7f6640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.350+0000 7f6665800640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 0x7f6640069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66601f7ad0 tx=0x7f6654008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.354+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f664c016090 con 0x7f66600ccfe0 2024-05-08T04:39:25.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.538+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f666015d040 con 0x7f66600ccfe0 2024-05-08T04:39:25.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.538+0000 7f665d600640 1 -- 172.21.15.164:0/1401767421 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f664c0efb90 con 0x7f66600ccfe0 2024-05-08T04:39:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:25.544 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:25.544 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 17 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 msgr2=0x7f6640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 0x7f6640069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66601f7ad0 tx=0x7f6654008040 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 msgr2=0x7f666015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f664c01a010 tx=0x7f664c019180 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 shutdown_connections 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6640066f60 0x7f6640069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f666015bc50 0x7f66600b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66600ccfe0 0x7f666015ec70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 --2- 172.21.15.164:0/1401767421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66600b4d50 0x7f66600c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 >> 172.21.15.164:0/1401767421 conn(0x7f66600b34f0 msgr2=0x7f66600b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:25.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 shutdown_connections 2024-05-08T04:39:25.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.542+0000 7f6668200640 1 -- 172.21.15.164:0/1401767421 wait complete. 2024-05-08T04:39:25.566 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:25.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 Processor -- start 2024-05-08T04:39:25.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 -- start start 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c780b26c0 0x7f1c780cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c78072640 con 0x7f1c780cfbd0 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c780d0060 con 0x7f1c780b26c0 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c7ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c770 con 0x7f1c7815ab60 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c780b26c0 0x7f1c780cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c780b26c0 0x7f1c780cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52082/0 (socket says 172.21.15.164:52082) 2024-05-08T04:39:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 -- 172.21.15.164:0/1554336025 learned_addr learned my addr 172.21.15.164:0/1554336025 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c76800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1c780cfbd0 0x7f1c780cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38990/0 (socket says 172.21.15.164:38990) 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75e00640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 -- 172.21.15.164:0/1554336025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 msgr2=0x7f1c78072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 -- 172.21.15.164:0/1554336025 >> v1:172.21.15.164:6789/0 conn(0x7f1c780cfbd0 legacy=0x7f1c780cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c74a00640 1 -- 172.21.15.164:0/1554336025 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801364669 0 0) 0x7f1c78072640 con 0x7f1c780cfbd0 2024-05-08T04:39:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.698+0000 7f1c75400640 1 -- 172.21.15.164:0/1554336025 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7806ca50 con 0x7f1c780b26c0 2024-05-08T04:39:25.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c75400640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780b26c0 0x7f1c780cf690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ae1393a269f96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c74a00640 1 -- 172.21.15.164:0/1554336025 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b2070 con 0x7f1c780b26c0 2024-05-08T04:39:25.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c74a00640 1 -- 172.21.15.164:0/1554336025 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640a9070 con 0x7f1c780b26c0 2024-05-08T04:39:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c74a00640 1 -- 172.21.15.164:0/1554336025 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640ad880 con 0x7f1c780b26c0 2024-05-08T04:39:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 -- 172.21.15.164:0/1554336025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780b26c0 msgr2=0x7f1c780cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780b26c0 0x7f1c780cf690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6401c560 comp rx=0 tx=0).stop 2024-05-08T04:39:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 -- 172.21.15.164:0/1554336025 shutdown_connections 2024-05-08T04:39:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780b26c0 0x7f1c780cf690 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/1554336025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 -- 172.21.15.164:0/1554336025 >> 172.21.15.164:0/1554336025 conn(0x7f1c7806ac90 msgr2=0x7f1c7806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 -- 172.21.15.164:0/1554336025 shutdown_connections 2024-05-08T04:39:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.702+0000 7f1c7ca00640 1 -- 172.21.15.164:0/1554336025 wait complete. 2024-05-08T04:39:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 Processor -- start 2024-05-08T04:39:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- start start 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c780b26c0 0x7f1c7814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c550 con 0x7f1c780cfbd0 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c3d0 con 0x7f1c780b26c0 2024-05-08T04:39:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780aed30 con 0x7f1c7815ab60 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52084/0 (socket says 172.21.15.164:52084) 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 -- 172.21.15.164:0/126284222 learned_addr learned my addr 172.21.15.164:0/126284222 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c76800640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75e00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c780b26c0 0x7f1c7814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 -- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 msgr2=0x7f1c78159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 -- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c780b26c0 msgr2=0x7f1c7814dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c780b26c0 0x7f1c7814dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 -- 172.21.15.164:0/126284222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78157f70 con 0x7f1c780cfbd0 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c76800640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c75400640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c640b0040 tx=0x7f1c64019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640bb070 con 0x7f1c780cfbd0 2024-05-08T04:39:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640add00 con 0x7f1c780cfbd0 2024-05-08T04:39:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b2040 con 0x7f1c780cfbd0 2024-05-08T04:39:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c781579f0 con 0x7f1c780cfbd0 2024-05-08T04:39:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c78158c80 con 0x7f1c780cfbd0 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.706+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c78139a10 con 0x7f1c780cfbd0 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.710+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c640ab070 con 0x7f1c780cfbd0 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.710+0000 7f1c6e000640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.710+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c6411df00 con 0x7f1c780cfbd0 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.710+0000 7f1c75e00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.710+0000 7f1c75e00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009990 tx=0x7f1c68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:25.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.714+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c78139a10 con 0x7f1c780cfbd0 2024-05-08T04:39:25.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.870+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c78069d60 con 0x7f1c780cfbd0 2024-05-08T04:39:25.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.870+0000 7f1c6e000640 1 -- 172.21.15.164:0/126284222 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f1c64016150 con 0x7f1c780cfbd0 2024-05-08T04:39:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:25.875 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 msgr2=0x7f1c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009990 tx=0x7f1c68008040 comp rx=0 tx=0).stop 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 msgr2=0x7f1c78154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c640b0040 tx=0x7f1c64019300 comp rx=0 tx=0).stop 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 shutdown_connections 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7815ab60 0x7f1c78159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c780cfbd0 0x7f1c78154be0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 --2- 172.21.15.164:0/126284222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c780b26c0 0x7f1c7814dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:25.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 >> 172.21.15.164:0/126284222 conn(0x7f1c7806ac90 msgr2=0x7f1c780cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:25.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 shutdown_connections 2024-05-08T04:39:25.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:25.874+0000 7f1c7ca00640 1 -- 172.21.15.164:0/126284222 wait complete. 2024-05-08T04:39:25.897 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:39:25.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:26.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 Processor -- start 2024-05-08T04:39:26.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 -- start start 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57a00cfda0 0x7f57a00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a0072640 con 0x7f57a00b26c0 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a006c770 con 0x7f57a00cfda0 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f57a5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a006ca50 con 0x7f57a015ab60 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57a00cfda0 0x7f57a00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57a00cfda0 0x7f57a00cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52092/0 (socket says 172.21.15.164:52092) 2024-05-08T04:39:26.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f57a00b26c0 0x7f57a00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39000/0 (socket says 172.21.15.164:39000) 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 -- 172.21.15.164:0/3824589496 learned_addr learned my addr 172.21.15.164:0/3824589496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579ea00640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 -- 172.21.15.164:0/3824589496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 msgr2=0x7f57a0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 -- 172.21.15.164:0/3824589496 >> v1:172.21.15.164:6789/0 conn(0x7f57a00b26c0 legacy=0x7f57a00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579f400640 1 -- 172.21.15.164:0/3824589496 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a006cd30 con 0x7f57a00cfda0 2024-05-08T04:39:26.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579d600640 1 -- 172.21.15.164:0/3824589496 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498747667 0 0) 0x7f57a0072640 con 0x7f57a00b26c0 2024-05-08T04:39:26.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.030+0000 7f579ea00640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:26.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f579f400640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00cd910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f45c4903dc33230 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f579d600640 1 -- 172.21.15.164:0/3824589496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a5700 con 0x7f57a00cfda0 2024-05-08T04:39:26.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f579d600640 1 -- 172.21.15.164:0/3824589496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980a58a0 con 0x7f57a00cfda0 2024-05-08T04:39:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f579d600640 1 -- 172.21.15.164:0/3824589496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a5c70 con 0x7f57a00cfda0 2024-05-08T04:39:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 -- 172.21.15.164:0/3824589496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 msgr2=0x7f57a00cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00cd910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579801ca30 comp rx=0 tx=0).stop 2024-05-08T04:39:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 -- 172.21.15.164:0/3824589496 shutdown_connections 2024-05-08T04:39:26.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00cd910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 --2- 172.21.15.164:0/3824589496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 -- 172.21.15.164:0/3824589496 >> 172.21.15.164:0/3824589496 conn(0x7f57a006ac90 msgr2=0x7f57a006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 -- 172.21.15.164:0/3824589496 shutdown_connections 2024-05-08T04:39:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 -- 172.21.15.164:0/3824589496 wait complete. 2024-05-08T04:39:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.034+0000 7f57a5c00640 1 Processor -- start 2024-05-08T04:39:26.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- start start 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a00b26c0 0x7f57a0142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00aeeb0 con 0x7f57a00cfda0 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00aed30 con 0x7f57a00b26c0 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cc1b0 con 0x7f57a015ab60 2024-05-08T04:39:26.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52096/0 (socket says 172.21.15.164:52096) 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 -- 172.21.15.164:0/2886168989 learned_addr learned my addr 172.21.15.164:0/2886168989 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 -- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 msgr2=0x7f57a00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 -- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a00b26c0 msgr2=0x7f57a0142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a00b26c0 0x7f57a0142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 -- 172.21.15.164:0/2886168989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a015b9f0 con 0x7f57a00cfda0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f579e000640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c520 tx=0x7f578c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c018070 con 0x7f57a00cfda0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0074e0 con 0x7f57a00cfda0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a015bcd0 con 0x7f57a00cfda0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0049f0 con 0x7f57a00cfda0 2024-05-08T04:39:26.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a015c290 con 0x7f57a00cfda0 2024-05-08T04:39:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.038+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0069d60 con 0x7f57a00cfda0 2024-05-08T04:39:26.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.042+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f578c01b070 con 0x7f57a00cfda0 2024-05-08T04:39:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.042+0000 7f5796a00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 0x7f5780069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.042+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f578c0845a0 con 0x7f57a00cfda0 2024-05-08T04:39:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.042+0000 7f579ea00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 0x7f5780069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.042+0000 7f579ea00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 0x7f5780069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5790006f10 tx=0x7f5790008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.046+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f578c055480 con 0x7f57a00cfda0 2024-05-08T04:39:26.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.198+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57a006d9a0 con 0x7f5780066f90 2024-05-08T04:39:26.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.202+0000 7f5796a00640 1 -- 172.21.15.164:0/2886168989 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251880 (secure 0 0 0) 0x7f57a006d9a0 con 0x7f5780066f90 2024-05-08T04:39:26.210 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:26.212 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 msgr2=0x7f5780069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 0x7f5780069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5790006f10 tx=0x7f5790008040 comp rx=0 tx=0).stop 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 msgr2=0x7f57a00b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c520 tx=0x7f578c00c9e0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 shutdown_connections 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5780066f90 0x7f5780069450 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a015ab60 0x7f57a00b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57a00cfda0 0x7f57a00b42a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 --2- 172.21.15.164:0/2886168989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a00b26c0 0x7f57a0142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 >> 172.21.15.164:0/2886168989 conn(0x7f57a006ac90 msgr2=0x7f57a015eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:26.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 shutdown_connections 2024-05-08T04:39:26.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.206+0000 7f57a5c00640 1 -- 172.21.15.164:0/2886168989 wait complete. 2024-05-08T04:39:26.231 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-08T04:39:25.442326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220088,"kb_used_data":3916,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754632,"statfs":{"total":773094113280,"available":772868743168,"internally_reserved":0,"allocated":4009984,"data_stored":2263356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.306933"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178960+0000","last_change":"2024-05-08T04:39:22.177752+0000","last_active":"2024-05-08T04:39:22.178960+0000","last_peered":"2024-05-08T04:39:22.178960+0000","last_clean":"2024-05-08T04:39:22.178960+0000","last_became_active":"2024-05-08T04:39:22.177009+0000","last_became_peered":"2024-05-08T04:39:22.177009+0000","last_unstale":"2024-05-08T04:39:22.178960+0000","last_undegraded":"2024-05-08T04:39:22.178960+0000","last_fullsized":"2024-05-08T04:39:22.178960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.170720+0000","last_change":"2024-05-08T04:39:22.166167+0000","last_active":"2024-05-08T04:39:22.170720+0000","last_peered":"2024-05-08T04:39:22.170720+0000","last_clean":"2024-05-08T04:39:22.170720+0000","last_became_active":"2024-05-08T04:39:22.165794+0000","last_became_peered":"2024-05-08T04:39:22.165794+0000","last_unstale":"2024-05-08T04:39:22.170720+0000","last_undegraded":"2024-05-08T04:39:22.170720+0000","last_fullsized":"2024-05-08T04:39:22.170720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:40.695059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185030+0000","last_change":"2024-05-08T04:39:22.177792+0000","last_active":"2024-05-08T04:39:22.185030+0000","last_peered":"2024-05-08T04:39:22.185030+0000","last_clean":"2024-05-08T04:39:22.185030+0000","last_became_active":"2024-05-08T04:39:22.172421+0000","last_became_peered":"2024-05-08T04:39:22.172421+0000","last_unstale":"2024-05-08T04:39:22.185030+0000","last_undegraded":"2024-05-08T04:39:22.185030+0000","last_fullsized":"2024-05-08T04:39:22.185030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.577250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175548+0000","last_change":"2024-05-08T04:39:22.173327+0000","last_active":"2024-05-08T04:39:22.175548+0000","last_peered":"2024-05-08T04:39:22.175548+0000","last_clean":"2024-05-08T04:39:22.175548+0000","last_became_active":"2024-05-08T04:39:22.172911+0000","last_became_peered":"2024-05-08T04:39:22.172911+0000","last_unstale":"2024-05-08T04:39:22.175548+0000","last_undegraded":"2024-05-08T04:39:22.175548+0000","last_fullsized":"2024-05-08T04:39:22.175548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174166+0000","last_change":"2024-05-08T04:39:22.169013+0000","last_active":"2024-05-08T04:39:22.174166+0000","last_peered":"2024-05-08T04:39:22.174166+0000","last_clean":"2024-05-08T04:39:22.174166+0000","last_became_active":"2024-05-08T04:39:22.167826+0000","last_became_peered":"2024-05-08T04:39:22.167826+0000","last_unstale":"2024-05-08T04:39:22.174166+0000","last_undegraded":"2024-05-08T04:39:22.174166+0000","last_fullsized":"2024-05-08T04:39:22.174166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:21.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.158475+0000","last_change":"2024-05-08T04:39:22.155434+0000","last_active":"2024-05-08T04:39:22.158475+0000","last_peered":"2024-05-08T04:39:22.158475+0000","last_clean":"2024-05-08T04:39:22.158475+0000","last_became_active":"2024-05-08T04:39:22.155185+0000","last_became_peered":"2024-05-08T04:39:22.155185+0000","last_unstale":"2024-05-08T04:39:22.158475+0000","last_undegraded":"2024-05-08T04:39:22.158475+0000","last_fullsized":"2024-05-08T04:39:22.158475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189514+0000","last_change":"2024-05-08T04:39:22.175415+0000","last_active":"2024-05-08T04:39:22.189514+0000","last_peered":"2024-05-08T04:39:22.189514+0000","last_clean":"2024-05-08T04:39:22.189514+0000","last_became_active":"2024-05-08T04:39:22.175134+0000","last_became_peered":"2024-05-08T04:39:22.175134+0000","last_unstale":"2024-05-08T04:39:22.189514+0000","last_undegraded":"2024-05-08T04:39:22.189514+0000","last_fullsized":"2024-05-08T04:39:22.189514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.187392+0000","last_change":"2024-05-08T04:39:22.169169+0000","last_active":"2024-05-08T04:39:22.187392+0000","last_peered":"2024-05-08T04:39:22.187392+0000","last_clean":"2024-05-08T04:39:22.187392+0000","last_became_active":"2024-05-08T04:39:22.168588+0000","last_became_peered":"2024-05-08T04:39:22.168588+0000","last_unstale":"2024-05-08T04:39:22.187392+0000","last_undegraded":"2024-05-08T04:39:22.187392+0000","last_fullsized":"2024-05-08T04:39:22.187392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172746+0000","last_change":"2024-05-08T04:39:22.169595+0000","last_active":"2024-05-08T04:39:22.172746+0000","last_peered":"2024-05-08T04:39:22.172746+0000","last_clean":"2024-05-08T04:39:22.172746+0000","last_became_active":"2024-05-08T04:39:22.169281+0000","last_became_peered":"2024-05-08T04:39:22.169281+0000","last_unstale":"2024-05-08T04:39:22.172746+0000","last_undegraded":"2024-05-08T04:39:22.172746+0000","last_fullsized":"2024-05-08T04:39:22.172746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.173944+0000","last_change":"2024-05-08T04:39:22.168829+0000","last_active":"2024-05-08T04:39:22.173944+0000","last_peered":"2024-05-08T04:39:22.173944+0000","last_clean":"2024-05-08T04:39:22.173944+0000","last_became_active":"2024-05-08T04:39:22.168552+0000","last_became_peered":"2024-05-08T04:39:22.168552+0000","last_unstale":"2024-05-08T04:39:22.173944+0000","last_undegraded":"2024-05-08T04:39:22.173944+0000","last_fullsized":"2024-05-08T04:39:22.173944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185912+0000","last_change":"2024-05-08T04:39:22.169889+0000","last_active":"2024-05-08T04:39:22.185912+0000","last_peered":"2024-05-08T04:39:22.185912+0000","last_clean":"2024-05-08T04:39:22.185912+0000","last_became_active":"2024-05-08T04:39:22.169585+0000","last_became_peered":"2024-05-08T04:39:22.169585+0000","last_unstale":"2024-05-08T04:39:22.185912+0000","last_undegraded":"2024-05-08T04:39:22.185912+0000","last_fullsized":"2024-05-08T04:39:22.185912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.161778+0000","last_change":"2024-05-08T04:39:22.156172+0000","last_active":"2024-05-08T04:39:22.161778+0000","last_peered":"2024-05-08T04:39:22.161778+0000","last_clean":"2024-05-08T04:39:22.161778+0000","last_became_active":"2024-05-08T04:39:22.155764+0000","last_became_peered":"2024-05-08T04:39:22.155764+0000","last_unstale":"2024-05-08T04:39:22.161778+0000","last_undegraded":"2024-05-08T04:39:22.161778+0000","last_fullsized":"2024-05-08T04:39:22.161778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174638+0000","last_change":"2024-05-08T04:39:22.169786+0000","last_active":"2024-05-08T04:39:22.174638+0000","last_peered":"2024-05-08T04:39:22.174638+0000","last_clean":"2024-05-08T04:39:22.174638+0000","last_became_active":"2024-05-08T04:39:22.169257+0000","last_became_peered":"2024-05-08T04:39:22.169257+0000","last_unstale":"2024-05-08T04:39:22.174638+0000","last_undegraded":"2024-05-08T04:39:22.174638+0000","last_fullsized":"2024-05-08T04:39:22.174638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:40.565012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.166959+0000","last_change":"2024-05-08T04:39:22.161334+0000","last_active":"2024-05-08T04:39:22.166959+0000","last_peered":"2024-05-08T04:39:22.166959+0000","last_clean":"2024-05-08T04:39:22.166959+0000","last_became_active":"2024-05-08T04:39:22.161089+0000","last_became_peered":"2024-05-08T04:39:22.161089+0000","last_unstale":"2024-05-08T04:39:22.166959+0000","last_undegraded":"2024-05-08T04:39:22.166959+0000","last_fullsized":"2024-05-08T04:39:22.166959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185086+0000","last_change":"2024-05-08T04:39:22.176111+0000","last_active":"2024-05-08T04:39:22.185086+0000","last_peered":"2024-05-08T04:39:22.185086+0000","last_clean":"2024-05-08T04:39:22.185086+0000","last_became_active":"2024-05-08T04:39:22.175856+0000","last_became_peered":"2024-05-08T04:39:22.175856+0000","last_unstale":"2024-05-08T04:39:22.185086+0000","last_undegraded":"2024-05-08T04:39:22.185086+0000","last_fullsized":"2024-05-08T04:39:22.185086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189842+0000","last_change":"2024-05-08T04:39:22.177462+0000","last_active":"2024-05-08T04:39:22.189842+0000","last_peered":"2024-05-08T04:39:22.189842+0000","last_clean":"2024-05-08T04:39:22.189842+0000","last_became_active":"2024-05-08T04:39:22.177203+0000","last_became_peered":"2024-05-08T04:39:22.177203+0000","last_unstale":"2024-05-08T04:39:22.189842+0000","last_undegraded":"2024-05-08T04:39:22.189842+0000","last_fullsized":"2024-05-08T04:39:22.189842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189401+0000","last_change":"2024-05-08T04:39:22.175253+0000","last_active":"2024-05-08T04:39:22.189401+0000","last_peered":"2024-05-08T04:39:22.189401+0000","last_clean":"2024-05-08T04:39:22.189401+0000","last_became_active":"2024-05-08T04:39:22.173287+0000","last_became_peered":"2024-05-08T04:39:22.173287+0000","last_unstale":"2024-05-08T04:39:22.189401+0000","last_undegraded":"2024-05-08T04:39:22.189401+0000","last_fullsized":"2024-05-08T04:39:22.189401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175924+0000","last_change":"2024-05-08T04:39:22.171023+0000","last_active":"2024-05-08T04:39:22.175924+0000","last_peered":"2024-05-08T04:39:22.175924+0000","last_clean":"2024-05-08T04:39:22.175924+0000","last_became_active":"2024-05-08T04:39:22.170791+0000","last_became_peered":"2024-05-08T04:39:22.170791+0000","last_unstale":"2024-05-08T04:39:22.175924+0000","last_undegraded":"2024-05-08T04:39:22.175924+0000","last_fullsized":"2024-05-08T04:39:22.175924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:51.194717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175652+0000","last_change":"2024-05-08T04:39:22.170343+0000","last_active":"2024-05-08T04:39:22.175652+0000","last_peered":"2024-05-08T04:39:22.175652+0000","last_clean":"2024-05-08T04:39:22.175652+0000","last_became_active":"2024-05-08T04:39:22.169887+0000","last_became_peered":"2024-05-08T04:39:22.169887+0000","last_unstale":"2024-05-08T04:39:22.175652+0000","last_undegraded":"2024-05-08T04:39:22.175652+0000","last_fullsized":"2024-05-08T04:39:22.175652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:33.012058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175110+0000","last_change":"2024-05-08T04:39:22.171762+0000","last_active":"2024-05-08T04:39:22.175110+0000","last_peered":"2024-05-08T04:39:22.175110+0000","last_clean":"2024-05-08T04:39:22.175110+0000","last_became_active":"2024-05-08T04:39:22.171384+0000","last_became_peered":"2024-05-08T04:39:22.171384+0000","last_unstale":"2024-05-08T04:39:22.175110+0000","last_undegraded":"2024-05-08T04:39:22.175110+0000","last_fullsized":"2024-05-08T04:39:22.175110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:03.121248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172651+0000","last_change":"2024-05-08T04:39:22.166276+0000","last_active":"2024-05-08T04:39:22.172651+0000","last_peered":"2024-05-08T04:39:22.172651+0000","last_clean":"2024-05-08T04:39:22.172651+0000","last_became_active":"2024-05-08T04:39:22.165998+0000","last_became_peered":"2024-05-08T04:39:22.165998+0000","last_unstale":"2024-05-08T04:39:22.172651+0000","last_undegraded":"2024-05-08T04:39:22.172651+0000","last_fullsized":"2024-05-08T04:39:22.172651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:45.570574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172959+0000","last_change":"2024-05-08T04:39:22.167910+0000","last_active":"2024-05-08T04:39:22.172959+0000","last_peered":"2024-05-08T04:39:22.172959+0000","last_clean":"2024-05-08T04:39:22.172959+0000","last_became_active":"2024-05-08T04:39:22.167621+0000","last_became_peered":"2024-05-08T04:39:22.167621+0000","last_unstale":"2024-05-08T04:39:22.172959+0000","last_undegraded":"2024-05-08T04:39:22.172959+0000","last_fullsized":"2024-05-08T04:39:22.172959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:30.689339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174445+0000","last_change":"2024-05-08T04:39:22.169160+0000","last_active":"2024-05-08T04:39:22.174445+0000","last_peered":"2024-05-08T04:39:22.174445+0000","last_clean":"2024-05-08T04:39:22.174445+0000","last_became_active":"2024-05-08T04:39:22.168870+0000","last_became_peered":"2024-05-08T04:39:22.168870+0000","last_unstale":"2024-05-08T04:39:22.174445+0000","last_undegraded":"2024-05-08T04:39:22.174445+0000","last_fullsized":"2024-05-08T04:39:22.174445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180619+0000","last_change":"2024-05-08T04:39:22.165534+0000","last_active":"2024-05-08T04:39:22.180619+0000","last_peered":"2024-05-08T04:39:22.180619+0000","last_clean":"2024-05-08T04:39:22.180619+0000","last_became_active":"2024-05-08T04:39:22.165269+0000","last_became_peered":"2024-05-08T04:39:22.165269+0000","last_unstale":"2024-05-08T04:39:22.180619+0000","last_undegraded":"2024-05-08T04:39:22.180619+0000","last_fullsized":"2024-05-08T04:39:22.180619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174073+0000","last_change":"2024-05-08T04:39:22.170978+0000","last_active":"2024-05-08T04:39:22.174073+0000","last_peered":"2024-05-08T04:39:22.174073+0000","last_clean":"2024-05-08T04:39:22.174073+0000","last_became_active":"2024-05-08T04:39:22.170597+0000","last_became_peered":"2024-05-08T04:39:22.170597+0000","last_unstale":"2024-05-08T04:39:22.174073+0000","last_undegraded":"2024-05-08T04:39:22.174073+0000","last_fullsized":"2024-05-08T04:39:22.174073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172513+0000","last_change":"2024-05-08T04:39:22.169063+0000","last_active":"2024-05-08T04:39:22.172513+0000","last_peered":"2024-05-08T04:39:22.172513+0000","last_clean":"2024-05-08T04:39:22.172513+0000","last_became_active":"2024-05-08T04:39:22.168810+0000","last_became_peered":"2024-05-08T04:39:22.168810+0000","last_unstale":"2024-05-08T04:39:22.172513+0000","last_undegraded":"2024-05-08T04:39:22.172513+0000","last_fullsized":"2024-05-08T04:39:22.172513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:13:22.836437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175142+0000","last_change":"2024-05-08T04:39:22.171564+0000","last_active":"2024-05-08T04:39:22.175142+0000","last_peered":"2024-05-08T04:39:22.175142+0000","last_clean":"2024-05-08T04:39:22.175142+0000","last_became_active":"2024-05-08T04:39:22.171305+0000","last_became_peered":"2024-05-08T04:39:22.171305+0000","last_unstale":"2024-05-08T04:39:22.175142+0000","last_undegraded":"2024-05-08T04:39:22.175142+0000","last_fullsized":"2024-05-08T04:39:22.175142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180569+0000","last_change":"2024-05-08T04:39:22.165080+0000","last_active":"2024-05-08T04:39:22.180569+0000","last_peered":"2024-05-08T04:39:22.180569+0000","last_clean":"2024-05-08T04:39:22.180569+0000","last_became_active":"2024-05-08T04:39:22.164840+0000","last_became_peered":"2024-05-08T04:39:22.164840+0000","last_unstale":"2024-05-08T04:39:22.180569+0000","last_undegraded":"2024-05-08T04:39:22.180569+0000","last_fullsized":"2024-05-08T04:39:22.180569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169498+0000","last_change":"2024-05-08T04:39:22.157527+0000","last_active":"2024-05-08T04:39:22.169498+0000","last_peered":"2024-05-08T04:39:22.169498+0000","last_clean":"2024-05-08T04:39:22.169498+0000","last_became_active":"2024-05-08T04:39:22.156747+0000","last_became_peered":"2024-05-08T04:39:22.156747+0000","last_unstale":"2024-05-08T04:39:22.169498+0000","last_undegraded":"2024-05-08T04:39:22.169498+0000","last_fullsized":"2024-05-08T04:39:22.169498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:06.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189569+0000","last_change":"2024-05-08T04:39:22.176471+0000","last_active":"2024-05-08T04:39:22.189569+0000","last_peered":"2024-05-08T04:39:22.189569+0000","last_clean":"2024-05-08T04:39:22.189569+0000","last_became_active":"2024-05-08T04:39:22.173355+0000","last_became_peered":"2024-05-08T04:39:22.173355+0000","last_unstale":"2024-05-08T04:39:22.189569+0000","last_undegraded":"2024-05-08T04:39:22.189569+0000","last_fullsized":"2024-05-08T04:39:22.189569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:22.275858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178399+0000","last_change":"2024-05-08T04:39:22.175105+0000","last_active":"2024-05-08T04:39:22.178399+0000","last_peered":"2024-05-08T04:39:22.178399+0000","last_clean":"2024-05-08T04:39:22.178399+0000","last_became_active":"2024-05-08T04:39:22.173670+0000","last_became_peered":"2024-05-08T04:39:22.173670+0000","last_unstale":"2024-05-08T04:39:22.178399+0000","last_undegraded":"2024-05-08T04:39:22.178399+0000","last_fullsized":"2024-05-08T04:39:22.178399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:30.689339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.188386+0000","last_change":"2024-05-08T04:39:22.173043+0000","last_active":"2024-05-08T04:39:22.188386+0000","last_peered":"2024-05-08T04:39:22.188386+0000","last_clean":"2024-05-08T04:39:22.188386+0000","last_became_active":"2024-05-08T04:39:22.172784+0000","last_became_peered":"2024-05-08T04:39:22.172784+0000","last_unstale":"2024-05-08T04:39:22.188386+0000","last_undegraded":"2024-05-08T04:39:22.188386+0000","last_fullsized":"2024-05-08T04:39:22.188386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178674+0000","last_change":"2024-05-08T04:39:22.173963+0000","last_active":"2024-05-08T04:39:22.178674+0000","last_peered":"2024-05-08T04:39:22.178674+0000","last_clean":"2024-05-08T04:39:22.178674+0000","last_became_active":"2024-05-08T04:39:22.173718+0000","last_became_peered":"2024-05-08T04:39:22.173718+0000","last_unstale":"2024-05-08T04:39:22.178674+0000","last_undegraded":"2024-05-08T04:39:22.178674+0000","last_fullsized":"2024-05-08T04:39:22.178674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191046+0000","last_change":"2024-05-08T04:39:21.164118+0000","last_active":"2024-05-08T04:39:22.191046+0000","last_peered":"2024-05-08T04:39:22.191046+0000","last_clean":"2024-05-08T04:39:22.191046+0000","last_became_active":"2024-05-08T04:39:21.163873+0000","last_became_peered":"2024-05-08T04:39:21.163873+0000","last_unstale":"2024-05-08T04:39:22.191046+0000","last_undegraded":"2024-05-08T04:39:22.191046+0000","last_fullsized":"2024-05-08T04:39:22.191046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:18.448263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178603+0000","last_change":"2024-05-08T04:39:22.176024+0000","last_active":"2024-05-08T04:39:22.178603+0000","last_peered":"2024-05-08T04:39:22.178603+0000","last_clean":"2024-05-08T04:39:22.178603+0000","last_became_active":"2024-05-08T04:39:22.173857+0000","last_became_peered":"2024-05-08T04:39:22.173857+0000","last_unstale":"2024-05-08T04:39:22.178603+0000","last_undegraded":"2024-05-08T04:39:22.178603+0000","last_fullsized":"2024-05-08T04:39:22.178603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189798+0000","last_change":"2024-05-08T04:39:21.157530+0000","last_active":"2024-05-08T04:39:22.189798+0000","last_peered":"2024-05-08T04:39:22.189798+0000","last_clean":"2024-05-08T04:39:22.189798+0000","last_became_active":"2024-05-08T04:39:21.157240+0000","last_became_peered":"2024-05-08T04:39:21.157240+0000","last_unstale":"2024-05-08T04:39:22.189798+0000","last_undegraded":"2024-05-08T04:39:22.189798+0000","last_fullsized":"2024-05-08T04:39:22.189798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:05.307289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171029+0000","last_change":"2024-05-08T04:39:22.158874+0000","last_active":"2024-05-08T04:39:22.171029+0000","last_peered":"2024-05-08T04:39:22.171029+0000","last_clean":"2024-05-08T04:39:22.171029+0000","last_became_active":"2024-05-08T04:39:22.158520+0000","last_became_peered":"2024-05-08T04:39:22.158520+0000","last_unstale":"2024-05-08T04:39:22.171029+0000","last_undegraded":"2024-05-08T04:39:22.171029+0000","last_fullsized":"2024-05-08T04:39:22.171029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:22.556792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190334+0000","last_change":"2024-05-08T04:39:21.166175+0000","last_active":"2024-05-08T04:39:22.190334+0000","last_peered":"2024-05-08T04:39:22.190334+0000","last_clean":"2024-05-08T04:39:22.190334+0000","last_became_active":"2024-05-08T04:39:21.165902+0000","last_became_peered":"2024-05-08T04:39:21.165902+0000","last_unstale":"2024-05-08T04:39:22.190334+0000","last_undegraded":"2024-05-08T04:39:22.190334+0000","last_fullsized":"2024-05-08T04:39:22.190334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:21.270084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171695+0000","last_change":"2024-05-08T04:39:22.166601+0000","last_active":"2024-05-08T04:39:22.171695+0000","last_peered":"2024-05-08T04:39:22.171695+0000","last_clean":"2024-05-08T04:39:22.171695+0000","last_became_active":"2024-05-08T04:39:22.165397+0000","last_became_peered":"2024-05-08T04:39:22.165397+0000","last_unstale":"2024-05-08T04:39:22.171695+0000","last_undegraded":"2024-05-08T04:39:22.171695+0000","last_fullsized":"2024-05-08T04:39:22.171695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.192997+0000","last_change":"2024-05-08T04:39:21.159405+0000","last_active":"2024-05-08T04:39:22.192997+0000","last_peered":"2024-05-08T04:39:22.192997+0000","last_clean":"2024-05-08T04:39:22.192997+0000","last_became_active":"2024-05-08T04:39:21.158942+0000","last_became_peered":"2024-05-08T04:39:21.158942+0000","last_unstale":"2024-05-08T04:39:22.192997+0000","last_undegraded":"2024-05-08T04:39:22.192997+0000","last_fullsized":"2024-05-08T04:39:22.192997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:56.924658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189853+0000","last_change":"2024-05-08T04:39:22.177782+0000","last_active":"2024-05-08T04:39:22.189853+0000","last_peered":"2024-05-08T04:39:22.189853+0000","last_clean":"2024-05-08T04:39:22.189853+0000","last_became_active":"2024-05-08T04:39:22.177615+0000","last_became_peered":"2024-05-08T04:39:22.177615+0000","last_unstale":"2024-05-08T04:39:22.189853+0000","last_undegraded":"2024-05-08T04:39:22.189853+0000","last_fullsized":"2024-05-08T04:39:22.189853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189902+0000","last_change":"2024-05-08T04:39:21.163324+0000","last_active":"2024-05-08T04:39:22.189902+0000","last_peered":"2024-05-08T04:39:22.189902+0000","last_clean":"2024-05-08T04:39:22.189902+0000","last_became_active":"2024-05-08T04:39:21.162979+0000","last_became_peered":"2024-05-08T04:39:21.162979+0000","last_unstale":"2024-05-08T04:39:22.189902+0000","last_undegraded":"2024-05-08T04:39:22.189902+0000","last_fullsized":"2024-05-08T04:39:22.189902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:00.568108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180708+0000","last_change":"2024-05-08T04:39:22.167005+0000","last_active":"2024-05-08T04:39:22.180708+0000","last_peered":"2024-05-08T04:39:22.180708+0000","last_clean":"2024-05-08T04:39:22.180708+0000","last_became_active":"2024-05-08T04:39:22.166685+0000","last_became_peered":"2024-05-08T04:39:22.166685+0000","last_unstale":"2024-05-08T04:39:22.180708+0000","last_undegraded":"2024-05-08T04:39:22.180708+0000","last_fullsized":"2024-05-08T04:39:22.180708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:22.275858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.188691+0000","last_change":"2024-05-08T04:39:21.158406+0000","last_active":"2024-05-08T04:39:22.188691+0000","last_peered":"2024-05-08T04:39:22.188691+0000","last_clean":"2024-05-08T04:39:22.188691+0000","last_became_active":"2024-05-08T04:39:21.158085+0000","last_became_peered":"2024-05-08T04:39:21.158085+0000","last_unstale":"2024-05-08T04:39:22.188691+0000","last_undegraded":"2024-05-08T04:39:22.188691+0000","last_fullsized":"2024-05-08T04:39:22.188691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:48.882556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.179477+0000","last_change":"2024-05-08T04:39:22.173698+0000","last_active":"2024-05-08T04:39:22.179477+0000","last_peered":"2024-05-08T04:39:22.179477+0000","last_clean":"2024-05-08T04:39:22.179477+0000","last_became_active":"2024-05-08T04:39:22.172731+0000","last_became_peered":"2024-05-08T04:39:22.172731+0000","last_unstale":"2024-05-08T04:39:22.179477+0000","last_undegraded":"2024-05-08T04:39:22.179477+0000","last_fullsized":"2024-05-08T04:39:22.179477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148763+0000","last_change":"2024-05-08T04:39:21.155478+0000","last_active":"2024-05-08T04:39:22.148763+0000","last_peered":"2024-05-08T04:39:22.148763+0000","last_clean":"2024-05-08T04:39:22.148763+0000","last_became_active":"2024-05-08T04:39:21.155218+0000","last_became_peered":"2024-05-08T04:39:21.155218+0000","last_unstale":"2024-05-08T04:39:22.148763+0000","last_undegraded":"2024-05-08T04:39:22.148763+0000","last_fullsized":"2024-05-08T04:39:22.148763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:35.594092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.168218+0000","last_change":"2024-05-08T04:39:22.159938+0000","last_active":"2024-05-08T04:39:22.168218+0000","last_peered":"2024-05-08T04:39:22.168218+0000","last_clean":"2024-05-08T04:39:22.168218+0000","last_became_active":"2024-05-08T04:39:22.159628+0000","last_became_peered":"2024-05-08T04:39:22.159628+0000","last_unstale":"2024-05-08T04:39:22.168218+0000","last_undegraded":"2024-05-08T04:39:22.168218+0000","last_fullsized":"2024-05-08T04:39:22.168218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:19.454037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.151953+0000","last_change":"2024-05-08T04:39:21.154784+0000","last_active":"2024-05-08T04:39:22.151953+0000","last_peered":"2024-05-08T04:39:22.151953+0000","last_clean":"2024-05-08T04:39:22.151953+0000","last_became_active":"2024-05-08T04:39:21.154506+0000","last_became_peered":"2024-05-08T04:39:21.154506+0000","last_unstale":"2024-05-08T04:39:22.151953+0000","last_undegraded":"2024-05-08T04:39:22.151953+0000","last_fullsized":"2024-05-08T04:39:22.151953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:39.559238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175627+0000","last_change":"2024-05-08T04:39:22.162798+0000","last_active":"2024-05-08T04:39:22.175627+0000","last_peered":"2024-05-08T04:39:22.175627+0000","last_clean":"2024-05-08T04:39:22.175627+0000","last_became_active":"2024-05-08T04:39:22.162498+0000","last_became_peered":"2024-05-08T04:39:22.162498+0000","last_unstale":"2024-05-08T04:39:22.175627+0000","last_undegraded":"2024-05-08T04:39:22.175627+0000","last_fullsized":"2024-05-08T04:39:22.175627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.149724+0000","last_change":"2024-05-08T04:39:21.161012+0000","last_active":"2024-05-08T04:39:22.149724+0000","last_peered":"2024-05-08T04:39:22.149724+0000","last_clean":"2024-05-08T04:39:22.149724+0000","last_became_active":"2024-05-08T04:39:21.160750+0000","last_became_peered":"2024-05-08T04:39:21.160750+0000","last_unstale":"2024-05-08T04:39:22.149724+0000","last_undegraded":"2024-05-08T04:39:22.149724+0000","last_fullsized":"2024-05-08T04:39:22.149724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:02.086524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175761+0000","last_change":"2024-05-08T04:39:22.170876+0000","last_active":"2024-05-08T04:39:22.175761+0000","last_peered":"2024-05-08T04:39:22.175761+0000","last_clean":"2024-05-08T04:39:22.175761+0000","last_became_active":"2024-05-08T04:39:22.170088+0000","last_became_peered":"2024-05-08T04:39:22.170088+0000","last_unstale":"2024-05-08T04:39:22.175761+0000","last_undegraded":"2024-05-08T04:39:22.175761+0000","last_fullsized":"2024-05-08T04:39:22.175761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.686125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153809+0000","last_change":"2024-05-08T04:39:21.164673+0000","last_active":"2024-05-08T04:39:22.153809+0000","last_peered":"2024-05-08T04:39:22.153809+0000","last_clean":"2024-05-08T04:39:22.153809+0000","last_became_active":"2024-05-08T04:39:21.164381+0000","last_became_peered":"2024-05-08T04:39:21.164381+0000","last_unstale":"2024-05-08T04:39:22.153809+0000","last_undegraded":"2024-05-08T04:39:22.153809+0000","last_fullsized":"2024-05-08T04:39:22.153809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:18.448263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176180+0000","last_change":"2024-05-08T04:39:22.172230+0000","last_active":"2024-05-08T04:39:22.176180+0000","last_peered":"2024-05-08T04:39:22.176180+0000","last_clean":"2024-05-08T04:39:22.176180+0000","last_became_active":"2024-05-08T04:39:22.171935+0000","last_became_peered":"2024-05-08T04:39:22.171935+0000","last_unstale":"2024-05-08T04:39:22.176180+0000","last_undegraded":"2024-05-08T04:39:22.176180+0000","last_fullsized":"2024-05-08T04:39:22.176180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:02.466282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190802+0000","last_change":"2024-05-08T04:39:21.160235+0000","last_active":"2024-05-08T04:39:22.190802+0000","last_peered":"2024-05-08T04:39:22.190802+0000","last_clean":"2024-05-08T04:39:22.190802+0000","last_became_active":"2024-05-08T04:39:21.159634+0000","last_became_peered":"2024-05-08T04:39:21.159634+0000","last_unstale":"2024-05-08T04:39:22.190802+0000","last_undegraded":"2024-05-08T04:39:22.190802+0000","last_fullsized":"2024-05-08T04:39:22.190802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.586008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.184412+0000","last_change":"2024-05-08T04:39:22.175415+0000","last_active":"2024-05-08T04:39:22.184412+0000","last_peered":"2024-05-08T04:39:22.184412+0000","last_clean":"2024-05-08T04:39:22.184412+0000","last_became_active":"2024-05-08T04:39:22.175148+0000","last_became_peered":"2024-05-08T04:39:22.175148+0000","last_unstale":"2024-05-08T04:39:22.184412+0000","last_undegraded":"2024-05-08T04:39:22.184412+0000","last_fullsized":"2024-05-08T04:39:22.184412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.193019+0000","last_change":"2024-05-08T04:39:21.166155+0000","last_active":"2024-05-08T04:39:22.193019+0000","last_peered":"2024-05-08T04:39:22.193019+0000","last_clean":"2024-05-08T04:39:22.193019+0000","last_became_active":"2024-05-08T04:39:21.165821+0000","last_became_peered":"2024-05-08T04:39:21.165821+0000","last_unstale":"2024-05-08T04:39:22.193019+0000","last_undegraded":"2024-05-08T04:39:22.193019+0000","last_fullsized":"2024-05-08T04:39:22.193019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:52.141830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.173938+0000","last_change":"2024-05-08T04:39:22.166970+0000","last_active":"2024-05-08T04:39:22.173938+0000","last_peered":"2024-05-08T04:39:22.173938+0000","last_clean":"2024-05-08T04:39:22.173938+0000","last_became_active":"2024-05-08T04:39:22.166716+0000","last_became_peered":"2024-05-08T04:39:22.166716+0000","last_unstale":"2024-05-08T04:39:22.173938+0000","last_undegraded":"2024-05-08T04:39:22.173938+0000","last_fullsized":"2024-05-08T04:39:22.173938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189643+0000","last_change":"2024-05-08T04:39:21.158038+0000","last_active":"2024-05-08T04:39:22.189643+0000","last_peered":"2024-05-08T04:39:22.189643+0000","last_clean":"2024-05-08T04:39:22.189643+0000","last_became_active":"2024-05-08T04:39:21.157181+0000","last_became_peered":"2024-05-08T04:39:21.157181+0000","last_unstale":"2024-05-08T04:39:22.189643+0000","last_undegraded":"2024-05-08T04:39:22.189643+0000","last_fullsized":"2024-05-08T04:39:22.189643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:35.637245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172030+0000","last_change":"2024-05-08T04:39:22.165838+0000","last_active":"2024-05-08T04:39:22.172030+0000","last_peered":"2024-05-08T04:39:22.172030+0000","last_clean":"2024-05-08T04:39:22.172030+0000","last_became_active":"2024-05-08T04:39:22.165559+0000","last_became_peered":"2024-05-08T04:39:22.165559+0000","last_unstale":"2024-05-08T04:39:22.172030+0000","last_undegraded":"2024-05-08T04:39:22.172030+0000","last_fullsized":"2024-05-08T04:39:22.172030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.577250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160692+0000","last_change":"2024-05-08T04:39:21.155859+0000","last_active":"2024-05-08T04:39:22.160692+0000","last_peered":"2024-05-08T04:39:22.160692+0000","last_clean":"2024-05-08T04:39:22.160692+0000","last_became_active":"2024-05-08T04:39:21.155578+0000","last_became_peered":"2024-05-08T04:39:21.155578+0000","last_unstale":"2024-05-08T04:39:22.160692+0000","last_undegraded":"2024-05-08T04:39:22.160692+0000","last_fullsized":"2024-05-08T04:39:22.160692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:21.551018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189865+0000","last_change":"2024-05-08T04:39:22.178731+0000","last_active":"2024-05-08T04:39:22.189865+0000","last_peered":"2024-05-08T04:39:22.189865+0000","last_clean":"2024-05-08T04:39:22.189865+0000","last_became_active":"2024-05-08T04:39:22.178108+0000","last_became_peered":"2024-05-08T04:39:22.178108+0000","last_unstale":"2024-05-08T04:39:22.189865+0000","last_undegraded":"2024-05-08T04:39:22.189865+0000","last_fullsized":"2024-05-08T04:39:22.189865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148773+0000","last_change":"2024-05-08T04:39:21.164025+0000","last_active":"2024-05-08T04:39:22.148773+0000","last_peered":"2024-05-08T04:39:22.148773+0000","last_clean":"2024-05-08T04:39:22.148773+0000","last_became_active":"2024-05-08T04:39:21.159659+0000","last_became_peered":"2024-05-08T04:39:21.159659+0000","last_unstale":"2024-05-08T04:39:22.148773+0000","last_undegraded":"2024-05-08T04:39:22.148773+0000","last_fullsized":"2024-05-08T04:39:22.148773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:16.100855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175716+0000","last_change":"2024-05-08T04:39:22.172851+0000","last_active":"2024-05-08T04:39:22.175716+0000","last_peered":"2024-05-08T04:39:22.175716+0000","last_clean":"2024-05-08T04:39:22.175716+0000","last_became_active":"2024-05-08T04:39:22.172535+0000","last_became_peered":"2024-05-08T04:39:22.172535+0000","last_unstale":"2024-05-08T04:39:22.175716+0000","last_undegraded":"2024-05-08T04:39:22.175716+0000","last_fullsized":"2024-05-08T04:39:22.175716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:22.556792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191053+0000","last_change":"2024-05-08T04:39:21.162955+0000","last_active":"2024-05-08T04:39:22.191053+0000","last_peered":"2024-05-08T04:39:22.191053+0000","last_clean":"2024-05-08T04:39:22.191053+0000","last_became_active":"2024-05-08T04:39:21.162495+0000","last_became_peered":"2024-05-08T04:39:21.162495+0000","last_unstale":"2024-05-08T04:39:22.191053+0000","last_undegraded":"2024-05-08T04:39:22.191053+0000","last_fullsized":"2024-05-08T04:39:22.191053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:56.102930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169384+0000","last_change":"2024-05-08T04:39:22.157147+0000","last_active":"2024-05-08T04:39:22.169384+0000","last_peered":"2024-05-08T04:39:22.169384+0000","last_clean":"2024-05-08T04:39:22.169384+0000","last_became_active":"2024-05-08T04:39:22.156863+0000","last_became_peered":"2024-05-08T04:39:22.156863+0000","last_unstale":"2024-05-08T04:39:22.169384+0000","last_undegraded":"2024-05-08T04:39:22.169384+0000","last_fullsized":"2024-05-08T04:39:22.169384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:09.449111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190813+0000","last_change":"2024-05-08T04:39:21.158688+0000","last_active":"2024-05-08T04:39:22.190813+0000","last_peered":"2024-05-08T04:39:22.190813+0000","last_clean":"2024-05-08T04:39:22.190813+0000","last_became_active":"2024-05-08T04:39:21.158322+0000","last_became_peered":"2024-05-08T04:39:21.158322+0000","last_unstale":"2024-05-08T04:39:22.190813+0000","last_undegraded":"2024-05-08T04:39:22.190813+0000","last_fullsized":"2024-05-08T04:39:22.190813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:23.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.164770+0000","last_change":"2024-05-08T04:39:22.158266+0000","last_active":"2024-05-08T04:39:22.164770+0000","last_peered":"2024-05-08T04:39:22.164770+0000","last_clean":"2024-05-08T04:39:22.164770+0000","last_became_active":"2024-05-08T04:39:22.157173+0000","last_became_peered":"2024-05-08T04:39:22.157173+0000","last_unstale":"2024-05-08T04:39:22.164770+0000","last_undegraded":"2024-05-08T04:39:22.164770+0000","last_fullsized":"2024-05-08T04:39:22.164770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189349+0000","last_change":"2024-05-08T04:39:21.157934+0000","last_active":"2024-05-08T04:39:22.189349+0000","last_peered":"2024-05-08T04:39:22.189349+0000","last_clean":"2024-05-08T04:39:22.189349+0000","last_became_active":"2024-05-08T04:39:21.157159+0000","last_became_peered":"2024-05-08T04:39:21.157159+0000","last_unstale":"2024-05-08T04:39:22.189349+0000","last_undegraded":"2024-05-08T04:39:22.189349+0000","last_fullsized":"2024-05-08T04:39:22.189349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:18.845857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171711+0000","last_change":"2024-05-08T04:39:22.165299+0000","last_active":"2024-05-08T04:39:22.171711+0000","last_peered":"2024-05-08T04:39:22.171711+0000","last_clean":"2024-05-08T04:39:22.171711+0000","last_became_active":"2024-05-08T04:39:22.165031+0000","last_became_peered":"2024-05-08T04:39:22.165031+0000","last_unstale":"2024-05-08T04:39:22.171711+0000","last_undegraded":"2024-05-08T04:39:22.171711+0000","last_fullsized":"2024-05-08T04:39:22.171711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189406+0000","last_change":"2024-05-08T04:39:21.154172+0000","last_active":"2024-05-08T04:39:22.189406+0000","last_peered":"2024-05-08T04:39:22.189406+0000","last_clean":"2024-05-08T04:39:22.189406+0000","last_became_active":"2024-05-08T04:39:21.153881+0000","last_became_peered":"2024-05-08T04:39:21.153881+0000","last_unstale":"2024-05-08T04:39:22.189406+0000","last_undegraded":"2024-05-08T04:39:22.189406+0000","last_fullsized":"2024-05-08T04:39:22.189406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:41:07.504382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174879+0000","last_change":"2024-05-08T04:39:22.169574+0000","last_active":"2024-05-08T04:39:22.174879+0000","last_peered":"2024-05-08T04:39:22.174879+0000","last_clean":"2024-05-08T04:39:22.174879+0000","last_became_active":"2024-05-08T04:39:22.168180+0000","last_became_peered":"2024-05-08T04:39:22.168180+0000","last_unstale":"2024-05-08T04:39:22.174879+0000","last_undegraded":"2024-05-08T04:39:22.174879+0000","last_fullsized":"2024-05-08T04:39:22.174879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:51.268620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153005+0000","last_change":"2024-05-08T04:39:21.163181+0000","last_active":"2024-05-08T04:39:22.153005+0000","last_peered":"2024-05-08T04:39:22.153005+0000","last_clean":"2024-05-08T04:39:22.153005+0000","last_became_active":"2024-05-08T04:39:21.162906+0000","last_became_peered":"2024-05-08T04:39:21.162906+0000","last_unstale":"2024-05-08T04:39:22.153005+0000","last_undegraded":"2024-05-08T04:39:22.153005+0000","last_fullsized":"2024-05-08T04:39:22.153005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:14:49.578673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176231+0000","last_change":"2024-05-08T04:39:22.172857+0000","last_active":"2024-05-08T04:39:22.176231+0000","last_peered":"2024-05-08T04:39:22.176231+0000","last_clean":"2024-05-08T04:39:22.176231+0000","last_became_active":"2024-05-08T04:39:22.172619+0000","last_became_peered":"2024-05-08T04:39:22.172619+0000","last_unstale":"2024-05-08T04:39:22.176231+0000","last_undegraded":"2024-05-08T04:39:22.176231+0000","last_fullsized":"2024-05-08T04:39:22.176231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:49.488484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189519+0000","last_change":"2024-05-08T04:39:21.155728+0000","last_active":"2024-05-08T04:39:22.189519+0000","last_peered":"2024-05-08T04:39:22.189519+0000","last_clean":"2024-05-08T04:39:22.189519+0000","last_became_active":"2024-05-08T04:39:21.155461+0000","last_became_peered":"2024-05-08T04:39:21.155461+0000","last_unstale":"2024-05-08T04:39:22.189519+0000","last_undegraded":"2024-05-08T04:39:22.189519+0000","last_fullsized":"2024-05-08T04:39:22.189519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:21.551018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169495+0000","last_change":"2024-05-08T04:39:22.163130+0000","last_active":"2024-05-08T04:39:22.169495+0000","last_peered":"2024-05-08T04:39:22.169495+0000","last_clean":"2024-05-08T04:39:22.169495+0000","last_became_active":"2024-05-08T04:39:22.162864+0000","last_became_peered":"2024-05-08T04:39:22.162864+0000","last_unstale":"2024-05-08T04:39:22.169495+0000","last_undegraded":"2024-05-08T04:39:22.169495+0000","last_fullsized":"2024-05-08T04:39:22.169495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:51.552344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.162058+0000","last_change":"2024-05-08T04:39:21.162175+0000","last_active":"2024-05-08T04:39:22.162058+0000","last_peered":"2024-05-08T04:39:22.162058+0000","last_clean":"2024-05-08T04:39:22.162058+0000","last_became_active":"2024-05-08T04:39:21.161881+0000","last_became_peered":"2024-05-08T04:39:21.161881+0000","last_unstale":"2024-05-08T04:39:22.162058+0000","last_undegraded":"2024-05-08T04:39:22.162058+0000","last_fullsized":"2024-05-08T04:39:22.162058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:08.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172139+0000","last_change":"2024-05-08T04:39:22.166231+0000","last_active":"2024-05-08T04:39:22.172139+0000","last_peered":"2024-05-08T04:39:22.172139+0000","last_clean":"2024-05-08T04:39:22.172139+0000","last_became_active":"2024-05-08T04:39:22.165981+0000","last_became_peered":"2024-05-08T04:39:22.165981+0000","last_unstale":"2024-05-08T04:39:22.172139+0000","last_undegraded":"2024-05-08T04:39:22.172139+0000","last_fullsized":"2024-05-08T04:39:22.172139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:59.269328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.152573+0000","last_change":"2024-05-08T04:39:21.157562+0000","last_active":"2024-05-08T04:39:22.152573+0000","last_peered":"2024-05-08T04:39:22.152573+0000","last_clean":"2024-05-08T04:39:22.152573+0000","last_became_active":"2024-05-08T04:39:21.156869+0000","last_became_peered":"2024-05-08T04:39:21.156869+0000","last_unstale":"2024-05-08T04:39:22.152573+0000","last_undegraded":"2024-05-08T04:39:22.152573+0000","last_fullsized":"2024-05-08T04:39:22.152573+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:02.115475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189336+0000","last_change":"2024-05-08T04:39:22.174739+0000","last_active":"2024-05-08T04:39:22.189336+0000","last_peered":"2024-05-08T04:39:22.189336+0000","last_clean":"2024-05-08T04:39:22.189336+0000","last_became_active":"2024-05-08T04:39:22.174472+0000","last_became_peered":"2024-05-08T04:39:22.174472+0000","last_unstale":"2024-05-08T04:39:22.189336+0000","last_undegraded":"2024-05-08T04:39:22.189336+0000","last_fullsized":"2024-05-08T04:39:22.189336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.154982+0000","last_change":"2024-05-08T04:39:21.160591+0000","last_active":"2024-05-08T04:39:22.154982+0000","last_peered":"2024-05-08T04:39:22.154982+0000","last_clean":"2024-05-08T04:39:22.154982+0000","last_became_active":"2024-05-08T04:39:21.160144+0000","last_became_peered":"2024-05-08T04:39:21.160144+0000","last_unstale":"2024-05-08T04:39:22.154982+0000","last_undegraded":"2024-05-08T04:39:22.154982+0000","last_fullsized":"2024-05-08T04:39:22.154982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:34.284988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176287+0000","last_change":"2024-05-08T04:39:22.174667+0000","last_active":"2024-05-08T04:39:22.176287+0000","last_peered":"2024-05-08T04:39:22.176287+0000","last_clean":"2024-05-08T04:39:22.176287+0000","last_became_active":"2024-05-08T04:39:22.173577+0000","last_became_peered":"2024-05-08T04:39:22.173577+0000","last_unstale":"2024-05-08T04:39:22.176287+0000","last_undegraded":"2024-05-08T04:39:22.176287+0000","last_fullsized":"2024-05-08T04:39:22.176287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:14:50.584447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191073+0000","last_change":"2024-05-08T04:39:21.159295+0000","last_active":"2024-05-08T04:39:22.191073+0000","last_peered":"2024-05-08T04:39:22.191073+0000","last_clean":"2024-05-08T04:39:22.191073+0000","last_became_active":"2024-05-08T04:39:21.159037+0000","last_became_peered":"2024-05-08T04:39:21.159037+0000","last_unstale":"2024-05-08T04:39:22.191073+0000","last_undegraded":"2024-05-08T04:39:22.191073+0000","last_fullsized":"2024-05-08T04:39:22.191073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:39.739479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180797+0000","last_change":"2024-05-08T04:39:22.173785+0000","last_active":"2024-05-08T04:39:22.180797+0000","last_peered":"2024-05-08T04:39:22.180797+0000","last_clean":"2024-05-08T04:39:22.180797+0000","last_became_active":"2024-05-08T04:39:22.173474+0000","last_became_peered":"2024-05-08T04:39:22.173474+0000","last_unstale":"2024-05-08T04:39:22.180797+0000","last_undegraded":"2024-05-08T04:39:22.180797+0000","last_fullsized":"2024-05-08T04:39:22.180797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153907+0000","last_change":"2024-05-08T04:39:21.161167+0000","last_active":"2024-05-08T04:39:22.153907+0000","last_peered":"2024-05-08T04:39:22.153907+0000","last_clean":"2024-05-08T04:39:22.153907+0000","last_became_active":"2024-05-08T04:39:21.160884+0000","last_became_peered":"2024-05-08T04:39:21.160884+0000","last_unstale":"2024-05-08T04:39:22.153907+0000","last_undegraded":"2024-05-08T04:39:22.153907+0000","last_fullsized":"2024-05-08T04:39:22.153907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:44.564800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.164772+0000","last_change":"2024-05-08T04:39:22.160398+0000","last_active":"2024-05-08T04:39:22.164772+0000","last_peered":"2024-05-08T04:39:22.164772+0000","last_clean":"2024-05-08T04:39:22.164772+0000","last_became_active":"2024-05-08T04:39:22.160110+0000","last_became_peered":"2024-05-08T04:39:22.160110+0000","last_unstale":"2024-05-08T04:39:22.164772+0000","last_undegraded":"2024-05-08T04:39:22.164772+0000","last_fullsized":"2024-05-08T04:39:22.164772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160962+0000","last_change":"2024-05-08T04:39:21.157088+0000","last_active":"2024-05-08T04:39:22.160962+0000","last_peered":"2024-05-08T04:39:22.160962+0000","last_clean":"2024-05-08T04:39:22.160962+0000","last_became_active":"2024-05-08T04:39:21.156840+0000","last_became_peered":"2024-05-08T04:39:21.156840+0000","last_unstale":"2024-05-08T04:39:22.160962+0000","last_undegraded":"2024-05-08T04:39:22.160962+0000","last_fullsized":"2024-05-08T04:39:22.160962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:29.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.177462+0000","last_change":"2024-05-08T04:39:22.173811+0000","last_active":"2024-05-08T04:39:22.177462+0000","last_peered":"2024-05-08T04:39:22.177462+0000","last_clean":"2024-05-08T04:39:22.177462+0000","last_became_active":"2024-05-08T04:39:22.173499+0000","last_became_peered":"2024-05-08T04:39:22.173499+0000","last_unstale":"2024-05-08T04:39:22.177462+0000","last_undegraded":"2024-05-08T04:39:22.177462+0000","last_fullsized":"2024-05-08T04:39:22.177462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:45.570574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153984+0000","last_change":"2024-05-08T04:39:21.148438+0000","last_active":"2024-05-08T04:39:22.153984+0000","last_peered":"2024-05-08T04:39:22.153984+0000","last_clean":"2024-05-08T04:39:22.153984+0000","last_became_active":"2024-05-08T04:39:21.148139+0000","last_became_peered":"2024-05-08T04:39:21.148139+0000","last_unstale":"2024-05-08T04:39:22.153984+0000","last_undegraded":"2024-05-08T04:39:22.153984+0000","last_fullsized":"2024-05-08T04:39:22.153984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:29.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178849+0000","last_change":"2024-05-08T04:39:22.174756+0000","last_active":"2024-05-08T04:39:22.178849+0000","last_peered":"2024-05-08T04:39:22.178849+0000","last_clean":"2024-05-08T04:39:22.178849+0000","last_became_active":"2024-05-08T04:39:22.174483+0000","last_became_peered":"2024-05-08T04:39:22.174483+0000","last_unstale":"2024-05-08T04:39:22.178849+0000","last_undegraded":"2024-05-08T04:39:22.178849+0000","last_fullsized":"2024-05-08T04:39:22.178849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160847+0000","last_change":"2024-05-08T04:39:21.155924+0000","last_active":"2024-05-08T04:39:22.160847+0000","last_peered":"2024-05-08T04:39:22.160847+0000","last_clean":"2024-05-08T04:39:22.160847+0000","last_became_active":"2024-05-08T04:39:21.155683+0000","last_became_peered":"2024-05-08T04:39:21.155683+0000","last_unstale":"2024-05-08T04:39:22.160847+0000","last_undegraded":"2024-05-08T04:39:22.160847+0000","last_fullsized":"2024-05-08T04:39:22.160847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:44.564800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.166892+0000","last_change":"2024-05-08T04:39:22.159438+0000","last_active":"2024-05-08T04:39:22.166892+0000","last_peered":"2024-05-08T04:39:22.166892+0000","last_clean":"2024-05-08T04:39:22.166892+0000","last_became_active":"2024-05-08T04:39:22.159155+0000","last_became_peered":"2024-05-08T04:39:22.159155+0000","last_unstale":"2024-05-08T04:39:22.166892+0000","last_undegraded":"2024-05-08T04:39:22.166892+0000","last_fullsized":"2024-05-08T04:39:22.166892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.152666+0000","last_change":"2024-05-08T04:39:21.156802+0000","last_active":"2024-05-08T04:39:22.152666+0000","last_peered":"2024-05-08T04:39:22.152666+0000","last_clean":"2024-05-08T04:39:22.152666+0000","last_became_active":"2024-05-08T04:39:21.156498+0000","last_became_peered":"2024-05-08T04:39:21.156498+0000","last_unstale":"2024-05-08T04:39:22.152666+0000","last_undegraded":"2024-05-08T04:39:22.152666+0000","last_fullsized":"2024-05-08T04:39:22.152666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:48.482710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.170758+0000","last_change":"2024-05-08T04:39:22.157966+0000","last_active":"2024-05-08T04:39:22.170758+0000","last_peered":"2024-05-08T04:39:22.170758+0000","last_clean":"2024-05-08T04:39:22.170758+0000","last_became_active":"2024-05-08T04:39:22.157699+0000","last_became_peered":"2024-05-08T04:39:22.157699+0000","last_unstale":"2024-05-08T04:39:22.170758+0000","last_undegraded":"2024-05-08T04:39:22.170758+0000","last_fullsized":"2024-05-08T04:39:22.170758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:19.454037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148736+0000","last_change":"2024-05-08T04:39:21.160773+0000","last_active":"2024-05-08T04:39:22.148736+0000","last_peered":"2024-05-08T04:39:22.148736+0000","last_clean":"2024-05-08T04:39:22.148736+0000","last_became_active":"2024-05-08T04:39:21.160535+0000","last_became_peered":"2024-05-08T04:39:21.160535+0000","last_unstale":"2024-05-08T04:39:22.148736+0000","last_undegraded":"2024-05-08T04:39:22.148736+0000","last_fullsized":"2024-05-08T04:39:22.148736+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:34.284988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189077+0000","last_change":"2024-05-08T04:39:22.173739+0000","last_active":"2024-05-08T04:39:22.189077+0000","last_peered":"2024-05-08T04:39:22.189077+0000","last_clean":"2024-05-08T04:39:22.189077+0000","last_became_active":"2024-05-08T04:39:22.171147+0000","last_became_peered":"2024-05-08T04:39:22.171147+0000","last_unstale":"2024-05-08T04:39:22.189077+0000","last_undegraded":"2024-05-08T04:39:22.189077+0000","last_fullsized":"2024-05-08T04:39:22.189077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.162092+0000","last_change":"2024-05-08T04:39:21.159551+0000","last_active":"2024-05-08T04:39:22.162092+0000","last_peered":"2024-05-08T04:39:22.162092+0000","last_clean":"2024-05-08T04:39:22.162092+0000","last_became_active":"2024-05-08T04:39:21.159161+0000","last_became_peered":"2024-05-08T04:39:21.159161+0000","last_unstale":"2024-05-08T04:39:22.162092+0000","last_undegraded":"2024-05-08T04:39:22.162092+0000","last_fullsized":"2024-05-08T04:39:22.162092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:05.307289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.159814+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:22.159814+0000","last_peered":"2024-05-08T04:39:22.159814+0000","last_clean":"2024-05-08T04:39:22.159814+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:22.159814+0000","last_undegraded":"2024-05-08T04:39:22.159814+0000","last_fullsized":"2024-05-08T04:39:22.159814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":140386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:39:26.235 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:26.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 Processor -- start 2024-05-08T04:39:26.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 -- start start 2024-05-08T04:39:26.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f806c7b0 0x7fe5f81620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f81625f0 0x7fe5f80b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f806cc70 con 0x7fe5f815dd70 2024-05-08T04:39:26.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f806cf50 con 0x7fe5f806c7b0 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5ffa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80b1570 con 0x7fe5f81625f0 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f806c7b0 0x7fe5f81620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f806c7b0 0x7fe5f81620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52100/0 (socket says 172.21.15.164:52100) 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 -- 172.21.15.164:0/4141018954 learned_addr learned my addr 172.21.15.164:0/4141018954 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5fda00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f81625f0 0x7fe5f80b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 -- 172.21.15.164:0/4141018954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f81625f0 msgr2=0x7fe5f80b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f81625f0 0x7fe5f80b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 -- 172.21.15.164:0/4141018954 >> v1:172.21.15.164:6789/0 conn(0x7fe5f815dd70 legacy=0x7fe5f81619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.406+0000 7fe5f7e00640 1 -- 172.21.15.164:0/4141018954 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80b1850 con 0x7fe5f806c7b0 2024-05-08T04:39:26.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5f7e00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f806c7b0 0x7fe5f81620b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009870 tx=0x7fe5e401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312cf347b77c9ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5f7400640 1 -- 172.21.15.164:0/4141018954 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40b5070 con 0x7fe5f806c7b0 2024-05-08T04:39:26.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5f7400640 1 -- 172.21.15.164:0/4141018954 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e40a8a50 con 0x7fe5f806c7b0 2024-05-08T04:39:26.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5f7400640 1 -- 172.21.15.164:0/4141018954 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40a8e20 con 0x7fe5f806c7b0 2024-05-08T04:39:26.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4141018954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f806c7b0 msgr2=0x7fe5f81620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f806c7b0 0x7fe5f81620b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009870 tx=0x7fe5e401ed50 comp rx=0 tx=0).stop 2024-05-08T04:39:26.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4141018954 shutdown_connections 2024-05-08T04:39:26.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f81625f0 0x7fe5f80b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4141018954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f806c7b0 0x7fe5f81620b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4141018954 >> 172.21.15.164:0/4141018954 conn(0x7fe5f80b34f0 msgr2=0x7fe5f80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:26.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4141018954 shutdown_connections 2024-05-08T04:39:26.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.410+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4141018954 wait complete. 2024-05-08T04:39:26.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 Processor -- start 2024-05-08T04:39:26.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- start start 2024-05-08T04:39:26.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 0x7fe5f8160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 0x7fe5f80b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815e2d0 con 0x7fe5f81625f0 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815e150 con 0x7fe5f806c7b0 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80b22b0 con 0x7fe5f815dd70 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52110/0 (socket says 172.21.15.164:52110) 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 -- 172.21.15.164:0/4049393127 learned_addr learned my addr 172.21.15.164:0/4049393127 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5f7e00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 0x7fe5f80b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fd000640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 0x7fe5f8160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 -- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 msgr2=0x7fe5f80b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 0x7fe5f80b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 -- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 msgr2=0x7fe5f8160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 0x7fe5f8160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 -- 172.21.15.164:0/4049393127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80c68c0 con 0x7fe5f81625f0 2024-05-08T04:39:26.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5f7e00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 0x7fe5f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:26.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fd000640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 0x7fe5f8160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:26.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5fda00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d560 tx=0x7fe5f000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00125f0 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f0019070 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00129f0 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f80c6ba0 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.414+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f80c7160 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f8069d60 con 0x7fe5f81625f0 2024-05-08T04:39:26.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5f0012b90 con 0x7fe5f81625f0 2024-05-08T04:39:26.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5f4c00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 0x7fe5d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5f00854e0 con 0x7fe5f81625f0 2024-05-08T04:39:26.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5fd000640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 0x7fe5d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.418+0000 7fe5fd000640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 0x7fe5d8069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8001e10 tx=0x7fe5e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.422+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5f0056450 con 0x7fe5f81625f0 2024-05-08T04:39:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.574+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5f80b4010 con 0x7fe5d8066f60 2024-05-08T04:39:26.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5f4c00640 1 -- 172.21.15.164:0/4049393127 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251880 (secure 0 0 0) 0x7fe5f80b4010 con 0x7fe5d8066f60 2024-05-08T04:39:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 msgr2=0x7fe5d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 0x7fe5d8069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8001e10 tx=0x7fe5e8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 msgr2=0x7fe5f80b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d560 tx=0x7fe5f000da20 comp rx=0 tx=0).stop 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 shutdown_connections 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe5d8066f60 0x7fe5d8069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f815dd70 0x7fe5f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f81625f0 0x7fe5f80b5730 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 --2- 172.21.15.164:0/4049393127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f806c7b0 0x7fe5f8160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 >> 172.21.15.164:0/4049393127 conn(0x7fe5f80b34f0 msgr2=0x7fe5f80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 shutdown_connections 2024-05-08T04:39:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.582+0000 7fe5ffa00640 1 -- 172.21.15.164:0/4049393127 wait complete. 2024-05-08T04:39:26.608 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-08T04:39:25.442326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220088,"kb_used_data":3916,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754632,"statfs":{"total":773094113280,"available":772868743168,"internally_reserved":0,"allocated":4009984,"data_stored":2263356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.306933"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178960+0000","last_change":"2024-05-08T04:39:22.177752+0000","last_active":"2024-05-08T04:39:22.178960+0000","last_peered":"2024-05-08T04:39:22.178960+0000","last_clean":"2024-05-08T04:39:22.178960+0000","last_became_active":"2024-05-08T04:39:22.177009+0000","last_became_peered":"2024-05-08T04:39:22.177009+0000","last_unstale":"2024-05-08T04:39:22.178960+0000","last_undegraded":"2024-05-08T04:39:22.178960+0000","last_fullsized":"2024-05-08T04:39:22.178960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.170720+0000","last_change":"2024-05-08T04:39:22.166167+0000","last_active":"2024-05-08T04:39:22.170720+0000","last_peered":"2024-05-08T04:39:22.170720+0000","last_clean":"2024-05-08T04:39:22.170720+0000","last_became_active":"2024-05-08T04:39:22.165794+0000","last_became_peered":"2024-05-08T04:39:22.165794+0000","last_unstale":"2024-05-08T04:39:22.170720+0000","last_undegraded":"2024-05-08T04:39:22.170720+0000","last_fullsized":"2024-05-08T04:39:22.170720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:40.695059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185030+0000","last_change":"2024-05-08T04:39:22.177792+0000","last_active":"2024-05-08T04:39:22.185030+0000","last_peered":"2024-05-08T04:39:22.185030+0000","last_clean":"2024-05-08T04:39:22.185030+0000","last_became_active":"2024-05-08T04:39:22.172421+0000","last_became_peered":"2024-05-08T04:39:22.172421+0000","last_unstale":"2024-05-08T04:39:22.185030+0000","last_undegraded":"2024-05-08T04:39:22.185030+0000","last_fullsized":"2024-05-08T04:39:22.185030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.577250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175548+0000","last_change":"2024-05-08T04:39:22.173327+0000","last_active":"2024-05-08T04:39:22.175548+0000","last_peered":"2024-05-08T04:39:22.175548+0000","last_clean":"2024-05-08T04:39:22.175548+0000","last_became_active":"2024-05-08T04:39:22.172911+0000","last_became_peered":"2024-05-08T04:39:22.172911+0000","last_unstale":"2024-05-08T04:39:22.175548+0000","last_undegraded":"2024-05-08T04:39:22.175548+0000","last_fullsized":"2024-05-08T04:39:22.175548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174166+0000","last_change":"2024-05-08T04:39:22.169013+0000","last_active":"2024-05-08T04:39:22.174166+0000","last_peered":"2024-05-08T04:39:22.174166+0000","last_clean":"2024-05-08T04:39:22.174166+0000","last_became_active":"2024-05-08T04:39:22.167826+0000","last_became_peered":"2024-05-08T04:39:22.167826+0000","last_unstale":"2024-05-08T04:39:22.174166+0000","last_undegraded":"2024-05-08T04:39:22.174166+0000","last_fullsized":"2024-05-08T04:39:22.174166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:21.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.158475+0000","last_change":"2024-05-08T04:39:22.155434+0000","last_active":"2024-05-08T04:39:22.158475+0000","last_peered":"2024-05-08T04:39:22.158475+0000","last_clean":"2024-05-08T04:39:22.158475+0000","last_became_active":"2024-05-08T04:39:22.155185+0000","last_became_peered":"2024-05-08T04:39:22.155185+0000","last_unstale":"2024-05-08T04:39:22.158475+0000","last_undegraded":"2024-05-08T04:39:22.158475+0000","last_fullsized":"2024-05-08T04:39:22.158475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189514+0000","last_change":"2024-05-08T04:39:22.175415+0000","last_active":"2024-05-08T04:39:22.189514+0000","last_peered":"2024-05-08T04:39:22.189514+0000","last_clean":"2024-05-08T04:39:22.189514+0000","last_became_active":"2024-05-08T04:39:22.175134+0000","last_became_peered":"2024-05-08T04:39:22.175134+0000","last_unstale":"2024-05-08T04:39:22.189514+0000","last_undegraded":"2024-05-08T04:39:22.189514+0000","last_fullsized":"2024-05-08T04:39:22.189514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.187392+0000","last_change":"2024-05-08T04:39:22.169169+0000","last_active":"2024-05-08T04:39:22.187392+0000","last_peered":"2024-05-08T04:39:22.187392+0000","last_clean":"2024-05-08T04:39:22.187392+0000","last_became_active":"2024-05-08T04:39:22.168588+0000","last_became_peered":"2024-05-08T04:39:22.168588+0000","last_unstale":"2024-05-08T04:39:22.187392+0000","last_undegraded":"2024-05-08T04:39:22.187392+0000","last_fullsized":"2024-05-08T04:39:22.187392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172746+0000","last_change":"2024-05-08T04:39:22.169595+0000","last_active":"2024-05-08T04:39:22.172746+0000","last_peered":"2024-05-08T04:39:22.172746+0000","last_clean":"2024-05-08T04:39:22.172746+0000","last_became_active":"2024-05-08T04:39:22.169281+0000","last_became_peered":"2024-05-08T04:39:22.169281+0000","last_unstale":"2024-05-08T04:39:22.172746+0000","last_undegraded":"2024-05-08T04:39:22.172746+0000","last_fullsized":"2024-05-08T04:39:22.172746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.173944+0000","last_change":"2024-05-08T04:39:22.168829+0000","last_active":"2024-05-08T04:39:22.173944+0000","last_peered":"2024-05-08T04:39:22.173944+0000","last_clean":"2024-05-08T04:39:22.173944+0000","last_became_active":"2024-05-08T04:39:22.168552+0000","last_became_peered":"2024-05-08T04:39:22.168552+0000","last_unstale":"2024-05-08T04:39:22.173944+0000","last_undegraded":"2024-05-08T04:39:22.173944+0000","last_fullsized":"2024-05-08T04:39:22.173944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185912+0000","last_change":"2024-05-08T04:39:22.169889+0000","last_active":"2024-05-08T04:39:22.185912+0000","last_peered":"2024-05-08T04:39:22.185912+0000","last_clean":"2024-05-08T04:39:22.185912+0000","last_became_active":"2024-05-08T04:39:22.169585+0000","last_became_peered":"2024-05-08T04:39:22.169585+0000","last_unstale":"2024-05-08T04:39:22.185912+0000","last_undegraded":"2024-05-08T04:39:22.185912+0000","last_fullsized":"2024-05-08T04:39:22.185912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.161778+0000","last_change":"2024-05-08T04:39:22.156172+0000","last_active":"2024-05-08T04:39:22.161778+0000","last_peered":"2024-05-08T04:39:22.161778+0000","last_clean":"2024-05-08T04:39:22.161778+0000","last_became_active":"2024-05-08T04:39:22.155764+0000","last_became_peered":"2024-05-08T04:39:22.155764+0000","last_unstale":"2024-05-08T04:39:22.161778+0000","last_undegraded":"2024-05-08T04:39:22.161778+0000","last_fullsized":"2024-05-08T04:39:22.161778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174638+0000","last_change":"2024-05-08T04:39:22.169786+0000","last_active":"2024-05-08T04:39:22.174638+0000","last_peered":"2024-05-08T04:39:22.174638+0000","last_clean":"2024-05-08T04:39:22.174638+0000","last_became_active":"2024-05-08T04:39:22.169257+0000","last_became_peered":"2024-05-08T04:39:22.169257+0000","last_unstale":"2024-05-08T04:39:22.174638+0000","last_undegraded":"2024-05-08T04:39:22.174638+0000","last_fullsized":"2024-05-08T04:39:22.174638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:40.565012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.166959+0000","last_change":"2024-05-08T04:39:22.161334+0000","last_active":"2024-05-08T04:39:22.166959+0000","last_peered":"2024-05-08T04:39:22.166959+0000","last_clean":"2024-05-08T04:39:22.166959+0000","last_became_active":"2024-05-08T04:39:22.161089+0000","last_became_peered":"2024-05-08T04:39:22.161089+0000","last_unstale":"2024-05-08T04:39:22.166959+0000","last_undegraded":"2024-05-08T04:39:22.166959+0000","last_fullsized":"2024-05-08T04:39:22.166959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.185086+0000","last_change":"2024-05-08T04:39:22.176111+0000","last_active":"2024-05-08T04:39:22.185086+0000","last_peered":"2024-05-08T04:39:22.185086+0000","last_clean":"2024-05-08T04:39:22.185086+0000","last_became_active":"2024-05-08T04:39:22.175856+0000","last_became_peered":"2024-05-08T04:39:22.175856+0000","last_unstale":"2024-05-08T04:39:22.185086+0000","last_undegraded":"2024-05-08T04:39:22.185086+0000","last_fullsized":"2024-05-08T04:39:22.185086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189842+0000","last_change":"2024-05-08T04:39:22.177462+0000","last_active":"2024-05-08T04:39:22.189842+0000","last_peered":"2024-05-08T04:39:22.189842+0000","last_clean":"2024-05-08T04:39:22.189842+0000","last_became_active":"2024-05-08T04:39:22.177203+0000","last_became_peered":"2024-05-08T04:39:22.177203+0000","last_unstale":"2024-05-08T04:39:22.189842+0000","last_undegraded":"2024-05-08T04:39:22.189842+0000","last_fullsized":"2024-05-08T04:39:22.189842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189401+0000","last_change":"2024-05-08T04:39:22.175253+0000","last_active":"2024-05-08T04:39:22.189401+0000","last_peered":"2024-05-08T04:39:22.189401+0000","last_clean":"2024-05-08T04:39:22.189401+0000","last_became_active":"2024-05-08T04:39:22.173287+0000","last_became_peered":"2024-05-08T04:39:22.173287+0000","last_unstale":"2024-05-08T04:39:22.189401+0000","last_undegraded":"2024-05-08T04:39:22.189401+0000","last_fullsized":"2024-05-08T04:39:22.189401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175924+0000","last_change":"2024-05-08T04:39:22.171023+0000","last_active":"2024-05-08T04:39:22.175924+0000","last_peered":"2024-05-08T04:39:22.175924+0000","last_clean":"2024-05-08T04:39:22.175924+0000","last_became_active":"2024-05-08T04:39:22.170791+0000","last_became_peered":"2024-05-08T04:39:22.170791+0000","last_unstale":"2024-05-08T04:39:22.175924+0000","last_undegraded":"2024-05-08T04:39:22.175924+0000","last_fullsized":"2024-05-08T04:39:22.175924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:51.194717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175652+0000","last_change":"2024-05-08T04:39:22.170343+0000","last_active":"2024-05-08T04:39:22.175652+0000","last_peered":"2024-05-08T04:39:22.175652+0000","last_clean":"2024-05-08T04:39:22.175652+0000","last_became_active":"2024-05-08T04:39:22.169887+0000","last_became_peered":"2024-05-08T04:39:22.169887+0000","last_unstale":"2024-05-08T04:39:22.175652+0000","last_undegraded":"2024-05-08T04:39:22.175652+0000","last_fullsized":"2024-05-08T04:39:22.175652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:33.012058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175110+0000","last_change":"2024-05-08T04:39:22.171762+0000","last_active":"2024-05-08T04:39:22.175110+0000","last_peered":"2024-05-08T04:39:22.175110+0000","last_clean":"2024-05-08T04:39:22.175110+0000","last_became_active":"2024-05-08T04:39:22.171384+0000","last_became_peered":"2024-05-08T04:39:22.171384+0000","last_unstale":"2024-05-08T04:39:22.175110+0000","last_undegraded":"2024-05-08T04:39:22.175110+0000","last_fullsized":"2024-05-08T04:39:22.175110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:03.121248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172651+0000","last_change":"2024-05-08T04:39:22.166276+0000","last_active":"2024-05-08T04:39:22.172651+0000","last_peered":"2024-05-08T04:39:22.172651+0000","last_clean":"2024-05-08T04:39:22.172651+0000","last_became_active":"2024-05-08T04:39:22.165998+0000","last_became_peered":"2024-05-08T04:39:22.165998+0000","last_unstale":"2024-05-08T04:39:22.172651+0000","last_undegraded":"2024-05-08T04:39:22.172651+0000","last_fullsized":"2024-05-08T04:39:22.172651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:45.570574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172959+0000","last_change":"2024-05-08T04:39:22.167910+0000","last_active":"2024-05-08T04:39:22.172959+0000","last_peered":"2024-05-08T04:39:22.172959+0000","last_clean":"2024-05-08T04:39:22.172959+0000","last_became_active":"2024-05-08T04:39:22.167621+0000","last_became_peered":"2024-05-08T04:39:22.167621+0000","last_unstale":"2024-05-08T04:39:22.172959+0000","last_undegraded":"2024-05-08T04:39:22.172959+0000","last_fullsized":"2024-05-08T04:39:22.172959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:30.689339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174445+0000","last_change":"2024-05-08T04:39:22.169160+0000","last_active":"2024-05-08T04:39:22.174445+0000","last_peered":"2024-05-08T04:39:22.174445+0000","last_clean":"2024-05-08T04:39:22.174445+0000","last_became_active":"2024-05-08T04:39:22.168870+0000","last_became_peered":"2024-05-08T04:39:22.168870+0000","last_unstale":"2024-05-08T04:39:22.174445+0000","last_undegraded":"2024-05-08T04:39:22.174445+0000","last_fullsized":"2024-05-08T04:39:22.174445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180619+0000","last_change":"2024-05-08T04:39:22.165534+0000","last_active":"2024-05-08T04:39:22.180619+0000","last_peered":"2024-05-08T04:39:22.180619+0000","last_clean":"2024-05-08T04:39:22.180619+0000","last_became_active":"2024-05-08T04:39:22.165269+0000","last_became_peered":"2024-05-08T04:39:22.165269+0000","last_unstale":"2024-05-08T04:39:22.180619+0000","last_undegraded":"2024-05-08T04:39:22.180619+0000","last_fullsized":"2024-05-08T04:39:22.180619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:03.092298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174073+0000","last_change":"2024-05-08T04:39:22.170978+0000","last_active":"2024-05-08T04:39:22.174073+0000","last_peered":"2024-05-08T04:39:22.174073+0000","last_clean":"2024-05-08T04:39:22.174073+0000","last_became_active":"2024-05-08T04:39:22.170597+0000","last_became_peered":"2024-05-08T04:39:22.170597+0000","last_unstale":"2024-05-08T04:39:22.174073+0000","last_undegraded":"2024-05-08T04:39:22.174073+0000","last_fullsized":"2024-05-08T04:39:22.174073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172513+0000","last_change":"2024-05-08T04:39:22.169063+0000","last_active":"2024-05-08T04:39:22.172513+0000","last_peered":"2024-05-08T04:39:22.172513+0000","last_clean":"2024-05-08T04:39:22.172513+0000","last_became_active":"2024-05-08T04:39:22.168810+0000","last_became_peered":"2024-05-08T04:39:22.168810+0000","last_unstale":"2024-05-08T04:39:22.172513+0000","last_undegraded":"2024-05-08T04:39:22.172513+0000","last_fullsized":"2024-05-08T04:39:22.172513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:13:22.836437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175142+0000","last_change":"2024-05-08T04:39:22.171564+0000","last_active":"2024-05-08T04:39:22.175142+0000","last_peered":"2024-05-08T04:39:22.175142+0000","last_clean":"2024-05-08T04:39:22.175142+0000","last_became_active":"2024-05-08T04:39:22.171305+0000","last_became_peered":"2024-05-08T04:39:22.171305+0000","last_unstale":"2024-05-08T04:39:22.175142+0000","last_undegraded":"2024-05-08T04:39:22.175142+0000","last_fullsized":"2024-05-08T04:39:22.175142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180569+0000","last_change":"2024-05-08T04:39:22.165080+0000","last_active":"2024-05-08T04:39:22.180569+0000","last_peered":"2024-05-08T04:39:22.180569+0000","last_clean":"2024-05-08T04:39:22.180569+0000","last_became_active":"2024-05-08T04:39:22.164840+0000","last_became_peered":"2024-05-08T04:39:22.164840+0000","last_unstale":"2024-05-08T04:39:22.180569+0000","last_undegraded":"2024-05-08T04:39:22.180569+0000","last_fullsized":"2024-05-08T04:39:22.180569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:53.147604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169498+0000","last_change":"2024-05-08T04:39:22.157527+0000","last_active":"2024-05-08T04:39:22.169498+0000","last_peered":"2024-05-08T04:39:22.169498+0000","last_clean":"2024-05-08T04:39:22.169498+0000","last_became_active":"2024-05-08T04:39:22.156747+0000","last_became_peered":"2024-05-08T04:39:22.156747+0000","last_unstale":"2024-05-08T04:39:22.169498+0000","last_undegraded":"2024-05-08T04:39:22.169498+0000","last_fullsized":"2024-05-08T04:39:22.169498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:06.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189569+0000","last_change":"2024-05-08T04:39:22.176471+0000","last_active":"2024-05-08T04:39:22.189569+0000","last_peered":"2024-05-08T04:39:22.189569+0000","last_clean":"2024-05-08T04:39:22.189569+0000","last_became_active":"2024-05-08T04:39:22.173355+0000","last_became_peered":"2024-05-08T04:39:22.173355+0000","last_unstale":"2024-05-08T04:39:22.189569+0000","last_undegraded":"2024-05-08T04:39:22.189569+0000","last_fullsized":"2024-05-08T04:39:22.189569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:22.275858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178399+0000","last_change":"2024-05-08T04:39:22.175105+0000","last_active":"2024-05-08T04:39:22.178399+0000","last_peered":"2024-05-08T04:39:22.178399+0000","last_clean":"2024-05-08T04:39:22.178399+0000","last_became_active":"2024-05-08T04:39:22.173670+0000","last_became_peered":"2024-05-08T04:39:22.173670+0000","last_unstale":"2024-05-08T04:39:22.178399+0000","last_undegraded":"2024-05-08T04:39:22.178399+0000","last_fullsized":"2024-05-08T04:39:22.178399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:30.689339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.188386+0000","last_change":"2024-05-08T04:39:22.173043+0000","last_active":"2024-05-08T04:39:22.188386+0000","last_peered":"2024-05-08T04:39:22.188386+0000","last_clean":"2024-05-08T04:39:22.188386+0000","last_became_active":"2024-05-08T04:39:22.172784+0000","last_became_peered":"2024-05-08T04:39:22.172784+0000","last_unstale":"2024-05-08T04:39:22.188386+0000","last_undegraded":"2024-05-08T04:39:22.188386+0000","last_fullsized":"2024-05-08T04:39:22.188386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178674+0000","last_change":"2024-05-08T04:39:22.173963+0000","last_active":"2024-05-08T04:39:22.178674+0000","last_peered":"2024-05-08T04:39:22.178674+0000","last_clean":"2024-05-08T04:39:22.178674+0000","last_became_active":"2024-05-08T04:39:22.173718+0000","last_became_peered":"2024-05-08T04:39:22.173718+0000","last_unstale":"2024-05-08T04:39:22.178674+0000","last_undegraded":"2024-05-08T04:39:22.178674+0000","last_fullsized":"2024-05-08T04:39:22.178674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191046+0000","last_change":"2024-05-08T04:39:21.164118+0000","last_active":"2024-05-08T04:39:22.191046+0000","last_peered":"2024-05-08T04:39:22.191046+0000","last_clean":"2024-05-08T04:39:22.191046+0000","last_became_active":"2024-05-08T04:39:21.163873+0000","last_became_peered":"2024-05-08T04:39:21.163873+0000","last_unstale":"2024-05-08T04:39:22.191046+0000","last_undegraded":"2024-05-08T04:39:22.191046+0000","last_fullsized":"2024-05-08T04:39:22.191046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:18.448263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178603+0000","last_change":"2024-05-08T04:39:22.176024+0000","last_active":"2024-05-08T04:39:22.178603+0000","last_peered":"2024-05-08T04:39:22.178603+0000","last_clean":"2024-05-08T04:39:22.178603+0000","last_became_active":"2024-05-08T04:39:22.173857+0000","last_became_peered":"2024-05-08T04:39:22.173857+0000","last_unstale":"2024-05-08T04:39:22.178603+0000","last_undegraded":"2024-05-08T04:39:22.178603+0000","last_fullsized":"2024-05-08T04:39:22.178603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189798+0000","last_change":"2024-05-08T04:39:21.157530+0000","last_active":"2024-05-08T04:39:22.189798+0000","last_peered":"2024-05-08T04:39:22.189798+0000","last_clean":"2024-05-08T04:39:22.189798+0000","last_became_active":"2024-05-08T04:39:21.157240+0000","last_became_peered":"2024-05-08T04:39:21.157240+0000","last_unstale":"2024-05-08T04:39:22.189798+0000","last_undegraded":"2024-05-08T04:39:22.189798+0000","last_fullsized":"2024-05-08T04:39:22.189798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:05.307289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171029+0000","last_change":"2024-05-08T04:39:22.158874+0000","last_active":"2024-05-08T04:39:22.171029+0000","last_peered":"2024-05-08T04:39:22.171029+0000","last_clean":"2024-05-08T04:39:22.171029+0000","last_became_active":"2024-05-08T04:39:22.158520+0000","last_became_peered":"2024-05-08T04:39:22.158520+0000","last_unstale":"2024-05-08T04:39:22.171029+0000","last_undegraded":"2024-05-08T04:39:22.171029+0000","last_fullsized":"2024-05-08T04:39:22.171029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:22.556792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190334+0000","last_change":"2024-05-08T04:39:21.166175+0000","last_active":"2024-05-08T04:39:22.190334+0000","last_peered":"2024-05-08T04:39:22.190334+0000","last_clean":"2024-05-08T04:39:22.190334+0000","last_became_active":"2024-05-08T04:39:21.165902+0000","last_became_peered":"2024-05-08T04:39:21.165902+0000","last_unstale":"2024-05-08T04:39:22.190334+0000","last_undegraded":"2024-05-08T04:39:22.190334+0000","last_fullsized":"2024-05-08T04:39:22.190334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:21.270084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171695+0000","last_change":"2024-05-08T04:39:22.166601+0000","last_active":"2024-05-08T04:39:22.171695+0000","last_peered":"2024-05-08T04:39:22.171695+0000","last_clean":"2024-05-08T04:39:22.171695+0000","last_became_active":"2024-05-08T04:39:22.165397+0000","last_became_peered":"2024-05-08T04:39:22.165397+0000","last_unstale":"2024-05-08T04:39:22.171695+0000","last_undegraded":"2024-05-08T04:39:22.171695+0000","last_fullsized":"2024-05-08T04:39:22.171695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.192997+0000","last_change":"2024-05-08T04:39:21.159405+0000","last_active":"2024-05-08T04:39:22.192997+0000","last_peered":"2024-05-08T04:39:22.192997+0000","last_clean":"2024-05-08T04:39:22.192997+0000","last_became_active":"2024-05-08T04:39:21.158942+0000","last_became_peered":"2024-05-08T04:39:21.158942+0000","last_unstale":"2024-05-08T04:39:22.192997+0000","last_undegraded":"2024-05-08T04:39:22.192997+0000","last_fullsized":"2024-05-08T04:39:22.192997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:56.924658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189853+0000","last_change":"2024-05-08T04:39:22.177782+0000","last_active":"2024-05-08T04:39:22.189853+0000","last_peered":"2024-05-08T04:39:22.189853+0000","last_clean":"2024-05-08T04:39:22.189853+0000","last_became_active":"2024-05-08T04:39:22.177615+0000","last_became_peered":"2024-05-08T04:39:22.177615+0000","last_unstale":"2024-05-08T04:39:22.189853+0000","last_undegraded":"2024-05-08T04:39:22.189853+0000","last_fullsized":"2024-05-08T04:39:22.189853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189902+0000","last_change":"2024-05-08T04:39:21.163324+0000","last_active":"2024-05-08T04:39:22.189902+0000","last_peered":"2024-05-08T04:39:22.189902+0000","last_clean":"2024-05-08T04:39:22.189902+0000","last_became_active":"2024-05-08T04:39:21.162979+0000","last_became_peered":"2024-05-08T04:39:21.162979+0000","last_unstale":"2024-05-08T04:39:22.189902+0000","last_undegraded":"2024-05-08T04:39:22.189902+0000","last_fullsized":"2024-05-08T04:39:22.189902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:00.568108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180708+0000","last_change":"2024-05-08T04:39:22.167005+0000","last_active":"2024-05-08T04:39:22.180708+0000","last_peered":"2024-05-08T04:39:22.180708+0000","last_clean":"2024-05-08T04:39:22.180708+0000","last_became_active":"2024-05-08T04:39:22.166685+0000","last_became_peered":"2024-05-08T04:39:22.166685+0000","last_unstale":"2024-05-08T04:39:22.180708+0000","last_undegraded":"2024-05-08T04:39:22.180708+0000","last_fullsized":"2024-05-08T04:39:22.180708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:22.275858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.188691+0000","last_change":"2024-05-08T04:39:21.158406+0000","last_active":"2024-05-08T04:39:22.188691+0000","last_peered":"2024-05-08T04:39:22.188691+0000","last_clean":"2024-05-08T04:39:22.188691+0000","last_became_active":"2024-05-08T04:39:21.158085+0000","last_became_peered":"2024-05-08T04:39:21.158085+0000","last_unstale":"2024-05-08T04:39:22.188691+0000","last_undegraded":"2024-05-08T04:39:22.188691+0000","last_fullsized":"2024-05-08T04:39:22.188691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:48.882556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.179477+0000","last_change":"2024-05-08T04:39:22.173698+0000","last_active":"2024-05-08T04:39:22.179477+0000","last_peered":"2024-05-08T04:39:22.179477+0000","last_clean":"2024-05-08T04:39:22.179477+0000","last_became_active":"2024-05-08T04:39:22.172731+0000","last_became_peered":"2024-05-08T04:39:22.172731+0000","last_unstale":"2024-05-08T04:39:22.179477+0000","last_undegraded":"2024-05-08T04:39:22.179477+0000","last_fullsized":"2024-05-08T04:39:22.179477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:40.745253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148763+0000","last_change":"2024-05-08T04:39:21.155478+0000","last_active":"2024-05-08T04:39:22.148763+0000","last_peered":"2024-05-08T04:39:22.148763+0000","last_clean":"2024-05-08T04:39:22.148763+0000","last_became_active":"2024-05-08T04:39:21.155218+0000","last_became_peered":"2024-05-08T04:39:21.155218+0000","last_unstale":"2024-05-08T04:39:22.148763+0000","last_undegraded":"2024-05-08T04:39:22.148763+0000","last_fullsized":"2024-05-08T04:39:22.148763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:35.594092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.168218+0000","last_change":"2024-05-08T04:39:22.159938+0000","last_active":"2024-05-08T04:39:22.168218+0000","last_peered":"2024-05-08T04:39:22.168218+0000","last_clean":"2024-05-08T04:39:22.168218+0000","last_became_active":"2024-05-08T04:39:22.159628+0000","last_became_peered":"2024-05-08T04:39:22.159628+0000","last_unstale":"2024-05-08T04:39:22.168218+0000","last_undegraded":"2024-05-08T04:39:22.168218+0000","last_fullsized":"2024-05-08T04:39:22.168218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:19.454037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.151953+0000","last_change":"2024-05-08T04:39:21.154784+0000","last_active":"2024-05-08T04:39:22.151953+0000","last_peered":"2024-05-08T04:39:22.151953+0000","last_clean":"2024-05-08T04:39:22.151953+0000","last_became_active":"2024-05-08T04:39:21.154506+0000","last_became_peered":"2024-05-08T04:39:21.154506+0000","last_unstale":"2024-05-08T04:39:22.151953+0000","last_undegraded":"2024-05-08T04:39:22.151953+0000","last_fullsized":"2024-05-08T04:39:22.151953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:39.559238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175627+0000","last_change":"2024-05-08T04:39:22.162798+0000","last_active":"2024-05-08T04:39:22.175627+0000","last_peered":"2024-05-08T04:39:22.175627+0000","last_clean":"2024-05-08T04:39:22.175627+0000","last_became_active":"2024-05-08T04:39:22.162498+0000","last_became_peered":"2024-05-08T04:39:22.162498+0000","last_unstale":"2024-05-08T04:39:22.175627+0000","last_undegraded":"2024-05-08T04:39:22.175627+0000","last_fullsized":"2024-05-08T04:39:22.175627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.149724+0000","last_change":"2024-05-08T04:39:21.161012+0000","last_active":"2024-05-08T04:39:22.149724+0000","last_peered":"2024-05-08T04:39:22.149724+0000","last_clean":"2024-05-08T04:39:22.149724+0000","last_became_active":"2024-05-08T04:39:21.160750+0000","last_became_peered":"2024-05-08T04:39:21.160750+0000","last_unstale":"2024-05-08T04:39:22.149724+0000","last_undegraded":"2024-05-08T04:39:22.149724+0000","last_fullsized":"2024-05-08T04:39:22.149724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:02.086524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175761+0000","last_change":"2024-05-08T04:39:22.170876+0000","last_active":"2024-05-08T04:39:22.175761+0000","last_peered":"2024-05-08T04:39:22.175761+0000","last_clean":"2024-05-08T04:39:22.175761+0000","last_became_active":"2024-05-08T04:39:22.170088+0000","last_became_peered":"2024-05-08T04:39:22.170088+0000","last_unstale":"2024-05-08T04:39:22.175761+0000","last_undegraded":"2024-05-08T04:39:22.175761+0000","last_fullsized":"2024-05-08T04:39:22.175761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.686125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153809+0000","last_change":"2024-05-08T04:39:21.164673+0000","last_active":"2024-05-08T04:39:22.153809+0000","last_peered":"2024-05-08T04:39:22.153809+0000","last_clean":"2024-05-08T04:39:22.153809+0000","last_became_active":"2024-05-08T04:39:21.164381+0000","last_became_peered":"2024-05-08T04:39:21.164381+0000","last_unstale":"2024-05-08T04:39:22.153809+0000","last_undegraded":"2024-05-08T04:39:22.153809+0000","last_fullsized":"2024-05-08T04:39:22.153809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:18.448263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176180+0000","last_change":"2024-05-08T04:39:22.172230+0000","last_active":"2024-05-08T04:39:22.176180+0000","last_peered":"2024-05-08T04:39:22.176180+0000","last_clean":"2024-05-08T04:39:22.176180+0000","last_became_active":"2024-05-08T04:39:22.171935+0000","last_became_peered":"2024-05-08T04:39:22.171935+0000","last_unstale":"2024-05-08T04:39:22.176180+0000","last_undegraded":"2024-05-08T04:39:22.176180+0000","last_fullsized":"2024-05-08T04:39:22.176180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:02.466282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190802+0000","last_change":"2024-05-08T04:39:21.160235+0000","last_active":"2024-05-08T04:39:22.190802+0000","last_peered":"2024-05-08T04:39:22.190802+0000","last_clean":"2024-05-08T04:39:22.190802+0000","last_became_active":"2024-05-08T04:39:21.159634+0000","last_became_peered":"2024-05-08T04:39:21.159634+0000","last_unstale":"2024-05-08T04:39:22.190802+0000","last_undegraded":"2024-05-08T04:39:22.190802+0000","last_fullsized":"2024-05-08T04:39:22.190802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.586008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.184412+0000","last_change":"2024-05-08T04:39:22.175415+0000","last_active":"2024-05-08T04:39:22.184412+0000","last_peered":"2024-05-08T04:39:22.184412+0000","last_clean":"2024-05-08T04:39:22.184412+0000","last_became_active":"2024-05-08T04:39:22.175148+0000","last_became_peered":"2024-05-08T04:39:22.175148+0000","last_unstale":"2024-05-08T04:39:22.184412+0000","last_undegraded":"2024-05-08T04:39:22.184412+0000","last_fullsized":"2024-05-08T04:39:22.184412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.193019+0000","last_change":"2024-05-08T04:39:21.166155+0000","last_active":"2024-05-08T04:39:22.193019+0000","last_peered":"2024-05-08T04:39:22.193019+0000","last_clean":"2024-05-08T04:39:22.193019+0000","last_became_active":"2024-05-08T04:39:21.165821+0000","last_became_peered":"2024-05-08T04:39:21.165821+0000","last_unstale":"2024-05-08T04:39:22.193019+0000","last_undegraded":"2024-05-08T04:39:22.193019+0000","last_fullsized":"2024-05-08T04:39:22.193019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:52.141830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.173938+0000","last_change":"2024-05-08T04:39:22.166970+0000","last_active":"2024-05-08T04:39:22.173938+0000","last_peered":"2024-05-08T04:39:22.173938+0000","last_clean":"2024-05-08T04:39:22.173938+0000","last_became_active":"2024-05-08T04:39:22.166716+0000","last_became_peered":"2024-05-08T04:39:22.166716+0000","last_unstale":"2024-05-08T04:39:22.173938+0000","last_undegraded":"2024-05-08T04:39:22.173938+0000","last_fullsized":"2024-05-08T04:39:22.173938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:36.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189643+0000","last_change":"2024-05-08T04:39:21.158038+0000","last_active":"2024-05-08T04:39:22.189643+0000","last_peered":"2024-05-08T04:39:22.189643+0000","last_clean":"2024-05-08T04:39:22.189643+0000","last_became_active":"2024-05-08T04:39:21.157181+0000","last_became_peered":"2024-05-08T04:39:21.157181+0000","last_unstale":"2024-05-08T04:39:22.189643+0000","last_undegraded":"2024-05-08T04:39:22.189643+0000","last_fullsized":"2024-05-08T04:39:22.189643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:35.637245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172030+0000","last_change":"2024-05-08T04:39:22.165838+0000","last_active":"2024-05-08T04:39:22.172030+0000","last_peered":"2024-05-08T04:39:22.172030+0000","last_clean":"2024-05-08T04:39:22.172030+0000","last_became_active":"2024-05-08T04:39:22.165559+0000","last_became_peered":"2024-05-08T04:39:22.165559+0000","last_unstale":"2024-05-08T04:39:22.172030+0000","last_undegraded":"2024-05-08T04:39:22.172030+0000","last_fullsized":"2024-05-08T04:39:22.172030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.577250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160692+0000","last_change":"2024-05-08T04:39:21.155859+0000","last_active":"2024-05-08T04:39:22.160692+0000","last_peered":"2024-05-08T04:39:22.160692+0000","last_clean":"2024-05-08T04:39:22.160692+0000","last_became_active":"2024-05-08T04:39:21.155578+0000","last_became_peered":"2024-05-08T04:39:21.155578+0000","last_unstale":"2024-05-08T04:39:22.160692+0000","last_undegraded":"2024-05-08T04:39:22.160692+0000","last_fullsized":"2024-05-08T04:39:22.160692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:21.551018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189865+0000","last_change":"2024-05-08T04:39:22.178731+0000","last_active":"2024-05-08T04:39:22.189865+0000","last_peered":"2024-05-08T04:39:22.189865+0000","last_clean":"2024-05-08T04:39:22.189865+0000","last_became_active":"2024-05-08T04:39:22.178108+0000","last_became_peered":"2024-05-08T04:39:22.178108+0000","last_unstale":"2024-05-08T04:39:22.189865+0000","last_undegraded":"2024-05-08T04:39:22.189865+0000","last_fullsized":"2024-05-08T04:39:22.189865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:52.698020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148773+0000","last_change":"2024-05-08T04:39:21.164025+0000","last_active":"2024-05-08T04:39:22.148773+0000","last_peered":"2024-05-08T04:39:22.148773+0000","last_clean":"2024-05-08T04:39:22.148773+0000","last_became_active":"2024-05-08T04:39:21.159659+0000","last_became_peered":"2024-05-08T04:39:21.159659+0000","last_unstale":"2024-05-08T04:39:22.148773+0000","last_undegraded":"2024-05-08T04:39:22.148773+0000","last_fullsized":"2024-05-08T04:39:22.148773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:16.100855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.175716+0000","last_change":"2024-05-08T04:39:22.172851+0000","last_active":"2024-05-08T04:39:22.175716+0000","last_peered":"2024-05-08T04:39:22.175716+0000","last_clean":"2024-05-08T04:39:22.175716+0000","last_became_active":"2024-05-08T04:39:22.172535+0000","last_became_peered":"2024-05-08T04:39:22.172535+0000","last_unstale":"2024-05-08T04:39:22.175716+0000","last_undegraded":"2024-05-08T04:39:22.175716+0000","last_fullsized":"2024-05-08T04:39:22.175716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:22.556792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191053+0000","last_change":"2024-05-08T04:39:21.162955+0000","last_active":"2024-05-08T04:39:22.191053+0000","last_peered":"2024-05-08T04:39:22.191053+0000","last_clean":"2024-05-08T04:39:22.191053+0000","last_became_active":"2024-05-08T04:39:21.162495+0000","last_became_peered":"2024-05-08T04:39:21.162495+0000","last_unstale":"2024-05-08T04:39:22.191053+0000","last_undegraded":"2024-05-08T04:39:22.191053+0000","last_fullsized":"2024-05-08T04:39:22.191053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:56.102930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169384+0000","last_change":"2024-05-08T04:39:22.157147+0000","last_active":"2024-05-08T04:39:22.169384+0000","last_peered":"2024-05-08T04:39:22.169384+0000","last_clean":"2024-05-08T04:39:22.169384+0000","last_became_active":"2024-05-08T04:39:22.156863+0000","last_became_peered":"2024-05-08T04:39:22.156863+0000","last_unstale":"2024-05-08T04:39:22.169384+0000","last_undegraded":"2024-05-08T04:39:22.169384+0000","last_fullsized":"2024-05-08T04:39:22.169384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:09.449111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.190813+0000","last_change":"2024-05-08T04:39:21.158688+0000","last_active":"2024-05-08T04:39:22.190813+0000","last_peered":"2024-05-08T04:39:22.190813+0000","last_clean":"2024-05-08T04:39:22.190813+0000","last_became_active":"2024-05-08T04:39:21.158322+0000","last_became_peered":"2024-05-08T04:39:21.158322+0000","last_unstale":"2024-05-08T04:39:22.190813+0000","last_undegraded":"2024-05-08T04:39:22.190813+0000","last_fullsized":"2024-05-08T04:39:22.190813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:23.218841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.164770+0000","last_change":"2024-05-08T04:39:22.158266+0000","last_active":"2024-05-08T04:39:22.164770+0000","last_peered":"2024-05-08T04:39:22.164770+0000","last_clean":"2024-05-08T04:39:22.164770+0000","last_became_active":"2024-05-08T04:39:22.157173+0000","last_became_peered":"2024-05-08T04:39:22.157173+0000","last_unstale":"2024-05-08T04:39:22.164770+0000","last_undegraded":"2024-05-08T04:39:22.164770+0000","last_fullsized":"2024-05-08T04:39:22.164770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189349+0000","last_change":"2024-05-08T04:39:21.157934+0000","last_active":"2024-05-08T04:39:22.189349+0000","last_peered":"2024-05-08T04:39:22.189349+0000","last_clean":"2024-05-08T04:39:22.189349+0000","last_became_active":"2024-05-08T04:39:21.157159+0000","last_became_peered":"2024-05-08T04:39:21.157159+0000","last_unstale":"2024-05-08T04:39:22.189349+0000","last_undegraded":"2024-05-08T04:39:22.189349+0000","last_fullsized":"2024-05-08T04:39:22.189349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:18.845857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.171711+0000","last_change":"2024-05-08T04:39:22.165299+0000","last_active":"2024-05-08T04:39:22.171711+0000","last_peered":"2024-05-08T04:39:22.171711+0000","last_clean":"2024-05-08T04:39:22.171711+0000","last_became_active":"2024-05-08T04:39:22.165031+0000","last_became_peered":"2024-05-08T04:39:22.165031+0000","last_unstale":"2024-05-08T04:39:22.171711+0000","last_undegraded":"2024-05-08T04:39:22.171711+0000","last_fullsized":"2024-05-08T04:39:22.171711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:28.591782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189406+0000","last_change":"2024-05-08T04:39:21.154172+0000","last_active":"2024-05-08T04:39:22.189406+0000","last_peered":"2024-05-08T04:39:22.189406+0000","last_clean":"2024-05-08T04:39:22.189406+0000","last_became_active":"2024-05-08T04:39:21.153881+0000","last_became_peered":"2024-05-08T04:39:21.153881+0000","last_unstale":"2024-05-08T04:39:22.189406+0000","last_undegraded":"2024-05-08T04:39:22.189406+0000","last_fullsized":"2024-05-08T04:39:22.189406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:41:07.504382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.174879+0000","last_change":"2024-05-08T04:39:22.169574+0000","last_active":"2024-05-08T04:39:22.174879+0000","last_peered":"2024-05-08T04:39:22.174879+0000","last_clean":"2024-05-08T04:39:22.174879+0000","last_became_active":"2024-05-08T04:39:22.168180+0000","last_became_peered":"2024-05-08T04:39:22.168180+0000","last_unstale":"2024-05-08T04:39:22.174879+0000","last_undegraded":"2024-05-08T04:39:22.174879+0000","last_fullsized":"2024-05-08T04:39:22.174879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:51.268620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153005+0000","last_change":"2024-05-08T04:39:21.163181+0000","last_active":"2024-05-08T04:39:22.153005+0000","last_peered":"2024-05-08T04:39:22.153005+0000","last_clean":"2024-05-08T04:39:22.153005+0000","last_became_active":"2024-05-08T04:39:21.162906+0000","last_became_peered":"2024-05-08T04:39:21.162906+0000","last_unstale":"2024-05-08T04:39:22.153005+0000","last_undegraded":"2024-05-08T04:39:22.153005+0000","last_fullsized":"2024-05-08T04:39:22.153005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:14:49.578673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176231+0000","last_change":"2024-05-08T04:39:22.172857+0000","last_active":"2024-05-08T04:39:22.176231+0000","last_peered":"2024-05-08T04:39:22.176231+0000","last_clean":"2024-05-08T04:39:22.176231+0000","last_became_active":"2024-05-08T04:39:22.172619+0000","last_became_peered":"2024-05-08T04:39:22.172619+0000","last_unstale":"2024-05-08T04:39:22.176231+0000","last_undegraded":"2024-05-08T04:39:22.176231+0000","last_fullsized":"2024-05-08T04:39:22.176231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:49.488484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189519+0000","last_change":"2024-05-08T04:39:21.155728+0000","last_active":"2024-05-08T04:39:22.189519+0000","last_peered":"2024-05-08T04:39:22.189519+0000","last_clean":"2024-05-08T04:39:22.189519+0000","last_became_active":"2024-05-08T04:39:21.155461+0000","last_became_peered":"2024-05-08T04:39:21.155461+0000","last_unstale":"2024-05-08T04:39:22.189519+0000","last_undegraded":"2024-05-08T04:39:22.189519+0000","last_fullsized":"2024-05-08T04:39:22.189519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:21.551018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.169495+0000","last_change":"2024-05-08T04:39:22.163130+0000","last_active":"2024-05-08T04:39:22.169495+0000","last_peered":"2024-05-08T04:39:22.169495+0000","last_clean":"2024-05-08T04:39:22.169495+0000","last_became_active":"2024-05-08T04:39:22.162864+0000","last_became_peered":"2024-05-08T04:39:22.162864+0000","last_unstale":"2024-05-08T04:39:22.169495+0000","last_undegraded":"2024-05-08T04:39:22.169495+0000","last_fullsized":"2024-05-08T04:39:22.169495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:51.552344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.162058+0000","last_change":"2024-05-08T04:39:21.162175+0000","last_active":"2024-05-08T04:39:22.162058+0000","last_peered":"2024-05-08T04:39:22.162058+0000","last_clean":"2024-05-08T04:39:22.162058+0000","last_became_active":"2024-05-08T04:39:21.161881+0000","last_became_peered":"2024-05-08T04:39:21.161881+0000","last_unstale":"2024-05-08T04:39:22.162058+0000","last_undegraded":"2024-05-08T04:39:22.162058+0000","last_fullsized":"2024-05-08T04:39:22.162058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:08.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.172139+0000","last_change":"2024-05-08T04:39:22.166231+0000","last_active":"2024-05-08T04:39:22.172139+0000","last_peered":"2024-05-08T04:39:22.172139+0000","last_clean":"2024-05-08T04:39:22.172139+0000","last_became_active":"2024-05-08T04:39:22.165981+0000","last_became_peered":"2024-05-08T04:39:22.165981+0000","last_unstale":"2024-05-08T04:39:22.172139+0000","last_undegraded":"2024-05-08T04:39:22.172139+0000","last_fullsized":"2024-05-08T04:39:22.172139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:59.269328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.152573+0000","last_change":"2024-05-08T04:39:21.157562+0000","last_active":"2024-05-08T04:39:22.152573+0000","last_peered":"2024-05-08T04:39:22.152573+0000","last_clean":"2024-05-08T04:39:22.152573+0000","last_became_active":"2024-05-08T04:39:21.156869+0000","last_became_peered":"2024-05-08T04:39:21.156869+0000","last_unstale":"2024-05-08T04:39:22.152573+0000","last_undegraded":"2024-05-08T04:39:22.152573+0000","last_fullsized":"2024-05-08T04:39:22.152573+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:02.115475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189336+0000","last_change":"2024-05-08T04:39:22.174739+0000","last_active":"2024-05-08T04:39:22.189336+0000","last_peered":"2024-05-08T04:39:22.189336+0000","last_clean":"2024-05-08T04:39:22.189336+0000","last_became_active":"2024-05-08T04:39:22.174472+0000","last_became_peered":"2024-05-08T04:39:22.174472+0000","last_unstale":"2024-05-08T04:39:22.189336+0000","last_undegraded":"2024-05-08T04:39:22.189336+0000","last_fullsized":"2024-05-08T04:39:22.189336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:57.108704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.154982+0000","last_change":"2024-05-08T04:39:21.160591+0000","last_active":"2024-05-08T04:39:22.154982+0000","last_peered":"2024-05-08T04:39:22.154982+0000","last_clean":"2024-05-08T04:39:22.154982+0000","last_became_active":"2024-05-08T04:39:21.160144+0000","last_became_peered":"2024-05-08T04:39:21.160144+0000","last_unstale":"2024-05-08T04:39:22.154982+0000","last_undegraded":"2024-05-08T04:39:22.154982+0000","last_fullsized":"2024-05-08T04:39:22.154982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:34.284988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.176287+0000","last_change":"2024-05-08T04:39:22.174667+0000","last_active":"2024-05-08T04:39:22.176287+0000","last_peered":"2024-05-08T04:39:22.176287+0000","last_clean":"2024-05-08T04:39:22.176287+0000","last_became_active":"2024-05-08T04:39:22.173577+0000","last_became_peered":"2024-05-08T04:39:22.173577+0000","last_unstale":"2024-05-08T04:39:22.176287+0000","last_undegraded":"2024-05-08T04:39:22.176287+0000","last_fullsized":"2024-05-08T04:39:22.176287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:14:50.584447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.191073+0000","last_change":"2024-05-08T04:39:21.159295+0000","last_active":"2024-05-08T04:39:22.191073+0000","last_peered":"2024-05-08T04:39:22.191073+0000","last_clean":"2024-05-08T04:39:22.191073+0000","last_became_active":"2024-05-08T04:39:21.159037+0000","last_became_peered":"2024-05-08T04:39:21.159037+0000","last_unstale":"2024-05-08T04:39:22.191073+0000","last_undegraded":"2024-05-08T04:39:22.191073+0000","last_fullsized":"2024-05-08T04:39:22.191073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:39.739479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.180797+0000","last_change":"2024-05-08T04:39:22.173785+0000","last_active":"2024-05-08T04:39:22.180797+0000","last_peered":"2024-05-08T04:39:22.180797+0000","last_clean":"2024-05-08T04:39:22.180797+0000","last_became_active":"2024-05-08T04:39:22.173474+0000","last_became_peered":"2024-05-08T04:39:22.173474+0000","last_unstale":"2024-05-08T04:39:22.180797+0000","last_undegraded":"2024-05-08T04:39:22.180797+0000","last_fullsized":"2024-05-08T04:39:22.180797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:35.438124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153907+0000","last_change":"2024-05-08T04:39:21.161167+0000","last_active":"2024-05-08T04:39:22.153907+0000","last_peered":"2024-05-08T04:39:22.153907+0000","last_clean":"2024-05-08T04:39:22.153907+0000","last_became_active":"2024-05-08T04:39:21.160884+0000","last_became_peered":"2024-05-08T04:39:21.160884+0000","last_unstale":"2024-05-08T04:39:22.153907+0000","last_undegraded":"2024-05-08T04:39:22.153907+0000","last_fullsized":"2024-05-08T04:39:22.153907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:44.564800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.164772+0000","last_change":"2024-05-08T04:39:22.160398+0000","last_active":"2024-05-08T04:39:22.164772+0000","last_peered":"2024-05-08T04:39:22.164772+0000","last_clean":"2024-05-08T04:39:22.164772+0000","last_became_active":"2024-05-08T04:39:22.160110+0000","last_became_peered":"2024-05-08T04:39:22.160110+0000","last_unstale":"2024-05-08T04:39:22.164772+0000","last_undegraded":"2024-05-08T04:39:22.164772+0000","last_fullsized":"2024-05-08T04:39:22.164772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:24.224615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160962+0000","last_change":"2024-05-08T04:39:21.157088+0000","last_active":"2024-05-08T04:39:22.160962+0000","last_peered":"2024-05-08T04:39:22.160962+0000","last_clean":"2024-05-08T04:39:22.160962+0000","last_became_active":"2024-05-08T04:39:21.156840+0000","last_became_peered":"2024-05-08T04:39:21.156840+0000","last_unstale":"2024-05-08T04:39:22.160962+0000","last_undegraded":"2024-05-08T04:39:22.160962+0000","last_fullsized":"2024-05-08T04:39:22.160962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:29.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.177462+0000","last_change":"2024-05-08T04:39:22.173811+0000","last_active":"2024-05-08T04:39:22.177462+0000","last_peered":"2024-05-08T04:39:22.177462+0000","last_clean":"2024-05-08T04:39:22.177462+0000","last_became_active":"2024-05-08T04:39:22.173499+0000","last_became_peered":"2024-05-08T04:39:22.173499+0000","last_unstale":"2024-05-08T04:39:22.177462+0000","last_undegraded":"2024-05-08T04:39:22.177462+0000","last_fullsized":"2024-05-08T04:39:22.177462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:45.570574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.153984+0000","last_change":"2024-05-08T04:39:21.148438+0000","last_active":"2024-05-08T04:39:22.153984+0000","last_peered":"2024-05-08T04:39:22.153984+0000","last_clean":"2024-05-08T04:39:22.153984+0000","last_became_active":"2024-05-08T04:39:21.148139+0000","last_became_peered":"2024-05-08T04:39:21.148139+0000","last_unstale":"2024-05-08T04:39:22.153984+0000","last_undegraded":"2024-05-08T04:39:22.153984+0000","last_fullsized":"2024-05-08T04:39:22.153984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:23:29.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.178849+0000","last_change":"2024-05-08T04:39:22.174756+0000","last_active":"2024-05-08T04:39:22.178849+0000","last_peered":"2024-05-08T04:39:22.178849+0000","last_clean":"2024-05-08T04:39:22.178849+0000","last_became_active":"2024-05-08T04:39:22.174483+0000","last_became_peered":"2024-05-08T04:39:22.174483+0000","last_unstale":"2024-05-08T04:39:22.178849+0000","last_undegraded":"2024-05-08T04:39:22.178849+0000","last_fullsized":"2024-05-08T04:39:22.178849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:35.290762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.160847+0000","last_change":"2024-05-08T04:39:21.155924+0000","last_active":"2024-05-08T04:39:22.160847+0000","last_peered":"2024-05-08T04:39:22.160847+0000","last_clean":"2024-05-08T04:39:22.160847+0000","last_became_active":"2024-05-08T04:39:21.155683+0000","last_became_peered":"2024-05-08T04:39:21.155683+0000","last_unstale":"2024-05-08T04:39:22.160847+0000","last_undegraded":"2024-05-08T04:39:22.160847+0000","last_fullsized":"2024-05-08T04:39:22.160847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:44.564800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.166892+0000","last_change":"2024-05-08T04:39:22.159438+0000","last_active":"2024-05-08T04:39:22.166892+0000","last_peered":"2024-05-08T04:39:22.166892+0000","last_clean":"2024-05-08T04:39:22.166892+0000","last_became_active":"2024-05-08T04:39:22.159155+0000","last_became_peered":"2024-05-08T04:39:22.159155+0000","last_unstale":"2024-05-08T04:39:22.166892+0000","last_undegraded":"2024-05-08T04:39:22.166892+0000","last_fullsized":"2024-05-08T04:39:22.166892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:18:39.892506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.152666+0000","last_change":"2024-05-08T04:39:21.156802+0000","last_active":"2024-05-08T04:39:22.152666+0000","last_peered":"2024-05-08T04:39:22.152666+0000","last_clean":"2024-05-08T04:39:22.152666+0000","last_became_active":"2024-05-08T04:39:21.156498+0000","last_became_peered":"2024-05-08T04:39:21.156498+0000","last_unstale":"2024-05-08T04:39:22.152666+0000","last_undegraded":"2024-05-08T04:39:22.152666+0000","last_fullsized":"2024-05-08T04:39:22.152666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:48.482710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.170758+0000","last_change":"2024-05-08T04:39:22.157966+0000","last_active":"2024-05-08T04:39:22.170758+0000","last_peered":"2024-05-08T04:39:22.170758+0000","last_clean":"2024-05-08T04:39:22.170758+0000","last_became_active":"2024-05-08T04:39:22.157699+0000","last_became_peered":"2024-05-08T04:39:22.157699+0000","last_unstale":"2024-05-08T04:39:22.170758+0000","last_undegraded":"2024-05-08T04:39:22.170758+0000","last_fullsized":"2024-05-08T04:39:22.170758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:19.454037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.148736+0000","last_change":"2024-05-08T04:39:21.160773+0000","last_active":"2024-05-08T04:39:22.148736+0000","last_peered":"2024-05-08T04:39:22.148736+0000","last_clean":"2024-05-08T04:39:22.148736+0000","last_became_active":"2024-05-08T04:39:21.160535+0000","last_became_peered":"2024-05-08T04:39:21.160535+0000","last_unstale":"2024-05-08T04:39:22.148736+0000","last_undegraded":"2024-05-08T04:39:22.148736+0000","last_fullsized":"2024-05-08T04:39:22.148736+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:34.284988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.189077+0000","last_change":"2024-05-08T04:39:22.173739+0000","last_active":"2024-05-08T04:39:22.189077+0000","last_peered":"2024-05-08T04:39:22.189077+0000","last_clean":"2024-05-08T04:39:22.189077+0000","last_became_active":"2024-05-08T04:39:22.171147+0000","last_became_peered":"2024-05-08T04:39:22.171147+0000","last_unstale":"2024-05-08T04:39:22.189077+0000","last_undegraded":"2024-05-08T04:39:22.189077+0000","last_fullsized":"2024-05-08T04:39:22.189077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:21.130203+0000","last_clean_scrub_stamp":"2024-05-08T04:39:21.130203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:17.106629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.162092+0000","last_change":"2024-05-08T04:39:21.159551+0000","last_active":"2024-05-08T04:39:22.162092+0000","last_peered":"2024-05-08T04:39:22.162092+0000","last_clean":"2024-05-08T04:39:22.162092+0000","last_became_active":"2024-05-08T04:39:21.159161+0000","last_became_peered":"2024-05-08T04:39:21.159161+0000","last_unstale":"2024-05-08T04:39:22.162092+0000","last_undegraded":"2024-05-08T04:39:22.162092+0000","last_fullsized":"2024-05-08T04:39:22.162092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:20.124429+0000","last_clean_scrub_stamp":"2024-05-08T04:39:20.124429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:05.307289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T04:39:22.159814+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:22.159814+0000","last_peered":"2024-05-08T04:39:22.159814+0000","last_clean":"2024-05-08T04:39:22.159814+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:22.159814+0000","last_undegraded":"2024-05-08T04:39:22.159814+0000","last_fullsized":"2024-05-08T04:39:22.159814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":140386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:39:26.611 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:39:26.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:39:26.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 Processor -- start 2024-05-08T04:39:26.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 -- start start 2024-05-08T04:39:26.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 0x7f56c015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c006da30 con 0x7f56c00affe0 2024-05-08T04:39:26.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c006dcd0 con 0x7f56c006ac90 2024-05-08T04:39:26.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56c4e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c006e320 con 0x7f56c006e7b0 2024-05-08T04:39:26.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bde00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f56c00affe0 0x7f56c006f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39022/0 (socket says 172.21.15.164:39022) 2024-05-08T04:39:26.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bde00640 1 -- 172.21.15.164:0/3273110123 learned_addr learned my addr 172.21.15.164:0/3273110123 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:26.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bd400640 1 --2- 172.21.15.164:0/3273110123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 0x7f56c015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500847243 0 0) 0x7f56c006da30 con 0x7f56c00affe0 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a4003530 con 0x7f56c00affe0 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bd400640 1 -- 172.21.15.164:0/3273110123 >> v1:172.21.15.172:6789/0 conn(0x7f56c006ac90 legacy=0x7f56c00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288100141 0 0) 0x7f56c006dcd0 con 0x7f56c006ac90 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bd400640 1 -- 172.21.15.164:0/3273110123 >> v1:172.21.15.164:6789/0 conn(0x7f56c00affe0 legacy=0x7f56c006f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.794+0000 7f56bd400640 1 -- 172.21.15.164:0/3273110123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c006dfb0 con 0x7f56c006e7b0 2024-05-08T04:39:26.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56bd400640 1 --2- 172.21.15.164:0/3273110123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 0x7f56c015cbd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56c006e4a0 tx=0x7f56ac01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7911a8c30c0a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0ab620 con 0x7f56c006e7b0 2024-05-08T04:39:26.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac0b0070 con 0x7f56c006e7b0 2024-05-08T04:39:26.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56bca00640 1 -- 172.21.15.164:0/3273110123 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0ab9e0 con 0x7f56c006e7b0 2024-05-08T04:39:26.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 -- 172.21.15.164:0/3273110123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 msgr2=0x7f56c015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 --2- 172.21.15.164:0/3273110123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 0x7f56c015cbd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56c006e4a0 tx=0x7f56ac01c4c0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 -- 172.21.15.164:0/3273110123 shutdown_connections 2024-05-08T04:39:26.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 --2- 172.21.15.164:0/3273110123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c006e7b0 0x7f56c015cbd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 -- 172.21.15.164:0/3273110123 >> 172.21.15.164:0/3273110123 conn(0x7f56c00b2d00 msgr2=0x7f56c00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:26.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 -- 172.21.15.164:0/3273110123 shutdown_connections 2024-05-08T04:39:26.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.798+0000 7f56c4e00640 1 -- 172.21.15.164:0/3273110123 wait complete. 2024-05-08T04:39:26.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 Processor -- start 2024-05-08T04:39:26.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- start start 2024-05-08T04:39:26.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c006ac90 0x7f56c01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c00affe0 0x7f56c013aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c01699b0 con 0x7f56c006e7b0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0169830 con 0x7f56c006ac90 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0169b30 con 0x7f56c00affe0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52120/0 (socket says 172.21.15.164:52120) 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 -- 172.21.15.164:0/1723192148 learned_addr learned my addr 172.21.15.164:0/1723192148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bde00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c006ac90 0x7f56c01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56be800640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c00affe0 0x7f56c013aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 -- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c00affe0 msgr2=0x7f56c013aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c00affe0 0x7f56c013aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 -- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c006ac90 msgr2=0x7f56c01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c006ac90 0x7f56c01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 -- 172.21.15.164:0/1723192148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0152f10 con 0x7f56c006e7b0 2024-05-08T04:39:26.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56bd400640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009740 tx=0x7f56ac019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0abbc0 con 0x7f56c006e7b0 2024-05-08T04:39:26.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac0abd60 con 0x7f56c006e7b0 2024-05-08T04:39:26.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac004260 con 0x7f56c006e7b0 2024-05-08T04:39:26.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c00b3f00 con 0x7f56c006e7b0 2024-05-08T04:39:26.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c00b47a0 con 0x7f56c006e7b0 2024-05-08T04:39:26.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.802+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0072e80 con 0x7f56c006e7b0 2024-05-08T04:39:26.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.806+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56ac003620 con 0x7f56c006e7b0 2024-05-08T04:39:26.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.806+0000 7f56b6000640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 0x7f56a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:26.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.806+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56ac11e920 con 0x7f56c006e7b0 2024-05-08T04:39:26.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.806+0000 7f56bde00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 0x7f56a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:26.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.806+0000 7f56bde00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 0x7f56a0069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56b0005c10 tx=0x7f56b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:26.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:26.810+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56ac0c4020 con 0x7f56c006e7b0 2024-05-08T04:39:26.965 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-05-08T04:39:26.964+0000 7f8be6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:39:27.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.014+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f56c0151b60 con 0x7f56c006e7b0 2024-05-08T04:39:27.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.018+0000 7f56b6000640 1 -- 172.21.15.164:0/1723192148 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f56ac0160b0 con 0x7f56c006e7b0 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:39:27.023 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 msgr2=0x7f56a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 0x7f56a0069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56b0005c10 tx=0x7f56b0009040 comp rx=0 tx=0).stop 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 msgr2=0x7f56c00ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009740 tx=0x7f56ac019260 comp rx=0 tx=0).stop 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 shutdown_connections 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f56a0066f90 0x7f56a0069450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56c00affe0 0x7f56c013aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56c006e7b0 0x7f56c00ccca0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 --2- 172.21.15.164:0/1723192148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c006ac90 0x7f56c01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 >> 172.21.15.164:0/1723192148 conn(0x7f56c00b2d00 msgr2=0x7f56c00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 shutdown_connections 2024-05-08T04:39:27.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.022+0000 7f56c4e00640 1 -- 172.21.15.164:0/1723192148 wait complete. 2024-05-08T04:39:27.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:27.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 Processor -- start 2024-05-08T04:39:27.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 -- start start 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf4415bc50 0x7faf440b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440ccfe0 0x7faf440d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faf440b4d50 0x7faf440d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440b42a0 con 0x7faf4415bc50 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440cd4e0 con 0x7faf440b4d50 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440cd7c0 con 0x7faf440ccfe0 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf4415bc50 0x7faf440b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf4415bc50 0x7faf440b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52128/0 (socket says 172.21.15.164:52128) 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 -- 172.21.15.164:0/2524828271 learned_addr learned my addr 172.21.15.164:0/2524828271 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49200640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440ccfe0 0x7faf440d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf4a600640 1 --2- 172.21.15.164:0/2524828271 >> v2:172.21.15.172:3300/0 conn(0x7faf440b4d50 0x7faf440d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 -- 172.21.15.164:0/2524828271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440ccfe0 msgr2=0x7faf440d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440ccfe0 0x7faf440d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 -- 172.21.15.164:0/2524828271 >> v2:172.21.15.172:3300/0 conn(0x7faf440b4d50 msgr2=0x7faf440d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 --2- 172.21.15.164:0/2524828271 >> v2:172.21.15.172:3300/0 conn(0x7faf440b4d50 0x7faf440d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.178+0000 7faf49c00640 1 -- 172.21.15.164:0/2524828271 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440cf590 con 0x7faf4415bc50 2024-05-08T04:39:27.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf49c00640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf440b4810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf440cbfa0 tx=0x7faf3801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e517663d321b53f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf43e00640 1 -- 172.21.15.164:0/2524828271 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380ab620 con 0x7faf4415bc50 2024-05-08T04:39:27.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf43e00640 1 -- 172.21.15.164:0/2524828271 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf380b0070 con 0x7faf4415bc50 2024-05-08T04:39:27.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf43e00640 1 -- 172.21.15.164:0/2524828271 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380ab9e0 con 0x7faf4415bc50 2024-05-08T04:39:27.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 -- 172.21.15.164:0/2524828271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 msgr2=0x7faf440b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf440b4810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf440cbfa0 tx=0x7faf3801c460 comp rx=0 tx=0).stop 2024-05-08T04:39:27.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 -- 172.21.15.164:0/2524828271 shutdown_connections 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 --2- 172.21.15.164:0/2524828271 >> v2:172.21.15.172:3300/0 conn(0x7faf440b4d50 0x7faf440d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440ccfe0 0x7faf440d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 --2- 172.21.15.164:0/2524828271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf440b4810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 -- 172.21.15.164:0/2524828271 >> 172.21.15.164:0/2524828271 conn(0x7faf440b34f0 msgr2=0x7faf440b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 -- 172.21.15.164:0/2524828271 shutdown_connections 2024-05-08T04:39:27.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.182+0000 7faf4c600640 1 -- 172.21.15.164:0/2524828271 wait complete. 2024-05-08T04:39:27.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 Processor -- start 2024-05-08T04:39:27.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- start start 2024-05-08T04:39:27.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440b4d50 0x7faf440c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf440ccfe0 0x7faf441528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440cea90 con 0x7faf4415bc50 2024-05-08T04:39:27.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440ce910 con 0x7faf440ccfe0 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440cec10 con 0x7faf440b4d50 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52134/0 (socket says 172.21.15.164:52134) 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 -- 172.21.15.164:0/2520420777 learned_addr learned my addr 172.21.15.164:0/2520420777 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf49c00640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440b4d50 0x7faf440c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf49200640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf440ccfe0 0x7faf441528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 -- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440b4d50 msgr2=0x7faf440c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440b4d50 0x7faf440c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 -- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf440ccfe0 msgr2=0x7faf441528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf440ccfe0 0x7faf441528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 -- 172.21.15.164:0/2520420777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4415ccf0 con 0x7faf4415bc50 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4a600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007e60 tx=0x7faf3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c014620 con 0x7faf4415bc50 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c019070 con 0x7faf4415bc50 2024-05-08T04:39:27.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c014a20 con 0x7faf4415bc50 2024-05-08T04:39:27.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf440b5380 con 0x7faf4415bc50 2024-05-08T04:39:27.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.186+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf440b5940 con 0x7faf4415bc50 2024-05-08T04:39:27.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf440b5b40 con 0x7faf4415bc50 2024-05-08T04:39:27.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf3c014bc0 con 0x7faf4415bc50 2024-05-08T04:39:27.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf41600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 0x7faf24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf3c085800 con 0x7faf4415bc50 2024-05-08T04:39:27.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf49c00640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 0x7faf24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.190+0000 7faf49c00640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 0x7faf24069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf380af000 tx=0x7faf380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.194+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf3c056120 con 0x7faf4415bc50 2024-05-08T04:39:27.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.378+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf44152e30 con 0x7faf4415bc50 2024-05-08T04:39:27.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.378+0000 7faf41600640 1 -- 172.21.15.164:0/2520420777 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7faf3c05a1c0 con 0x7faf4415bc50 2024-05-08T04:39:27.382 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:27.383 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 17 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 msgr2=0x7faf24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 0x7faf24069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf380af000 tx=0x7faf380a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 msgr2=0x7faf4415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf3c007e60 tx=0x7faf3c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 shutdown_connections 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf24066f60 0x7faf24069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf4415bc50 0x7faf4415c2a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf440ccfe0 0x7faf441528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 --2- 172.21.15.164:0/2520420777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf440b4d50 0x7faf440c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 >> 172.21.15.164:0/2520420777 conn(0x7faf440b34f0 msgr2=0x7faf440d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 shutdown_connections 2024-05-08T04:39:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.382+0000 7faf4c600640 1 -- 172.21.15.164:0/2520420777 wait complete. 2024-05-08T04:39:27.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:39:21.526329+0000', 'modified': '2024-05-08T04:39:24.549738+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5064}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5064': {'gid': 5064, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/3451370637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6833', 'nonce': 3451370637}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 3451370637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:39:27.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:39:27.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T04:39:27.828 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:39:27.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:39:27.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 Processor -- start 2024-05-08T04:39:27.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 -- start start 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec7406f450 0x7fec7415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec740b4fa0 0x7fec7406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740b5740 con 0x7fec741580d0 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec740b6350 con 0x7fec740b4fa0 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec73200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec740b5a20 con 0x7fec7406f450 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec740b4fa0 0x7fec7406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec740b4fa0 0x7fec7406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52148/0 (socket says 172.21.15.164:52148) 2024-05-08T04:39:27.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 -- 172.21.15.164:0/1510246729 learned_addr learned my addr 172.21.15.164:0/1510246729 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec69400640 1 --2- 172.21.15.164:0/1510246729 >> v2:172.21.15.172:3300/0 conn(0x7fec7406f450 0x7fec7415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 -- 172.21.15.164:0/1510246729 >> v2:172.21.15.172:3300/0 conn(0x7fec7406f450 msgr2=0x7fec7415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 --2- 172.21.15.164:0/1510246729 >> v2:172.21.15.172:3300/0 conn(0x7fec7406f450 0x7fec7415ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 -- 172.21.15.164:0/1510246729 >> v1:172.21.15.172:6789/0 conn(0x7fec741580d0 legacy=0x7fec7406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.942+0000 7fec72800640 1 -- 172.21.15.164:0/1510246729 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740b5d00 con 0x7fec740b4fa0 2024-05-08T04:39:27.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec72800640 1 --2- 172.21.15.164:0/1510246729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7406bda0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fec7406e2f0 tx=0x7fec6c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7828605f6501917 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec71400640 1 -- 172.21.15.164:0/1510246729 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0a5710 con 0x7fec740b4fa0 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec71400640 1 -- 172.21.15.164:0/1510246729 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0a58b0 con 0x7fec740b4fa0 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec71400640 1 -- 172.21.15.164:0/1510246729 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0a5c80 con 0x7fec740b4fa0 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 -- 172.21.15.164:0/1510246729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 msgr2=0x7fec7406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 --2- 172.21.15.164:0/1510246729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7406bda0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fec7406e2f0 tx=0x7fec6c01c9f0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 -- 172.21.15.164:0/1510246729 shutdown_connections 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 --2- 172.21.15.164:0/1510246729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7406bda0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 --2- 172.21.15.164:0/1510246729 >> v2:172.21.15.172:3300/0 conn(0x7fec7406f450 0x7fec7415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 -- 172.21.15.164:0/1510246729 >> 172.21.15.164:0/1510246729 conn(0x7fec7409aa20 msgr2=0x7fec740b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 -- 172.21.15.164:0/1510246729 shutdown_connections 2024-05-08T04:39:27.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.946+0000 7fec73200640 1 -- 172.21.15.164:0/1510246729 wait complete. 2024-05-08T04:39:27.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 Processor -- start 2024-05-08T04:39:27.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- start start 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec7406f450 0x7fec74153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 0x7fec7409bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74158630 con 0x7fec740b4fa0 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741584b0 con 0x7fec7406f450 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7406caf0 con 0x7fec741580d0 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52158/0 (socket says 172.21.15.164:52158) 2024-05-08T04:39:27.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 -- 172.21.15.164:0/346504361 learned_addr learned my addr 172.21.15.164:0/346504361 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec71e00640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec7406f450 0x7fec74153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec72800640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 0x7fec7409bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 -- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 msgr2=0x7fec7409bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 0x7fec7409bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 -- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec7406f450 msgr2=0x7fec74153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec7406f450 0x7fec74153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 -- 172.21.15.164:0/346504361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec741f7610 con 0x7fec740b4fa0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec72800640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 0x7fec7409bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec69400640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d830 tx=0x7fec5c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec6aa00640 1 -- 172.21.15.164:0/346504361 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00b6c0 con 0x7fec740b4fa0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec6aa00640 1 -- 172.21.15.164:0/346504361 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c019070 con 0x7fec740b4fa0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec6aa00640 1 -- 172.21.15.164:0/346504361 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00ba60 con 0x7fec740b4fa0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec741f78f0 con 0x7fec740b4fa0 2024-05-08T04:39:27.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.950+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec741f7eb0 con 0x7fec740b4fa0 2024-05-08T04:39:27.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec6aa00640 1 -- 172.21.15.164:0/346504361 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec5c01e460 con 0x7fec740b4fa0 2024-05-08T04:39:27.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec6aa00640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec6aa00640 1 -- 172.21.15.164:0/346504361 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec5c085690 con 0x7fec740b4fa0 2024-05-08T04:39:27.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 Processor -- start 2024-05-08T04:39:27.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 0x7fec50069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec64001770 tx=0x7fec64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 -- start start 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 0x560af8b339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af8a1b5f0 con 0x560af8b7d920 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af8982da0 con 0x560af8b7e240 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af890b150 con 0x560af8b33f30 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52172/0 (socket says 172.21.15.164:52172) 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 -- 172.21.15.164:0/1317383357 learned_addr learned my addr 172.21.15.164:0/1317383357 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec69400640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 0x560af8b339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec72800640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 -- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 msgr2=0x560af8b34310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.954+0000 7fec71e00640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b34310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 msgr2=0x560af8b339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 0x560af8b339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/1317383357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af8924ca0 con 0x560af8b7d920 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec69400640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 0x560af8b339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:27.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fec6400fae0 tx=0x7fec64000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d801c71f597e50c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec62a00640 1 -- 172.21.15.164:0/1317383357 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64025070 con 0x560af8b7d920 2024-05-08T04:39:27.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec62a00640 1 -- 172.21.15.164:0/1317383357 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64015bf0 con 0x560af8b7d920 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec62a00640 1 -- 172.21.15.164:0/1317383357 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64020400 con 0x560af8b7d920 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- 172.21.15.164:0/1317383357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 msgr2=0x560af8b7dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fec6400fae0 tx=0x7fec64000ee0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- 172.21.15.164:0/1317383357 shutdown_connections 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b34310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7e240 0x560af8b339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- 172.21.15.164:0/1317383357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7d920 0x560af8b7dd00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- 172.21.15.164:0/1317383357 >> 172.21.15.164:0/1317383357 conn(0x560af8b3a2a0 msgr2=0x560af8b322d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- 172.21.15.164:0/1317383357 shutdown_connections 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- 172.21.15.164:0/1317383357 wait complete. 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- start start 2024-05-08T04:39:27.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 Processor -- start 2024-05-08T04:39:27.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b7e240 0x560af8b89e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af893d990 con 0x560af8b33f30 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af897ee40 con 0x560af8b7d920 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af890e900 con 0x560af8b7e240 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec7c731000 10 client.0 _ll_register_callbacks cb 0x560af8a361c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52186/0 (socket says 172.21.15.164:52186) 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/124174586 learned_addr learned my addr 172.21.15.164:0/124174586 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec69400640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec72800640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b7e240 0x560af8b89e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b7e240 msgr2=0x560af8b89e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b7e240 0x560af8b89e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 msgr2=0x560af8b854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec71e00640 1 -- 172.21.15.164:0/124174586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af893b630 con 0x560af8b33f30 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.958+0000 7fec69400640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec71e00640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec64000f20 tx=0x7fec64007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90012f4dbb8eb428 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64025050 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec6402c650 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec64015bf0 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec64039050 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 10 client.5244 fetch_fsmap learned FSMap version 17 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x560af890b3a0 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64015bf0 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fec6403c020 con 0x560af8b33f30 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 10 client.5244 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 10 client.5244 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 20 client.5244 populate_metadata read hostname 'smithi164' 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8b8c670 0x560af8b8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 4 client.5244 mds_command: new command op to 5064 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:39:27.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 --> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af891ea50 con 0x560af8b8c670 2024-05-08T04:39:27.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec69400640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8b8c670 0x560af8b8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec69400640 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8b8c670 0x560af8b8ea60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948bb367ed3277fa server_cookie=9207d21e36ca7801 in_seq=0 out_seq=0 2024-05-08T04:39:27.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 10 client.5244 ms_handle_connect on v2:172.21.15.172:6833/3451370637 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 1 -- 172.21.15.164:0/124174586 <== mds.0 v2:172.21.15.172:6833/3451370637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560af891ea50 con 0x560af8b8c670 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 10 client.5244 handle_command_reply: tid=0 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 client.5244 shutdown 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 msgr2=0x560af8b84f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec64000f20 tx=0x7fec64007f90 comp rx=0 tx=0).stop 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 shutdown_connections 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8b8c670 0x560af8b8ea60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b7e240 0x560af8b89e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 --2- 172.21.15.164:0/124174586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b33f30 0x560af8b84f70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec57400640 0 client.5244 ms_handle_reset on v2:172.21.15.172:6833/3451370637 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 >> 172.21.15.164:0/124174586 conn(0x560af8b3a2a0 msgr2=0x560af8b329e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 shutdown_connections 2024-05-08T04:39:27.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 1 -- 172.21.15.164:0/124174586 wait complete. 2024-05-08T04:39:27.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.962+0000 7fec7c731000 20 client.5244 trim_cache size 0 max 16384 2024-05-08T04:39:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 Processor -- start 2024-05-08T04:39:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 -- start start 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b7c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b7c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74201b10 con 0x560af8b7e240 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74200fd0 con 0x560af8b7d920 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec74201510 con 0x560af8b33f30 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.978+0000 7fec72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52188/0 (socket says 172.21.15.164:52188) 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 -- 172.21.15.164:0/2711234533 learned_addr learned my addr 172.21.15.164:0/2711234533 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec69400640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b7c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec71e00640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b7c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 -- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 msgr2=0x560af8b7c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b7c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 -- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 msgr2=0x560af8b7c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b7c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 -- 172.21.15.164:0/2711234533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af8b85820 con 0x560af8b7e240 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec69400640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b7c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0af040 tx=0x7fec6c0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe63258844e7b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec62000640 1 -- 172.21.15.164:0/2711234533 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0a5710 con 0x560af8b7e240 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec62000640 1 -- 172.21.15.164:0/2711234533 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0a58b0 con 0x560af8b7e240 2024-05-08T04:39:27.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec62000640 1 -- 172.21.15.164:0/2711234533 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0a5c60 con 0x560af8b7e240 2024-05-08T04:39:27.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- 172.21.15.164:0/2711234533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 msgr2=0x560af8b36320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0af040 tx=0x7fec6c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- 172.21.15.164:0/2711234533 shutdown_connections 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8b36320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8b7c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- 172.21.15.164:0/2711234533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8b7c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- 172.21.15.164:0/2711234533 >> 172.21.15.164:0/2711234533 conn(0x560af8b3a2a0 msgr2=0x560af8b331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- 172.21.15.164:0/2711234533 shutdown_connections 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- 172.21.15.164:0/2711234533 wait complete. 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- start start 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 Processor -- start 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8bde890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8bdedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af89873d0 con 0x560af8b7e240 2024-05-08T04:39:27.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af8a3ed50 con 0x560af8b7d920 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af891d980 con 0x560af8b33f30 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec7c731000 10 client.0 _ll_register_callbacks cb 0x7fec64001df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52196/0 (socket says 172.21.15.164:52196) 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec72800640 1 -- 172.21.15.164:0/2883114494 learned_addr learned my addr 172.21.15.164:0/2883114494 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec71e00640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8bde890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.982+0000 7fec69400640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8bdedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 -- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 msgr2=0x560af8bde890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8bde890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 -- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 msgr2=0x560af8bdedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8bdedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 -- 172.21.15.164:0/2883114494 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af8b7e070 con 0x560af8b7e240 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec72800640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009680 tx=0x7fec6c01b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee2586d60b15f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:39:27.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0c7070 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec6c003ba0 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6c0a58b0 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6c0d3050 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 10 client.5250 fetch_fsmap learned FSMap version 17 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x560af8a04600 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c0a58b0 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fec6c0d6020 con 0x560af8b7e240 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 10 client.5250 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 10 client.5250 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 20 client.5250 populate_metadata read hostname 'smithi164' 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8bc2c00 0x560af8bc2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 4 client.5250 mds_command: new command op to 5064 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 --> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560af89902e0 con 0x560af8bc2c00 2024-05-08T04:39:27.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec71e00640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8bc2c00 0x560af8bc2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec71e00640 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8bc2c00 0x560af8bc2fe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa5e9d01a9cab5a server_cookie=7e198a5c85311dd4 in_seq=0 out_seq=0 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 10 client.5250 ms_handle_connect on v2:172.21.15.172:6833/3451370637 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 1 -- 172.21.15.164:0/2883114494 <== mds.0 v2:172.21.15.172:6833/3451370637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560af89902e0 con 0x560af8bc2c00 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 10 client.5250 handle_command_reply: tid=0 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 client.5250 shutdown 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 msgr2=0x560af8bc26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009680 tx=0x7fec6c01b230 comp rx=0 tx=0).stop 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 shutdown_connections 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:6833/3451370637,v1:172.21.15.172:6835/3451370637] conn(0x560af8bc2c00 0x560af8bc2fe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560af8b7e240 0x560af8bc26c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560af8b7d920 0x560af8bdedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec57400640 0 client.5250 ms_handle_reset on v2:172.21.15.172:6833/3451370637 2024-05-08T04:39:27.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 --2- 172.21.15.164:0/2883114494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560af8b33f30 0x560af8bde890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 >> 172.21.15.164:0/2883114494 conn(0x560af8b3a2a0 msgr2=0x560af8b331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 shutdown_connections 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 1 -- 172.21.15.164:0/2883114494 wait complete. 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec7c731000 20 client.5250 trim_cache size 0 max 16384 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 5064, 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:39:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 5.8281734250000001, 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 17, 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 33, 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 33, 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 12.860150425 2024-05-08T04:39:27.996 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 msgr2=0x7fec50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 0x7fec50069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec64001770 tx=0x7fec64009040 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 msgr2=0x7fec7409b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d830 tx=0x7fec5c00dcf0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 shutdown_connections 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec50066f60 0x7fec50069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec741580d0 0x7fec7409bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec740b4fa0 0x7fec7409b660 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 --2- 172.21.15.164:0/346504361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec7406f450 0x7fec74153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 >> 172.21.15.164:0/346504361 conn(0x7fec7409aa20 msgr2=0x7fec740ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 shutdown_connections 2024-05-08T04:39:27.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:27.986+0000 7fec73200640 1 -- 172.21.15.164:0/346504361 wait complete. 2024-05-08T04:39:28.010 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:39:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 Processor -- start 2024-05-08T04:39:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 -- start start 2024-05-08T04:39:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f39740ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39740d07d0 0x7f39740d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397415b1d0 con 0x7f397406ecd0 2024-05-08T04:39:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3974071e90 con 0x7f39740d07d0 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f397a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740cd540 con 0x7f3974155170 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39740d07d0 0x7f39740d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39740d07d0 0x7f39740d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52212/0 (socket says 172.21.15.164:52212) 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 -- 172.21.15.164:0/500650870 learned_addr learned my addr 172.21.15.164:0/500650870 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973400640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f39740ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 -- 172.21.15.164:0/500650870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 msgr2=0x7f39740ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f39740ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 -- 172.21.15.164:0/500650870 >> v1:172.21.15.172:6789/0 conn(0x7f397406ecd0 legacy=0x7f39740cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.142+0000 7f3973e00640 1 -- 172.21.15.164:0/500650870 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3974155550 con 0x7f39740d07d0 2024-05-08T04:39:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f3973e00640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39740d07d0 0x7f39740d0c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f46fd057e27a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f3972000640 1 -- 172.21.15.164:0/500650870 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0b2070 con 0x7f39740d07d0 2024-05-08T04:39:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f3972000640 1 -- 172.21.15.164:0/500650870 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c0a9070 con 0x7f39740d07d0 2024-05-08T04:39:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f3972000640 1 -- 172.21.15.164:0/500650870 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0ad880 con 0x7f39740d07d0 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 -- 172.21.15.164:0/500650870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39740d07d0 msgr2=0x7f39740d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39740d07d0 0x7f39740d0c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c01c560 comp rx=0 tx=0).stop 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 -- 172.21.15.164:0/500650870 shutdown_connections 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39740d07d0 0x7f39740d0c60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 --2- 172.21.15.164:0/500650870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f39740ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 -- 172.21.15.164:0/500650870 >> 172.21.15.164:0/500650870 conn(0x7f3974066fa0 msgr2=0x7f397406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 -- 172.21.15.164:0/500650870 shutdown_connections 2024-05-08T04:39:28.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.146+0000 7f397a200640 1 -- 172.21.15.164:0/500650870 wait complete. 2024-05-08T04:39:28.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 Processor -- start 2024-05-08T04:39:28.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- start start 2024-05-08T04:39:28.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 0x7f3974131f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f3974159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397406f250 con 0x7f397406ecd0 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397406f0d0 con 0x7f39740d07d0 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740b48e0 con 0x7f3974155170 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52224/0 (socket says 172.21.15.164:52224) 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 -- 172.21.15.164:0/3006352050 learned_addr learned my addr 172.21.15.164:0/3006352050 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3972a00640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 0x7f3974131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973e00640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f3974159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 -- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 msgr2=0x7f3974159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f3974159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 -- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 msgr2=0x7f3974131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 0x7f3974131f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 -- 172.21.15.164:0/3006352050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39741329f0 con 0x7f397406ecd0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3972a00640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 0x7f3974131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f3973400640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f396400efc0 tx=0x7f396400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964018020 con 0x7f397406ecd0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3964014480 con 0x7f397406ecd0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964014850 con 0x7f397406ecd0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3974132cd0 con 0x7f397406ecd0 2024-05-08T04:39:28.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.150+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39741422b0 con 0x7f397406ecd0 2024-05-08T04:39:28.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3974069d60 con 0x7f397406ecd0 2024-05-08T04:39:28.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f396401b020 con 0x7f397406ecd0 2024-05-08T04:39:28.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f396b400640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 0x7f3954069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:28.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3964085b50 con 0x7f397406ecd0 2024-05-08T04:39:28.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f3972a00640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 0x7f3954069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:28.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.154+0000 7f3972a00640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 0x7f3954069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39740b2430 tx=0x7f3960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:28.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.158+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3964022070 con 0x7f397406ecd0 2024-05-08T04:39:28.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39740cd750 con 0x7f397406ecd0 2024-05-08T04:39:28.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f396b400640 1 -- 172.21.15.164:0/3006352050 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3964010040 con 0x7f397406ecd0 2024-05-08T04:39:28.354 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 msgr2=0x7f3954069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 0x7f3954069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39740b2430 tx=0x7f3960009040 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 msgr2=0x7f3974141d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f396400efc0 tx=0x7f396400c490 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 shutdown_connections 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3954066f90 0x7f3954069450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3974155170 0x7f3974159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39740d07d0 0x7f3974131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 --2- 172.21.15.164:0/3006352050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397406ecd0 0x7f3974141d70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 >> 172.21.15.164:0/3006352050 conn(0x7f3974066fa0 msgr2=0x7f397406b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 shutdown_connections 2024-05-08T04:39:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:28.350+0000 7f397a200640 1 -- 172.21.15.164:0/3006352050 wait complete. 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:39:28.375 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:39:28.376 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:39:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.376 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:39:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-05-08T04:39:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:39:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:28.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.388 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:39:28.449 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:39:28.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.452 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:39:28.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:39:28.508 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:39:28.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.596 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:39:28.605 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:39:28.605 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:39:28.605 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:39:28.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.605 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:39:28.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:39:28.671 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:39:28.723 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:39:28.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:39:28.724 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:39:28.775 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:39:28.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:39:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:39:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:39:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:39:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:39:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:39:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:39:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:39:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:39:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:39:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:39:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:39:28.893 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:39:28.893 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:39:28.893 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:39:28.893 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:39:28.893 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:39:28.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:39:28.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:39:28.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:28.963 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:39:29.024 INFO:teuthology.orchestra.run.smithi164.stdout:client4907 2024-05-08T04:39:29.024 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:29.025 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:29.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:39:29.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 Processor -- start 2024-05-08T04:39:29.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 -- start start 2024-05-08T04:39:29.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1000cfda0 0x7fd100160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd10015be50 0x7fd1000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1000cd9f0 con 0x7fd10015ba70 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10015fe70 con 0x7fd10015be50 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.254+0000 7fd107a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cdcd0 con 0x7fd1000cfda0 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd10015be50 0x7fd1000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd10015be50 0x7fd1000b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43392/0 (socket says 172.21.15.164:43392) 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 -- 172.21.15.164:0/48222040 learned_addr learned my addr 172.21.15.164:0/48222040 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd0ffe00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1000cfda0 0x7fd100160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 -- 172.21.15.164:0/48222040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1000cfda0 msgr2=0x7fd100160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1000cfda0 0x7fd100160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 -- 172.21.15.164:0/48222040 >> v1:172.21.15.172:6789/0 conn(0x7fd10015ba70 legacy=0x7fd1000cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 -- 172.21.15.164:0/48222040 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10015f820 con 0x7fd10015be50 2024-05-08T04:39:29.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd105a00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015be50 0x7fd1000b31a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009870 tx=0x7fd0f801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588dc64e2576dfa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:29.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd0ff400640 1 -- 172.21.15.164:0/48222040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80ab640 con 0x7fd10015be50 2024-05-08T04:39:29.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd0ff400640 1 -- 172.21.15.164:0/48222040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80b0070 con 0x7fd10015be50 2024-05-08T04:39:29.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.258+0000 7fd0ff400640 1 -- 172.21.15.164:0/48222040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80aba00 con 0x7fd10015be50 2024-05-08T04:39:29.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- 172.21.15.164:0/48222040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015be50 msgr2=0x7fd1000b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015be50 0x7fd1000b31a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009870 tx=0x7fd0f801c530 comp rx=0 tx=0).stop 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- 172.21.15.164:0/48222040 shutdown_connections 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015be50 0x7fd1000b31a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- 172.21.15.164:0/48222040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1000cfda0 0x7fd100160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- 172.21.15.164:0/48222040 >> 172.21.15.164:0/48222040 conn(0x7fd10006ac90 msgr2=0x7fd10006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- 172.21.15.164:0/48222040 shutdown_connections 2024-05-08T04:39:29.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- 172.21.15.164:0/48222040 wait complete. 2024-05-08T04:39:29.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 Processor -- start 2024-05-08T04:39:29.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- start start 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 0x7fd100147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 0x7fd100148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100168d40 con 0x7fd10015ba70 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100168bc0 con 0x7fd1000cfda0 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd107a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd100168ec0 con 0x7fd10015be50 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43394/0 (socket says 172.21.15.164:43394) 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 -- 172.21.15.164:0/511689485 learned_addr learned my addr 172.21.15.164:0/511689485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd105000640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 0x7fd100147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd105a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 0x7fd100148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 -- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 msgr2=0x7fd100148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 0x7fd100148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 -- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 msgr2=0x7fd100147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 0x7fd100147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 -- 172.21.15.164:0/511689485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10015c640 con 0x7fd10015ba70 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd105000640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 0x7fd100147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd105a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 0x7fd100148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.262+0000 7fd0ffe00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d7d0 tx=0x7fd0ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00b6c0 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec019070 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00baa0 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10015c920 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1001f8b40 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd100158770 con 0x7fd10015ba70 2024-05-08T04:39:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0ec00bc40 con 0x7fd10015ba70 2024-05-08T04:39:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 0x7fd0e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec085480 con 0x7fd10015ba70 2024-05-08T04:39:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.266+0000 7fd105000640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 0x7fd0e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:29.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.270+0000 7fd105000640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 0x7fd0e0069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007fd0 tx=0x7fd0f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.270+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd100158770 con 0x7fd10015ba70 2024-05-08T04:39:29.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd10006cb10 con 0x7fd10015ba70 2024-05-08T04:39:29.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd0fcc00640 1 -- 172.21.15.164:0/511689485 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd0ec056480 con 0x7fd10015ba70 2024-05-08T04:39:29.467 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 msgr2=0x7fd0e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 0x7fd0e0069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007fd0 tx=0x7fd0f0009040 comp rx=0 tx=0).stop 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 msgr2=0x7fd100150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d7d0 tx=0x7fd0ec00dc90 comp rx=0 tx=0).stop 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 shutdown_connections 2024-05-08T04:39:29.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd0e0066f90 0x7fd0e0069450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd10015be50 0x7fd100148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd10015ba70 0x7fd100150b40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 --2- 172.21.15.164:0/511689485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1000cfda0 0x7fd100147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 >> 172.21.15.164:0/511689485 conn(0x7fd10006ac90 msgr2=0x7fd1001553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 shutdown_connections 2024-05-08T04:39:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:29.462+0000 7fd107a00640 1 -- 172.21.15.164:0/511689485 wait complete. 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:39:29.488 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:39:29.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.489 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:39:29.498 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:39:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:39:29.498 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:39:29.499 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42302sec preferred_lft 42302sec 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.500 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:39:29.501 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:39:29.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.502 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:39:29.566 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:39:29.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.569 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:39:29.623 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:39:29.623 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:39:29.623 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:39:29.623 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:39:29.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.624 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:39:29.624 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:39:29.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:39:29.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:39:29.625 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:39:29.625 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:39:29.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.719 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:39:29.729 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:39:29.729 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:39:29.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:39:29.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.729 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:39:29.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:39:29.791 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:39:29.846 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:39:29.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:39:29.847 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:39:29.902 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:39:29.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:29.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:39:29.997 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:39:29.997 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:39:29.997 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:39:30.014 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:39:30.014 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:39:30.014 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:39:30.014 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:39:30.014 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:39:30.023 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:39:30.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:39:30.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:30.027 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:39:30.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:30.048 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:39:30.100 INFO:teuthology.orchestra.run.smithi172.stdout:client4913 2024-05-08T04:39:30.101 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:30.101 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:30.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:39:30.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 Processor -- start 2024-05-08T04:39:30.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 -- start start 2024-05-08T04:39:30.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ce10 0x7fdbf406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdbf406e390 0x7fdbf40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf40c6600 con 0x7fdbf415ba20 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf415fbe0 con 0x7fdbf406e390 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbfa400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40c6040 con 0x7fdbf415ce10 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdbf406e390 0x7fdbf40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdbf415ba20 0x7fdbf4160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44922/0 (socket says 172.21.15.164:44922) 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3400640 1 -- 172.21.15.164:0/1534236317 learned_addr learned my addr 172.21.15.164:0/1534236317 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:30.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf2a00640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ce10 0x7fdbf406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3e00640 1 -- 172.21.15.164:0/1534236317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ce10 msgr2=0x7fdbf406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:30.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3e00640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ce10 0x7fdbf406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3e00640 1 -- 172.21.15.164:0/1534236317 >> v1:172.21.15.164:6789/0 conn(0x7fdbf415ba20 legacy=0x7fdbf4160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:30.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf3e00640 1 -- 172.21.15.164:0/1534236317 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40c6320 con 0x7fdbf406e390 2024-05-08T04:39:30.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.302+0000 7fdbf2000640 1 -- 172.21.15.164:0/1534236317 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226657955 0 0) 0x7fdbf40c6600 con 0x7fdbf415ba20 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbf3e00640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf40ae6e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009d90 tx=0x7fdbec01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592613dac12c3059 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbf2000640 1 -- 172.21.15.164:0/1534236317 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0b2070 con 0x7fdbf406e390 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbf2000640 1 -- 172.21.15.164:0/1534236317 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec0a57a0 con 0x7fdbf406e390 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbf2000640 1 -- 172.21.15.164:0/1534236317 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0a5b70 con 0x7fdbf406e390 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 -- 172.21.15.164:0/1534236317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 msgr2=0x7fdbf40ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf40ae6e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009d90 tx=0x7fdbec01c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 -- 172.21.15.164:0/1534236317 shutdown_connections 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf40ae6e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 --2- 172.21.15.164:0/1534236317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ce10 0x7fdbf406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 -- 172.21.15.164:0/1534236317 >> 172.21.15.164:0/1534236317 conn(0x7fdbf40cf770 msgr2=0x7fdbf40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:30.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 -- 172.21.15.164:0/1534236317 shutdown_connections 2024-05-08T04:39:30.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.306+0000 7fdbfa400640 1 -- 172.21.15.164:0/1534236317 wait complete. 2024-05-08T04:39:30.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 Processor -- start 2024-05-08T04:39:30.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- start start 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ba20 0x7fdbf4143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf415ce10 0x7fdbf40712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4168b70 con 0x7fdbf406e390 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40ce090 con 0x7fdbf415ce10 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4168cf0 con 0x7fdbf415ba20 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43418/0 (socket says 172.21.15.164:43418) 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 -- 172.21.15.164:0/2927643348 learned_addr learned my addr 172.21.15.164:0/2927643348 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf2a00640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ba20 0x7fdbf4143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3e00640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf415ce10 0x7fdbf40712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 -- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ba20 msgr2=0x7fdbf4143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ba20 0x7fdbf4143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 -- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf415ce10 msgr2=0x7fdbf40712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf415ce10 0x7fdbf40712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 -- 172.21.15.164:0/2927643348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4071d80 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbf3400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40011f0 tx=0x7fdbe400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4018070 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40074e0 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004a10 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf4157a40 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf4158000 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.310+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf4158440 con 0x7fdbf406e390 2024-05-08T04:39:30.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.314+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbe401b070 con 0x7fdbf406e390 2024-05-08T04:39:30.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.314+0000 7fdbeb400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:30.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.314+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbe4084340 con 0x7fdbf406e390 2024-05-08T04:39:30.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.314+0000 7fdbf2a00640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:30.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.314+0000 7fdbf2a00640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 0x7fdbd4069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40b0690 tx=0x7fdbe0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:30.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.318+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbe4054c60 con 0x7fdbf406e390 2024-05-08T04:39:30.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:30.506+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdbf404f010 con 0x7fdbf406e390 2024-05-08T04:39:31.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.194+0000 7fdbeb400640 1 -- 172.21.15.164:0/2927643348 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdbe4058d00 con 0x7fdbf406e390 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 msgr2=0x7fdbd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 0x7fdbd4069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40b0690 tx=0x7fdbe0009190 comp rx=0 tx=0).stop 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 msgr2=0x7fdbf4158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40011f0 tx=0x7fdbe400c9b0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 shutdown_connections 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbf415ba20 0x7fdbf4143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf415ce10 0x7fdbf40712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 --2- 172.21.15.164:0/2927643348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbf406e390 0x7fdbf4158800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 >> 172.21.15.164:0/2927643348 conn(0x7fdbf40cf770 msgr2=0x7fdbf40cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:31.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 shutdown_connections 2024-05-08T04:39:31.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.198+0000 7fdbfa400640 1 -- 172.21.15.164:0/2927643348 wait complete. 2024-05-08T04:39:31.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:39:31.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 Processor -- start 2024-05-08T04:39:31.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 -- start start 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa25015ab60 0x7fa250072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 0x7fa2500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa250072640 con 0x7fa2500cfbd0 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500d0060 con 0x7fa25015ab60 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa258600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006c770 con 0x7fa2500b26c0 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa256600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa2500cfbd0 0x7fa2500cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44936/0 (socket says 172.21.15.164:44936) 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa256600640 1 -- 172.21.15.164:0/1702197692 learned_addr learned my addr 172.21.15.164:0/1702197692 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa255200640 1 --2- 172.21.15.164:0/1702197692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 0x7fa2500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa255c00640 1 --2- 172.21.15.164:0/1702197692 >> v2:172.21.15.172:3300/0 conn(0x7fa25015ab60 0x7fa250072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619932630 0 0) 0x7fa250072640 con 0x7fa2500cfbd0 2024-05-08T04:39:31.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.362+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa238003530 con 0x7fa2500cfbd0 2024-05-08T04:39:31.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3192426149 0 0) 0x7fa238003530 con 0x7fa2500cfbd0 2024-05-08T04:39:31.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa255200640 1 -- 172.21.15.164:0/1702197692 >> v2:172.21.15.172:3300/0 conn(0x7fa25015ab60 msgr2=0x7fa250072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:31.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa255200640 1 --2- 172.21.15.164:0/1702197692 >> v2:172.21.15.172:3300/0 conn(0x7fa25015ab60 0x7fa250072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa255200640 1 -- 172.21.15.164:0/1702197692 >> v1:172.21.15.164:6789/0 conn(0x7fa2500cfbd0 legacy=0x7fa2500cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa255200640 1 -- 172.21.15.164:0/1702197692 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25006ca50 con 0x7fa2500b26c0 2024-05-08T04:39:31.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa255200640 1 --2- 172.21.15.164:0/1702197692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 0x7fa2500cf690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00ba60 tx=0x7fa23c01ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebcfc82a7d815fce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0b4070 con 0x7fa2500b26c0 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c0a76a0 con 0x7fa2500b26c0 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.366+0000 7fa24fe00640 1 -- 172.21.15.164:0/1702197692 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248001a00 con 0x7fa2500b26c0 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- 172.21.15.164:0/1702197692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 msgr2=0x7fa2500cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- 172.21.15.164:0/1702197692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 0x7fa2500cf690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00ba60 tx=0x7fa23c01ea90 comp rx=0 tx=0).stop 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- 172.21.15.164:0/1702197692 shutdown_connections 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- 172.21.15.164:0/1702197692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500b26c0 0x7fa2500cf690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- 172.21.15.164:0/1702197692 >> v2:172.21.15.172:3300/0 conn(0x7fa25015ab60 0x7fa250072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- 172.21.15.164:0/1702197692 >> 172.21.15.164:0/1702197692 conn(0x7fa25006ac90 msgr2=0x7fa25006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- 172.21.15.164:0/1702197692 shutdown_connections 2024-05-08T04:39:31.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- 172.21.15.164:0/1702197692 wait complete. 2024-05-08T04:39:31.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 Processor -- start 2024-05-08T04:39:31.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- start start 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2500b26c0 0x7fa25013c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 0x7fa250159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006c550 con 0x7fa25015ab60 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006c3d0 con 0x7fa2500b26c0 2024-05-08T04:39:31.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa258600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500aed30 con 0x7fa2500cfbd0 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43428/0 (socket says 172.21.15.164:43428) 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 -- 172.21.15.164:0/1986051776 learned_addr learned my addr 172.21.15.164:0/1986051776 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa255200640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 0x7fa250159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa255c00640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2500b26c0 0x7fa25013c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 -- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 msgr2=0x7fa250159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 0x7fa250159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 -- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2500b26c0 msgr2=0x7fa25013c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2500b26c0 0x7fa25013c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 -- 172.21.15.164:0/1986051776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25013c6c0 con 0x7fa25015ab60 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa255200640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 0x7fa250159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.370+0000 7fa256600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2480018a0 tx=0x7fa24800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248012400 con 0x7fa25015ab60 2024-05-08T04:39:31.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2480125c0 con 0x7fa25015ab60 2024-05-08T04:39:31.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2480129b0 con 0x7fa25015ab60 2024-05-08T04:39:31.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25015a290 con 0x7fa25015ab60 2024-05-08T04:39:31.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa250159270 con 0x7fa25015ab60 2024-05-08T04:39:31.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa250069d60 con 0x7fa25015ab60 2024-05-08T04:39:31.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa248012b50 con 0x7fa25015ab60 2024-05-08T04:39:31.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 0x7fa230069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2480858a0 con 0x7fa25015ab60 2024-05-08T04:39:31.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.374+0000 7fa255c00640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 0x7fa230069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.378+0000 7fa255c00640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 0x7fa230069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa244009a50 tx=0x7fa244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.378+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2480568a0 con 0x7fa25015ab60 2024-05-08T04:39:31.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.554+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2501549a0 con 0x7fa230066f90 2024-05-08T04:39:31.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa24d600640 1 -- 172.21.15.164:0/1986051776 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa2501549a0 con 0x7fa230066f90 2024-05-08T04:39:31.560 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:39:31.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:31.561 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:39:31.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:31.561 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 msgr2=0x7fa230069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 0x7fa230069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa244009a50 tx=0x7fa244008040 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 msgr2=0x7fa2500cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2480018a0 tx=0x7fa24800a430 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 shutdown_connections 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa230066f90 0x7fa230069450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2500cfbd0 0x7fa250159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa25015ab60 0x7fa2500cc5b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 --2- 172.21.15.164:0/1986051776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2500b26c0 0x7fa25013c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 >> 172.21.15.164:0/1986051776 conn(0x7fa25006ac90 msgr2=0x7fa2501509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:31.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 shutdown_connections 2024-05-08T04:39:31.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.558+0000 7fa258600640 1 -- 172.21.15.164:0/1986051776 wait complete. 2024-05-08T04:39:31.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:39:31.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 Processor -- start 2024-05-08T04:39:31.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 -- start start 2024-05-08T04:39:31.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6e815bbf0 0x7fa6e80cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e806f100 0x7fa6e80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e815bfd0 con 0x7fa6e80d0280 2024-05-08T04:39:31.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e80d06e0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.718+0000 7fa6ef800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80b0ef0 con 0x7fa6e806f100 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6e815bbf0 0x7fa6e80cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6e815bbf0 0x7fa6e80cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43438/0 (socket says 172.21.15.164:43438) 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 -- 172.21.15.164:0/283844223 learned_addr learned my addr 172.21.15.164:0/283844223 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ed800640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e806f100 0x7fa6e80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 -- 172.21.15.164:0/283844223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e806f100 msgr2=0x7fa6e80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e806f100 0x7fa6e80ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 -- 172.21.15.164:0/283844223 >> v1:172.21.15.172:6789/0 conn(0x7fa6e80d0280 legacy=0x7fa6e806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 -- 172.21.15.164:0/283844223 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e806f5d0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6ece00640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e80cfd40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009870 tx=0x7fa6d801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b36461603cb1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6e7e00640 1 -- 172.21.15.164:0/283844223 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80b1070 con 0x7fa6e815bbf0 2024-05-08T04:39:31.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6e7e00640 1 -- 172.21.15.164:0/283844223 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d80ac620 con 0x7fa6e815bbf0 2024-05-08T04:39:31.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.722+0000 7fa6e7e00640 1 -- 172.21.15.164:0/283844223 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80ac9f0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- 172.21.15.164:0/283844223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 msgr2=0x7fa6e80cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e80cfd40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009870 tx=0x7fa6d801c4d0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- 172.21.15.164:0/283844223 shutdown_connections 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e806f100 0x7fa6e80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- 172.21.15.164:0/283844223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e80cfd40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- 172.21.15.164:0/283844223 >> 172.21.15.164:0/283844223 conn(0x7fa6e809aa20 msgr2=0x7fa6e809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- 172.21.15.164:0/283844223 shutdown_connections 2024-05-08T04:39:31.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- 172.21.15.164:0/283844223 wait complete. 2024-05-08T04:39:31.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 Processor -- start 2024-05-08T04:39:31.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- start start 2024-05-08T04:39:31.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 0x7fa6e815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e80d0280 0x7fa6e8155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80af530 con 0x7fa6e815bbf0 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80af3b0 con 0x7fa6e806f100 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ef800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80af6b0 con 0x7fa6e80d0280 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ed800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ece00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 0x7fa6e815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ed800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43454/0 (socket says 172.21.15.164:43454) 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6ed800640 1 -- 172.21.15.164:0/36637457 learned_addr learned my addr 172.21.15.164:0/36637457 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.726+0000 7fa6dfe00640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e80d0280 0x7fa6e8155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 -- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e80d0280 msgr2=0x7fa6e8155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e80d0280 0x7fa6e8155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 -- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 msgr2=0x7fa6e815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 0x7fa6e815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 -- 172.21.15.164:0/36637457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80b4ed0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ece00640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 0x7fa6e815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ed800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0007e40 tx=0x7fa6e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0014620 con 0x7fa6e815bbf0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e0019070 con 0x7fa6e815bbf0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e814ce40 con 0x7fa6e815bbf0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0014a20 con 0x7fa6e815bbf0 2024-05-08T04:39:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80b6fd0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e809f9e0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6e0014bc0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 0x7fa6c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6e0085930 con 0x7fa6e815bbf0 2024-05-08T04:39:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.730+0000 7fa6ece00640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 0x7fa6c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.734+0000 7fa6ece00640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 0x7fa6c4069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80ae000 tx=0x7fa6d80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.734+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6e814ce40 con 0x7fa6e815bbf0 2024-05-08T04:39:31.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa6e806e7d0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6e5600640 1 -- 172.21.15.164:0/36637457 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa6e80b4ed0 con 0x7fa6e815bbf0 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 msgr2=0x7fa6c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 0x7fa6c4069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80ae000 tx=0x7fa6d80a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 msgr2=0x7fa6e8155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0007e40 tx=0x7fa6e000e4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 shutdown_connections 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6c4066f60 0x7fa6c4069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e80d0280 0x7fa6e8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6e815bbf0 0x7fa6e8155a40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 --2- 172.21.15.164:0/36637457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e806f100 0x7fa6e815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:31.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 >> 172.21.15.164:0/36637457 conn(0x7fa6e809aa20 msgr2=0x7fa6e8073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:31.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 shutdown_connections 2024-05-08T04:39:31.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:31.894+0000 7fa6ef800640 1 -- 172.21.15.164:0/36637457 wait complete. 2024-05-08T04:39:31.918 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp29280 2024-05-08T04:39:32.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 Processor -- start 2024-05-08T04:39:32.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 -- start start 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a880af6d0 0x7f3a880b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a8806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8815e300 con 0x7f3a880b4dc0 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8815e570 con 0x7f3a880af6d0 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a8d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8815e850 con 0x7f3a880adfa0 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a7fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3a880b4dc0 0x7f3a880ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44940/0 (socket says 172.21.15.164:44940) 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.054+0000 7f3a7fe00640 1 -- 172.21.15.164:0/4053430626 learned_addr learned my addr 172.21.15.164:0/4053430626 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 --2- 172.21.15.164:0/4053430626 >> v2:172.21.15.172:3300/0 conn(0x7f3a880af6d0 0x7f3a880b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a87400640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a8806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86000640 1 -- 172.21.15.164:0/4053430626 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824149872 0 0) 0x7f3a8815e300 con 0x7f3a880b4dc0 2024-05-08T04:39:32.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86000640 1 -- 172.21.15.164:0/4053430626 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a6c003530 con 0x7f3a880b4dc0 2024-05-08T04:39:32.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 -- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 msgr2=0x7f3a8806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:32.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a8806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 -- 172.21.15.164:0/4053430626 >> v1:172.21.15.164:6789/0 conn(0x7f3a880b4dc0 legacy=0x7f3a880ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 -- 172.21.15.164:0/4053430626 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880b5220 con 0x7f3a880af6d0 2024-05-08T04:39:32.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a87400640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a8806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:32.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86a00640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880af6d0 0x7f3a880b2ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097c0 tx=0x7f3a7801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1303e96cff9495aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.058+0000 7f3a86000640 1 -- 172.21.15.164:0/4053430626 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780b0070 con 0x7f3a880af6d0 2024-05-08T04:39:32.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a86000640 1 -- 172.21.15.164:0/4053430626 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a780ab600 con 0x7f3a880af6d0 2024-05-08T04:39:32.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a86000640 1 -- 172.21.15.164:0/4053430626 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780ab9b0 con 0x7f3a880af6d0 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880af6d0 msgr2=0x7f3a880b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880af6d0 0x7f3a880b2ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097c0 tx=0x7f3a7801c410 comp rx=0 tx=0).stop 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- 172.21.15.164:0/4053430626 shutdown_connections 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a8806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- 172.21.15.164:0/4053430626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880af6d0 0x7f3a880b2ff0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- 172.21.15.164:0/4053430626 >> 172.21.15.164:0/4053430626 conn(0x7f3a8806cca0 msgr2=0x7f3a8806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- 172.21.15.164:0/4053430626 shutdown_connections 2024-05-08T04:39:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- 172.21.15.164:0/4053430626 wait complete. 2024-05-08T04:39:32.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 Processor -- start 2024-05-08T04:39:32.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- start start 2024-05-08T04:39:32.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a880b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 0x7f3a88145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8816a950 con 0x7f3a880af6d0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8806bc30 con 0x7f3a880b4dc0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a8d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8816aad0 con 0x7f3a880adfa0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43468/0 (socket says 172.21.15.164:43468) 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 -- 172.21.15.164:0/1905095585 learned_addr learned my addr 172.21.15.164:0/1905095585 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a87400640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 0x7f3a88145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a86a00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a880b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 -- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 msgr2=0x7f3a880b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a880b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 -- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 msgr2=0x7f3a88145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 0x7f3a88145600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a7fe00640 1 -- 172.21.15.164:0/1905095585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88146150 con 0x7f3a880af6d0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.062+0000 7f3a87400640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 0x7f3a88145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7fe00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c450 tx=0x7f3a7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70017070 con 0x7f3a880af6d0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70005cf0 con 0x7f3a880af6d0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70007290 con 0x7f3a880af6d0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a881446a0 con 0x7f3a880af6d0 2024-05-08T04:39:32.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a88144b20 con 0x7f3a880af6d0 2024-05-08T04:39:32.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8806ddf0 con 0x7f3a880af6d0 2024-05-08T04:39:32.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a7001a070 con 0x7f3a880af6d0 2024-05-08T04:39:32.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 0x7f3a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a70084330 con 0x7f3a880af6d0 2024-05-08T04:39:32.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.066+0000 7f3a86a00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 0x7f3a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.070+0000 7f3a86a00640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 0x7f3a64069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a780ad000 tx=0x7f3a780a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.070+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a70054ce0 con 0x7f3a880af6d0 2024-05-08T04:39:32.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.246+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp29280", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a8804f010 con 0x7f3a64066f60 2024-05-08T04:39:32.257 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.254+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:39:32.257 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.254+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:39:32.257 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.254+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:39:32.257 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.254+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:39:32.257 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.254+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:39:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.270+0000 7f3a7ea00640 1 -- 172.21.15.164:0/1905095585 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3a8804f010 con 0x7f3a64066f60 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 msgr2=0x7f3a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 0x7f3a64069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a780ad000 tx=0x7f3a780a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 msgr2=0x7f3a88144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c450 tx=0x7f3a7000c910 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 shutdown_connections 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a64066f60 0x7f3a64069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a880b4dc0 0x7f3a88145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a880af6d0 0x7f3a88144160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 --2- 172.21.15.164:0/1905095585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a880adfa0 0x7f3a880b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 >> 172.21.15.164:0/1905095585 conn(0x7f3a8806cca0 msgr2=0x7f3a8815be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 shutdown_connections 2024-05-08T04:39:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.274+0000 7f3a8d600640 1 -- 172.21.15.164:0/1905095585 wait complete. 2024-05-08T04:39:32.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp29280 subvol_snap59320 2024-05-08T04:39:32.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 Processor -- start 2024-05-08T04:39:32.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 -- start start 2024-05-08T04:39:32.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01a415a200 0x7f01a40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01a40ca850 0x7f01a40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a406d150 con 0x7f01a406dfa0 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a406bca0 con 0x7f01a40ca850 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01ab600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40af150 con 0x7f01a415a200 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01a40ca850 0x7f01a40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01a40ca850 0x7f01a40cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43488/0 (socket says 172.21.15.164:43488) 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 -- 172.21.15.164:0/1752317243 learned_addr learned my addr 172.21.15.164:0/1752317243 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a8c00640 1 --2- 172.21.15.164:0/1752317243 >> v2:172.21.15.172:3300/0 conn(0x7f01a415a200 0x7f01a40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 -- 172.21.15.164:0/1752317243 >> v2:172.21.15.172:3300/0 conn(0x7f01a415a200 msgr2=0x7f01a40aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 --2- 172.21.15.164:0/1752317243 >> v2:172.21.15.172:3300/0 conn(0x7f01a415a200 0x7f01a40aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 -- 172.21.15.164:0/1752317243 >> v1:172.21.15.172:6789/0 conn(0x7f01a406dfa0 legacy=0x7f01a40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.434+0000 7f01a9600640 1 -- 172.21.15.164:0/1752317243 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a406d430 con 0x7f01a40ca850 2024-05-08T04:39:32.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01a9600640 1 --2- 172.21.15.164:0/1752317243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a40ca850 0x7f01a40cd670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cc150 tx=0x7f019c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba3dabed652b8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01a3400640 1 -- 172.21.15.164:0/1752317243 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0a5660 con 0x7f01a40ca850 2024-05-08T04:39:32.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01a3400640 1 -- 172.21.15.164:0/1752317243 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c0a5800 con 0x7f01a40ca850 2024-05-08T04:39:32.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01a3400640 1 -- 172.21.15.164:0/1752317243 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0a5bd0 con 0x7f01a40ca850 2024-05-08T04:39:32.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 -- 172.21.15.164:0/1752317243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a40ca850 msgr2=0x7f01a40cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 --2- 172.21.15.164:0/1752317243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a40ca850 0x7f01a40cd670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cc150 tx=0x7f019c01c930 comp rx=0 tx=0).stop 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 -- 172.21.15.164:0/1752317243 shutdown_connections 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 --2- 172.21.15.164:0/1752317243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a40ca850 0x7f01a40cd670 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 --2- 172.21.15.164:0/1752317243 >> v2:172.21.15.172:3300/0 conn(0x7f01a415a200 0x7f01a40aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 -- 172.21.15.164:0/1752317243 >> 172.21.15.164:0/1752317243 conn(0x7f01a40cf770 msgr2=0x7f01a40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 -- 172.21.15.164:0/1752317243 shutdown_connections 2024-05-08T04:39:32.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 -- 172.21.15.164:0/1752317243 wait complete. 2024-05-08T04:39:32.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.438+0000 7f01ab600640 1 Processor -- start 2024-05-08T04:39:32.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- start start 2024-05-08T04:39:32.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01a406dfa0 0x7f01a40c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a40ca850 0x7f01a40c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a415a200 0x7f01a4167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40cdd30 con 0x7f01a415a200 2024-05-08T04:39:32.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40cdbb0 con 0x7f01a40ca850 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40cdeb0 con 0x7f01a406dfa0 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a415a200 0x7f01a4167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a415a200 0x7f01a4167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43498/0 (socket says 172.21.15.164:43498) 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 -- 172.21.15.164:0/4206883184 learned_addr learned my addr 172.21.15.164:0/4206883184 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01a406dfa0 0x7f01a40c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a3e00640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a40ca850 0x7f01a40c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 -- 172.21.15.164:0/4206883184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01a406dfa0 msgr2=0x7f01a40c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01a406dfa0 0x7f01a40c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 -- 172.21.15.164:0/4206883184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a40ca850 msgr2=0x7f01a40c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a40ca850 0x7f01a40c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 -- 172.21.15.164:0/4206883184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a41521e0 con 0x7f01a415a200 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a9600640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01a415a200 0x7f01a4167a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f019c019540 tx=0x7f019c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0bc020 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c0b8670 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c009510 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- 172.21.15.164:0/4206883184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a41524c0 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- 172.21.15.164:0/4206883184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a4152a80 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.442+0000 7f01ab600640 1 -- 172.21.15.164:0/4206883184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4072d30 con 0x7f01a415a200 2024-05-08T04:39:32.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.446+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f019c0bf020 con 0x7f01a415a200 2024-05-08T04:39:32.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.446+0000 7f01a0c00640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0184066f60 0x7f0184069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.446+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f019c120090 con 0x7f01a415a200 2024-05-08T04:39:32.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.446+0000 7f01a8c00640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0184066f60 0x7f0184069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.446+0000 7f01a8c00640 1 --2- 172.21.15.164:0/4206883184 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0184066f60 0x7f0184069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01980079c0 tx=0x7f0198008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.450+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f019c0c6030 con 0x7f01a415a200 2024-05-08T04:39:32.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.630+0000 7f01ab600640 1 -- 172.21.15.164:0/4206883184 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp29280", "snap_name": "subvol_snap59320", "target": ["mon-mgr", ""]}) v1 -- 0x7f01a40af5b0 con 0x7f0184066f60 2024-05-08T04:39:32.634 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:39:32.630+0000 7ffa56200640 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2024-05-08T04:39:32.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.630+0000 7f01a0c00640 1 -- 172.21.15.164:0/4206883184 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7f01a40af5b0 con 0x7f0184066f60 2024-05-08T04:39:32.635 INFO:teuthology.orchestra.run.smithi164.stderr:Error ENOSYS: subvolume group snapshots are not supported 2024-05-08T04:39:32.644 DEBUG:teuthology.orchestra.run:got remote process result: 38 2024-05-08T04:39:32.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp29280 2024-05-08T04:39:32.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.778+0000 7f106b200640 1 Processor -- start 2024-05-08T04:39:32.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.778+0000 7f106b200640 1 -- start start 2024-05-08T04:39:32.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f106c06ac90 0x7f106c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f106c15e170 0x7f106c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c06b630 con 0x7f106c15e170 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c0cce80 con 0x7f106c06ac90 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0d05b0 con 0x7f106c0ccaa0 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f106c15e170 0x7f106c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f106c15e170 0x7f106c15e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43514/0 (socket says 172.21.15.164:43514) 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 -- 172.21.15.164:0/3932725993 learned_addr learned my addr 172.21.15.164:0/3932725993 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f1069e00640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f1061400640 1 --2- 172.21.15.164:0/3932725993 >> v2:172.21.15.172:3300/0 conn(0x7f106c06ac90 0x7f106c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 -- 172.21.15.164:0/3932725993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 msgr2=0x7f106c0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c0d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 -- 172.21.15.164:0/3932725993 >> v2:172.21.15.172:3300/0 conn(0x7f106c06ac90 msgr2=0x7f106c06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 --2- 172.21.15.164:0/3932725993 >> v2:172.21.15.172:3300/0 conn(0x7f106c06ac90 0x7f106c06b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.782+0000 7f106a800640 1 -- 172.21.15.164:0/3932725993 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c15ef10 con 0x7f106c15e170 2024-05-08T04:39:32.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106a800640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c15e580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f106c0d17b0 tx=0x7f106401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a803863c5ede2ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f1069400640 1 -- 172.21.15.164:0/3932725993 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10640a56f0 con 0x7f106c15e170 2024-05-08T04:39:32.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f1069400640 1 -- 172.21.15.164:0/3932725993 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10640a5890 con 0x7f106c15e170 2024-05-08T04:39:32.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f1069400640 1 -- 172.21.15.164:0/3932725993 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10640a5c60 con 0x7f106c15e170 2024-05-08T04:39:32.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- 172.21.15.164:0/3932725993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 msgr2=0x7f106c15e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c15e580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f106c0d17b0 tx=0x7f106401c960 comp rx=0 tx=0).stop 2024-05-08T04:39:32.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- 172.21.15.164:0/3932725993 shutdown_connections 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c15e580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- 172.21.15.164:0/3932725993 >> v2:172.21.15.172:3300/0 conn(0x7f106c06ac90 0x7f106c06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- 172.21.15.164:0/3932725993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c0d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- 172.21.15.164:0/3932725993 >> 172.21.15.164:0/3932725993 conn(0x7f106c066fa0 msgr2=0x7f106c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- 172.21.15.164:0/3932725993 shutdown_connections 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- 172.21.15.164:0/3932725993 wait complete. 2024-05-08T04:39:32.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 Processor -- start 2024-05-08T04:39:32.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- start start 2024-05-08T04:39:32.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f106c06ac90 0x7f106c15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0cf830 con 0x7f106c15e170 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0cf6b0 con 0x7f106c06ac90 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0cf9b0 con 0x7f106c0ccaa0 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43524/0 (socket says 172.21.15.164:43524) 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.786+0000 7f106a800640 1 -- 172.21.15.164:0/2276910969 learned_addr learned my addr 172.21.15.164:0/2276910969 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1069e00640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f106c06ac90 0x7f106c15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1061400640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 -- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 msgr2=0x7f106c159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 -- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f106c06ac90 msgr2=0x7f106c15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f106c06ac90 0x7f106c15b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 -- 172.21.15.164:0/2276910969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c153d00 con 0x7f106c15e170 2024-05-08T04:39:32.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106a800640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f106401d200 tx=0x7f1064019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10640bc020 con 0x7f106c15e170 2024-05-08T04:39:32.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10640b8690 con 0x7f106c15e170 2024-05-08T04:39:32.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064009510 con 0x7f106c15e170 2024-05-08T04:39:32.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c153fe0 con 0x7f106c15e170 2024-05-08T04:39:32.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c0b3800 con 0x7f106c15e170 2024-05-08T04:39:32.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c04ef30 con 0x7f106c15e170 2024-05-08T04:39:32.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.790+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10640bf020 con 0x7f106c15e170 2024-05-08T04:39:32.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.794+0000 7f1062a00640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 0x7f1048069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:32.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.794+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f106411fca0 con 0x7f106c15e170 2024-05-08T04:39:32.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.794+0000 7f1069e00640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 0x7f1048069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.794+0000 7f1069e00640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 0x7f1048069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f105c009ac0 tx=0x7f105c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:32.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.798+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10640c6030 con 0x7f106c15e170 2024-05-08T04:39:32.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.970+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp29280", "target": ["mon-mgr", ""]}) v1 -- 0x7f106c06e010 con 0x7f1048066f60 2024-05-08T04:39:32.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f1062a00640 1 -- 172.21.15.164:0/2276910969 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f106c06e010 con 0x7f1048066f60 2024-05-08T04:39:32.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 msgr2=0x7f1048069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 0x7f1048069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f105c009ac0 tx=0x7f105c009190 comp rx=0 tx=0).stop 2024-05-08T04:39:32.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 msgr2=0x7f106c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f106401d200 tx=0x7f1064019700 comp rx=0 tx=0).stop 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 shutdown_connections 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1048066f60 0x7f1048069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f106c0ccaa0 0x7f106c159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f106c15e170 0x7f106c0b3d00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 --2- 172.21.15.164:0/2276910969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f106c06ac90 0x7f106c15b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 >> 172.21.15.164:0/2276910969 conn(0x7f106c066fa0 msgr2=0x7f106c158900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 shutdown_connections 2024-05-08T04:39:32.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:32.978+0000 7f106b200640 1 -- 172.21.15.164:0/2276910969 wait complete. 2024-05-08T04:39:33.001 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:39:33.001 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:33.086 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:33.185 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:39:33.185 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:33.185 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:33.194 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:33.194 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:33.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:33.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 Processor -- start 2024-05-08T04:39:33.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 -- start start 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 0x7f4fd409ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40cc350 con 0x7f4fd40b06b0 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40d2110 con 0x7f4fd415cd80 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40d1ec0 con 0x7f4fd40ce820 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4fd40b06b0 0x7f4fd40ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44952/0 (socket says 172.21.15.164:44952) 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2a00640 1 -- 172.21.15.164:0/3204678086 learned_addr learned my addr 172.21.15.164:0/3204678086 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:33.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd3e00640 1 --2- 172.21.15.164:0/3204678086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 0x7f4fd409ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722510134 0 0) 0x7f4fd40cc350 con 0x7f4fd40b06b0 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8003530 con 0x7f4fd40b06b0 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355828590 0 0) 0x7f4fd40d2110 con 0x7f4fd415cd80 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd40cc350 con 0x7f4fd415cd80 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd3e00640 1 -- 172.21.15.164:0/3204678086 >> v1:172.21.15.172:6789/0 conn(0x7f4fd415cd80 legacy=0x7f4fd415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd3e00640 1 -- 172.21.15.164:0/3204678086 >> v1:172.21.15.164:6789/0 conn(0x7f4fd40b06b0 legacy=0x7f4fd40ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.426+0000 7f4fd3e00640 1 -- 172.21.15.164:0/3204678086 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40d23f0 con 0x7f4fd40ce820 2024-05-08T04:39:33.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd3e00640 1 --2- 172.21.15.164:0/3204678086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 0x7f4fd409ae30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009dc0 tx=0x7f4fcc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3146b6116dd681a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0b2070 con 0x7f4fd40ce820 2024-05-08T04:39:33.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc0a5960 con 0x7f4fd40ce820 2024-05-08T04:39:33.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd2000640 1 -- 172.21.15.164:0/3204678086 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0a5d30 con 0x7f4fd40ce820 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 -- 172.21.15.164:0/3204678086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 msgr2=0x7f4fd409ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/3204678086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 0x7f4fd409ae30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009dc0 tx=0x7f4fcc01c990 comp rx=0 tx=0).stop 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 -- 172.21.15.164:0/3204678086 shutdown_connections 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/3204678086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ce820 0x7f4fd409ae30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 -- 172.21.15.164:0/3204678086 >> 172.21.15.164:0/3204678086 conn(0x7f4fd409aa20 msgr2=0x7f4fd40d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 -- 172.21.15.164:0/3204678086 shutdown_connections 2024-05-08T04:39:33.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.430+0000 7f4fd8a00640 1 -- 172.21.15.164:0/3204678086 wait complete. 2024-05-08T04:39:33.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 Processor -- start 2024-05-08T04:39:33.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- start start 2024-05-08T04:39:33.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40b06b0 0x7f4fd4152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd40ce820 0x7f4fd4166bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40cc960 con 0x7f4fd415cd80 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40cd8b0 con 0x7f4fd40ce820 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4169bf0 con 0x7f4fd40b06b0 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43530/0 (socket says 172.21.15.164:43530) 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 -- 172.21.15.164:0/268259141 learned_addr learned my addr 172.21.15.164:0/268259141 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3400640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40b06b0 0x7f4fd4152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd2a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd40ce820 0x7f4fd4166bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 -- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40b06b0 msgr2=0x7f4fd4152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40b06b0 0x7f4fd4152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 -- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd40ce820 msgr2=0x7f4fd4166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd40ce820 0x7f4fd4166bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 -- 172.21.15.164:0/268259141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd406dba0 con 0x7f4fd415cd80 2024-05-08T04:39:33.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd3e00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc01ce70 tx=0x7f4fcc019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0bd020 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc0b8470 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0b2040 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd406d210 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd4153b50 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.434+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd404ef30 con 0x7f4fd415cd80 2024-05-08T04:39:33.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.438+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fcc0c0020 con 0x7f4fd415cd80 2024-05-08T04:39:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.438+0000 7f4fcb400640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 0x7f4fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.438+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fcc11fce0 con 0x7f4fd415cd80 2024-05-08T04:39:33.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.438+0000 7f4fd3400640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 0x7f4fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.438+0000 7f4fd3400640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 0x7f4fb4069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40011f0 tx=0x7f4fc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.442+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fcc0c7030 con 0x7f4fd415cd80 2024-05-08T04:39:33.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.598+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fd40cf000 con 0x7f4fd415cd80 2024-05-08T04:39:33.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fcb400640 1 -- 172.21.15.164:0/268259141 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f4fcc016580 con 0x7f4fd415cd80 2024-05-08T04:39:33.604 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:33.604 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:33.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 msgr2=0x7f4fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 0x7f4fb4069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40011f0 tx=0x7f4fc4009040 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 msgr2=0x7f4fd4153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc01ce70 tx=0x7f4fcc019730 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 shutdown_connections 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb4066f60 0x7f4fb4069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd415cd80 0x7f4fd4153610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd40ce820 0x7f4fd4166bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 --2- 172.21.15.164:0/268259141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40b06b0 0x7f4fd4152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 >> 172.21.15.164:0/268259141 conn(0x7f4fd409aa20 msgr2=0x7f4fd40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 shutdown_connections 2024-05-08T04:39:33.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.602+0000 7f4fd8a00640 1 -- 172.21.15.164:0/268259141 wait complete. 2024-05-08T04:39:33.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:20.033719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:20.517477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:33.627 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:33.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 Processor -- start 2024-05-08T04:39:33.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 -- start start 2024-05-08T04:39:33.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 0x7f3e0006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e0006e390 0x7f3e000ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e000c6600 con 0x7f3e0015ba20 2024-05-08T04:39:33.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0015fbe0 con 0x7f3e0006e390 2024-05-08T04:39:33.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3e05a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000c6040 con 0x7f3e0015ce10 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3dfe000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 0x7f3e0006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3dfea00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3e0015ba20 0x7f3e00160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:37014/0 (socket says 172.21.15.164:37014) 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3dfea00640 1 -- 172.21.15.164:0/1007962459 learned_addr learned my addr 172.21.15.164:0/1007962459 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.762+0000 7f3dff400640 1 --2- 172.21.15.164:0/1007962459 >> v2:172.21.15.172:3300/0 conn(0x7f3e0006e390 0x7f3e000ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dff400640 1 -- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 msgr2=0x7f3e0006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dff400640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 0x7f3e0006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dff400640 1 -- 172.21.15.164:0/1007962459 >> v1:172.21.15.172:6789/0 conn(0x7f3e0015ba20 legacy=0x7f3e00160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dff400640 1 -- 172.21.15.164:0/1007962459 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000c6320 con 0x7f3e0006e390 2024-05-08T04:39:33.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dfd600640 1 -- 172.21.15.164:0/1007962459 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249246015 0 0) 0x7f3e000c6600 con 0x7f3e0015ba20 2024-05-08T04:39:33.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dfe000640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 0x7f3e0006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:33.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dff400640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0006e390 0x7f3e000ae6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097c0 tx=0x7f3df801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769216e587f3d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dfd600640 1 -- 172.21.15.164:0/1007962459 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df80b0070 con 0x7f3e0006e390 2024-05-08T04:39:33.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dfd600640 1 -- 172.21.15.164:0/1007962459 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df80ab600 con 0x7f3e0006e390 2024-05-08T04:39:33.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3dfd600640 1 -- 172.21.15.164:0/1007962459 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df80ab9d0 con 0x7f3e0006e390 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 -- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0006e390 msgr2=0x7f3e000ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0006e390 0x7f3e000ae6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097c0 tx=0x7f3df801c410 comp rx=0 tx=0).stop 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 -- 172.21.15.164:0/1007962459 shutdown_connections 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0015ce10 0x7f3e0006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 --2- 172.21.15.164:0/1007962459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0006e390 0x7f3e000ae6e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.766+0000 7f3e05a00640 1 -- 172.21.15.164:0/1007962459 >> 172.21.15.164:0/1007962459 conn(0x7f3e000cf770 msgr2=0x7f3e000cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:33.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- 172.21.15.164:0/1007962459 shutdown_connections 2024-05-08T04:39:33.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- 172.21.15.164:0/1007962459 wait complete. 2024-05-08T04:39:33.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 Processor -- start 2024-05-08T04:39:33.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- start start 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0006e390 0x7f3e001f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0015ce10 0x7f3e00145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e00168b70 con 0x7f3e0015ba20 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e000ce090 con 0x7f3e0015ce10 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3e05a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e00168cf0 con 0x7f3e0006e390 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43536/0 (socket says 172.21.15.164:43536) 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 -- 172.21.15.164:0/351842968 learned_addr learned my addr 172.21.15.164:0/351842968 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dff400640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0015ce10 0x7f3e00145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfea00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0006e390 0x7f3e001f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 -- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0006e390 msgr2=0x7f3e001f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0006e390 0x7f3e001f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 -- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0015ce10 msgr2=0x7f3e00145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0015ce10 0x7f3e00145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 -- 172.21.15.164:0/351842968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e00150cf0 con 0x7f3e0015ba20 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.770+0000 7f3dfe000640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e930 tx=0x7f3dec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec017070 con 0x7f3e0015ba20 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec00ca00 con 0x7f3e0015ba20 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e000b2880 con 0x7f3e0015ba20 2024-05-08T04:39:33.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00cdd0 con 0x7f3e0015ba20 2024-05-08T04:39:33.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e00150350 con 0x7f3e0015ba20 2024-05-08T04:39:33.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e00072d30 con 0x7f3e0015ba20 2024-05-08T04:39:33.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3dec01a070 con 0x7f3e0015ba20 2024-05-08T04:39:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 0x7f3de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dec085230 con 0x7f3e0015ba20 2024-05-08T04:39:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3dfea00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 0x7f3de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:33.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.774+0000 7f3dfea00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 0x7f3de0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3df0004fd0 tx=0x7f3df0001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:33.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.778+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e00150cf0 con 0x7f3e0015ba20 2024-05-08T04:39:33.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e001540f0 con 0x7f3e0015ba20 2024-05-08T04:39:33.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3df6a00640 1 -- 172.21.15.164:0/351842968 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dec055b60 con 0x7f3e0015ba20 2024-05-08T04:39:33.937 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 msgr2=0x7f3de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 0x7f3de0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3df0004fd0 tx=0x7f3df0001770 comp rx=0 tx=0).stop 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 msgr2=0x7f3e000afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e930 tx=0x7f3dec00edf0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 shutdown_connections 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3de0066f60 0x7f3de0069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e0015ce10 0x7f3e00145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e0015ba20 0x7f3e000afea0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 --2- 172.21.15.164:0/351842968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e0006e390 0x7f3e001f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:33.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 >> 172.21.15.164:0/351842968 conn(0x7f3e000cf770 msgr2=0x7f3e0006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:33.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 shutdown_connections 2024-05-08T04:39:33.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:33.934+0000 7f3e05a00640 1 -- 172.21.15.164:0/351842968 wait complete. 2024-05-08T04:39:33.957 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:34.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 Processor -- start 2024-05-08T04:39:34.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 -- start start 2024-05-08T04:39:34.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05dc15ab10 0x7f05dc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 0x7f05dc0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc15e870 con 0x7f05dc0ae6e0 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc06bb80 con 0x7f05dc15ab10 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05e2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc15eb50 con 0x7f05dc0cee90 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05dc15ab10 0x7f05dc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05dc15ab10 0x7f05dc06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43552/0 (socket says 172.21.15.164:43552) 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 -- 172.21.15.164:0/3765615352 learned_addr learned my addr 172.21.15.164:0/3765615352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05dbe00640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 0x7f05dc0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 -- 172.21.15.164:0/3765615352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 msgr2=0x7f05dc0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 0x7f05dc0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 -- 172.21.15.164:0/3765615352 >> v1:172.21.15.172:6789/0 conn(0x7f05dc0ae6e0 legacy=0x7f05dc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05db400640 1 -- 172.21.15.164:0/3765615352 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc15ee30 con 0x7f05dc15ab10 2024-05-08T04:39:34.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.094+0000 7f05dbe00640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 0x7f05dc0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:34.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05db400640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc15ab10 0x7f05dc06cce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009870 tx=0x7f05cc01e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bc0fdc307be88d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05da000640 1 -- 172.21.15.164:0/3765615352 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0ae070 con 0x7f05dc15ab10 2024-05-08T04:39:34.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05da000640 1 -- 172.21.15.164:0/3765615352 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc0a8640 con 0x7f05dc15ab10 2024-05-08T04:39:34.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05da000640 1 -- 172.21.15.164:0/3765615352 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0a8a10 con 0x7f05dc15ab10 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 -- 172.21.15.164:0/3765615352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc15ab10 msgr2=0x7f05dc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc15ab10 0x7f05dc06cce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009870 tx=0x7f05cc01e6b0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 -- 172.21.15.164:0/3765615352 shutdown_connections 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0cee90 0x7f05dc0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 --2- 172.21.15.164:0/3765615352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc15ab10 0x7f05dc06cce0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 -- 172.21.15.164:0/3765615352 >> 172.21.15.164:0/3765615352 conn(0x7f05dc0cf770 msgr2=0x7f05dc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 -- 172.21.15.164:0/3765615352 shutdown_connections 2024-05-08T04:39:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 -- 172.21.15.164:0/3765615352 wait complete. 2024-05-08T04:39:34.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.098+0000 7f05e2400640 1 Processor -- start 2024-05-08T04:39:34.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- start start 2024-05-08T04:39:34.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0ae6e0 0x7f05dc14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05dc15ab10 0x7f05dc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0cb070 con 0x7f05dc0cee90 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0caef0 con 0x7f05dc15ab10 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0cb1f0 con 0x7f05dc0ae6e0 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43564/0 (socket says 172.21.15.164:43564) 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 -- 172.21.15.164:0/2096896724 learned_addr learned my addr 172.21.15.164:0/2096896724 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05dbe00640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05dc15ab10 0x7f05dc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05db400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0ae6e0 0x7f05dc14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 -- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0ae6e0 msgr2=0x7f05dc14af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0ae6e0 0x7f05dc14af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 -- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05dc15ab10 msgr2=0x7f05dc0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05dc15ab10 0x7f05dc0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 -- 172.21.15.164:0/2096896724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc15c910 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05daa00640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c420 tx=0x7f05c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80052d0 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8005470 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc15cbf0 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8007260 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc15d1b0 con 0x7f05dc0cee90 2024-05-08T04:39:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.102+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc072d30 con 0x7f05dc0cee90 2024-05-08T04:39:34.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.106+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05c8007400 con 0x7f05dc0cee90 2024-05-08T04:39:34.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.106+0000 7f05d3400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 0x7f05bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.106+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05c8084490 con 0x7f05dc0cee90 2024-05-08T04:39:34.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.106+0000 7f05db400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 0x7f05bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.106+0000 7f05db400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 0x7f05bc069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0b0000 tx=0x7f05cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.110+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05c8054e40 con 0x7f05dc0cee90 2024-05-08T04:39:34.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.214+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f05c80514b0 con 0x7f05dc0cee90 2024-05-08T04:39:34.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05dc15d380 con 0x7f05dc0cee90 2024-05-08T04:39:34.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05d3400640 1 -- 172.21.15.164:0/2096896724 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05c805b3d0 con 0x7f05dc0cee90 2024-05-08T04:39:34.274 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:34.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 msgr2=0x7f05bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 0x7f05bc069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0b0000 tx=0x7f05cc009040 comp rx=0 tx=0).stop 2024-05-08T04:39:34.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 msgr2=0x7f05dc15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c420 tx=0x7f05c800c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 shutdown_connections 2024-05-08T04:39:34.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05bc066f60 0x7f05bc069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05dc15ab10 0x7f05dc0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05dc0cee90 0x7f05dc15c220 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 --2- 172.21.15.164:0/2096896724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05dc0ae6e0 0x7f05dc14af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 >> 172.21.15.164:0/2096896724 conn(0x7f05dc0cf770 msgr2=0x7f05dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 shutdown_connections 2024-05-08T04:39:34.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.270+0000 7f05e2400640 1 -- 172.21.15.164:0/2096896724 wait complete. 2024-05-08T04:39:34.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:34.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 Processor -- start 2024-05-08T04:39:34.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 -- start start 2024-05-08T04:39:34.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7bc15ab10 0x7fd7bc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc15e870 con 0x7fd7bc0cee90 2024-05-08T04:39:34.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc15eb50 con 0x7fd7bc0ae6e0 2024-05-08T04:39:34.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc06bb80 con 0x7fd7bc15ab10 2024-05-08T04:39:34.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd7bc0cee90 0x7fd7bc0ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44962/0 (socket says 172.21.15.164:44962) 2024-05-08T04:39:34.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1e00640 1 -- 172.21.15.164:0/1563134494 learned_addr learned my addr 172.21.15.164:0/1563134494 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> v2:172.21.15.172:3300/0 conn(0x7fd7bc15ab10 0x7fd7bc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14279335 0 0) 0x7fd7bc15e870 con 0x7fd7bc0cee90 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003530 con 0x7fd7bc0cee90 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355027419 0 0) 0x7fd7bc15eb50 con 0x7fd7bc0ae6e0 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc15e870 con 0x7fd7bc0ae6e0 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 352927491 0 0) 0x7fd7a4003530 con 0x7fd7bc0cee90 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 >> v2:172.21.15.172:3300/0 conn(0x7fd7bc15ab10 msgr2=0x7fd7bc06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 --2- 172.21.15.164:0/1563134494 >> v2:172.21.15.172:3300/0 conn(0x7fd7bc15ab10 0x7fd7bc06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 >> v1:172.21.15.172:6789/0 conn(0x7fd7bc0ae6e0 legacy=0x7fd7bc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc15ee30 con 0x7fd7bc0cee90 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd7b4001a20 con 0x7fd7bc0cee90 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> v2:172.21.15.172:3300/0 conn(0x7fd7bc15ab10 0x7fd7bc06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 >> v1:172.21.15.164:6789/0 conn(0x7fd7bc0cee90 legacy=0x7fd7bc0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 0x7fd7a4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4004810 0x7fd7a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a40051d0 0x7fd7a400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc06bb80 con 0x7fd7a4003b50 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a400e310 con 0x7fd7a40051d0 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a400e510 con 0x7fd7a4004810 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 0x7fd7a4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1e00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a40051d0 0x7fd7a400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c0a00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4004810 0x7fd7a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.430+0000 7fd7c1400640 1 -- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4004810 msgr2=0x7fd7a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4004810 0x7fd7a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1400640 1 -- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a40051d0 msgr2=0x7fd7a400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a40051d0 0x7fd7a400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1400640 1 -- 172.21.15.164:0/1563134494 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4003530 con 0x7fd7a4003b50 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1e00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a40051d0 0x7fd7a400dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:34.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1e00640 1 -- 172.21.15.164:0/1563134494 reap_dead start 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c1400640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 0x7fd7a4003f50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009cb0 tx=0x7fd7b001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff25eb9c289f8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00b2070 con 0x7fd7a4003b50 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b00a5790 con 0x7fd7a4003b50 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7bbe00640 1 -- 172.21.15.164:0/1563134494 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0009880 con 0x7fd7a4003b50 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 -- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 msgr2=0x7fd7a4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 0x7fd7a4003f50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009cb0 tx=0x7fd7b001cbe0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 -- 172.21.15.164:0/1563134494 shutdown_connections 2024-05-08T04:39:34.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/1563134494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a4003b50 0x7fd7a4003f50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 -- 172.21.15.164:0/1563134494 >> 172.21.15.164:0/1563134494 conn(0x7fd7bc0cf770 msgr2=0x7fd7bc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 -- 172.21.15.164:0/1563134494 shutdown_connections 2024-05-08T04:39:34.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.434+0000 7fd7c3e00640 1 -- 172.21.15.164:0/1563134494 wait complete. 2024-05-08T04:39:34.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 Processor -- start 2024-05-08T04:39:34.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- start start 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 0x7fd7bc1fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 0x7fd7bc070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c5b70 con 0x7fd7bc15da20 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc15aef0 con 0x7fd7bc070920 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c5cf0 con 0x7fd7a4003b50 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43582/0 (socket says 172.21.15.164:43582) 2024-05-08T04:39:34.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 -- 172.21.15.164:0/2994807095 learned_addr learned my addr 172.21.15.164:0/2994807095 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c1400640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 0x7fd7bc1fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c1e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 0x7fd7bc070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 -- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 msgr2=0x7fd7bc1fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 0x7fd7bc1fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 -- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 msgr2=0x7fd7bc070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 0x7fd7bc070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 -- 172.21.15.164:0/2994807095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc154290 con 0x7fd7bc15da20 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c1400640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 0x7fd7bc1fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c1e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 0x7fd7bc070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c0a00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e580 tx=0x7fd7a800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80052e0 con 0x7fd7bc15da20 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8018070 con 0x7fd7bc15da20 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc15dea0 con 0x7fd7bc15da20 2024-05-08T04:39:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80054a0 con 0x7fd7bc15da20 2024-05-08T04:39:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.438+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc154810 con 0x7fd7bc15da20 2024-05-08T04:39:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc072d30 con 0x7fd7bc15da20 2024-05-08T04:39:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fd7a8003840 con 0x7fd7bc15da20 2024-05-08T04:39:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7b9600640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 0x7fd79c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a8084ef0 con 0x7fd7bc15da20 2024-05-08T04:39:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7c1400640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 0x7fd79c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.442+0000 7fd7c1400640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 0x7fd79c0694b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00af000 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.446+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a8055e20 con 0x7fd7bc15da20 2024-05-08T04:39:34.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7bc0c9b30 con 0x7fd7bc15da20 2024-05-08T04:39:34.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7b9600640 1 -- 172.21.15.164:0/2994807095 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7a8055720 con 0x7fd7bc15da20 2024-05-08T04:39:34.609 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 msgr2=0x7fd79c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 0x7fd79c0694b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00af000 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 msgr2=0x7fd7bc1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e580 tx=0x7fd7a800ea40 comp rx=0 tx=0).stop 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 shutdown_connections 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd79c066ff0 0x7fd79c0694b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7bc070920 0x7fd7bc070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7bc15da20 0x7fd7bc1539f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 --2- 172.21.15.164:0/2994807095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a4003b50 0x7fd7bc1fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 >> 172.21.15.164:0/2994807095 conn(0x7fd7bc0cf770 msgr2=0x7fd7bc0ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 shutdown_connections 2024-05-08T04:39:34.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.606+0000 7fd7c3e00640 1 -- 172.21.15.164:0/2994807095 wait complete. 2024-05-08T04:39:34.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 Processor -- start 2024-05-08T04:39:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 -- start start 2024-05-08T04:39:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f06e015be50 0x7f06e00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e00cd9f0 con 0x7f06e00aeda0 2024-05-08T04:39:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e015fe70 con 0x7f06e015be50 2024-05-08T04:39:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06e5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e00cdcd0 con 0x7f06e015ba70 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06d6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f06e015be50 0x7f06e00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06d6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f06e015be50 0x7f06e00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43596/0 (socket says 172.21.15.164:43596) 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.762+0000 7f06d6000640 1 -- 172.21.15.164:0/3431699624 learned_addr learned my addr 172.21.15.164:0/3431699624 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06dea00640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06d6000640 1 -- 172.21.15.164:0/3431699624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 msgr2=0x7f06e0160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06d6000640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e0160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06d6000640 1 -- 172.21.15.164:0/3431699624 >> v1:172.21.15.172:6789/0 conn(0x7f06e00aeda0 legacy=0x7f06e00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06d6000640 1 -- 172.21.15.164:0/3431699624 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e015f820 con 0x7f06e015be50 2024-05-08T04:39:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06d6000640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e015be50 0x7f06e00cf690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009870 tx=0x7f06c801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb444e8e35f4c009 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06de000640 1 -- 172.21.15.164:0/3431699624 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80ab640 con 0x7f06e015be50 2024-05-08T04:39:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06de000640 1 -- 172.21.15.164:0/3431699624 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c80b0070 con 0x7f06e015be50 2024-05-08T04:39:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.766+0000 7f06de000640 1 -- 172.21.15.164:0/3431699624 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80aba00 con 0x7f06e015be50 2024-05-08T04:39:34.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- 172.21.15.164:0/3431699624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e015be50 msgr2=0x7f06e00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e015be50 0x7f06e00cf690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009870 tx=0x7f06c801c530 comp rx=0 tx=0).stop 2024-05-08T04:39:34.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- 172.21.15.164:0/3431699624 shutdown_connections 2024-05-08T04:39:34.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e015be50 0x7f06e00cf690 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- 172.21.15.164:0/3431699624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e0160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- 172.21.15.164:0/3431699624 >> 172.21.15.164:0/3431699624 conn(0x7f06e006ac90 msgr2=0x7f06e006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- 172.21.15.164:0/3431699624 shutdown_connections 2024-05-08T04:39:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- 172.21.15.164:0/3431699624 wait complete. 2024-05-08T04:39:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 Processor -- start 2024-05-08T04:39:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- start start 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06e015be50 0x7f06e00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0168d40 con 0x7f06e00aeda0 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0168bc0 con 0x7f06e015be50 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06e5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0168ec0 con 0x7f06e015ba70 2024-05-08T04:39:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43606/0 (socket says 172.21.15.164:43606) 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 -- 172.21.15.164:0/3249231159 learned_addr learned my addr 172.21.15.164:0/3249231159 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06d6000640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06df400640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06e015be50 0x7f06e00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 -- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 msgr2=0x7f06e015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e015cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 -- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06e015be50 msgr2=0x7f06e00b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06e015be50 0x7f06e00b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 -- 172.21.15.164:0/3249231159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e015d770 con 0x7f06e00aeda0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.770+0000 7f06dea00640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d800 tx=0x7f06d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000b6c0 con 0x7f06e00aeda0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d0019070 con 0x7f06e00aeda0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000ba60 con 0x7f06e00aeda0 2024-05-08T04:39:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e00af4c0 con 0x7f06e00aeda0 2024-05-08T04:39:34.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e00afa80 con 0x7f06e00aeda0 2024-05-08T04:39:34.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e0069d60 con 0x7f06e00aeda0 2024-05-08T04:39:34.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f06d000bc00 con 0x7f06e00aeda0 2024-05-08T04:39:34.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 0x7f06bc0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:34.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06d00855b0 con 0x7f06e00aeda0 2024-05-08T04:39:34.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d6000640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 0x7f06bc0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:34.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.774+0000 7f06d6000640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 0x7f06bc0694e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06c80af000 tx=0x7f06c80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:34.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.778+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06d00564d0 con 0x7f06e00aeda0 2024-05-08T04:39:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.941+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f06e006e210 con 0x7f06e00aeda0 2024-05-08T04:39:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.941+0000 7f06d7400640 1 -- 172.21.15.164:0/3249231159 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f06d0055dd0 con 0x7f06e00aeda0 2024-05-08T04:39:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 msgr2=0x7f06bc0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 0x7f06bc0694e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06c80af000 tx=0x7f06c80a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 msgr2=0x7f06e0141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d800 tx=0x7f06d000dcc0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 shutdown_connections 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f06bc067020 0x7f06bc0694e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06e015ba70 0x7f06e015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06e015be50 0x7f06e00b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 --2- 172.21.15.164:0/3249231159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e00aeda0 0x7f06e0141890 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 >> 172.21.15.164:0/3249231159 conn(0x7f06e006ac90 msgr2=0x7f06e01553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:34.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 shutdown_connections 2024-05-08T04:39:34.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:34.945+0000 7f06e5600640 1 -- 172.21.15.164:0/3249231159 wait complete. 2024-05-08T04:39:34.968 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:39:34.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:34.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:39:34.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:39:34.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:34.997 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:39:35.052 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:35.052 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:35.062 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:35.062 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:35.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:35.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 Processor -- start 2024-05-08T04:39:35.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 -- start start 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 0x7f5858160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58580cd9f0 con 0x7f58580cfda0 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58580cdcd0 con 0x7f585815be50 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5860600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585815fe70 con 0x7f585815ba70 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585e600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f58580cfda0 0x7f58580b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44966/0 (socket says 172.21.15.164:44966) 2024-05-08T04:39:35.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585e600640 1 -- 172.21.15.164:0/1245264351 learned_addr learned my addr 172.21.15.164:0/1245264351 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:35.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585dc00640 1 --2- 172.21.15.164:0/1245264351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 0x7f5858160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114676942 0 0) 0x7f58580cd9f0 con 0x7f58580cfda0 2024-05-08T04:39:35.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5840003530 con 0x7f58580cfda0 2024-05-08T04:39:35.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585dc00640 1 -- 172.21.15.164:0/1245264351 >> v1:172.21.15.172:6789/0 conn(0x7f585815be50 legacy=0x7f58580cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518628234 0 0) 0x7f58580cdcd0 con 0x7f585815be50 2024-05-08T04:39:35.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585dc00640 1 -- 172.21.15.164:0/1245264351 >> v1:172.21.15.164:6789/0 conn(0x7f58580cfda0 legacy=0x7f58580b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.245+0000 7f585dc00640 1 -- 172.21.15.164:0/1245264351 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585815f820 con 0x7f585815ba70 2024-05-08T04:39:35.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f585dc00640 1 --2- 172.21.15.164:0/1245264351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 0x7f5858160300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f584c009dc0 tx=0x7f584c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4200182d20674e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0b2070 con 0x7f585815ba70 2024-05-08T04:39:35.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c0a5930 con 0x7f585815ba70 2024-05-08T04:39:35.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5857e00640 1 -- 172.21.15.164:0/1245264351 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0a5ce0 con 0x7f585815ba70 2024-05-08T04:39:35.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 -- 172.21.15.164:0/1245264351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 msgr2=0x7f5858160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 --2- 172.21.15.164:0/1245264351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 0x7f5858160300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f584c009dc0 tx=0x7f584c01c960 comp rx=0 tx=0).stop 2024-05-08T04:39:35.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 -- 172.21.15.164:0/1245264351 shutdown_connections 2024-05-08T04:39:35.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 --2- 172.21.15.164:0/1245264351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815ba70 0x7f5858160300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 -- 172.21.15.164:0/1245264351 >> 172.21.15.164:0/1245264351 conn(0x7f585806ac90 msgr2=0x7f585806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:35.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.249+0000 7f5860600640 1 -- 172.21.15.164:0/1245264351 shutdown_connections 2024-05-08T04:39:35.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- 172.21.15.164:0/1245264351 wait complete. 2024-05-08T04:39:35.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 Processor -- start 2024-05-08T04:39:35.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- start start 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58580cfda0 0x7f585814a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815be50 0x7f58580713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858169f60 con 0x7f585815ba70 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858169de0 con 0x7f58580cfda0 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585816a0e0 con 0x7f585815be50 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43618/0 (socket says 172.21.15.164:43618) 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 -- 172.21.15.164:0/4007433607 learned_addr learned my addr 172.21.15.164:0/4007433607 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:35.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585dc00640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58580cfda0 0x7f585814a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585e600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815be50 0x7f58580713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 -- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815be50 msgr2=0x7f58580713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815be50 0x7f58580713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 -- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58580cfda0 msgr2=0x7f585814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58580cfda0 0x7f585814a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 -- 172.21.15.164:0/4007433607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580751c0 con 0x7f585815ba70 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f585d200640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f584400c3f0 tx=0x7f584400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844017070 con 0x7f585815ba70 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844003600 con 0x7f585815ba70 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440039f0 con 0x7f585815ba70 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580b2050 con 0x7f585815ba70 2024-05-08T04:39:35.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.253+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5858151270 con 0x7f585815ba70 2024-05-08T04:39:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5858067820 con 0x7f585815ba70 2024-05-08T04:39:35.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f584401a070 con 0x7f585815ba70 2024-05-08T04:39:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f5855600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 0x7f58380694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5844084400 con 0x7f585815ba70 2024-05-08T04:39:35.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f585dc00640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 0x7f58380694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.257+0000 7f585dc00640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 0x7f58380694b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f584c0af000 tx=0x7f584c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.261+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5844054d60 con 0x7f585815ba70 2024-05-08T04:39:35.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.417+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5858069d60 con 0x7f585815ba70 2024-05-08T04:39:35.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5855600640 1 -- 172.21.15.164:0/4007433607 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f5844058e00 con 0x7f585815ba70 2024-05-08T04:39:35.424 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:35.424 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 msgr2=0x7f58380694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 0x7f58380694b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f584c0af000 tx=0x7f584c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 msgr2=0x7f58580b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f584400c3f0 tx=0x7f584400c8b0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 shutdown_connections 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5838066ff0 0x7f58380694b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f585815be50 0x7f58580713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f585815ba70 0x7f58580b1b10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 --2- 172.21.15.164:0/4007433607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58580cfda0 0x7f585814a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 >> 172.21.15.164:0/4007433607 conn(0x7f585806ac90 msgr2=0x7f58580b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:35.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 shutdown_connections 2024-05-08T04:39:35.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.421+0000 7f5860600640 1 -- 172.21.15.164:0/4007433607 wait complete. 2024-05-08T04:39:35.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:20.033719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:20.517477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:35.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:35.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 Processor -- start 2024-05-08T04:39:35.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 -- start start 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 0x7f3f8c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8c15e870 con 0x7f3f8c0ce780 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8c06bb80 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f94400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c15eb50 con 0x7f3f8c15ab10 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43626/0 (socket says 172.21.15.164:43626) 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 -- 172.21.15.164:0/2817239896 learned_addr learned my addr 172.21.15.164:0/2817239896 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91a00640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 0x7f3f8c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 -- 172.21.15.164:0/2817239896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 msgr2=0x7f3f8c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 0x7f3f8c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 -- 172.21.15.164:0/2817239896 >> v1:172.21.15.172:6789/0 conn(0x7f3f8c0ce780 legacy=0x7f3f8c0c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:35.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91000640 1 -- 172.21.15.164:0/2817239896 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c15ee30 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.581+0000 7f3f91a00640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 0x7f3f8c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f91000640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800bb10 tx=0x7f3f7801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85736920222a17ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f8be00640 1 -- 172.21.15.164:0/2817239896 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780b4070 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f8be00640 1 -- 172.21.15.164:0/2817239896 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f780a7bc0 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f8be00640 1 -- 172.21.15.164:0/2817239896 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780af440 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 -- 172.21.15.164:0/2817239896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 msgr2=0x7f3f8c0aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800bb10 tx=0x7f3f7801eb30 comp rx=0 tx=0).stop 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 -- 172.21.15.164:0/2817239896 shutdown_connections 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c0aeb40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 --2- 172.21.15.164:0/2817239896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15ab10 0x7f3f8c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 -- 172.21.15.164:0/2817239896 >> 172.21.15.164:0/2817239896 conn(0x7f3f8c0cf770 msgr2=0x7f3f8c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 -- 172.21.15.164:0/2817239896 shutdown_connections 2024-05-08T04:39:35.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.585+0000 7f3f94400640 1 -- 172.21.15.164:0/2817239896 wait complete. 2024-05-08T04:39:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 Processor -- start 2024-05-08T04:39:35.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- start start 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c0ce780 0x7f3f8c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c15ab10 0x7f3f8c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0cd670 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c15aef0 con 0x7f3f8c15ab10 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0cd7f0 con 0x7f3f8c0ce780 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43640/0 (socket says 172.21.15.164:43640) 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 -- 172.21.15.164:0/1635799895 learned_addr learned my addr 172.21.15.164:0/1635799895 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:35.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91000640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c0ce780 0x7f3f8c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f92400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c15ab10 0x7f3f8c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 -- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c0ce780 msgr2=0x7f3f8c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c0ce780 0x7f3f8c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 -- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c15ab10 msgr2=0x7f3f8c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c15ab10 0x7f3f8c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 -- 172.21.15.164:0/1635799895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c158ad0 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f91a00640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000ce30 tx=0x7f3f8000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000f890 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8000fa30 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c158db0 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000fe00 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c159370 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.589+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8c072d30 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.593+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f3f80018400 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.593+0000 7f3f89600640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 0x7f3f6c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.593+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f80085e80 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.593+0000 7f3f91000640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 0x7f3f6c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.593+0000 7f3f91000640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 0x7f3f6c0694b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f780b1000 tx=0x7f3f78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.597+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f80014030 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f8c04f010 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f89600640 1 -- 172.21.15.164:0/1635799895 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f8000ae20 con 0x7f3f8c0ae6e0 2024-05-08T04:39:35.758 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 msgr2=0x7f3f6c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 0x7f3f6c0694b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f780b1000 tx=0x7f3f78008040 comp rx=0 tx=0).stop 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 msgr2=0x7f3f8c15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000ce30 tx=0x7f3f8000a650 comp rx=0 tx=0).stop 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 shutdown_connections 2024-05-08T04:39:35.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f6c066ff0 0x7f3f6c0694b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c0ce780 0x7f3f8c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c15ab10 0x7f3f8c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 --2- 172.21.15.164:0/1635799895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f8c0ae6e0 0x7f3f8c15b2d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 >> 172.21.15.164:0/1635799895 conn(0x7f3f8c0cf770 msgr2=0x7f3f8c0cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.753+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 shutdown_connections 2024-05-08T04:39:35.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.757+0000 7f3f94400640 1 -- 172.21.15.164:0/1635799895 wait complete. 2024-05-08T04:39:35.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 Processor -- start 2024-05-08T04:39:35.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 -- start start 2024-05-08T04:39:35.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 0x7f25000b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25000b4dc0 0x7f250006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250015e300 con 0x7f25000af6d0 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250015e570 con 0x7f25000b4dc0 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f2506000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250015e850 con 0x7f25000adfa0 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ff400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f25000af6d0 0x7f25000ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44972/0 (socket says 172.21.15.164:44972) 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ff400640 1 -- 172.21.15.164:0/2666855102 learned_addr learned my addr 172.21.15.164:0/2666855102 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ffe00640 1 --2- 172.21.15.164:0/2666855102 >> v2:172.21.15.164:3300/0 conn(0x7f25000b4dc0 0x7f250006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24f6a00640 1 --2- 172.21.15.164:0/2666855102 >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 0x7f25000b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24fea00640 1 -- 172.21.15.164:0/2666855102 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329071915 0 0) 0x7f250015e300 con 0x7f25000af6d0 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24fea00640 1 -- 172.21.15.164:0/2666855102 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003530 con 0x7f25000af6d0 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ffe00640 1 -- 172.21.15.164:0/2666855102 >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 msgr2=0x7f25000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ffe00640 1 --2- 172.21.15.164:0/2666855102 >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 0x7f25000b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ffe00640 1 -- 172.21.15.164:0/2666855102 >> v1:172.21.15.164:6789/0 conn(0x7f25000af6d0 legacy=0x7f25000ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24ffe00640 1 -- 172.21.15.164:0/2666855102 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000b5250 con 0x7f25000b4dc0 2024-05-08T04:39:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.913+0000 7f24f6a00640 1 --2- 172.21.15.164:0/2666855102 >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 0x7f25000b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f24ffe00640 1 --2- 172.21.15.164:0/2666855102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000b4dc0 0x7f250006ac90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2500070b80 tx=0x7f24f801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829ef13d976685ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f24fea00640 1 -- 172.21.15.164:0/2666855102 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80b0070 con 0x7f25000b4dc0 2024-05-08T04:39:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f24fea00640 1 -- 172.21.15.164:0/2666855102 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f80ab600 con 0x7f25000b4dc0 2024-05-08T04:39:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f24fea00640 1 -- 172.21.15.164:0/2666855102 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80ab9d0 con 0x7f25000b4dc0 2024-05-08T04:39:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 -- 172.21.15.164:0/2666855102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000b4dc0 msgr2=0x7f250006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 --2- 172.21.15.164:0/2666855102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000b4dc0 0x7f250006ac90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2500070b80 tx=0x7f24f801c3f0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 -- 172.21.15.164:0/2666855102 shutdown_connections 2024-05-08T04:39:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 --2- 172.21.15.164:0/2666855102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000b4dc0 0x7f250006ac90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 --2- 172.21.15.164:0/2666855102 >> v2:172.21.15.172:3300/0 conn(0x7f25000adfa0 0x7f25000b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 -- 172.21.15.164:0/2666855102 >> 172.21.15.164:0/2666855102 conn(0x7f250006cca0 msgr2=0x7f250006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 -- 172.21.15.164:0/2666855102 shutdown_connections 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.917+0000 7f2506000640 1 -- 172.21.15.164:0/2666855102 wait complete. 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 Processor -- start 2024-05-08T04:39:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- start start 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25000af6d0 0x7f250014f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 0x7f250015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250016a950 con 0x7f25000adfa0 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250006bbf0 con 0x7f25000af6d0 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250016aad0 con 0x7f25000b4dc0 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43652/0 (socket says 172.21.15.164:43652) 2024-05-08T04:39:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 -- 172.21.15.164:0/2706040654 learned_addr learned my addr 172.21.15.164:0/2706040654 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ffe00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 0x7f250015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24f6a00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25000af6d0 0x7f250014f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 -- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 msgr2=0x7f250015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 0x7f250015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 -- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25000af6d0 msgr2=0x7f250014f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25000af6d0 0x7f250014f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 -- 172.21.15.164:0/2706040654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25001518f0 con 0x7f25000adfa0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ffe00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 0x7f250015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24ff400640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24f000cb40 tx=0x7f24f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0017070 con 0x7f25000adfa0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f000aca0 con 0x7f25000adfa0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2500151bd0 con 0x7f25000adfa0 2024-05-08T04:39:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0012620 con 0x7f25000adfa0 2024-05-08T04:39:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2500152190 con 0x7f25000adfa0 2024-05-08T04:39:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.921+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250006e5b0 con 0x7f25000adfa0 2024-05-08T04:39:35.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.925+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f24f001a070 con 0x7f25000adfa0 2024-05-08T04:39:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.925+0000 7f24f7e00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 0x7f24dc0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.925+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24f0084dd0 con 0x7f25000adfa0 2024-05-08T04:39:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.925+0000 7f24f6a00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 0x7f24dc0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:35.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.925+0000 7f24f6a00640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 0x7f24dc0694e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2500150f60 tx=0x7f24e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:35.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:35.929+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24f0055e80 con 0x7f25000adfa0 2024-05-08T04:39:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.081+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f250004f010 con 0x7f25000adfa0 2024-05-08T04:39:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.081+0000 7f24f7e00640 1 -- 172.21.15.164:0/2706040654 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24f0055780 con 0x7f25000adfa0 2024-05-08T04:39:36.086 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 msgr2=0x7f24dc0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 0x7f24dc0694e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2500150f60 tx=0x7f24e8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 msgr2=0x7f250015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24f000cb40 tx=0x7f24f000a060 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 shutdown_connections 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f24dc067020 0x7f24dc0694e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25000b4dc0 0x7f250015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25000af6d0 0x7f250014f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 --2- 172.21.15.164:0/2706040654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25000adfa0 0x7f250015ddb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 >> 172.21.15.164:0/2706040654 conn(0x7f250006cca0 msgr2=0x7f2500072330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 shutdown_connections 2024-05-08T04:39:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.085+0000 7f2506000640 1 -- 172.21.15.164:0/2706040654 wait complete. 2024-05-08T04:39:36.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:36.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.241+0000 7f2a83200640 1 Processor -- start 2024-05-08T04:39:36.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.241+0000 7f2a83200640 1 -- start start 2024-05-08T04:39:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a83200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a7c15aa10 0x7f2a7c071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a83200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c0b48b0 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a83200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c06d480 con 0x7f2a7c071b40 2024-05-08T04:39:36.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a83200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c06d760 con 0x7f2a7c15aa10 2024-05-08T04:39:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a81200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2a7c0b3ed0 0x7f2a7c06e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44988/0 (socket says 172.21.15.164:44988) 2024-05-08T04:39:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a81200640 1 -- 172.21.15.164:0/687849757 learned_addr learned my addr 172.21.15.164:0/687849757 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7be00640 1 --2- 172.21.15.164:0/687849757 >> v2:172.21.15.172:3300/0 conn(0x7f2a7c15aa10 0x7f2a7c071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378596164 0 0) 0x7f2a7c0b48b0 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a60005740 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7be00640 1 -- 172.21.15.164:0/687849757 >> v1:172.21.15.172:6789/0 conn(0x7f2a7c071b40 legacy=0x7f2a7c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234224266 0 0) 0x7f2a7c06d480 con 0x7f2a7c071b40 2024-05-08T04:39:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7be00640 1 -- 172.21.15.164:0/687849757 >> v1:172.21.15.164:6789/0 conn(0x7f2a7c0b3ed0 legacy=0x7f2a7c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.245+0000 7f2a7be00640 1 -- 172.21.15.164:0/687849757 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c06da40 con 0x7f2a7c15aa10 2024-05-08T04:39:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a7be00640 1 --2- 172.21.15.164:0/687849757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c071600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ffd6033c78612d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0b2070 con 0x7f2a7c15aa10 2024-05-08T04:39:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0a57b0 con 0x7f2a7c15aa10 2024-05-08T04:39:36.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a7aa00640 1 -- 172.21.15.164:0/687849757 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0a5b80 con 0x7f2a7c15aa10 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- 172.21.15.164:0/687849757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 msgr2=0x7f2a7c071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 --2- 172.21.15.164:0/687849757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c071600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009d90 tx=0x7f2a6c01c8e0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- 172.21.15.164:0/687849757 shutdown_connections 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 --2- 172.21.15.164:0/687849757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c071600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- 172.21.15.164:0/687849757 >> 172.21.15.164:0/687849757 conn(0x7f2a7c0cffd0 msgr2=0x7f2a7c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:36.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- 172.21.15.164:0/687849757 shutdown_connections 2024-05-08T04:39:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- 172.21.15.164:0/687849757 wait complete. 2024-05-08T04:39:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 Processor -- start 2024-05-08T04:39:36.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.249+0000 7f2a83200640 1 -- start start 2024-05-08T04:39:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a7c071b40 0x7f2a7c147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0ae1f0 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0ae070 con 0x7f2a7c15aa10 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c06b4a0 con 0x7f2a7c071b40 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43668/0 (socket says 172.21.15.164:43668) 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 -- 172.21.15.164:0/1511199482 learned_addr learned my addr 172.21.15.164:0/1511199482 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a81200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7be00640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a7c071b40 0x7f2a7c147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 -- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a7c071b40 msgr2=0x7f2a7c147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a7c071b40 0x7f2a7c147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 -- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 msgr2=0x7f2a7c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 -- 172.21.15.164:0/1511199482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c15e260 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a7b400640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c480 tx=0x7f2a6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68005480 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a68003600 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a680039f0 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c146260 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c146820 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c09e220 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.253+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f2a68003b90 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.257+0000 7f2a73e00640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 0x7f2a5c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.257+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a68084dc0 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.257+0000 7f2a7be00640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 0x7f2a5c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.257+0000 7f2a7be00640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 0x7f2a5c0694e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0af000 tx=0x7f2a6c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.261+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a68055d70 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.413+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a7c06e220 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.413+0000 7f2a73e00640 1 -- 172.21.15.164:0/1511199482 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a68055670 con 0x7f2a7c0b3ed0 2024-05-08T04:39:36.418 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 msgr2=0x7f2a5c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 0x7f2a5c0694e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0af000 tx=0x7f2a6c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 msgr2=0x7f2a7c145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c480 tx=0x7f2a6800c940 comp rx=0 tx=0).stop 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 shutdown_connections 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a5c067020 0x7f2a5c0694e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a7c15aa10 0x7f2a7c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7c0b3ed0 0x7f2a7c145a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 --2- 172.21.15.164:0/1511199482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a7c071b40 0x7f2a7c147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 >> 172.21.15.164:0/1511199482 conn(0x7f2a7c0cffd0 msgr2=0x7f2a7c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:36.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 shutdown_connections 2024-05-08T04:39:36.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.417+0000 7f2a83200640 1 -- 172.21.15.164:0/1511199482 wait complete. 2024-05-08T04:39:36.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:39:36.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 Processor -- start 2024-05-08T04:39:36.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 -- start start 2024-05-08T04:39:36.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58fc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc156700 con 0x7f58fc156320 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc15c210 con 0x7f58fc06d6f0 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f5901c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc06dc90 con 0x7f58fc15d3d0 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58faa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f58fc156320 0x7f58fc0cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44998/0 (socket says 172.21.15.164:44998) 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58faa00640 1 -- 172.21.15.164:0/2504400241 learned_addr learned my addr 172.21.15.164:0/2504400241 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fb400640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58fc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148814037 0 0) 0x7f58fc156700 con 0x7f58fc156320 2024-05-08T04:39:36.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003530 con 0x7f58fc156320 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143576817 0 0) 0x7f58e0003530 con 0x7f58fc156320 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 msgr2=0x7f58fc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58fc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 >> v1:172.21.15.172:6789/0 conn(0x7f58fc06d6f0 legacy=0x7f58fc15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc06df70 con 0x7f58fc156320 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f58ec001a00 con 0x7f58fc156320 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 >> v1:172.21.15.164:6789/0 conn(0x7f58fc156320 legacy=0x7f58fc0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58e0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc06dc90 con 0x7f58e00043a0 2024-05-08T04:39:36.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000ba80 con 0x7f58e00039f0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000bc80 con 0x7f58fc15d3d0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58faa00640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58fb400640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58e0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 -- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 msgr2=0x7f58e0005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58e0005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 -- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00039f0 msgr2=0x7f58e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00039f0 0x7f58e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.573+0000 7f58f2000640 1 -- 172.21.15.164:0/2504400241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0003530 con 0x7f58e00043a0 2024-05-08T04:39:36.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58fb400640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc15d3d0 0x7f58e0005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:36.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58f2000640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f58e4001ae0 tx=0x7f58e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a426a6653a6fac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40b0070 con 0x7f58e00043a0 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40ab620 con 0x7f58e00043a0 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58fa000640 1 -- 172.21.15.164:0/2504400241 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40016e0 con 0x7f58e00043a0 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 -- 172.21.15.164:0/2504400241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e00043a0 msgr2=0x7f58e0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 --2- 172.21.15.164:0/2504400241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58e00043a0 0x7f58e0004870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f58e4001ae0 tx=0x7f58e401c560 comp rx=0 tx=0).stop 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f58fb400640 1 -- 172.21.15.164:0/2504400241 reap_dead start 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 -- 172.21.15.164:0/2504400241 shutdown_connections 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 -- 172.21.15.164:0/2504400241 >> 172.21.15.164:0/2504400241 conn(0x7f58fc066fa0 msgr2=0x7f58fc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 -- 172.21.15.164:0/2504400241 shutdown_connections 2024-05-08T04:39:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.577+0000 7f5901c00640 1 -- 172.21.15.164:0/2504400241 wait complete. 2024-05-08T04:39:36.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 Processor -- start 2024-05-08T04:39:36.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- start start 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00043a0 0x7f58fc142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc157590 0x7f58fc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc167850 con 0x7f58fc157110 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc15dbf0 con 0x7f58e00043a0 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc1679d0 con 0x7f58fc157590 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43684/0 (socket says 172.21.15.164:43684) 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 -- 172.21.15.164:0/966519351 learned_addr learned my addr 172.21.15.164:0/966519351 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58fb400640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc157590 0x7f58fc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58faa00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00043a0 0x7f58fc142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 -- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc157590 msgr2=0x7f58fc0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc157590 0x7f58fc0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 -- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00043a0 msgr2=0x7f58fc142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00043a0 0x7f58fc142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 -- 172.21.15.164:0/966519351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc157b60 con 0x7f58fc157110 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f2000640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58e401ca40 tx=0x7f58e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40b9020 con 0x7f58fc157110 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4001120 con 0x7f58fc157110 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40b0040 con 0x7f58fc157110 2024-05-08T04:39:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58fc157e40 con 0x7f58fc157110 2024-05-08T04:39:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.581+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc158400 con 0x7f58fc157110 2024-05-08T04:39:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc069d60 con 0x7f58fc157110 2024-05-08T04:39:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f58e40bc020 con 0x7f58fc157110 2024-05-08T04:39:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f58f3400640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 0x7f58d80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58e411ed80 con 0x7f58fc157110 2024-05-08T04:39:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f58faa00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 0x7f58d80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.585+0000 7f58faa00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 0x7f58d80694b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f58ec002240 tx=0x7f58ec006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:36.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.589+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58e40c3030 con 0x7f58fc157110 2024-05-08T04:39:36.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.753+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58fc155740 con 0x7f58fc157110 2024-05-08T04:39:36.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.753+0000 7f58f3400640 1 -- 172.21.15.164:0/966519351 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f58fc15c210 con 0x7f58fc157110 2024-05-08T04:39:36.760 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:39:36.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 msgr2=0x7f58d80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 0x7f58d80694b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f58ec002240 tx=0x7f58ec006040 comp rx=0 tx=0).stop 2024-05-08T04:39:36.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 msgr2=0x7f58fc0cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:36.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58e401ca40 tx=0x7f58e0005840 comp rx=0 tx=0).stop 2024-05-08T04:39:36.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 shutdown_connections 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58d8066ff0 0x7f58d80694b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58fc157590 0x7f58fc0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58fc157110 0x7f58fc0cfb30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 --2- 172.21.15.164:0/966519351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58e00043a0 0x7f58fc142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 >> 172.21.15.164:0/966519351 conn(0x7f58fc066fa0 msgr2=0x7f58fc154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 shutdown_connections 2024-05-08T04:39:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:36.757+0000 7f5901c00640 1 -- 172.21.15.164:0/966519351 wait complete. 2024-05-08T04:39:36.781 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:39:36.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:36.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:39:36.811 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:39:36.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:39:36.811 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:39:36.867 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:36.867 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:36.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:39:36.876 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:36.876 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:36.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:39:36.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:37.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 Processor -- start 2024-05-08T04:39:37.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 -- start start 2024-05-08T04:39:37.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73040b2d80 0x7f7304168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730406cc70 con 0x7f730415dd70 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730406cf50 con 0x7f730406c7b0 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f730b800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73040b1570 con 0x7f73040b2d80 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7308e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f730415dd70 0x7f73041619a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45004/0 (socket says 172.21.15.164:45004) 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7308e00640 1 -- 172.21.15.164:0/2836255755 learned_addr learned my addr 172.21.15.164:0/2836255755 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7309800640 1 --2- 172.21.15.164:0/2836255755 >> v2:172.21.15.172:3300/0 conn(0x7f73040b2d80 0x7f7304168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733782081 0 0) 0x7f730406cc70 con 0x7f730415dd70 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72ec003530 con 0x7f730415dd70 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012535216 0 0) 0x7f730406cf50 con 0x7f730406c7b0 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f730406cc70 con 0x7f730406c7b0 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3831739236 0 0) 0x7f72ec003530 con 0x7f730415dd70 2024-05-08T04:39:37.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7309800640 1 -- 172.21.15.164:0/2836255755 >> v1:172.21.15.172:6789/0 conn(0x7f730406c7b0 legacy=0x7f73041620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7309800640 1 -- 172.21.15.164:0/2836255755 >> v1:172.21.15.164:6789/0 conn(0x7f730415dd70 legacy=0x7f73041619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.053+0000 7f7309800640 1 -- 172.21.15.164:0/2836255755 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040b1850 con 0x7f73040b2d80 2024-05-08T04:39:37.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f7309800640 1 --2- 172.21.15.164:0/2836255755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f7304168cd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009d90 tx=0x7f72fc01f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea7f88fb0c2ac72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0b5070 con 0x7f73040b2d80 2024-05-08T04:39:37.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc0a8c10 con 0x7f73040b2d80 2024-05-08T04:39:37.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f7303400640 1 -- 172.21.15.164:0/2836255755 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8001680 con 0x7f73040b2d80 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 -- 172.21.15.164:0/2836255755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 msgr2=0x7f7304168cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 --2- 172.21.15.164:0/2836255755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f7304168cd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009d90 tx=0x7f72fc01f080 comp rx=0 tx=0).stop 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 -- 172.21.15.164:0/2836255755 shutdown_connections 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 --2- 172.21.15.164:0/2836255755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f7304168cd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 -- 172.21.15.164:0/2836255755 >> 172.21.15.164:0/2836255755 conn(0x7f73040b34f0 msgr2=0x7f73040b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 -- 172.21.15.164:0/2836255755 shutdown_connections 2024-05-08T04:39:37.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.057+0000 7f730b800640 1 -- 172.21.15.164:0/2836255755 wait complete. 2024-05-08T04:39:37.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 Processor -- start 2024-05-08T04:39:37.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- start start 2024-05-08T04:39:37.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730406c7b0 0x7f730415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f73040b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730415e150 con 0x7f730415dd70 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73040b4230 con 0x7f73040b2d80 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730415e2d0 con 0x7f730406c7b0 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43692/0 (socket says 172.21.15.164:43692) 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 -- 172.21.15.164:0/2684810674 learned_addr learned my addr 172.21.15.164:0/2684810674 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7303e00640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f73040b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7308e00640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730406c7b0 0x7f730415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 -- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730406c7b0 msgr2=0x7f730415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730406c7b0 0x7f730415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 -- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 msgr2=0x7f73040b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f73040b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 -- 172.21.15.164:0/2684810674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730415fa80 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7309800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b3040 tx=0x7f72fc01be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0b5050 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc0b6040 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc009450 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730415fd60 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.061+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7304160320 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7304069d60 con 0x7f730415dd70 2024-05-08T04:39:37.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f72fc0c3020 con 0x7f730415dd70 2024-05-08T04:39:37.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f7300c00640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 0x7f72e40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72fc122f70 con 0x7f730415dd70 2024-05-08T04:39:37.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f7308e00640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 0x7f72e40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.065+0000 7f7308e00640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 0x7f72e40694b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72f80011f0 tx=0x7f72f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.069+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72fc0ca020 con 0x7f730415dd70 2024-05-08T04:39:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.265+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73040b7940 con 0x7f730415dd70 2024-05-08T04:39:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.265+0000 7f7300c00640 1 -- 172.21.15.164:0/2684810674 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f72fc018be0 con 0x7f730415dd70 2024-05-08T04:39:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 18 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 msgr2=0x7f72e40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 0x7f72e40694b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72f80011f0 tx=0x7f72f8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 msgr2=0x7f73040c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b3040 tx=0x7f72fc01be20 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 shutdown_connections 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f72e4066ff0 0x7f72e40694b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f730415dd70 0x7f73040c5420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73040b2d80 0x7f73040b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 --2- 172.21.15.164:0/2684810674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f730406c7b0 0x7f730415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 >> 172.21.15.164:0/2684810674 conn(0x7f73040b34f0 msgr2=0x7f730415ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 shutdown_connections 2024-05-08T04:39:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.269+0000 7f730b800640 1 -- 172.21.15.164:0/2684810674 wait complete. 2024-05-08T04:39:37.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:37.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 Processor -- start 2024-05-08T04:39:37.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 -- start start 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f983806c7b0 0x7f98381620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98381625f0 0x7f98380b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983806cc70 con 0x7f983815dd70 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983806cf50 con 0x7f98381625f0 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f983d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380b1570 con 0x7f983806c7b0 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98381625f0 0x7f98380b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98381625f0 0x7f98380b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43698/0 (socket says 172.21.15.164:43698) 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 -- 172.21.15.164:0/623321747 learned_addr learned my addr 172.21.15.164:0/623321747 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:37.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9835e00640 1 --2- 172.21.15.164:0/623321747 >> v2:172.21.15.172:3300/0 conn(0x7f983806c7b0 0x7f98381620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 -- 172.21.15.164:0/623321747 >> v2:172.21.15.172:3300/0 conn(0x7f983806c7b0 msgr2=0x7f98381620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 --2- 172.21.15.164:0/623321747 >> v2:172.21.15.172:3300/0 conn(0x7f983806c7b0 0x7f98381620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 -- 172.21.15.164:0/623321747 >> v1:172.21.15.172:6789/0 conn(0x7f983815dd70 legacy=0x7f98381619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.425+0000 7f9837200640 1 -- 172.21.15.164:0/623321747 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380b1850 con 0x7f98381625f0 2024-05-08T04:39:37.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f9837200640 1 --2- 172.21.15.164:0/623321747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98381625f0 0x7f98380b1d70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9838070fa0 tx=0x7f983001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9bea512bfb3b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f9835400640 1 -- 172.21.15.164:0/623321747 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300a56c0 con 0x7f98381625f0 2024-05-08T04:39:37.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f9835400640 1 -- 172.21.15.164:0/623321747 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98300a5860 con 0x7f98381625f0 2024-05-08T04:39:37.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f9835400640 1 -- 172.21.15.164:0/623321747 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300a5c30 con 0x7f98381625f0 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 -- 172.21.15.164:0/623321747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98381625f0 msgr2=0x7f98380b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 --2- 172.21.15.164:0/623321747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98381625f0 0x7f98380b1d70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9838070fa0 tx=0x7f983001c9f0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 -- 172.21.15.164:0/623321747 shutdown_connections 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 --2- 172.21.15.164:0/623321747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98381625f0 0x7f98380b1d70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 --2- 172.21.15.164:0/623321747 >> v2:172.21.15.172:3300/0 conn(0x7f983806c7b0 0x7f98381620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 -- 172.21.15.164:0/623321747 >> 172.21.15.164:0/623321747 conn(0x7f98380b34f0 msgr2=0x7f98380b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 -- 172.21.15.164:0/623321747 shutdown_connections 2024-05-08T04:39:37.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.429+0000 7f983d400640 1 -- 172.21.15.164:0/623321747 wait complete. 2024-05-08T04:39:37.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 Processor -- start 2024-05-08T04:39:37.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- start start 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983815dd70 0x7f983814b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381625f0 0x7f983814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b2e50 con 0x7f983806c7b0 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b2cd0 con 0x7f98381625f0 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b2fd0 con 0x7f983815dd70 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43708/0 (socket says 172.21.15.164:43708) 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 -- 172.21.15.164:0/4213138174 learned_addr learned my addr 172.21.15.164:0/4213138174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9835e00640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983815dd70 0x7f983814b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9837200640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381625f0 0x7f983814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 -- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983815dd70 msgr2=0x7f983814b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983815dd70 0x7f983814b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 -- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381625f0 msgr2=0x7f983814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381625f0 0x7f983814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 -- 172.21.15.164:0/4213138174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983814bc00 con 0x7f983806c7b0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f9836800640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f982800c370 tx=0x7f982800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280052d0 con 0x7f983806c7b0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828005470 con 0x7f983806c7b0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828007260 con 0x7f983806c7b0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983815bd00 con 0x7f983806c7b0 2024-05-08T04:39:37.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.433+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983815c2c0 con 0x7f983806c7b0 2024-05-08T04:39:37.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838069d60 con 0x7f983806c7b0 2024-05-08T04:39:37.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f9828007400 con 0x7f983806c7b0 2024-05-08T04:39:37.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f982ea00640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 0x7f98180694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98280844f0 con 0x7f983806c7b0 2024-05-08T04:39:37.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f9835e00640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 0x7f98180694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.437+0000 7f9835e00640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 0x7f98180694b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98380d0840 tx=0x7f9824009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.441+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9828054dc0 con 0x7f983806c7b0 2024-05-08T04:39:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.597+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98380b4010 con 0x7f983806c7b0 2024-05-08T04:39:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.597+0000 7f982ea00640 1 -- 172.21.15.164:0/4213138174 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f9828058e60 con 0x7f983806c7b0 2024-05-08T04:39:37.603 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:37.603 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 msgr2=0x7f98180694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 0x7f98180694b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98380d0840 tx=0x7f9824009190 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 msgr2=0x7f9838160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f982800c370 tx=0x7f982800c830 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 shutdown_connections 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9818066ff0 0x7f98180694b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983815dd70 0x7f983814b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381625f0 0x7f983814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 --2- 172.21.15.164:0/4213138174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f983806c7b0 0x7f9838160dc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 >> 172.21.15.164:0/4213138174 conn(0x7f98380b34f0 msgr2=0x7f983806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 shutdown_connections 2024-05-08T04:39:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.601+0000 7f983d400640 1 -- 172.21.15.164:0/4213138174 wait complete. 2024-05-08T04:39:37.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:20.033719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:20.517477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:37.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:37.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.757+0000 7f9bda200640 1 Processor -- start 2024-05-08T04:39:37.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.757+0000 7f9bda200640 1 -- start start 2024-05-08T04:39:37.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bda200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9bd4144110 0x7f9bd40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bda200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9bd415e760 0x7f9bd4071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bda200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd415ebe0 con 0x7f9bd40b2180 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bda200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd415eec0 con 0x7f9bd4144110 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bda200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd40b1440 con 0x7f9bd415e760 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9bd4144110 0x7f9bd40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9bd4144110 0x7f9bd40b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43714/0 (socket says 172.21.15.164:43714) 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 -- 172.21.15.164:0/2304958915 learned_addr learned my addr 172.21.15.164:0/2304958915 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 -- 172.21.15.164:0/2304958915 >> v2:172.21.15.172:3300/0 conn(0x7f9bd415e760 msgr2=0x7f9bd4071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 --2- 172.21.15.164:0/2304958915 >> v2:172.21.15.172:3300/0 conn(0x7f9bd415e760 0x7f9bd4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 -- 172.21.15.164:0/2304958915 >> v1:172.21.15.172:6789/0 conn(0x7f9bd40b2180 legacy=0x7f9bd415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.761+0000 7f9bd8e00640 1 -- 172.21.15.164:0/2304958915 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd40b1720 con 0x7f9bd4144110 2024-05-08T04:39:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd8e00640 1 --2- 172.21.15.164:0/2304958915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd4144110 0x7f9bd40b1c40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40b0c60 tx=0x7f9bc401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed9db5706379f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3400640 1 -- 172.21.15.164:0/2304958915 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40a5660 con 0x7f9bd4144110 2024-05-08T04:39:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3400640 1 -- 172.21.15.164:0/2304958915 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc40a5800 con 0x7f9bd4144110 2024-05-08T04:39:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3400640 1 -- 172.21.15.164:0/2304958915 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40a5bd0 con 0x7f9bd4144110 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- 172.21.15.164:0/2304958915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd4144110 msgr2=0x7f9bd40b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- 172.21.15.164:0/2304958915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd4144110 0x7f9bd40b1c40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40b0c60 tx=0x7f9bc401c850 comp rx=0 tx=0).stop 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- 172.21.15.164:0/2304958915 shutdown_connections 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- 172.21.15.164:0/2304958915 >> v2:172.21.15.172:3300/0 conn(0x7f9bd415e760 0x7f9bd4071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- 172.21.15.164:0/2304958915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd4144110 0x7f9bd40b1c40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- 172.21.15.164:0/2304958915 >> 172.21.15.164:0/2304958915 conn(0x7f9bd406ac90 msgr2=0x7f9bd406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- 172.21.15.164:0/2304958915 shutdown_connections 2024-05-08T04:39:37.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- 172.21.15.164:0/2304958915 wait complete. 2024-05-08T04:39:37.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 Processor -- start 2024-05-08T04:39:37.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- start start 2024-05-08T04:39:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bd4144110 0x7f9bd4140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd4143140 0x7f9bd4137490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd416a850 con 0x7f9bd415e760 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd416a6d0 con 0x7f9bd4143140 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bda200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd416a9d0 con 0x7f9bd4144110 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43718/0 (socket says 172.21.15.164:43718) 2024-05-08T04:39:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.765+0000 7f9bd3e00640 1 -- 172.21.15.164:0/78199967 learned_addr learned my addr 172.21.15.164:0/78199967 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd9800640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd4143140 0x7f9bd4137490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd8e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bd4144110 0x7f9bd4140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 -- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bd4144110 msgr2=0x7f9bd4140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bd4144110 0x7f9bd4140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 -- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd4143140 msgr2=0x7f9bd4137490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd4143140 0x7f9bd4137490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 -- 172.21.15.164:0/78199967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd413b240 con 0x7f9bd415e760 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd3e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c410 tx=0x7f9bc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd415e760 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0003600 con 0x7f9bd415e760 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd4141030 con 0x7f9bd415e760 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00039f0 con 0x7f9bd415e760 2024-05-08T04:39:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd413b7e0 con 0x7f9bd415e760 2024-05-08T04:39:37.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd404ef30 con 0x7f9bd415e760 2024-05-08T04:39:37.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f9bc001a070 con 0x7f9bd415e760 2024-05-08T04:39:37.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 0x7f9bb40694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bc00842c0 con 0x7f9bd415e760 2024-05-08T04:39:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.769+0000 7f9bd8e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 0x7f9bb40694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.773+0000 7f9bd8e00640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 0x7f9bb40694e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40ae000 tx=0x7f9bc40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.773+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bc0054d20 con 0x7f9bd415e760 2024-05-08T04:39:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bd406b2c0 con 0x7f9bd415e760 2024-05-08T04:39:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bd0c00640 1 -- 172.21.15.164:0/78199967 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bc0058dc0 con 0x7f9bd415e760 2024-05-08T04:39:37.934 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:37.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 msgr2=0x7f9bb40694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 0x7f9bb40694e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40ae000 tx=0x7f9bc40a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:37.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 msgr2=0x7f9bd413bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.929+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c410 tx=0x7f9bc000c8d0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 shutdown_connections 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9bb4067020 0x7f9bb40694e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd4143140 0x7f9bd4137490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bd415e760 0x7f9bd413bfe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 --2- 172.21.15.164:0/78199967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bd4144110 0x7f9bd4140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 >> 172.21.15.164:0/78199967 conn(0x7f9bd406ac90 msgr2=0x7f9bd40af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 shutdown_connections 2024-05-08T04:39:37.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:37.933+0000 7f9bda200640 1 -- 172.21.15.164:0/78199967 wait complete. 2024-05-08T04:39:37.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:39:38.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 Processor -- start 2024-05-08T04:39:38.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 -- start start 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c1415ba20 0x7f1c14160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1c1406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140c6600 con 0x7f1c140ae6e0 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1415fbe0 con 0x7f1c1415ba20 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c19600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140c6040 con 0x7f1c1406de50 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c13400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1c140ae6e0 0x7f1c140cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45006/0 (socket says 172.21.15.164:45006) 2024-05-08T04:39:38.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c13400640 1 -- 172.21.15.164:0/1149703816 learned_addr learned my addr 172.21.15.164:0/1149703816 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:38.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 --2- 172.21.15.164:0/1149703816 >> v2:172.21.15.172:3300/0 conn(0x7f1c1415ba20 0x7f1c14160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12000640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1c1406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510662495 0 0) 0x7f1c140c6600 con 0x7f1c140ae6e0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf8003530 con 0x7f1c140ae6e0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712927244 0 0) 0x7f1bf8003530 con 0x7f1c140ae6e0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 msgr2=0x7f1c1406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1c1406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 >> v2:172.21.15.172:3300/0 conn(0x7f1c1415ba20 msgr2=0x7f1c14160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 --2- 172.21.15.164:0/1149703816 >> v2:172.21.15.172:3300/0 conn(0x7f1c1415ba20 0x7f1c14160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c140c6320 con 0x7f1c140ae6e0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1c0c001680 con 0x7f1c140ae6e0 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 >> v1:172.21.15.164:6789/0 conn(0x7f1c140ae6e0 legacy=0x7f1c140cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bf80039c0 0x7f1bf8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1bf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf8004d50 0x7f1bf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1415fbe0 con 0x7f1bf80039c0 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140c6040 con 0x7f1bf8004d50 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf800b9d0 con 0x7f1c1406de50 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bf80039c0 0x7f1bf8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c13400640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf8004d50 0x7f1bf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12000640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1bf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 -- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 msgr2=0x7f1bf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c1406de50 0x7f1bf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 -- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf8004d50 msgr2=0x7f1bf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf8004d50 0x7f1bf80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.089+0000 7f1c12a00640 1 -- 172.21.15.164:0/1149703816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8003530 con 0x7f1bf80039c0 2024-05-08T04:39:38.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c12a00640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bf80039c0 0x7f1bf8003e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009fc0 tx=0x7f1c0401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a8a7773272a230 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040b2070 con 0x7f1bf80039c0 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c040a5c20 con 0x7f1bf80039c0 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c11600640 1 -- 172.21.15.164:0/1149703816 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04009c10 con 0x7f1bf80039c0 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 -- 172.21.15.164:0/1149703816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bf80039c0 msgr2=0x7f1bf8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 --2- 172.21.15.164:0/1149703816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bf80039c0 0x7f1bf8003e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009fc0 tx=0x7f1c0401cc90 comp rx=0 tx=0).stop 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c13400640 1 -- 172.21.15.164:0/1149703816 reap_dead start 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 -- 172.21.15.164:0/1149703816 shutdown_connections 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 -- 172.21.15.164:0/1149703816 >> 172.21.15.164:0/1149703816 conn(0x7f1c140cf770 msgr2=0x7f1c140cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 -- 172.21.15.164:0/1149703816 shutdown_connections 2024-05-08T04:39:38.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.093+0000 7f1c19600640 1 -- 172.21.15.164:0/1149703816 wait complete. 2024-05-08T04:39:38.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 Processor -- start 2024-05-08T04:39:38.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- start start 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf80039c0 0x7f1c14153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c14161550 0x7f1c14156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406e3b0 con 0x7f1c141610d0 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406e230 con 0x7f1bf80039c0 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406e530 con 0x7f1c14161550 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43738/0 (socket says 172.21.15.164:43738) 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 -- 172.21.15.164:0/2160112269 learned_addr learned my addr 172.21.15.164:0/2160112269 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12a00640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf80039c0 0x7f1c14153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c13400640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c14161550 0x7f1c14156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 -- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c14161550 msgr2=0x7f1c14156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c14161550 0x7f1c14156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 -- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf80039c0 msgr2=0x7f1c14153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf80039c0 0x7f1c14153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 -- 172.21.15.164:0/2160112269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c14153660 con 0x7f1c141610d0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c12000640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c000115e0 tx=0x7f1c00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00015870 con 0x7f1c141610d0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00015a10 con 0x7f1c141610d0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c140caea0 con 0x7f1c141610d0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00015dc0 con 0x7f1c141610d0 2024-05-08T04:39:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.097+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c140cb460 con 0x7f1c141610d0 2024-05-08T04:39:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14072d30 con 0x7f1c141610d0 2024-05-08T04:39:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f1c0000f480 con 0x7f1c141610d0 2024-05-08T04:39:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c0aa00640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 0x7f1bf40694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c00085a90 con 0x7f1c141610d0 2024-05-08T04:39:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c12a00640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 0x7f1bf40694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.101+0000 7f1c12a00640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 0x7f1bf40694e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af000 tx=0x7f1c040a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.105+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c14153660 con 0x7f1c141610d0 2024-05-08T04:39:38.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c140cea50 con 0x7f1c141610d0 2024-05-08T04:39:38.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c0aa00640 1 -- 172.21.15.164:0/2160112269 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c00019030 con 0x7f1c141610d0 2024-05-08T04:39:38.269 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:39:38.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 msgr2=0x7f1bf40694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 0x7f1bf40694e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af000 tx=0x7f1c040a7040 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 msgr2=0x7f1c141568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c000115e0 tx=0x7f1c00011aa0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 shutdown_connections 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bf4067020 0x7f1bf40694e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c14161550 0x7f1c14156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c141610d0 0x7f1c141568a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 --2- 172.21.15.164:0/2160112269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bf80039c0 0x7f1c14153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 >> 172.21.15.164:0/2160112269 conn(0x7f1c140cf770 msgr2=0x7f1c1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 shutdown_connections 2024-05-08T04:39:38.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.265+0000 7f1c19600640 1 -- 172.21.15.164:0/2160112269 wait complete. 2024-05-08T04:39:38.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:39:38.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 Processor -- start 2024-05-08T04:39:38.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 -- start start 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcaa406cce0 0x7fcaa40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa415e870 con 0x7fcaa415ab10 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cf130 con 0x7fcaa406cce0 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaac000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa406bb80 con 0x7fcaa40cecc0 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcaa406cce0 0x7fcaa40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcaa406cce0 0x7fcaa40ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43754/0 (socket says 172.21.15.164:43754) 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 -- 172.21.15.164:0/48830461 learned_addr learned my addr 172.21.15.164:0/48830461 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaaa000640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 -- 172.21.15.164:0/48830461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 msgr2=0x7fcaa40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 -- 172.21.15.164:0/48830461 >> v1:172.21.15.172:6789/0 conn(0x7fcaa415ab10 legacy=0x7fcaa40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.425+0000 7fcaa8c00640 1 -- 172.21.15.164:0/48830461 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa415eb50 con 0x7fcaa406cce0 2024-05-08T04:39:38.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaa8c00640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa406cce0 0x7fcaa40ce780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd6b7132bc3ba99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaa3e00640 1 -- 172.21.15.164:0/48830461 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900aa600 con 0x7fcaa406cce0 2024-05-08T04:39:38.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaa3e00640 1 -- 172.21.15.164:0/48830461 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900af070 con 0x7fcaa406cce0 2024-05-08T04:39:38.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaa3e00640 1 -- 172.21.15.164:0/48830461 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900aa9c0 con 0x7fcaa406cce0 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 -- 172.21.15.164:0/48830461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa406cce0 msgr2=0x7fcaa40ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa406cce0 0x7fcaa40ce780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 -- 172.21.15.164:0/48830461 shutdown_connections 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 --2- 172.21.15.164:0/48830461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa406cce0 0x7fcaa40ce780 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 -- 172.21.15.164:0/48830461 >> 172.21.15.164:0/48830461 conn(0x7fcaa40cf770 msgr2=0x7fcaa40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 -- 172.21.15.164:0/48830461 shutdown_connections 2024-05-08T04:39:38.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.429+0000 7fcaac000640 1 -- 172.21.15.164:0/48830461 wait complete. 2024-05-08T04:39:38.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 Processor -- start 2024-05-08T04:39:38.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 -- start start 2024-05-08T04:39:38.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa406cce0 0x7fcaa41f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40aedf0 con 0x7fcaa415ab10 2024-05-08T04:39:38.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415aef0 con 0x7fcaa406cce0 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40aef70 con 0x7fcaa40cecc0 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43764/0 (socket says 172.21.15.164:43764) 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 -- 172.21.15.164:0/846039456 learned_addr learned my addr 172.21.15.164:0/846039456 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaa9600640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa406cce0 0x7fcaa41f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaa8c00640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 -- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 msgr2=0x7fcaa40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 -- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa406cce0 msgr2=0x7fcaa41f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa406cce0 0x7fcaa41f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 -- 172.21.15.164:0/846039456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40b54d0 con 0x7fcaa415ab10 2024-05-08T04:39:38.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaaa000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca9c007e60 tx=0x7fca9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c014420 con 0x7fcaa415ab10 2024-05-08T04:39:38.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c019070 con 0x7fcaa415ab10 2024-05-08T04:39:38.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c014820 con 0x7fcaa415ab10 2024-05-08T04:39:38.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.433+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40b4c70 con 0x7fcaa415ab10 2024-05-08T04:39:38.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa40b4f10 con 0x7fcaa415ab10 2024-05-08T04:39:38.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa4072d30 con 0x7fcaa415ab10 2024-05-08T04:39:38.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fca9c0149c0 con 0x7fcaa415ab10 2024-05-08T04:39:38.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaa1600640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 0x7fca840694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca9c085710 con 0x7fcaa415ab10 2024-05-08T04:39:38.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaa9600640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 0x7fca840694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.437+0000 7fcaa9600640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 0x7fca840694b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca98001770 tx=0x7fca98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.441+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca9c056070 con 0x7fcaa415ab10 2024-05-08T04:39:38.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.593+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcaa404f010 con 0x7fcaa415ab10 2024-05-08T04:39:38.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.593+0000 7fcaa1600640 1 -- 172.21.15.164:0/846039456 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca9c05a110 con 0x7fcaa415ab10 2024-05-08T04:39:38.600 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:39:38.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 msgr2=0x7fca840694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 0x7fca840694b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca98001770 tx=0x7fca98009040 comp rx=0 tx=0).stop 2024-05-08T04:39:38.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 msgr2=0x7fcaa40b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca9c007e60 tx=0x7fca9c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 shutdown_connections 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca84066ff0 0x7fca840694b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cecc0 0x7fcaa40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa415ab10 0x7fcaa40b29f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 --2- 172.21.15.164:0/846039456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa406cce0 0x7fcaa41f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 >> 172.21.15.164:0/846039456 conn(0x7fcaa40cf770 msgr2=0x7fcaa40ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 shutdown_connections 2024-05-08T04:39:38.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.597+0000 7fcaac000640 1 -- 172.21.15.164:0/846039456 wait complete. 2024-05-08T04:39:38.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:38.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 Processor -- start 2024-05-08T04:39:38.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 -- start start 2024-05-08T04:39:38.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93c8144740 0x7f93c8160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93c80cf9b0 0x7f93c80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c8145540 con 0x7f93c806ac90 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c80ce970 con 0x7f93c80cf9b0 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c8144b20 con 0x7f93c8144740 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93bd400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93c80cf9b0 0x7f93c80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c6800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f93c806ac90 0x7f93c806d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45016/0 (socket says 172.21.15.164:45016) 2024-05-08T04:39:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c6800640 1 -- 172.21.15.164:0/1567373314 learned_addr learned my addr 172.21.15.164:0/1567373314 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c5e00640 1 --2- 172.21.15.164:0/1567373314 >> v2:172.21.15.172:3300/0 conn(0x7f93c8144740 0x7f93c8160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93bd400640 1 -- 172.21.15.164:0/1567373314 >> v2:172.21.15.172:3300/0 conn(0x7f93c8144740 msgr2=0x7f93c8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93bd400640 1 --2- 172.21.15.164:0/1567373314 >> v2:172.21.15.172:3300/0 conn(0x7f93c8144740 0x7f93c8160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93bd400640 1 -- 172.21.15.164:0/1567373314 >> v1:172.21.15.164:6789/0 conn(0x7f93c806ac90 legacy=0x7f93c806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93bd400640 1 -- 172.21.15.164:0/1567373314 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80cebf0 con 0x7f93c80cf9b0 2024-05-08T04:39:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.753+0000 7f93c5400640 1 -- 172.21.15.164:0/1567373314 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406213434 0 0) 0x7f93c00014a0 con 0x7f93c806ac90 2024-05-08T04:39:38.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93bd400640 1 --2- 172.21.15.164:0/1567373314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c80d0150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93c80b5b40 tx=0x7f93c806e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75f72cd3958cf21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c5400640 1 -- 172.21.15.164:0/1567373314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b1070 con 0x7f93c80cf9b0 2024-05-08T04:39:38.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c5400640 1 -- 172.21.15.164:0/1567373314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b00ac620 con 0x7f93c80cf9b0 2024-05-08T04:39:38.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c5400640 1 -- 172.21.15.164:0/1567373314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00ac9f0 con 0x7f93c80cf9b0 2024-05-08T04:39:38.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- 172.21.15.164:0/1567373314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 msgr2=0x7f93c80d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 --2- 172.21.15.164:0/1567373314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c80d0150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93c80b5b40 tx=0x7f93c806e180 comp rx=0 tx=0).stop 2024-05-08T04:39:38.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- 172.21.15.164:0/1567373314 shutdown_connections 2024-05-08T04:39:38.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 --2- 172.21.15.164:0/1567373314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c80d0150 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 --2- 172.21.15.164:0/1567373314 >> v2:172.21.15.172:3300/0 conn(0x7f93c8144740 0x7f93c8160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- 172.21.15.164:0/1567373314 >> 172.21.15.164:0/1567373314 conn(0x7f93c806c3a0 msgr2=0x7f93c80b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- 172.21.15.164:0/1567373314 shutdown_connections 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- 172.21.15.164:0/1567373314 wait complete. 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 Processor -- start 2024-05-08T04:39:38.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.757+0000 7f93c7200640 1 -- start start 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c806ac90 0x7f93c8165600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93c8144740 0x7f93c815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8168990 con 0x7f93c80cf9b0 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c806b4d0 con 0x7f93c806ac90 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8168b10 con 0x7f93c8144740 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43780/0 (socket says 172.21.15.164:43780) 2024-05-08T04:39:38.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 -- 172.21.15.164:0/2462820956 learned_addr learned my addr 172.21.15.164:0/2462820956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:38.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 -- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93c8144740 msgr2=0x7f93c815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93c8144740 0x7f93c815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 -- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c806ac90 msgr2=0x7f93c8165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c806ac90 0x7f93c8165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 -- 172.21.15.164:0/2462820956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c815ee00 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bd400640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f93b00af040 tx=0x7f93b0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b9020 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b00b5410 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b1040 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c8164b20 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c81650e0 con 0x7f93c80cf9b0 2024-05-08T04:39:38.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c804ef30 con 0x7f93c80cf9b0 2024-05-08T04:39:38.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.761+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f93b00bc020 con 0x7f93c80cf9b0 2024-05-08T04:39:38.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.765+0000 7f93bea00640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 0x7f93a40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:38.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.765+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b011ef20 con 0x7f93c80cf9b0 2024-05-08T04:39:38.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.765+0000 7f93c5e00640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 0x7f93a40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:38.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.765+0000 7f93c5e00640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 0x7f93a40694b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f93c813d040 tx=0x7f93b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:38.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.769+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93b00c3030 con 0x7f93c80cf9b0 2024-05-08T04:39:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93c81652b0 con 0x7f93c80cf9b0 2024-05-08T04:39:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93bea00640 1 -- 172.21.15.164:0/2462820956 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f93b0016070 con 0x7f93c80cf9b0 2024-05-08T04:39:38.957 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:38.957 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 18 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 msgr2=0x7f93a40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 0x7f93a40694b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f93c813d040 tx=0x7f93b8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 msgr2=0x7f93c815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f93b00af040 tx=0x7f93b0019160 comp rx=0 tx=0).stop 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 shutdown_connections 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93a4066ff0 0x7f93a40694b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93c8144740 0x7f93c815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c80cf9b0 0x7f93c815b010 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 --2- 172.21.15.164:0/2462820956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c806ac90 0x7f93c8165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 >> 172.21.15.164:0/2462820956 conn(0x7f93c806c3a0 msgr2=0x7f93c8141eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 shutdown_connections 2024-05-08T04:39:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:38.953+0000 7f93c7200640 1 -- 172.21.15.164:0/2462820956 wait complete. 2024-05-08T04:39:38.978 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:39.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 Processor -- start 2024-05-08T04:39:39.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 -- start start 2024-05-08T04:39:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f034c15ab60 0x7f034c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c072640 con 0x7f034c0cfda0 2024-05-08T04:39:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c06c770 con 0x7f034c0b26c0 2024-05-08T04:39:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0353000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c06ca50 con 0x7f034c15ab60 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f034be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f034c15ab60 0x7f034c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0351000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f034c0cfda0 0x7f034c0cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45032/0 (socket says 172.21.15.164:45032) 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f0351000640 1 -- 172.21.15.164:0/4281916488 learned_addr learned my addr 172.21.15.164:0/4281916488 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f034be00640 1 --2- 172.21.15.164:0/4281916488 >> v2:172.21.15.164:3300/0 conn(0x7f034c15ab60 0x7f034c072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43782/0 (socket says 172.21.15.164:43782) 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.109+0000 7f034be00640 1 -- 172.21.15.164:0/4281916488 >> v1:172.21.15.172:6789/0 conn(0x7f034c0b26c0 legacy=0x7f034c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034be00640 1 -- 172.21.15.164:0/4281916488 >> v1:172.21.15.164:6789/0 conn(0x7f034c0cfda0 legacy=0x7f034c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034be00640 1 -- 172.21.15.164:0/4281916488 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c06cd30 con 0x7f034c15ab60 2024-05-08T04:39:39.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034be00640 1 --2- 172.21.15.164:0/4281916488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c072100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f033c0097c0 tx=0x7f033c01e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5671b4d12ca08b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034b400640 1 -- 172.21.15.164:0/4281916488 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0ac620 con 0x7f034c15ab60 2024-05-08T04:39:39.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034b400640 1 -- 172.21.15.164:0/4281916488 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c0b1070 con 0x7f034c15ab60 2024-05-08T04:39:39.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f034b400640 1 -- 172.21.15.164:0/4281916488 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0ac9e0 con 0x7f034c15ab60 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 -- 172.21.15.164:0/4281916488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 msgr2=0x7f034c072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 --2- 172.21.15.164:0/4281916488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c072100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f033c0097c0 tx=0x7f033c01e6b0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 -- 172.21.15.164:0/4281916488 shutdown_connections 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 --2- 172.21.15.164:0/4281916488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c072100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 -- 172.21.15.164:0/4281916488 >> 172.21.15.164:0/4281916488 conn(0x7f034c06ac90 msgr2=0x7f034c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.113+0000 7f0353000640 1 -- 172.21.15.164:0/4281916488 shutdown_connections 2024-05-08T04:39:39.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 -- 172.21.15.164:0/4281916488 wait complete. 2024-05-08T04:39:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 Processor -- start 2024-05-08T04:39:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 -- start start 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f034c0b26c0 0x7f034c15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c0cfda0 0x7f034c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0aeeb0 con 0x7f034c15ab60 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0aed30 con 0x7f034c0cfda0 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0353000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0cc1b0 con 0x7f034c0b26c0 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43796/0 (socket says 172.21.15.164:43796) 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 -- 172.21.15.164:0/3135758344 learned_addr learned my addr 172.21.15.164:0/3135758344 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:39.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f034be00640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f034c0b26c0 0x7f034c15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0343400640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c0cfda0 0x7f034c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 -- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f034c0b26c0 msgr2=0x7f034c15bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f034c0b26c0 0x7f034c15bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 -- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c0cfda0 msgr2=0x7f034c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c0cfda0 0x7f034c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 -- 172.21.15.164:0/3135758344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c15dc10 con 0x7f034c15ab60 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0351000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f034400caf0 tx=0x7f034400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.117+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344017070 con 0x7f034c15ab60 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034400aca0 con 0x7f034c15ab60 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c1fa440 con 0x7f034c15ab60 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344012600 con 0x7f034c15ab60 2024-05-08T04:39:39.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c14d600 con 0x7f034c15ab60 2024-05-08T04:39:39.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c069d60 con 0x7f034c15ab60 2024-05-08T04:39:39.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f034401a070 con 0x7f034c15ab60 2024-05-08T04:39:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0348c00640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 0x7f03280694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0344084da0 con 0x7f034c15ab60 2024-05-08T04:39:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f034be00640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 0x7f03280694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.121+0000 7f034be00640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 0x7f03280694b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f033c0b0000 tx=0x7f033c0a9000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.125+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0344055670 con 0x7f034c15ab60 2024-05-08T04:39:39.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.285+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f034c06d9a0 con 0x7f034c15ab60 2024-05-08T04:39:39.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0348c00640 1 -- 172.21.15.164:0/3135758344 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f0344059710 con 0x7f034c15ab60 2024-05-08T04:39:39.292 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:39.293 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:39.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 msgr2=0x7f03280694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 0x7f03280694b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f033c0b0000 tx=0x7f033c0a9000 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 msgr2=0x7f034c14d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f034400caf0 tx=0x7f034400a060 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 shutdown_connections 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0328066ff0 0x7f03280694b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034c15ab60 0x7f034c14d0c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c0cfda0 0x7f034c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 --2- 172.21.15.164:0/3135758344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f034c0b26c0 0x7f034c15bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 >> 172.21.15.164:0/3135758344 conn(0x7f034c06ac90 msgr2=0x7f034c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 shutdown_connections 2024-05-08T04:39:39.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.289+0000 7f0353000640 1 -- 172.21.15.164:0/3135758344 wait complete. 2024-05-08T04:39:39.313 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:39:39.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 Processor -- start 2024-05-08T04:39:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 -- start start 2024-05-08T04:39:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2bfc15ba70 0x7f2bfc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2bfc0aeda0 0x7f2bfc0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc15fe70 con 0x7f2bfc15ba70 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cd9f0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c03600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cdcd0 con 0x7f2bfc15be50 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2bfc15ba70 0x7f2bfc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2bfc15ba70 0x7f2bfc160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49878/0 (socket says 172.21.15.164:49878) 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 -- 172.21.15.164:0/169730975 learned_addr learned my addr 172.21.15.164:0/169730975 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2bfbe00640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c01600640 1 --2- 172.21.15.164:0/169730975 >> v2:172.21.15.172:3300/0 conn(0x7f2bfc0aeda0 0x7f2bfc0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 -- 172.21.15.164:0/169730975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 msgr2=0x7f2bfc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 -- 172.21.15.164:0/169730975 >> v2:172.21.15.172:3300/0 conn(0x7f2bfc0aeda0 msgr2=0x7f2bfc0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 --2- 172.21.15.164:0/169730975 >> v2:172.21.15.172:3300/0 conn(0x7f2bfc0aeda0 0x7f2bfc0cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.445+0000 7f2c00c00640 1 -- 172.21.15.164:0/169730975 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc15f820 con 0x7f2bfc15ba70 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c00c00640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc160300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b4830 tx=0x7f2bf001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ae60df1dfe3054 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2bfb400640 1 -- 172.21.15.164:0/169730975 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00b1070 con 0x7f2bfc15ba70 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2bfb400640 1 -- 172.21.15.164:0/169730975 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00ac620 con 0x7f2bfc15ba70 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2bfb400640 1 -- 172.21.15.164:0/169730975 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00ac9f0 con 0x7f2bfc15ba70 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 -- 172.21.15.164:0/169730975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc15ba70 msgr2=0x7f2bfc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc160300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b4830 tx=0x7f2bf001c4f0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 -- 172.21.15.164:0/169730975 shutdown_connections 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 --2- 172.21.15.164:0/169730975 >> v2:172.21.15.172:3300/0 conn(0x7f2bfc0aeda0 0x7f2bfc0cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 --2- 172.21.15.164:0/169730975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc160300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 -- 172.21.15.164:0/169730975 >> 172.21.15.164:0/169730975 conn(0x7f2bfc06ac90 msgr2=0x7f2bfc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:39.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 -- 172.21.15.164:0/169730975 shutdown_connections 2024-05-08T04:39:39.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.449+0000 7f2c03600640 1 -- 172.21.15.164:0/169730975 wait complete. 2024-05-08T04:39:39.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 Processor -- start 2024-05-08T04:39:39.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- start start 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b3d40 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b3bc0 con 0x7f2bfc15ba70 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b3ec0 con 0x7f2bfc15be50 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49886/0 (socket says 172.21.15.164:49886) 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 -- 172.21.15.164:0/2686296380 learned_addr learned my addr 172.21.15.164:0/2686296380 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2bfbe00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c01600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 -- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 msgr2=0x7f2bfc0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 -- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bfc15ba70 msgr2=0x7f2bfc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 -- 172.21.15.164:0/2686296380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc1fb9f0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c00c00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bf001cd90 tx=0x7f2bf001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00b1040 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00bd070 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc070910 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00b7880 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.453+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc070ed0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc069d60 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f2bf00ac4a0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2bf8c00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 0x7f2bdc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf011f7b0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2bfbe00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 0x7f2bdc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.457+0000 7f2bfbe00640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 0x7f2bdc0694b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc15ad70 tx=0x7f2be8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.461+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bfc1fb9f0 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bfc06e210 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2bf8c00640 1 -- 172.21.15.164:0/2686296380 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f2bf0016110 con 0x7f2bfc0aeda0 2024-05-08T04:39:39.661 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:39.661 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 18 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 msgr2=0x7f2bdc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 0x7f2bdc0694b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc15ad70 tx=0x7f2be8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 msgr2=0x7f2bfc1fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bf001cd90 tx=0x7f2bf001ad30 comp rx=0 tx=0).stop 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 shutdown_connections 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2bdc066ff0 0x7f2bdc0694b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bfc15be50 0x7f2bfc0719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bfc15ba70 0x7f2bfc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 --2- 172.21.15.164:0/2686296380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bfc0aeda0 0x7f2bfc1fb4b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 >> 172.21.15.164:0/2686296380 conn(0x7f2bfc06ac90 msgr2=0x7f2bfc0ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:39.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 shutdown_connections 2024-05-08T04:39:39.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.657+0000 7f2c03600640 1 -- 172.21.15.164:0/2686296380 wait complete. 2024-05-08T04:39:39.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:39.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 Processor -- start 2024-05-08T04:39:39.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 -- start start 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f309c0cfd60 0x7f309c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 0x7f309c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c162880 con 0x7f309c0cfd60 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c161fe0 con 0x7f309c0cc5f0 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a4600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1622c0 con 0x7f309c144200 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f309c0cfd60 0x7f309c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f309c0cfd60 0x7f309c0cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49898/0 (socket says 172.21.15.164:49898) 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 -- 172.21.15.164:0/471284789 learned_addr learned my addr 172.21.15.164:0/471284789 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1c00640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a2600640 1 --2- 172.21.15.164:0/471284789 >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 0x7f309c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 -- 172.21.15.164:0/471284789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 msgr2=0x7f309c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 -- 172.21.15.164:0/471284789 >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 msgr2=0x7f309c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 --2- 172.21.15.164:0/471284789 >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 0x7f309c15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a1200640 1 -- 172.21.15.164:0/471284789 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c1625a0 con 0x7f309c0cfd60 2024-05-08T04:39:39.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.813+0000 7f30a2600640 1 --2- 172.21.15.164:0/471284789 >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 0x7f309c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:39.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a1200640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c0cc0b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f309c0b4d90 tx=0x7f308801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e20a073679c533 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f309be00640 1 -- 172.21.15.164:0/471284789 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880b0070 con 0x7f309c0cfd60 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f309be00640 1 -- 172.21.15.164:0/471284789 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30880ab600 con 0x7f309c0cfd60 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f309be00640 1 -- 172.21.15.164:0/471284789 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880ab9d0 con 0x7f309c0cfd60 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 -- 172.21.15.164:0/471284789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 msgr2=0x7f309c0cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c0cc0b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f309c0b4d90 tx=0x7f308801c490 comp rx=0 tx=0).stop 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 -- 172.21.15.164:0/471284789 shutdown_connections 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 --2- 172.21.15.164:0/471284789 >> v2:172.21.15.172:3300/0 conn(0x7f309c0cc5f0 0x7f309c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c0cc0b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 --2- 172.21.15.164:0/471284789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 -- 172.21.15.164:0/471284789 >> 172.21.15.164:0/471284789 conn(0x7f309c06ac90 msgr2=0x7f309c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 -- 172.21.15.164:0/471284789 shutdown_connections 2024-05-08T04:39:39.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.817+0000 7f30a4600640 1 -- 172.21.15.164:0/471284789 wait complete. 2024-05-08T04:39:39.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 Processor -- start 2024-05-08T04:39:39.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- start start 2024-05-08T04:39:39.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f309c0cc5f0 0x7f309c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c06c3d0 con 0x7f309c0cfd60 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cf400 con 0x7f309c0cc5f0 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c06c550 con 0x7f309c144200 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49906/0 (socket says 172.21.15.164:49906) 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 -- 172.21.15.164:0/1617827592 learned_addr learned my addr 172.21.15.164:0/1617827592 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1c00640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f309c0cc5f0 0x7f309c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a2600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 -- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 msgr2=0x7f309c13c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c13c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 -- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f309c0cc5f0 msgr2=0x7f309c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f309c0cc5f0 0x7f309c15d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 -- 172.21.15.164:0/1617827592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c139fb0 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a1200640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f308801ab40 tx=0x7f3088019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880b9070 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30880b4400 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c13a290 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880b0040 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.821+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c13a850 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c069d60 con 0x7f309c0cfd60 2024-05-08T04:39:39.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f30880bc070 con 0x7f309c0cfd60 2024-05-08T04:39:39.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f3099600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 0x7f307c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:39.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f308811e370 con 0x7f309c0cfd60 2024-05-08T04:39:39.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f30a1c00640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 0x7f307c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:39.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.825+0000 7f30a1c00640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 0x7f307c0694b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f309000d8d0 tx=0x7f3090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:39.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:39.829+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3088016110 con 0x7f309c0cfd60 2024-05-08T04:39:40.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.013+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f309c0af110 con 0x7f309c0cfd60 2024-05-08T04:39:40.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.013+0000 7f3099600640 1 -- 172.21.15.164:0/1617827592 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f30880ef330 con 0x7f309c0cfd60 2024-05-08T04:39:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:40.019 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:21.526329+0000","modified":"2024-05-08T04:39:24.549738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/3451370637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3451370637},{"type":"v1","addr":"172.21.15.172:6835","nonce":3451370637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T04:39:40.019 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 18 2024-05-08T04:39:40.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 msgr2=0x7f307c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 0x7f307c0694b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f309000d8d0 tx=0x7f3090008040 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 msgr2=0x7f309c13ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f308801ab40 tx=0x7f3088019230 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 shutdown_connections 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f307c066ff0 0x7f307c0694b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309c144200 0x7f309c13c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f309c0cfd60 0x7f309c13ad70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 --2- 172.21.15.164:0/1617827592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f309c0cc5f0 0x7f309c15d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 >> 172.21.15.164:0/1617827592 conn(0x7f309c06ac90 msgr2=0x7f309c0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 shutdown_connections 2024-05-08T04:39:40.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.017+0000 7f30a4600640 1 -- 172.21.15.164:0/1617827592 wait complete. 2024-05-08T04:39:40.040 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:40.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 Processor -- start 2024-05-08T04:39:40.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 -- start start 2024-05-08T04:39:40.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e1c0afe50 0x7f8e1c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 0x7f8e1c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c1625b0 con 0x7f8e1c1621d0 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0b02b0 con 0x7f8e1c0afe50 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e21800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c071480 con 0x7f8e1c06b1d0 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e1c0afe50 0x7f8e1c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e1c0afe50 0x7f8e1c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49912/0 (socket says 172.21.15.164:49912) 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 -- 172.21.15.164:0/1447664274 learned_addr learned my addr 172.21.15.164:0/1447664274 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 -- 172.21.15.164:0/1447664274 >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 msgr2=0x7f8e1c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:40.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1b400640 1 --2- 172.21.15.164:0/1447664274 >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 0x7f8e1c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 --2- 172.21.15.164:0/1447664274 >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 0x7f8e1c0b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 -- 172.21.15.164:0/1447664274 >> v1:172.21.15.172:6789/0 conn(0x7f8e1c1621d0 legacy=0x7f8e1c0721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:40.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1a000640 1 -- 172.21.15.164:0/1447664274 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c06d900 con 0x7f8e1c0afe50 2024-05-08T04:39:40.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.173+0000 7f8e1b400640 1 --2- 172.21.15.164:0/1447664274 >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 0x7f8e1c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:40.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e1a000640 1 --2- 172.21.15.164:0/1447664274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c06ac90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0d1800 tx=0x7f8e0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f89dd590a49aba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e19600640 1 -- 172.21.15.164:0/1447664274 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b0070 con 0x7f8e1c0afe50 2024-05-08T04:39:40.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e19600640 1 -- 172.21.15.164:0/1447664274 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0ab600 con 0x7f8e1c0afe50 2024-05-08T04:39:40.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e19600640 1 -- 172.21.15.164:0/1447664274 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0ab9d0 con 0x7f8e1c0afe50 2024-05-08T04:39:40.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 -- 172.21.15.164:0/1447664274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 msgr2=0x7f8e1c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 --2- 172.21.15.164:0/1447664274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c06ac90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0d1800 tx=0x7f8e0c01c450 comp rx=0 tx=0).stop 2024-05-08T04:39:40.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 -- 172.21.15.164:0/1447664274 shutdown_connections 2024-05-08T04:39:40.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 --2- 172.21.15.164:0/1447664274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c06ac90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 --2- 172.21.15.164:0/1447664274 >> v2:172.21.15.172:3300/0 conn(0x7f8e1c06b1d0 0x7f8e1c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 -- 172.21.15.164:0/1447664274 >> 172.21.15.164:0/1447664274 conn(0x7f8e1c066fa0 msgr2=0x7f8e1c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 -- 172.21.15.164:0/1447664274 shutdown_connections 2024-05-08T04:39:40.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.177+0000 7f8e21800640 1 -- 172.21.15.164:0/1447664274 wait complete. 2024-05-08T04:39:40.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 Processor -- start 2024-05-08T04:39:40.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- start start 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 0x7f8e1c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 0x7f8e1c162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c06d660 con 0x7f8e1c0afe50 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c06d4e0 con 0x7f8e1c06b1d0 2024-05-08T04:39:40.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0b4120 con 0x7f8e1c1621d0 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49922/0 (socket says 172.21.15.164:49922) 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 -- 172.21.15.164:0/1533320648 learned_addr learned my addr 172.21.15.164:0/1533320648 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1aa00640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 0x7f8e1c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1b400640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 0x7f8e1c162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 -- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 msgr2=0x7f8e1c162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 0x7f8e1c162780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 -- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 msgr2=0x7f8e1c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 0x7f8e1c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 -- 172.21.15.164:0/1533320648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c15f780 con 0x7f8e1c0afe50 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1aa00640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 0x7f8e1c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1b400640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 0x7f8e1c162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:40.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e1a000640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ae040 tx=0x7f8e0c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0ba020 con 0x7f8e1c0afe50 2024-05-08T04:39:40.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0b6640 con 0x7f8e1c0afe50 2024-05-08T04:39:40.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b0040 con 0x7f8e1c0afe50 2024-05-08T04:39:40.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c15fa60 con 0x7f8e1c0afe50 2024-05-08T04:39:40.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c160020 con 0x7f8e1c0afe50 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.181+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c0cd810 con 0x7f8e1c0afe50 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.185+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f8e0c0bd020 con 0x7f8e1c0afe50 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.185+0000 7f8e12a00640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 0x7f8df80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.185+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e0c11f520 con 0x7f8e1c0afe50 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.185+0000 7f8e1aa00640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 0x7f8df80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.185+0000 7f8e1aa00640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 0x7f8df80694b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c1fe8b0 tx=0x7f8e04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.189+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e0c0c4030 con 0x7f8e1c0afe50 2024-05-08T04:39:40.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e1c069d60 con 0x7f8e1c0afe50 2024-05-08T04:39:40.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e12a00640 1 -- 172.21.15.164:0/1533320648 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f8e0c0160d0 con 0x7f8e1c0afe50 2024-05-08T04:39:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:40.354 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:23.146668+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:20.033719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:20.517477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 msgr2=0x7f8df80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 0x7f8df80694b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c1fe8b0 tx=0x7f8e04008040 comp rx=0 tx=0).stop 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 msgr2=0x7f8e1c15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ae040 tx=0x7f8e0c01ac90 comp rx=0 tx=0).stop 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 shutdown_connections 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8df8066ff0 0x7f8df80694b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e1c1621d0 0x7f8e1c162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e1c0afe50 0x7f8e1c15bbf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 --2- 172.21.15.164:0/1533320648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1c06b1d0 0x7f8e1c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 >> 172.21.15.164:0/1533320648 conn(0x7f8e1c066fa0 msgr2=0x7f8e1c0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.349+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 shutdown_connections 2024-05-08T04:39:40.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.353+0000 7f8e21800640 1 -- 172.21.15.164:0/1533320648 wait complete. 2024-05-08T04:39:40.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:39:40.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 Processor -- start 2024-05-08T04:39:40.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 -- start start 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48c015be50 0x7f48c0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00cd9f0 con 0x7f48c015ba70 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00cdcd0 con 0x7f48c00cfda0 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48c5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c015fe70 con 0x7f48c015be50 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48bea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f48c015ba70 0x7f48c00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55070/0 (socket says 172.21.15.164:55070) 2024-05-08T04:39:40.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48bea00640 1 -- 172.21.15.164:0/1639926059 learned_addr learned my addr 172.21.15.164:0/1639926059 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48bf400640 1 --2- 172.21.15.164:0/1639926059 >> v2:172.21.15.172:3300/0 conn(0x7f48c015be50 0x7f48c0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604608011 0 0) 0x7f48c00cd9f0 con 0x7f48c015ba70 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.509+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a4003530 con 0x7f48c015ba70 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49138628 0 0) 0x7f48a4003530 con 0x7f48c015ba70 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 >> v2:172.21.15.172:3300/0 conn(0x7f48c015be50 msgr2=0x7f48c0160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 --2- 172.21.15.164:0/1639926059 >> v2:172.21.15.172:3300/0 conn(0x7f48c015be50 0x7f48c0160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 >> v1:172.21.15.172:6789/0 conn(0x7f48c00cfda0 legacy=0x7f48c00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c015f820 con 0x7f48c015ba70 2024-05-08T04:39:40.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f48b0001a00 con 0x7f48c015ba70 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 >> v1:172.21.15.164:6789/0 conn(0x7f48c015ba70 legacy=0x7f48c00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c015fe70 con 0x7f48a4004810 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a400c070 con 0x7f48a40051d0 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a400c270 con 0x7f48a40039c0 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48bf400640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48bea00640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 -- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40039c0 msgr2=0x7f48a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40039c0 0x7f48a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 -- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a40051d0 msgr2=0x7f48a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:40.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a40051d0 0x7f48a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48bf400640 1 -- 172.21.15.164:0/1639926059 reap_dead start 2024-05-08T04:39:40.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 -- 172.21.15.164:0/1639926059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4003530 con 0x7f48a4004810 2024-05-08T04:39:40.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.513+0000 7f48b6000640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48a8001780 tx=0x7f48a801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b46d0cb9f7a246 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a80013a0 con 0x7f48a4004810 2024-05-08T04:39:40.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a80052f0 con 0x7f48a4004810 2024-05-08T04:39:40.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48be000640 1 -- 172.21.15.164:0/1639926059 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003840 con 0x7f48a4004810 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 msgr2=0x7f48a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48a8001780 tx=0x7f48a801c400 comp rx=0 tx=0).stop 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- 172.21.15.164:0/1639926059 shutdown_connections 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 --2- 172.21.15.164:0/1639926059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- 172.21.15.164:0/1639926059 >> 172.21.15.164:0/1639926059 conn(0x7f48c006ac90 msgr2=0x7f48c006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- 172.21.15.164:0/1639926059 shutdown_connections 2024-05-08T04:39:40.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- 172.21.15.164:0/1639926059 wait complete. 2024-05-08T04:39:40.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 Processor -- start 2024-05-08T04:39:40.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- start start 2024-05-08T04:39:40.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 0x7f48c00ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c015c2a0 0x7f48c015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00cd7b0 con 0x7f48a4004810 2024-05-08T04:39:40.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00cd630 con 0x7f48c015c2a0 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48c5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00aeda0 con 0x7f48c00ade30 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48bea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48bea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49938/0 (socket says 172.21.15.164:49938) 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.517+0000 7f48bea00640 1 -- 172.21.15.164:0/3025159057 learned_addr learned my addr 172.21.15.164:0/3025159057 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bf400640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c015c2a0 0x7f48c015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b6000640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 0x7f48c00ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 -- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 msgr2=0x7f48c00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 0x7f48c00ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 -- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c015c2a0 msgr2=0x7f48c015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c015c2a0 0x7f48c015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 -- 172.21.15.164:0/3025159057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c01559f0 con 0x7f48a4004810 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b6000640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 0x7f48c00ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48bea00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f48b0001b70 tx=0x7f48b0007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b001c070 con 0x7f48a4004810 2024-05-08T04:39:40.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0011950 con 0x7f48a4004810 2024-05-08T04:39:40.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0010040 con 0x7f48a4004810 2024-05-08T04:39:40.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c015cfb0 con 0x7f48a4004810 2024-05-08T04:39:40.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c015d570 con 0x7f48a4004810 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c0069d60 con 0x7f48a4004810 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f48b0011d40 con 0x7f48a4004810 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 0x7f489c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.521+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48b00881b0 con 0x7f48a4004810 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.525+0000 7f48b6000640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 0x7f489c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.525+0000 7f48b6000640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 0x7f489c0694b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48a8004000 tx=0x7f48a80011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.525+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48c01559f0 con 0x7f48a4004810 2024-05-08T04:39:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.713+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f48c015b0e0 con 0x7f48a4004810 2024-05-08T04:39:40.717 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:39:40.713+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:39:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.713+0000 7f48b7400640 1 -- 172.21.15.164:0/3025159057 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48b0058a80 con 0x7f48a4004810 2024-05-08T04:39:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 msgr2=0x7f489c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 0x7f489c0694b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48a8004000 tx=0x7f48a80011a0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 msgr2=0x7f48c01554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f48b0001b70 tx=0x7f48b0007bb0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 shutdown_connections 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f489c066ff0 0x7f489c0694b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ade30 0x7f48c00ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c015c2a0 0x7f48c015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 --2- 172.21.15.164:0/3025159057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a4004810 0x7f48c01554b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 >> 172.21.15.164:0/3025159057 conn(0x7f48c006ac90 msgr2=0x7f48c0157780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 shutdown_connections 2024-05-08T04:39:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.717+0000 7f48c5c00640 1 -- 172.21.15.164:0/3025159057 wait complete. 2024-05-08T04:39:40.731 INFO:tasks.ceph.mds.d.smithi172.stderr: -4134> 2024-05-08T04:39:26.964+0000 7f8be6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:39:40.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:39:40.782 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:39:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.873+0000 7fb490c00640 1 Processor -- start 2024-05-08T04:39:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.873+0000 7fb490c00640 1 -- start start 2024-05-08T04:39:40.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb490c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 0x7fb48c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb490c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0c6600 con 0x7fb48c15ba20 2024-05-08T04:39:40.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb490c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0c6040 con 0x7fb48c06e390 2024-05-08T04:39:40.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb490c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c15fbe0 con 0x7fb48c15ce10 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb48c15ba20 0x7fb48c160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55086/0 (socket says 172.21.15.164:55086) 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489e00640 1 -- 172.21.15.164:0/3261794221 learned_addr learned my addr 172.21.15.164:0/3261794221 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb481400640 1 --2- 172.21.15.164:0/3261794221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 0x7fb48c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320811091 0 0) 0x7fb48c0c6600 con 0x7fb48c15ba20 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb470003530 con 0x7fb48c15ba20 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675068662 0 0) 0x7fb48c0c6040 con 0x7fb48c06e390 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c0c6600 con 0x7fb48c06e390 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb481400640 1 -- 172.21.15.164:0/3261794221 >> v1:172.21.15.172:6789/0 conn(0x7fb48c06e390 legacy=0x7fb48c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb481400640 1 -- 172.21.15.164:0/3261794221 >> v1:172.21.15.164:6789/0 conn(0x7fb48c15ba20 legacy=0x7fb48c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.877+0000 7fb481400640 1 -- 172.21.15.164:0/3261794221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0c6320 con 0x7fb48c15ce10 2024-05-08T04:39:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb481400640 1 --2- 172.21.15.164:0/3261794221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 0x7fb48c06de50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097f0 tx=0x7fb47401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a407537d2b249e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b0070 con 0x7fb48c15ce10 2024-05-08T04:39:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740ab600 con 0x7fb48c15ce10 2024-05-08T04:39:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb489400640 1 -- 172.21.15.164:0/3261794221 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740ab9b0 con 0x7fb48c15ce10 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- 172.21.15.164:0/3261794221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 msgr2=0x7fb48c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 --2- 172.21.15.164:0/3261794221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 0x7fb48c06de50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4740097f0 tx=0x7fb47401c450 comp rx=0 tx=0).stop 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- 172.21.15.164:0/3261794221 shutdown_connections 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 --2- 172.21.15.164:0/3261794221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c15ce10 0x7fb48c06de50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- 172.21.15.164:0/3261794221 >> 172.21.15.164:0/3261794221 conn(0x7fb48c0cf770 msgr2=0x7fb48c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- 172.21.15.164:0/3261794221 shutdown_connections 2024-05-08T04:39:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- 172.21.15.164:0/3261794221 wait complete. 2024-05-08T04:39:40.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 Processor -- start 2024-05-08T04:39:40.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.881+0000 7fb490c00640 1 -- start start 2024-05-08T04:39:40.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c06e390 0x7fb48c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 0x7fb48c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0cde80 con 0x7fb48c15ce10 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0af040 con 0x7fb48c15ba20 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0ce000 con 0x7fb48c06e390 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49944/0 (socket says 172.21.15.164:49944) 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 -- 172.21.15.164:0/2631394500 learned_addr learned my addr 172.21.15.164:0/2631394500 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb489e00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c06e390 0x7fb48c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb481400640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 0x7fb48c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 -- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c06e390 msgr2=0x7fb48c15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c06e390 0x7fb48c15e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 -- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 msgr2=0x7fb48c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 0x7fb48c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 -- 172.21.15.164:0/2631394500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0afe70 con 0x7fb48c15ce10 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb481400640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 0x7fb48c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb48a800640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb48400c3c0 tx=0x7fb48400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484017070 con 0x7fb48c15ce10 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4840052d0 con 0x7fb48c15ce10 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c0b0110 con 0x7fb48c15ce10 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840054c0 con 0x7fb48c15ce10 2024-05-08T04:39:40.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c0b06d0 con 0x7fb48c15ce10 2024-05-08T04:39:40.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c072d30 con 0x7fb48c15ce10 2024-05-08T04:39:40.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.885+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fb48401a070 con 0x7fb48c15ce10 2024-05-08T04:39:40.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.889+0000 7fb482a00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 0x7fb4680694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:40.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.889+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb484084360 con 0x7fb48c15ce10 2024-05-08T04:39:40.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.889+0000 7fb489e00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 0x7fb4680694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:40.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.889+0000 7fb489e00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 0x7fb4680694e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0012f0 tx=0x7fb47c001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:40.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:40.893+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb484054c60 con 0x7fb48c15ce10 2024-05-08T04:39:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.081+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb48c167950 con 0x7fb48c15ce10 2024-05-08T04:39:41.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.717+0000 7fb482a00640 1 -- 172.21.15.164:0/2631394500 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb484058d00 con 0x7fb48c15ce10 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 msgr2=0x7fb4680694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 0x7fb4680694e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0012f0 tx=0x7fb47c001aa0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 msgr2=0x7fb48c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb48400c3c0 tx=0x7fb48400c880 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 shutdown_connections 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb468067020 0x7fb4680694e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb48c15ce10 0x7fb48c0cafe0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c15ba20 0x7fb48c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 --2- 172.21.15.164:0/2631394500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb48c06e390 0x7fb48c15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 >> 172.21.15.164:0/2631394500 conn(0x7fb48c0cf770 msgr2=0x7fb48c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:41.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 shutdown_connections 2024-05-08T04:39:41.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.721+0000 7fb490c00640 1 -- 172.21.15.164:0/2631394500 wait complete. 2024-05-08T04:39:41.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:39:41.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 Processor -- start 2024-05-08T04:39:41.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 -- start start 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd16015ab10 0x7fd16006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd1600aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1600ce780 0x7fd1600cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1600cf130 con 0x7fd1600ce780 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16006bb80 con 0x7fd16015ab10 2024-05-08T04:39:41.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd166a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16015e870 con 0x7fd1600ae6e0 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd164a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1600ce780 0x7fd1600cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd164a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1600ce780 0x7fd1600cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49952/0 (socket says 172.21.15.164:49952) 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd164a00640 1 -- 172.21.15.164:0/2316865087 learned_addr learned my addr 172.21.15.164:0/2316865087 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd157400640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd1600aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd15fe00640 1 --2- 172.21.15.164:0/2316865087 >> v2:172.21.15.172:3300/0 conn(0x7fd16015ab10 0x7fd16006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd164a00640 1 -- 172.21.15.164:0/2316865087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 msgr2=0x7fd1600aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.885+0000 7fd164a00640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd1600aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd164a00640 1 -- 172.21.15.164:0/2316865087 >> v2:172.21.15.172:3300/0 conn(0x7fd16015ab10 msgr2=0x7fd16006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd164a00640 1 --2- 172.21.15.164:0/2316865087 >> v2:172.21.15.172:3300/0 conn(0x7fd16015ab10 0x7fd16006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd164a00640 1 -- 172.21.15.164:0/2316865087 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16015eb50 con 0x7fd1600ce780 2024-05-08T04:39:41.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd164a00640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1600ce780 0x7fd1600cebf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd1600c4920 tx=0x7fd15801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599d0aa243d79cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:41.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd15f400640 1 -- 172.21.15.164:0/2316865087 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580a57b0 con 0x7fd1600ce780 2024-05-08T04:39:41.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd15f400640 1 -- 172.21.15.164:0/2316865087 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580a5950 con 0x7fd1600ce780 2024-05-08T04:39:41.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd15f400640 1 -- 172.21.15.164:0/2316865087 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580a5d20 con 0x7fd1600ce780 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 -- 172.21.15.164:0/2316865087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1600ce780 msgr2=0x7fd1600cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1600ce780 0x7fd1600cebf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd1600c4920 tx=0x7fd15801c930 comp rx=0 tx=0).stop 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 -- 172.21.15.164:0/2316865087 shutdown_connections 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1600ce780 0x7fd1600cebf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 --2- 172.21.15.164:0/2316865087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd1600aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 --2- 172.21.15.164:0/2316865087 >> v2:172.21.15.172:3300/0 conn(0x7fd16015ab10 0x7fd16006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 -- 172.21.15.164:0/2316865087 >> 172.21.15.164:0/2316865087 conn(0x7fd1600cf770 msgr2=0x7fd1600cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:41.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 -- 172.21.15.164:0/2316865087 shutdown_connections 2024-05-08T04:39:41.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.889+0000 7fd166a00640 1 -- 172.21.15.164:0/2316865087 wait complete. 2024-05-08T04:39:41.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 Processor -- start 2024-05-08T04:39:41.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- start start 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd160158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1600ce780 0x7fd16014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c5b70 con 0x7fd16015ab10 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16015aef0 con 0x7fd1600ce780 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c5cf0 con 0x7fd1600ae6e0 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49966/0 (socket says 172.21.15.164:49966) 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 -- 172.21.15.164:0/3285698782 learned_addr learned my addr 172.21.15.164:0/3285698782 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:41.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd160158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd157400640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1600ce780 0x7fd16014dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 -- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 msgr2=0x7fd160158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd160158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 -- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1600ce780 msgr2=0x7fd16014dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1600ce780 0x7fd16014dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 -- 172.21.15.164:0/3285698782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1601504f0 con 0x7fd16015ab10 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd164a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd1580af040 tx=0x7fd1580196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580bc020 con 0x7fd16015ab10 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580b8670 con 0x7fd16015ab10 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158009510 con 0x7fd16015ab10 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd160150800 con 0x7fd16015ab10 2024-05-08T04:39:41.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.893+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd160151040 con 0x7fd16015ab10 2024-05-08T04:39:41.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd160072d30 con 0x7fd16015ab10 2024-05-08T04:39:41.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fd1580bf020 con 0x7fd16015ab10 2024-05-08T04:39:41.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd15cc00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 0x7fd13c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:41.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd15811ffd0 con 0x7fd16015ab10 2024-05-08T04:39:41.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd15fe00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 0x7fd13c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:41.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.897+0000 7fd15fe00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 0x7fd13c0694e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd150009a40 tx=0x7fd150009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:41.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:41.901+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1580c6030 con 0x7fd16015ab10 2024-05-08T04:39:42.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.057+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd16004f010 con 0x7fd16015ab10 2024-05-08T04:39:42.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.717+0000 7fd15cc00640 1 -- 172.21.15.164:0/3285698782 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd160150800 con 0x7fd16015ab10 2024-05-08T04:39:42.724 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:39:42.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 msgr2=0x7fd13c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:42.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 0x7fd13c0694e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd150009a40 tx=0x7fd150009190 comp rx=0 tx=0).stop 2024-05-08T04:39:42.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 msgr2=0x7fd160152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:42.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd1580af040 tx=0x7fd1580196d0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 shutdown_connections 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd13c067020 0x7fd13c0694e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd16015ab10 0x7fd160152330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1600ce780 0x7fd16014dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 --2- 172.21.15.164:0/3285698782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1600ae6e0 0x7fd160158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 >> 172.21.15.164:0/3285698782 conn(0x7fd1600cf770 msgr2=0x7fd16014ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.721+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 shutdown_connections 2024-05-08T04:39:42.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.725+0000 7fd166a00640 1 -- 172.21.15.164:0/3285698782 wait complete. 2024-05-08T04:39:42.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:39:42.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.889+0000 7f9499200640 1 Processor -- start 2024-05-08T04:39:42.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.889+0000 7f9499200640 1 -- start start 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f949415ab10 0x7f949406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94940ae6e0 0x7f94940aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 0x7f94940cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940cf130 con 0x7f94940ae6e0 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949406bb80 con 0x7f94940ce780 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9499200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949415e870 con 0x7f949415ab10 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94940ae6e0 0x7f94940aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94940ae6e0 0x7f94940aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49972/0 (socket says 172.21.15.164:49972) 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 -- 172.21.15.164:0/766301796 learned_addr learned my addr 172.21.15.164:0/766301796 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:42.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9492800640 1 --2- 172.21.15.164:0/766301796 >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 0x7f94940cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9491e00640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f949415ab10 0x7f949406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 -- 172.21.15.164:0/766301796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f949415ab10 msgr2=0x7f949406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f949415ab10 0x7f949406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 -- 172.21.15.164:0/766301796 >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 msgr2=0x7f94940cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 --2- 172.21.15.164:0/766301796 >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 0x7f94940cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9489400640 1 -- 172.21.15.164:0/766301796 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949415eb50 con 0x7f94940ae6e0 2024-05-08T04:39:42.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.893+0000 7f9492800640 1 --2- 172.21.15.164:0/766301796 >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 0x7f94940cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:42.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9489400640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f94940aeb40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f949406be30 tx=0x7f947c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621f405d5af9dec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:42.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9491400640 1 -- 172.21.15.164:0/766301796 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0ab620 con 0x7f94940ae6e0 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9491400640 1 -- 172.21.15.164:0/766301796 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c0b0070 con 0x7f94940ae6e0 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9491400640 1 -- 172.21.15.164:0/766301796 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0ab9e0 con 0x7f94940ae6e0 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- 172.21.15.164:0/766301796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 msgr2=0x7f94940aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f94940aeb40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f949406be30 tx=0x7f947c01c460 comp rx=0 tx=0).stop 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- 172.21.15.164:0/766301796 shutdown_connections 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 --2- 172.21.15.164:0/766301796 >> v2:172.21.15.172:3300/0 conn(0x7f94940ce780 0x7f94940cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f94940aeb40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 --2- 172.21.15.164:0/766301796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f949415ab10 0x7f949406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- 172.21.15.164:0/766301796 >> 172.21.15.164:0/766301796 conn(0x7f94940cf770 msgr2=0x7f94940cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- 172.21.15.164:0/766301796 shutdown_connections 2024-05-08T04:39:42.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- 172.21.15.164:0/766301796 wait complete. 2024-05-08T04:39:42.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 Processor -- start 2024-05-08T04:39:42.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.897+0000 7f9499200640 1 -- start start 2024-05-08T04:39:42.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 0x7f949414eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 0x7f949406d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940c5b70 con 0x7f94940ae6e0 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949415aef0 con 0x7f949415ab10 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94940c5cf0 con 0x7f94940ce780 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49974/0 (socket says 172.21.15.164:49974) 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 -- 172.21.15.164:0/2048703665 learned_addr learned my addr 172.21.15.164:0/2048703665 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9492800640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 0x7f949406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9489400640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 0x7f949414eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 -- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 msgr2=0x7f949414eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 0x7f949414eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 -- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 msgr2=0x7f949406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:42.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 0x7f949406d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 -- 172.21.15.164:0/2048703665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494158dc0 con 0x7f94940ae6e0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9489400640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 0x7f949414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9492800640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 0x7f949406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9491e00640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e40 tx=0x7f948400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484014620 con 0x7f94940ae6e0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9484019070 con 0x7f94940ae6e0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94941590a0 con 0x7f94940ae6e0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484014a00 con 0x7f94940ae6e0 2024-05-08T04:39:42.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949415a0c0 con 0x7f94940ae6e0 2024-05-08T04:39:42.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494072d30 con 0x7f94940ae6e0 2024-05-08T04:39:42.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.901+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f9484014ba0 con 0x7f94940ae6e0 2024-05-08T04:39:42.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.905+0000 7f948aa00640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 0x7f94700694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:42.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.905+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f9484085770 con 0x7f94940ae6e0 2024-05-08T04:39:42.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.905+0000 7f9489400640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 0x7f94700694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:42.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.905+0000 7f9489400640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 0x7f94700694b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f947c0af000 tx=0x7f947c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:42.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:42.909+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9484056880 con 0x7f94940ae6e0 2024-05-08T04:39:43.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.061+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94940b53d0 con 0x7f94940ae6e0 2024-05-08T04:39:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.721+0000 7f948aa00640 1 -- 172.21.15.164:0/2048703665 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9484056180 con 0x7f94940ae6e0 2024-05-08T04:39:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:39:43.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 msgr2=0x7f94700694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:43.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 0x7f94700694b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f947c0af000 tx=0x7f947c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:43.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 msgr2=0x7f9494159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:43.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e40 tx=0x7f948400e4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 shutdown_connections 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9470066ff0 0x7f94700694b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94940ce780 0x7f949414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f949415ab10 0x7f949406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 --2- 172.21.15.164:0/2048703665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94940ae6e0 0x7f9494159b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 >> 172.21.15.164:0/2048703665 conn(0x7f94940cf770 msgr2=0x7f9494152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 shutdown_connections 2024-05-08T04:39:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.725+0000 7f9499200640 1 -- 172.21.15.164:0/2048703665 wait complete. 2024-05-08T04:39:43.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:39:43.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.885+0000 7fe181a00640 1 Processor -- start 2024-05-08T04:39:43.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.885+0000 7fe181a00640 1 -- start start 2024-05-08T04:39:43.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe17c15ba20 0x7fe17c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe17c06de50 0x7fe17c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c0ae6e0 0x7fe17c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c15fbe0 con 0x7fe17c15ba20 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0af060 con 0x7fe17c06de50 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe181a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0c6600 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe17c15ba20 0x7fe17c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe17c15ba20 0x7fe17c160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49980/0 (socket says 172.21.15.164:49980) 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 -- 172.21.15.164:0/1202257342 learned_addr learned my addr 172.21.15.164:0/1202257342 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17b400640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c0ae6e0 0x7fe17c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe172000640 1 --2- 172.21.15.164:0/1202257342 >> v2:172.21.15.172:3300/0 conn(0x7fe17c06de50 0x7fe17c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 -- 172.21.15.164:0/1202257342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c0ae6e0 msgr2=0x7fe17c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c0ae6e0 0x7fe17c0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 -- 172.21.15.164:0/1202257342 >> v2:172.21.15.172:3300/0 conn(0x7fe17c06de50 msgr2=0x7fe17c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 --2- 172.21.15.164:0/1202257342 >> v2:172.21.15.172:3300/0 conn(0x7fe17c06de50 0x7fe17c06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.889+0000 7fe17aa00640 1 -- 172.21.15.164:0/1202257342 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0c6040 con 0x7fe17c15ba20 2024-05-08T04:39:43.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe17aa00640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c15ba20 0x7fe17c160070 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0b5980 tx=0x7fe16c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313e823cb8b750cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:43.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe17a000640 1 -- 172.21.15.164:0/1202257342 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0aa610 con 0x7fe17c15ba20 2024-05-08T04:39:43.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe17a000640 1 -- 172.21.15.164:0/1202257342 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c0af070 con 0x7fe17c15ba20 2024-05-08T04:39:43.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe17a000640 1 -- 172.21.15.164:0/1202257342 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0aa9d0 con 0x7fe17c15ba20 2024-05-08T04:39:43.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- 172.21.15.164:0/1202257342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c15ba20 msgr2=0x7fe17c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:43.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c15ba20 0x7fe17c160070 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0b5980 tx=0x7fe16c01c4f0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- 172.21.15.164:0/1202257342 shutdown_connections 2024-05-08T04:39:43.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c0ae6e0 0x7fe17c0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- 172.21.15.164:0/1202257342 >> v2:172.21.15.172:3300/0 conn(0x7fe17c06de50 0x7fe17c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- 172.21.15.164:0/1202257342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c15ba20 0x7fe17c160070 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- 172.21.15.164:0/1202257342 >> 172.21.15.164:0/1202257342 conn(0x7fe17c0cf770 msgr2=0x7fe17c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:43.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- 172.21.15.164:0/1202257342 shutdown_connections 2024-05-08T04:39:43.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- 172.21.15.164:0/1202257342 wait complete. 2024-05-08T04:39:43.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 Processor -- start 2024-05-08T04:39:43.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- start start 2024-05-08T04:39:43.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe17c06de50 0x7fe17c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c15ba20 0x7fe17c070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cde80 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1605b0 con 0x7fe17c06de50 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.893+0000 7fe181a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0ce000 con 0x7fe17c15ba20 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49992/0 (socket says 172.21.15.164:49992) 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 -- 172.21.15.164:0/4142870531 learned_addr learned my addr 172.21.15.164:0/4142870531 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe17aa00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe17c06de50 0x7fe17c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe17b400640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c15ba20 0x7fe17c070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 -- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c15ba20 msgr2=0x7fe17c070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c15ba20 0x7fe17c070fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 -- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe17c06de50 msgr2=0x7fe17c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:43.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe17c06de50 0x7fe17c0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:43.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 -- 172.21.15.164:0/4142870531 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c153cf0 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe172000640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe16400d830 tx=0x7fe16400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:43.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164018020 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1640096a0 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164009a50 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c071510 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0b4ad0 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c072d30 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fe16401b020 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.897+0000 7fe173400640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 0x7fe1580694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:43.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.901+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe1640853e0 con 0x7fe17c0ae6e0 2024-05-08T04:39:43.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.901+0000 7fe17aa00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 0x7fe1580694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:43.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.901+0000 7fe17aa00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 0x7fe1580694e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ae000 tx=0x7fe16c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:43.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:43.901+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe164022020 con 0x7fe17c0ae6e0 2024-05-08T04:39:44.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.061+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe17c04f010 con 0x7fe17c0ae6e0 2024-05-08T04:39:44.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.061+0000 7fe173400640 1 -- 172.21.15.164:0/4142870531 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe164056690 con 0x7fe17c0ae6e0 2024-05-08T04:39:44.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 msgr2=0x7fe1580694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 0x7fe1580694e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ae000 tx=0x7fe16c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 msgr2=0x7fe17c1537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe16400d830 tx=0x7fe16400dcf0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 shutdown_connections 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe158067020 0x7fe1580694e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe17c15ba20 0x7fe17c070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe17c0ae6e0 0x7fe17c1537b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 --2- 172.21.15.164:0/4142870531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe17c06de50 0x7fe17c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 >> 172.21.15.164:0/4142870531 conn(0x7fe17c0cf770 msgr2=0x7fe17c15eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 shutdown_connections 2024-05-08T04:39:44.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.065+0000 7fe181a00640 1 -- 172.21.15.164:0/4142870531 wait complete. 2024-05-08T04:39:44.089 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2024-05-08T04:39:44.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 Processor -- start 2024-05-08T04:39:44.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 -- start start 2024-05-08T04:39:44.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b6015dd70 0x7f0b601619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b60161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b601628b0 con 0x7f0b60162420 2024-05-08T04:39:44.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6006cc70 con 0x7f0b6015dd70 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b66e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6006cf50 con 0x7f0b6006c7b0 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b6015dd70 0x7f0b601619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b6015dd70 0x7f0b601619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50000/0 (socket says 172.21.15.164:50000) 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 -- 172.21.15.164:0/2272669166 learned_addr learned my addr 172.21.15.164:0/2272669166 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5f400640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b60161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 -- 172.21.15.164:0/2272669166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 msgr2=0x7f0b60161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b60161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 -- 172.21.15.164:0/2272669166 >> v1:172.21.15.172:6789/0 conn(0x7f0b60162420 legacy=0x7f0b600cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5fe00640 1 -- 172.21.15.164:0/2272669166 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600b1570 con 0x7f0b6015dd70 2024-05-08T04:39:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.221+0000 7f0b5f400640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b60161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:44.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6015dd70 0x7f0b601619a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009870 tx=0x7f0b5001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab264db81408f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b5ea00640 1 -- 172.21.15.164:0/2272669166 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500b5070 con 0x7f0b6015dd70 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b5ea00640 1 -- 172.21.15.164:0/2272669166 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b500a87f0 con 0x7f0b6015dd70 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b5ea00640 1 -- 172.21.15.164:0/2272669166 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500a8bc0 con 0x7f0b6015dd70 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- 172.21.15.164:0/2272669166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6015dd70 msgr2=0x7f0b601619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6015dd70 0x7f0b601619a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009870 tx=0x7f0b5001eb30 comp rx=0 tx=0).stop 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- 172.21.15.164:0/2272669166 shutdown_connections 2024-05-08T04:39:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b60161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 --2- 172.21.15.164:0/2272669166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6015dd70 0x7f0b601619a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- 172.21.15.164:0/2272669166 >> 172.21.15.164:0/2272669166 conn(0x7f0b600b34f0 msgr2=0x7f0b600b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- 172.21.15.164:0/2272669166 shutdown_connections 2024-05-08T04:39:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- 172.21.15.164:0/2272669166 wait complete. 2024-05-08T04:39:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 Processor -- start 2024-05-08T04:39:44.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.225+0000 7f0b66e00640 1 -- start start 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b601581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 0x7f0b6015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6015e150 con 0x7f0b60162420 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600b4230 con 0x7f0b6015dd70 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6015e2d0 con 0x7f0b6006c7b0 2024-05-08T04:39:44.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50014/0 (socket says 172.21.15.164:50014) 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 -- 172.21.15.164:0/324981904 learned_addr learned my addr 172.21.15.164:0/324981904 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b601581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b5f400640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 0x7f0b6015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 -- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 msgr2=0x7f0b601581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b601581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 -- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 msgr2=0x7f0b6015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 0x7f0b6015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 -- 172.21.15.164:0/324981904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60158b20 con 0x7f0b60162420 2024-05-08T04:39:44.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b601581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b5f400640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 0x7f0b6015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b64e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800b970 tx=0x7f0b5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58019070 con 0x7f0b60162420 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5800cb40 con 0x7f0b60162420 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58014440 con 0x7f0b60162420 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6015fcf0 con 0x7f0b60162420 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b600c58a0 con 0x7f0b60162420 2024-05-08T04:39:44.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.229+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b60069d60 con 0x7f0b60162420 2024-05-08T04:39:44.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.233+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f0b5801c070 con 0x7f0b60162420 2024-05-08T04:39:44.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.233+0000 7f0b57e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 0x7f0b400694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.233+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b58084a70 con 0x7f0b60162420 2024-05-08T04:39:44.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.233+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 0x7f0b400694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.233+0000 7f0b5fe00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 0x7f0b400694b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b500b2000 tx=0x7f0b500aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.237+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b58055c90 con 0x7f0b60162420 2024-05-08T04:39:44.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.445+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f0b6015d510 con 0x7f0b60162420 2024-05-08T04:39:44.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.721+0000 7f0b57e00640 1 -- 172.21.15.164:0/324981904 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0b58055590 con 0x7f0b60162420 2024-05-08T04:39:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 msgr2=0x7f0b400694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 0x7f0b400694b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b500b2000 tx=0x7f0b500aa070 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 msgr2=0x7f0b6015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800b970 tx=0x7f0b5800be30 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 shutdown_connections 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b40066ff0 0x7f0b400694b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b60162420 0x7f0b6015f6a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b6015dd70 0x7f0b6015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 --2- 172.21.15.164:0/324981904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b6006c7b0 0x7f0b601581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 >> 172.21.15.164:0/324981904 conn(0x7f0b600b34f0 msgr2=0x7f0b6006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 shutdown_connections 2024-05-08T04:39:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.725+0000 7f0b66e00640 1 -- 172.21.15.164:0/324981904 wait complete. 2024-05-08T04:39:44.751 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2024-05-08T04:39:44.751 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2024-05-08T04:39:44.751 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:39:44.751 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:44.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.881+0000 7f7ab7a00640 1 Processor -- start 2024-05-08T04:39:44.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.881+0000 7f7ab7a00640 1 -- start start 2024-05-08T04:39:44.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab7a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ab00b26c0 0x7f7ab00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab7a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab006c770 con 0x7f7ab015ab60 2024-05-08T04:39:44.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab7a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab00d0060 con 0x7f7ab00b26c0 2024-05-08T04:39:44.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab006ca50 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:47492/0 (socket says 172.21.15.164:47492) 2024-05-08T04:39:44.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 -- 172.21.15.164:0/1451441515 learned_addr learned my addr 172.21.15.164:0/1451441515 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:44.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7aafe00640 1 --2- 172.21.15.164:0/1451441515 >> v2:172.21.15.172:3300/0 conn(0x7f7ab00b26c0 0x7f7ab00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 -- 172.21.15.164:0/1451441515 >> v2:172.21.15.172:3300/0 conn(0x7f7ab00b26c0 msgr2=0x7f7ab00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:44.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 --2- 172.21.15.164:0/1451441515 >> v2:172.21.15.172:3300/0 conn(0x7f7ab00b26c0 0x7f7ab00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 -- 172.21.15.164:0/1451441515 >> v1:172.21.15.172:6789/0 conn(0x7f7ab015ab60 legacy=0x7f7ab0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.885+0000 7f7ab5a00640 1 -- 172.21.15.164:0/1451441515 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab006cd30 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab5a00640 1 --2- 172.21.15.164:0/1451441515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009870 tx=0x7f7aa801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c7e244a89424c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7aaf400640 1 -- 172.21.15.164:0/1451441515 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80b2070 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7aaf400640 1 -- 172.21.15.164:0/1451441515 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa80a9070 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7aaf400640 1 -- 172.21.15.164:0/1451441515 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80ad880 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 -- 172.21.15.164:0/1451441515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 msgr2=0x7f7ab00cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/1451441515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009870 tx=0x7f7aa801c560 comp rx=0 tx=0).stop 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 -- 172.21.15.164:0/1451441515 shutdown_connections 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/1451441515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab00cfbd0 0x7f7ab00cd910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/1451441515 >> v2:172.21.15.172:3300/0 conn(0x7f7ab00b26c0 0x7f7ab00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 -- 172.21.15.164:0/1451441515 >> 172.21.15.164:0/1451441515 conn(0x7f7ab006ac90 msgr2=0x7f7ab006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 -- 172.21.15.164:0/1451441515 shutdown_connections 2024-05-08T04:39:44.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.889+0000 7f7ab7a00640 1 -- 172.21.15.164:0/1451441515 wait complete. 2024-05-08T04:39:44.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 Processor -- start 2024-05-08T04:39:44.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- start start 2024-05-08T04:39:44.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ab00b26c0 0x7f7ab014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab015ab60 0x7f7ab015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00aeeb0 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00aed30 con 0x7f7ab00b26c0 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00cc1b0 con 0x7f7ab015ab60 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50026/0 (socket says 172.21.15.164:50026) 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 -- 172.21.15.164:0/547054971 learned_addr learned my addr 172.21.15.164:0/547054971 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab5a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab015ab60 0x7f7ab015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab5000640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ab00b26c0 0x7f7ab014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 -- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab015ab60 msgr2=0x7f7ab015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab015ab60 0x7f7ab015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 -- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ab00b26c0 msgr2=0x7f7ab014e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:44.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ab00b26c0 0x7f7ab014e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:44.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 -- 172.21.15.164:0/547054971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab006d790 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aafe00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e930 tx=0x7f7a9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c017070 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c00ca00 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab014db80 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00cdb0 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab01552c0 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.893+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab0069d60 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.897+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f7a9c01a070 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.897+0000 7f7aacc00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 0x7f7a900694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:44.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.897+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a9c084de0 con 0x7f7ab00cfbd0 2024-05-08T04:39:44.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.897+0000 7f7ab5000640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 0x7f7a900694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:44.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.897+0000 7f7ab5000640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 0x7f7a900694b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ab015e050 tx=0x7f7aa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:44.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:44.901+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a9c055a70 con 0x7f7ab00cfbd0 2024-05-08T04:39:45.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ab015e290 con 0x7f7ab00cfbd0 2024-05-08T04:39:45.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7aacc00640 1 -- 172.21.15.164:0/547054971 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13082 (secure 0 0 0) 0x7f7a9c059b10 con 0x7f7ab00cfbd0 2024-05-08T04:39:45.073 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:45.073 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":36,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:43.723797+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/3451370637":"2024-05-09T04:39:40.716898+0000","172.21.15.172:6833/3451370637":"2024-05-09T04:39:40.716898+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 msgr2=0x7f7a900694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 0x7f7a900694b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ab015e050 tx=0x7f7aa0009040 comp rx=0 tx=0).stop 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 msgr2=0x7f7ab014d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e930 tx=0x7f7a9c00edf0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 shutdown_connections 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a90066ff0 0x7f7a900694b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ab015ab60 0x7f7ab015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ab00cfbd0 0x7f7ab014d640 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 --2- 172.21.15.164:0/547054971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ab00b26c0 0x7f7ab014e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 >> 172.21.15.164:0/547054971 conn(0x7f7ab006ac90 msgr2=0x7f7ab00cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:45.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 shutdown_connections 2024-05-08T04:39:45.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.069+0000 7f7ab7a00640 1 -- 172.21.15.164:0/547054971 wait complete. 2024-05-08T04:39:45.095 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:45.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:45.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 Processor -- start 2024-05-08T04:39:45.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 -- start start 2024-05-08T04:39:45.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 0x7f048c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f048c06de50 0x7f048c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c0c6600 con 0x7f048c0ae6e0 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c15fbe0 con 0x7f048c06de50 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0494400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c0c6040 con 0x7f048c15ba20 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0491000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f048c06de50 0x7f048c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0491000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f048c06de50 0x7f048c06e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50032/0 (socket says 172.21.15.164:50032) 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0491000640 1 -- 172.21.15.164:0/385753742 learned_addr learned my addr 172.21.15.164:0/385753742 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:45.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0492400640 1 --1- 172.21.15.164:0/385753742 >> v1:172.21.15.164:6789/0 conn(0x7f048c0ae6e0 0x7f048c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55094/0 (socket says 172.21.15.164:55094) 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0491000640 1 -- 172.21.15.164:0/385753742 >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 msgr2=0x7f048c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.229+0000 7f0491a00640 1 --2- 172.21.15.164:0/385753742 >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 0x7f048c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0491000640 1 --2- 172.21.15.164:0/385753742 >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 0x7f048c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0491000640 1 -- 172.21.15.164:0/385753742 >> v1:172.21.15.164:6789/0 conn(0x7f048c0ae6e0 legacy=0x7f048c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0491000640 1 -- 172.21.15.164:0/385753742 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0c6320 con 0x7f048c06de50 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0491a00640 1 --2- 172.21.15.164:0/385753742 >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 0x7f048c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:45.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f048be00640 1 -- 172.21.15.164:0/385753742 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776374068 0 0) 0x7f048c0c6600 con 0x7f048c0ae6e0 2024-05-08T04:39:45.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0491000640 1 --2- 172.21.15.164:0/385753742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06e2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cc0e0 tx=0x7f047801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3a9cc8872f4d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f048be00640 1 -- 172.21.15.164:0/385753742 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780aa5f0 con 0x7f048c06de50 2024-05-08T04:39:45.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f048be00640 1 -- 172.21.15.164:0/385753742 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04780af070 con 0x7f048c06de50 2024-05-08T04:39:45.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f048be00640 1 -- 172.21.15.164:0/385753742 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04780aa9b0 con 0x7f048c06de50 2024-05-08T04:39:45.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 -- 172.21.15.164:0/385753742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 msgr2=0x7f048c06e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 --2- 172.21.15.164:0/385753742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06e2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f048c0cc0e0 tx=0x7f047801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 -- 172.21.15.164:0/385753742 shutdown_connections 2024-05-08T04:39:45.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 --2- 172.21.15.164:0/385753742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06e2c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 --2- 172.21.15.164:0/385753742 >> v2:172.21.15.172:3300/0 conn(0x7f048c15ba20 0x7f048c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 -- 172.21.15.164:0/385753742 >> 172.21.15.164:0/385753742 conn(0x7f048c0cf770 msgr2=0x7f048c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 -- 172.21.15.164:0/385753742 shutdown_connections 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.233+0000 7f0494400640 1 -- 172.21.15.164:0/385753742 wait complete. 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 Processor -- start 2024-05-08T04:39:45.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- start start 2024-05-08T04:39:45.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f048c0ae6e0 0x7f048c144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f048c15ba20 0x7f048c144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0aec60 con 0x7f048c06de50 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0aeae0 con 0x7f048c15ba20 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0aede0 con 0x7f048c0ae6e0 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50046/0 (socket says 172.21.15.164:50046) 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 -- 172.21.15.164:0/1372417 learned_addr learned my addr 172.21.15.164:0/1372417 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0492400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f048c15ba20 0x7f048c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491000640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f048c0ae6e0 0x7f048c144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 -- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f048c0ae6e0 msgr2=0x7f048c144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f048c0ae6e0 0x7f048c144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 -- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f048c15ba20 msgr2=0x7f048c144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f048c15ba20 0x7f048c144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 -- 172.21.15.164:0/1372417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0b0170 con 0x7f048c06de50 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0491a00640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f048000c7f0 tx=0x7f048000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480015070 con 0x7f048c06de50 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480007820 con 0x7f048c06de50 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480007c10 con 0x7f048c06de50 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c0b06f0 con 0x7f048c06de50 2024-05-08T04:39:45.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.237+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0b0cb0 con 0x7f048c06de50 2024-05-08T04:39:45.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c072d30 con 0x7f048c06de50 2024-05-08T04:39:45.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f0480018070 con 0x7f048c06de50 2024-05-08T04:39:45.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0489600640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 0x7f046c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0480084ff0 con 0x7f048c06de50 2024-05-08T04:39:45.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0491000640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 0x7f046c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.241+0000 7f0491000640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 0x7f046c0694b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04780ae000 tx=0x7f04780a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.245+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0480056180 con 0x7f048c06de50 2024-05-08T04:39:45.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.397+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f048c04f010 con 0x7f048c06de50 2024-05-08T04:39:45.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0489600640 1 -- 172.21.15.164:0/1372417 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0480055a80 con 0x7f048c06de50 2024-05-08T04:39:45.405 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 msgr2=0x7f046c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 0x7f046c0694b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04780ae000 tx=0x7f04780a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 msgr2=0x7f048c06aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f048000c7f0 tx=0x7f048000ccb0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 shutdown_connections 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f046c066ff0 0x7f046c0694b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f048c0ae6e0 0x7f048c144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f048c15ba20 0x7f048c144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 --2- 172.21.15.164:0/1372417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f048c06de50 0x7f048c06aef0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 >> 172.21.15.164:0/1372417 conn(0x7f048c0cf770 msgr2=0x7f048c152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 shutdown_connections 2024-05-08T04:39:45.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.401+0000 7f0494400640 1 -- 172.21.15.164:0/1372417 wait complete. 2024-05-08T04:39:45.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2024-05-08T04:39:45.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 Processor -- start 2024-05-08T04:39:45.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 -- start start 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b415dd70 0x7ff7b41619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7b41625f0 0x7ff7b40b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b406cc70 con 0x7ff7b406c7b0 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b406cf50 con 0x7ff7b41625f0 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7bc200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40b1570 con 0x7ff7b415dd70 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7ba200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7b41625f0 0x7ff7b40b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7b8e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff7b406c7b0 0x7ff7b4161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55096/0 (socket says 172.21.15.164:55096) 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7b8e00640 1 -- 172.21.15.164:0/3133591777 learned_addr learned my addr 172.21.15.164:0/3133591777 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:45.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.557+0000 7ff7b9800640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b415dd70 0x7ff7b41619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7ba200640 1 -- 172.21.15.164:0/3133591777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b415dd70 msgr2=0x7ff7b41619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:45.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7ba200640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b415dd70 0x7ff7b41619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7ba200640 1 -- 172.21.15.164:0/3133591777 >> v1:172.21.15.164:6789/0 conn(0x7ff7b406c7b0 legacy=0x7ff7b4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7ba200640 1 -- 172.21.15.164:0/3133591777 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40b1850 con 0x7ff7b41625f0 2024-05-08T04:39:45.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7b3e00640 1 -- 172.21.15.164:0/3133591777 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665808940 0 0) 0x7ff7b406cc70 con 0x7ff7b406c7b0 2024-05-08T04:39:45.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7ba200640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b41625f0 0x7ff7b40b1d70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009d90 tx=0x7ff7ac01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4c600b993a5589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7b3e00640 1 -- 172.21.15.164:0/3133591777 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0b4070 con 0x7ff7b41625f0 2024-05-08T04:39:45.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7b3e00640 1 -- 172.21.15.164:0/3133591777 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac0af600 con 0x7ff7b41625f0 2024-05-08T04:39:45.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7b3e00640 1 -- 172.21.15.164:0/3133591777 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0af9d0 con 0x7ff7b41625f0 2024-05-08T04:39:45.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 -- 172.21.15.164:0/3133591777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b41625f0 msgr2=0x7ff7b40b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b41625f0 0x7ff7b40b1d70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009d90 tx=0x7ff7ac01f150 comp rx=0 tx=0).stop 2024-05-08T04:39:45.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 -- 172.21.15.164:0/3133591777 shutdown_connections 2024-05-08T04:39:45.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b41625f0 0x7ff7b40b1d70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 --2- 172.21.15.164:0/3133591777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b415dd70 0x7ff7b41619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.561+0000 7ff7bc200640 1 -- 172.21.15.164:0/3133591777 >> 172.21.15.164:0/3133591777 conn(0x7ff7b40b34f0 msgr2=0x7ff7b40b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:45.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- 172.21.15.164:0/3133591777 shutdown_connections 2024-05-08T04:39:45.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- 172.21.15.164:0/3133591777 wait complete. 2024-05-08T04:39:45.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 Processor -- start 2024-05-08T04:39:45.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- start start 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b406c7b0 0x7ff7b4152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 0x7ff7b41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415e2d0 con 0x7ff7b415dd70 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415e150 con 0x7ff7b406c7b0 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40b22b0 con 0x7ff7b41625f0 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50064/0 (socket says 172.21.15.164:50064) 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 -- 172.21.15.164:0/1571396449 learned_addr learned my addr 172.21.15.164:0/1571396449 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b9800640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b406c7b0 0x7ff7b4152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7ba200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 0x7ff7b41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 -- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 msgr2=0x7ff7b41581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 0x7ff7b41581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 -- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b406c7b0 msgr2=0x7ff7b4152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b406c7b0 0x7ff7b4152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 -- 172.21.15.164:0/1571396449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4157120 con 0x7ff7b415dd70 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7ba200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 0x7ff7b41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b8e00640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c520 tx=0x7ff7a000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0018070 con 0x7ff7b415dd70 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a00074e0 con 0x7ff7b415dd70 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00049f0 con 0x7ff7b415dd70 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b4157400 con 0x7ff7b415dd70 2024-05-08T04:39:45.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.565+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b41579c0 con 0x7ff7b415dd70 2024-05-08T04:39:45.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4069d60 con 0x7ff7b415dd70 2024-05-08T04:39:45.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7ff7a0004b90 con 0x7ff7b415dd70 2024-05-08T04:39:45.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7b1600640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 0x7ff7940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:45.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7a0084060 con 0x7ff7b415dd70 2024-05-08T04:39:45.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7b9800640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 0x7ff7940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:45.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.569+0000 7ff7b9800640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 0x7ff7940694b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4157050 tx=0x7ff7a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:45.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.573+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7a0054cc0 con 0x7ff7b415dd70 2024-05-08T04:39:45.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:45.785+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7ff7b40b4010 con 0x7ff7b415dd70 2024-05-08T04:39:46.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.733+0000 7ff7b1600640 1 -- 172.21.15.164:0/1571396449 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v97) v1 ==== 175+0+0 (secure 0 0 0) 0x7ff7a0058d60 con 0x7ff7b415dd70 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 msgr2=0x7ff7940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 0x7ff7940694b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4157050 tx=0x7ff7a8008040 comp rx=0 tx=0).stop 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 msgr2=0x7ff7b40b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c520 tx=0x7ff7a000c9e0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 shutdown_connections 2024-05-08T04:39:46.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff794066ff0 0x7ff7940694b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7b41625f0 0x7ff7b41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7b415dd70 0x7ff7b40b4810 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 --2- 172.21.15.164:0/1571396449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b406c7b0 0x7ff7b4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 >> 172.21.15.164:0/1571396449 conn(0x7ff7b40b34f0 msgr2=0x7ff7b40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 shutdown_connections 2024-05-08T04:39:46.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.737+0000 7ff7bc200640 1 -- 172.21.15.164:0/1571396449 wait complete. 2024-05-08T04:39:46.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:39:46.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 Processor -- start 2024-05-08T04:39:46.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 -- start start 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f07780ae6e0 0x7f07780aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07780c6600 con 0x7f077815ba20 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07780c6040 con 0x7f077815ce10 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f077d800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077815fbe0 con 0x7f07780ae6e0 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0777400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f07780ae6e0 0x7f07780aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0776a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f077815ba20 0x7f0778160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55104/0 (socket says 172.21.15.164:55104) 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0777400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f07780ae6e0 0x7f07780aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50072/0 (socket says 172.21.15.164:50072) 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0776a00640 1 -- 172.21.15.164:0/2435169955 learned_addr learned my addr 172.21.15.164:0/2435169955 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:46.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0777400640 1 -- 172.21.15.164:0/2435169955 >> v1:172.21.15.172:6789/0 conn(0x7f077815ce10 legacy=0x7f077806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0777400640 1 -- 172.21.15.164:0/2435169955 >> v1:172.21.15.164:6789/0 conn(0x7f077815ba20 legacy=0x7f0778160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0777400640 1 -- 172.21.15.164:0/2435169955 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780c6320 con 0x7f07780ae6e0 2024-05-08T04:39:46.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.909+0000 7f0776000640 1 -- 172.21.15.164:0/2435169955 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153683971 0 0) 0x7f07780c6600 con 0x7f077815ba20 2024-05-08T04:39:46.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f0777400640 1 --2- 172.21.15.164:0/2435169955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780ae6e0 0x7f07780aeb20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0770009d90 tx=0x7f077001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac0f7273a41677c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:46.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f0776000640 1 -- 172.21.15.164:0/2435169955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700b2070 con 0x7f07780ae6e0 2024-05-08T04:39:46.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f0776000640 1 -- 172.21.15.164:0/2435169955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07700a5830 con 0x7f07780ae6e0 2024-05-08T04:39:46.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f0776000640 1 -- 172.21.15.164:0/2435169955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700a5c00 con 0x7f07780ae6e0 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 -- 172.21.15.164:0/2435169955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780ae6e0 msgr2=0x7f07780aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 --2- 172.21.15.164:0/2435169955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780ae6e0 0x7f07780aeb20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0770009d90 tx=0x7f077001c970 comp rx=0 tx=0).stop 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 -- 172.21.15.164:0/2435169955 shutdown_connections 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 --2- 172.21.15.164:0/2435169955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780ae6e0 0x7f07780aeb20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 -- 172.21.15.164:0/2435169955 >> 172.21.15.164:0/2435169955 conn(0x7f07780cf770 msgr2=0x7f07780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 -- 172.21.15.164:0/2435169955 shutdown_connections 2024-05-08T04:39:46.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.913+0000 7f077d800640 1 -- 172.21.15.164:0/2435169955 wait complete. 2024-05-08T04:39:46.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 Processor -- start 2024-05-08T04:39:46.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- start start 2024-05-08T04:39:46.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780ae6e0 0x7f077814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ba20 0x7f077814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780ce000 con 0x7f077815ce10 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780cde80 con 0x7f07780ae6e0 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0778169b80 con 0x7f077815ba20 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50088/0 (socket says 172.21.15.164:50088) 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 -- 172.21.15.164:0/1914237602 learned_addr learned my addr 172.21.15.164:0/1914237602 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0776a00640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780ae6e0 0x7f077814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f076e000640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ba20 0x7f077814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 -- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ba20 msgr2=0x7f077814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ba20 0x7f077814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 -- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780ae6e0 msgr2=0x7f077814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780ae6e0 0x7f077814f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 -- 172.21.15.164:0/1914237602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780b5130 con 0x7f077815ce10 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f0777400640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07700b0040 tx=0x7f0770019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700b2050 con 0x7f077815ce10 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07700b3040 con 0x7f077815ce10 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07780b5410 con 0x7f077815ce10 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770003e20 con 0x7f077815ce10 2024-05-08T04:39:46.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.917+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07780c8080 con 0x7f077815ce10 2024-05-08T04:39:46.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0778072d30 con 0x7f077815ce10 2024-05-08T04:39:46.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f07700c1060 con 0x7f077815ce10 2024-05-08T04:39:46.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f076f400640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 0x7f07540694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:46.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f077011ee70 con 0x7f077815ce10 2024-05-08T04:39:46.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f0776a00640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 0x7f07540694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:46.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.921+0000 7f0776a00640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 0x7f07540694b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0768007a90 tx=0x7f07680079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:46.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:46.925+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0770016560 con 0x7f077815ce10 2024-05-08T04:39:47.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.081+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f077804f010 con 0x7f077815ce10 2024-05-08T04:39:47.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.085+0000 7f076f400640 1 -- 172.21.15.164:0/1914237602 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07700f0090 con 0x7f077815ce10 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 msgr2=0x7f07540694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 0x7f07540694b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0768007a90 tx=0x7f07680079e0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 msgr2=0x7f0778151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07700b0040 tx=0x7f0770019710 comp rx=0 tx=0).stop 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 shutdown_connections 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0754066ff0 0x7f07540694b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ba20 0x7f077814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077815ce10 0x7f0778151d60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 --2- 172.21.15.164:0/1914237602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780ae6e0 0x7f077814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 >> 172.21.15.164:0/1914237602 conn(0x7f07780cf770 msgr2=0x7f07780c52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:47.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 shutdown_connections 2024-05-08T04:39:47.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.089+0000 7f077d800640 1 -- 172.21.15.164:0/1914237602 wait complete. 2024-05-08T04:39:47.112 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:39:47.112 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:47.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:39:47.297 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:39:47.298 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:47.298 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:47.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:39:47.306 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:47.306 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:47.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:39:47.316 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:39:47.316 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:47.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:39:47.359 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:39:47.359 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:39:47.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:39:47.368 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:39:47.368 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:47.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 Processor -- start 2024-05-08T04:39:47.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 -- start start 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 0x7f99f80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99f80cfbd0 0x7f99f80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f806c770 con 0x7f99f815ab60 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f80d0060 con 0x7f99f80cfbd0 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99fde00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f806ca50 con 0x7f99f80b26c0 2024-05-08T04:39:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99f80cfbd0 0x7f99f80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99f80cfbd0 0x7f99f80cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50102/0 (socket says 172.21.15.164:50102) 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 -- 172.21.15.164:0/1866388937 learned_addr learned my addr 172.21.15.164:0/1866388937 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99ee000640 1 --2- 172.21.15.164:0/1866388937 >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 0x7f99f80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 -- 172.21.15.164:0/1866388937 >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 msgr2=0x7f99f80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 --2- 172.21.15.164:0/1866388937 >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 0x7f99f80cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 -- 172.21.15.164:0/1866388937 >> v1:172.21.15.172:6789/0 conn(0x7f99f815ab60 legacy=0x7f99f8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:47.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99f7400640 1 -- 172.21.15.164:0/1866388937 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f806cd30 con 0x7f99f80cfbd0 2024-05-08T04:39:47.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.533+0000 7f99ee000640 1 --2- 172.21.15.164:0/1866388937 >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 0x7f99f80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:47.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99f7400640 1 --2- 172.21.15.164:0/1866388937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80cfbd0 0x7f99f80cd910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f99f806e880 tx=0x7f99f001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82ff14926585dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99f6000640 1 -- 172.21.15.164:0/1866388937 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00a5670 con 0x7f99f80cfbd0 2024-05-08T04:39:47.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99f6000640 1 -- 172.21.15.164:0/1866388937 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f00a5810 con 0x7f99f80cfbd0 2024-05-08T04:39:47.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99f6000640 1 -- 172.21.15.164:0/1866388937 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00a5be0 con 0x7f99f80cfbd0 2024-05-08T04:39:47.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 -- 172.21.15.164:0/1866388937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80cfbd0 msgr2=0x7f99f80cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 --2- 172.21.15.164:0/1866388937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80cfbd0 0x7f99f80cd910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f99f806e880 tx=0x7f99f001c9b0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 -- 172.21.15.164:0/1866388937 shutdown_connections 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 --2- 172.21.15.164:0/1866388937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80cfbd0 0x7f99f80cd910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 --2- 172.21.15.164:0/1866388937 >> v2:172.21.15.172:3300/0 conn(0x7f99f80b26c0 0x7f99f80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 -- 172.21.15.164:0/1866388937 >> 172.21.15.164:0/1866388937 conn(0x7f99f806ac90 msgr2=0x7f99f806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 -- 172.21.15.164:0/1866388937 shutdown_connections 2024-05-08T04:39:47.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.537+0000 7f99fde00640 1 -- 172.21.15.164:0/1866388937 wait complete. 2024-05-08T04:39:47.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 Processor -- start 2024-05-08T04:39:47.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- start start 2024-05-08T04:39:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f80cfbd0 0x7f99f81503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f815ab60 0x7f99f814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80aeeb0 con 0x7f99f80b26c0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80aed30 con 0x7f99f80cfbd0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8167970 con 0x7f99f815ab60 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50112/0 (socket says 172.21.15.164:50112) 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 -- 172.21.15.164:0/3375672341 learned_addr learned my addr 172.21.15.164:0/3375672341 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99ee000640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f80cfbd0 0x7f99f81503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f7400640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f815ab60 0x7f99f814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 -- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f815ab60 msgr2=0x7f99f814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f815ab60 0x7f99f814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 -- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f80cfbd0 msgr2=0x7f99f81503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f80cfbd0 0x7f99f81503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 -- 172.21.15.164:0/3375672341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8152820 con 0x7f99f80b26c0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99f6a00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b440 tx=0x7f99e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8015020 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8011890 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f8152b00 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8011c80 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.541+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f81530c0 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f8069d60 con 0x7f99f80b26c0 2024-05-08T04:39:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f99e8018020 con 0x7f99f80b26c0 2024-05-08T04:39:47.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99efe00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 0x7f99d40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f99e8084b70 con 0x7f99f80b26c0 2024-05-08T04:39:47.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99ee000640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 0x7f99d40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.545+0000 7f99ee000640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 0x7f99d40694b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99f81573b0 tx=0x7f99e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.549+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99e801f030 con 0x7f99f80b26c0 2024-05-08T04:39:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.737+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99f806d9a0 con 0x7f99f80b26c0 2024-05-08T04:39:47.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.737+0000 7f99efe00640 1 -- 172.21.15.164:0/3375672341 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f99e800d040 con 0x7f99f80b26c0 2024-05-08T04:39:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:47.743 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2245412911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2245412911},{"type":"v1","addr":"172.21.15.172:6834","nonce":2245412911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/414723531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":414723531},{"type":"v1","addr":"172.21.15.164:6836","nonce":414723531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5042,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2357960819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2357960819},{"type":"v1","addr":"172.21.15.172:6835","nonce":2357960819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5055,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/308505231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":308505231},{"type":"v1","addr":"172.21.15.164:6837","nonce":308505231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-05-08T04:39:47.743 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 20 2024-05-08T04:39:47.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 msgr2=0x7f99d40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 0x7f99d40694b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99f81573b0 tx=0x7f99e0008040 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 msgr2=0x7f99f81538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b440 tx=0x7f99e800b900 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 shutdown_connections 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99d4066ff0 0x7f99d40694b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f815ab60 0x7f99f814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f80cfbd0 0x7f99f81503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 --2- 172.21.15.164:0/3375672341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f80b26c0 0x7f99f81538c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 >> 172.21.15.164:0/3375672341 conn(0x7f99f806ac90 msgr2=0x7f99f80cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 shutdown_connections 2024-05-08T04:39:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.741+0000 7f99fde00640 1 -- 172.21.15.164:0/3375672341 wait complete. 2024-05-08T04:39:47.764 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:39:47.764 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:39:47.764 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:39:47.764 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:47.764 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:39:47.765 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:39:47.765 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:39:47.765 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:47.765 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:39:47.765 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:39:47.765 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:39:47.765 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:47.765 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:39:47.766 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:39:47.766 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:39:47.766 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:39:47.778 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:39:47.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:39:47.779 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:39:47.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:39:47.780 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:39:47.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:39:47.781 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:39:47.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:39:47.825 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:39:47.826 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:47.827 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:39:47.827 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:47.828 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:39:47.828 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:47.829 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:39:47.829 DEBUG:teuthology.parallel:result is None 2024-05-08T04:39:47.829 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:39:47.913 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:39:47.914 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:39:47.917 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:39:47.918 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:39:47.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.969+0000 7fa3c7200640 1 Processor -- start 2024-05-08T04:39:47.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.969+0000 7fa3c7200640 1 -- start start 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.969+0000 7fa3c7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3c00ccdf0 0x7fa3c00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.969+0000 7fa3c7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3c006b3a0 0x7fa3c006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.969+0000 7fa3c7200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c0072320 con 0x7fa3c00addd0 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c006d4e0 con 0x7fa3c006b3a0 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0072600 con 0x7fa3c00ccdf0 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bf400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa3c00addd0 0x7fa3c006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:55128/0 (socket says 172.21.15.164:55128) 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bf400640 1 -- 172.21.15.164:0/377523093 learned_addr learned my addr 172.21.15.164:0/377523093 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 --2- 172.21.15.164:0/377523093 >> v2:172.21.15.164:3300/0 conn(0x7fa3c006b3a0 0x7fa3c006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bfe00640 1 --2- 172.21.15.164:0/377523093 >> v2:172.21.15.172:3300/0 conn(0x7fa3c00ccdf0 0x7fa3c00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bea00640 1 -- 172.21.15.164:0/377523093 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272566228 0 0) 0x7fa3c0072320 con 0x7fa3c00addd0 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bea00640 1 -- 172.21.15.164:0/377523093 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a4003530 con 0x7fa3c00addd0 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 -- 172.21.15.164:0/377523093 >> v2:172.21.15.172:3300/0 conn(0x7fa3c00ccdf0 msgr2=0x7fa3c00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 --2- 172.21.15.164:0/377523093 >> v2:172.21.15.172:3300/0 conn(0x7fa3c00ccdf0 0x7fa3c00ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 -- 172.21.15.164:0/377523093 >> v1:172.21.15.164:6789/0 conn(0x7fa3c00addd0 legacy=0x7fa3c006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 -- 172.21.15.164:0/377523093 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c006d730 con 0x7fa3c006b3a0 2024-05-08T04:39:47.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3c5200640 1 --2- 172.21.15.164:0/377523093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c006df30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009d90 tx=0x7fa3b801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bea25d2230cb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bea00640 1 -- 172.21.15.164:0/377523093 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80b2070 con 0x7fa3c006b3a0 2024-05-08T04:39:47.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bea00640 1 -- 172.21.15.164:0/377523093 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80a5870 con 0x7fa3c006b3a0 2024-05-08T04:39:47.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.973+0000 7fa3bea00640 1 -- 172.21.15.164:0/377523093 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80a5c40 con 0x7fa3c006b3a0 2024-05-08T04:39:47.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- 172.21.15.164:0/377523093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 msgr2=0x7fa3c006df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- 172.21.15.164:0/377523093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c006df30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009d90 tx=0x7fa3b801c8a0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- 172.21.15.164:0/377523093 shutdown_connections 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- 172.21.15.164:0/377523093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c006df30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- 172.21.15.164:0/377523093 >> v2:172.21.15.172:3300/0 conn(0x7fa3c00ccdf0 0x7fa3c00ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- 172.21.15.164:0/377523093 >> 172.21.15.164:0/377523093 conn(0x7fa3c0066fa0 msgr2=0x7fa3c006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- 172.21.15.164:0/377523093 shutdown_connections 2024-05-08T04:39:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- 172.21.15.164:0/377523093 wait complete. 2024-05-08T04:39:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 Processor -- start 2024-05-08T04:39:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- start start 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c00addd0 0x7fa3c014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c00ccdf0 0x7fa3c00ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0157af0 con 0x7fa3c006b3a0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0071ff0 con 0x7fa3c00addd0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0157c70 con 0x7fa3c00ccdf0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50152/0 (socket says 172.21.15.164:50152) 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 -- 172.21.15.164:0/1488159874 learned_addr learned my addr 172.21.15.164:0/1488159874 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3c5200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c00ccdf0 0x7fa3c00ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bf400640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c00addd0 0x7fa3c014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 -- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c00ccdf0 msgr2=0x7fa3c00ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c00ccdf0 0x7fa3c00ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 -- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c00addd0 msgr2=0x7fa3c014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c00addd0 0x7fa3c014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.977+0000 7fa3bfe00640 1 -- 172.21.15.164:0/1488159874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00b5120 con 0x7fa3c006b3a0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3bfe00640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d5f0 tx=0x7fa3b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00125f0 con 0x7fa3c006b3a0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0019070 con 0x7fa3c006b3a0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00129b0 con 0x7fa3c006b3a0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c00b45a0 con 0x7fa3c006b3a0 2024-05-08T04:39:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c00b4b60 con 0x7fa3c006b3a0 2024-05-08T04:39:47.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c0069d60 con 0x7fa3c006b3a0 2024-05-08T04:39:47.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fa3b0012b50 con 0x7fa3c006b3a0 2024-05-08T04:39:47.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 0x7fa3a00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:47.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3b0085130 con 0x7fa3c006b3a0 2024-05-08T04:39:47.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.981+0000 7fa3bf400640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 0x7fa3a00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:47.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.985+0000 7fa3bf400640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 0x7fa3a00694b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0150770 tx=0x7fa3ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:47.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:47.985+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3b00563e0 con 0x7fa3c006b3a0 2024-05-08T04:39:48.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:48.161+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa3c014f280 con 0x7fa3c006b3a0 2024-05-08T04:39:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.117+0000 7fa3b7e00640 1 -- 172.21.15.164:0/1488159874 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa3b0055ce0 con 0x7fa3c006b3a0 2024-05-08T04:39:49.122 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 msgr2=0x7fa3a00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 0x7fa3a00694b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0150770 tx=0x7fa3ac009040 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 msgr2=0x7fa3c014f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d5f0 tx=0x7fa3b000dab0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 shutdown_connections 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3a0066ff0 0x7fa3a00694b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3c00ccdf0 0x7fa3c00ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3c00addd0 0x7fa3c014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 --2- 172.21.15.164:0/1488159874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c006b3a0 0x7fa3c014f510 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 >> 172.21.15.164:0/1488159874 conn(0x7fa3c0066fa0 msgr2=0x7fa3c01539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 shutdown_connections 2024-05-08T04:39:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.121+0000 7fa3c7200640 1 -- 172.21.15.164:0/1488159874 wait complete. 2024-05-08T04:39:49.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:39:49.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.277+0000 7f58db600640 1 Processor -- start 2024-05-08T04:39:49.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.277+0000 7f58db600640 1 -- start start 2024-05-08T04:39:49.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58db600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d40ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58db600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d4161fe0 con 0x7f58d4161950 2024-05-08T04:39:49.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58db600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d41622c0 con 0x7f58d4144200 2024-05-08T04:39:49.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58db600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4162880 con 0x7f58d40cfd60 2024-05-08T04:39:49.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f58d4161950 0x7f58d415ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34066/0 (socket says 172.21.15.164:34066) 2024-05-08T04:39:49.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3e00640 1 -- 172.21.15.164:0/468381537 learned_addr learned my addr 172.21.15.164:0/468381537 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:49.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d9600640 1 --2- 172.21.15.164:0/468381537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d40ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548895255 0 0) 0x7f58d4161fe0 con 0x7f58d4161950 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b8003530 con 0x7f58d4161950 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394092484 0 0) 0x7f58d41622c0 con 0x7f58d4144200 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d4161fe0 con 0x7f58d4144200 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d9600640 1 -- 172.21.15.164:0/468381537 >> v1:172.21.15.172:6789/0 conn(0x7f58d4144200 legacy=0x7f58d40cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d9600640 1 -- 172.21.15.164:0/468381537 >> v1:172.21.15.164:6789/0 conn(0x7f58d4161950 legacy=0x7f58d415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.281+0000 7f58d9600640 1 -- 172.21.15.164:0/468381537 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d41625a0 con 0x7f58d40cfd60 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58d9600640 1 --2- 172.21.15.164:0/468381537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d40ce9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0097f0 tx=0x7f58cc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5e71bf678541fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0b0070 con 0x7f58d40cfd60 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc0ab600 con 0x7f58d40cfd60 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58d3400640 1 -- 172.21.15.164:0/468381537 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0ab9b0 con 0x7f58d40cfd60 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- 172.21.15.164:0/468381537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 msgr2=0x7f58d40ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 --2- 172.21.15.164:0/468381537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d40ce9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0097f0 tx=0x7f58cc01c450 comp rx=0 tx=0).stop 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- 172.21.15.164:0/468381537 shutdown_connections 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 --2- 172.21.15.164:0/468381537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d40ce9e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- 172.21.15.164:0/468381537 >> 172.21.15.164:0/468381537 conn(0x7f58d406ac90 msgr2=0x7f58d406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- 172.21.15.164:0/468381537 shutdown_connections 2024-05-08T04:39:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- 172.21.15.164:0/468381537 wait complete. 2024-05-08T04:39:49.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 Processor -- start 2024-05-08T04:39:49.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.285+0000 7f58db600640 1 -- start start 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d413f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4161950 0x7f58d415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40cc7c0 con 0x7f58d4144200 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4161d80 con 0x7f58d4161950 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40cc940 con 0x7f58d40cfd60 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41534/0 (socket says 172.21.15.164:41534) 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 -- 172.21.15.164:0/3991140312 learned_addr learned my addr 172.21.15.164:0/3991140312 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:49.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d8c00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d413f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d9600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4161950 0x7f58d415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 -- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 msgr2=0x7f58d413f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d413f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 -- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4161950 msgr2=0x7f58d415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4161950 0x7f58d415cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 -- 172.21.15.164:0/3991140312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d413e0c0 con 0x7f58d4144200 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d8c00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d413f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d3e00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c530 tx=0x7f58c000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0017070 con 0x7f58d4144200 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c0005cf0 con 0x7f58d4144200 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d413e3a0 con 0x7f58d4144200 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0007290 con 0x7f58d4144200 2024-05-08T04:39:49.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d413e960 con 0x7f58d4144200 2024-05-08T04:39:49.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d4069d60 con 0x7f58d4144200 2024-05-08T04:39:49.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.289+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f58c001a070 con 0x7f58d4144200 2024-05-08T04:39:49.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.293+0000 7f58d0c00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 0x7f58b40694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.293+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58c00841d0 con 0x7f58d4144200 2024-05-08T04:39:49.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.293+0000 7f58d8c00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 0x7f58b40694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.293+0000 7f58d8c00640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 0x7f58b40694e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007fd0 tx=0x7f58c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.297+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58c0055440 con 0x7f58d4144200 2024-05-08T04:39:49.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.505+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f58d41402f0 con 0x7f58d4144200 2024-05-08T04:39:49.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.505+0000 7f58d0c00640 1 -- 172.21.15.164:0/3991140312 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f58c0054d40 con 0x7f58d4144200 2024-05-08T04:39:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:49.511 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:39:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:39:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:39:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:39:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:39:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:39:49.524 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 msgr2=0x7f58b40694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 0x7f58b40694e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007fd0 tx=0x7f58c4009040 comp rx=0 tx=0).stop 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 msgr2=0x7f58d413d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c530 tx=0x7f58c000c9f0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 shutdown_connections 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58b4067020 0x7f58b40694e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4161950 0x7f58d415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d4144200 0x7f58d413d930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 --2- 172.21.15.164:0/3991140312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58d40cfd60 0x7f58d413f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 >> 172.21.15.164:0/3991140312 conn(0x7f58d406ac90 msgr2=0x7f58d4071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 shutdown_connections 2024-05-08T04:39:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.509+0000 7f58db600640 1 -- 172.21.15.164:0/3991140312 wait complete. 2024-05-08T04:39:49.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:49.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 Processor -- start 2024-05-08T04:39:49.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 -- start start 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d40b26c0 0x7fd9d40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9d40cfbd0 0x7fd9d40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d406c770 con 0x7fd9d415ab60 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40d0060 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d406ca50 con 0x7fd9d40b26c0 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9d40cfbd0 0x7fd9d40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9d40cfbd0 0x7fd9d40cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41546/0 (socket says 172.21.15.164:41546) 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 -- 172.21.15.164:0/165941060 learned_addr learned my addr 172.21.15.164:0/165941060 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:49.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9ca000640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d40b26c0 0x7fd9d40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 -- 172.21.15.164:0/165941060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d40b26c0 msgr2=0x7fd9d40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:49.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d40b26c0 0x7fd9d40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 -- 172.21.15.164:0/165941060 >> v1:172.21.15.172:6789/0 conn(0x7fd9d415ab60 legacy=0x7fd9d4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.665+0000 7fd9d3400640 1 -- 172.21.15.164:0/165941060 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d406cd30 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d3400640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d40cd910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e50 tx=0x7fd9cc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e60b90346a3aceb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d2000640 1 -- 172.21.15.164:0/165941060 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0a5700 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d2000640 1 -- 172.21.15.164:0/165941060 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc0a58a0 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d2000640 1 -- 172.21.15.164:0/165941060 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0a5c70 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 -- 172.21.15.164:0/165941060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 msgr2=0x7fd9d40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d40cd910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e50 tx=0x7fd9cc01ca30 comp rx=0 tx=0).stop 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 -- 172.21.15.164:0/165941060 shutdown_connections 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d40cd910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/165941060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d40b26c0 0x7fd9d40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 -- 172.21.15.164:0/165941060 >> 172.21.15.164:0/165941060 conn(0x7fd9d406ac90 msgr2=0x7fd9d406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 -- 172.21.15.164:0/165941060 shutdown_connections 2024-05-08T04:39:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.669+0000 7fd9d9c00640 1 -- 172.21.15.164:0/165941060 wait complete. 2024-05-08T04:39:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 Processor -- start 2024-05-08T04:39:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- start start 2024-05-08T04:39:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9d40b26c0 0x7fd9d414b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d415ab60 0x7fd9d40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40aeeb0 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40aed30 con 0x7fd9d40b26c0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4167970 con 0x7fd9d415ab60 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41562/0 (socket says 172.21.15.164:41562) 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 -- 172.21.15.164:0/1134324134 learned_addr learned my addr 172.21.15.164:0/1134324134 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d2a00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9d40b26c0 0x7fd9d414b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d3400640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d415ab60 0x7fd9d40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 -- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d415ab60 msgr2=0x7fd9d40ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d415ab60 0x7fd9d40ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 -- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9d40b26c0 msgr2=0x7fd9d414b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9d40b26c0 0x7fd9d414b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 -- 172.21.15.164:0/1134324134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d414be70 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9ca000640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00d800 tx=0x7fd9bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00b6c0 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc019070 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d4158270 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00ba60 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d4152380 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.673+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4069d60 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.677+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fd9bc00bc00 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.677+0000 7fd9cb400640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 0x7fd9b00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.677+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd9bc085200 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.677+0000 7fd9d2a00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 0x7fd9b00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.677+0000 7fd9d2a00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 0x7fd9b00694b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40015b0 tx=0x7fd9c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.681+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9bc056500 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.837+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9d406d9a0 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.837+0000 7fd9cb400640 1 -- 172.21.15.164:0/1134324134 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12951 (secure 0 0 0) 0x7fd9bc055e00 con 0x7fd9d40cfbd0 2024-05-08T04:39:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:49.844 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":41,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:49.118762+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:49.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 msgr2=0x7fd9b00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 0x7fd9b00694b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40015b0 tx=0x7fd9c4009040 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 msgr2=0x7fd9d414cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00d800 tx=0x7fd9bc00dcc0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 shutdown_connections 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9b0066ff0 0x7fd9b00694b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d415ab60 0x7fd9d40ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9d40cfbd0 0x7fd9d414cab0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 --2- 172.21.15.164:0/1134324134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9d40b26c0 0x7fd9d414b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 >> 172.21.15.164:0/1134324134 conn(0x7fd9d406ac90 msgr2=0x7fd9d406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 shutdown_connections 2024-05-08T04:39:49.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.841+0000 7fd9d9c00640 1 -- 172.21.15.164:0/1134324134 wait complete. 2024-05-08T04:39:49.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:39:49.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:39:50.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 Processor -- start 2024-05-08T04:39:50.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 -- start start 2024-05-08T04:39:50.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc20806cce0 0x7fc2080ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2080cecc0 0x7fc2080ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20815e870 con 0x7fc20815ab10 2024-05-08T04:39:50.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20806bb80 con 0x7fc20806cce0 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:49.997+0000 7fc20dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080cf0f0 con 0x7fc2080cecc0 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc20806cce0 0x7fc2080ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc20806cce0 0x7fc2080ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41578/0 (socket says 172.21.15.164:41578) 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 -- 172.21.15.164:0/1124856064 learned_addr learned my addr 172.21.15.164:0/1124856064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc206a00640 1 --1- 172.21.15.164:0/1124856064 >> v1:172.21.15.164:6789/0 conn(0x7fc20815ab10 0x7fc2080ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34074/0 (socket says 172.21.15.164:34074) 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc207400640 1 --2- 172.21.15.164:0/1124856064 >> v2:172.21.15.172:3300/0 conn(0x7fc2080cecc0 0x7fc2080ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 -- 172.21.15.164:0/1124856064 >> v2:172.21.15.172:3300/0 conn(0x7fc2080cecc0 msgr2=0x7fc2080ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/1124856064 >> v2:172.21.15.172:3300/0 conn(0x7fc2080cecc0 0x7fc2080ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 -- 172.21.15.164:0/1124856064 >> v1:172.21.15.164:6789/0 conn(0x7fc20815ab10 legacy=0x7fc2080ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 -- 172.21.15.164:0/1124856064 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20815eb50 con 0x7fc20806cce0 2024-05-08T04:39:50.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/1124856064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc20806cce0 0x7fc2080ce780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ca1f0 tx=0x7fc1f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e2db243fba8152 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc206000640 1 -- 172.21.15.164:0/1124856064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00ab660 con 0x7fc20806cce0 2024-05-08T04:39:50.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc206000640 1 -- 172.21.15.164:0/1124856064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f00b0070 con 0x7fc20806cce0 2024-05-08T04:39:50.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.001+0000 7fc206000640 1 -- 172.21.15.164:0/1124856064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00aba20 con 0x7fc20806cce0 2024-05-08T04:39:50.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- 172.21.15.164:0/1124856064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc20806cce0 msgr2=0x7fc2080ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- 172.21.15.164:0/1124856064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc20806cce0 0x7fc2080ce780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ca1f0 tx=0x7fc1f001c460 comp rx=0 tx=0).stop 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- 172.21.15.164:0/1124856064 shutdown_connections 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- 172.21.15.164:0/1124856064 >> v2:172.21.15.172:3300/0 conn(0x7fc2080cecc0 0x7fc2080ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- 172.21.15.164:0/1124856064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc20806cce0 0x7fc2080ce780 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- 172.21.15.164:0/1124856064 >> 172.21.15.164:0/1124856064 conn(0x7fc2080cf770 msgr2=0x7fc2080cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- 172.21.15.164:0/1124856064 shutdown_connections 2024-05-08T04:39:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- 172.21.15.164:0/1124856064 wait complete. 2024-05-08T04:39:50.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 Processor -- start 2024-05-08T04:39:50.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- start start 2024-05-08T04:39:50.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc20806cce0 0x7fc208154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20815ab10 0x7fc2080c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080aedf0 con 0x7fc2080cecc0 2024-05-08T04:39:50.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815aef0 con 0x7fc20815ab10 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc20dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080aef70 con 0x7fc20806cce0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41582/0 (socket says 172.21.15.164:41582) 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 -- 172.21.15.164:0/2422629955 learned_addr learned my addr 172.21.15.164:0/2422629955 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 -- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc20806cce0 msgr2=0x7fc208154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc20806cce0 0x7fc208154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 -- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20815ab10 msgr2=0x7fc2080c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20815ab10 0x7fc2080c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.005+0000 7fc1ffe00640 1 -- 172.21.15.164:0/2422629955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20815b210 con 0x7fc2080cecc0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1ffe00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0019070 tx=0x7fc1f001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00be5f0 con 0x7fc2080cecc0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f00be790 con 0x7fc2080cecc0 2024-05-08T04:39:50.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00beb80 con 0x7fc2080cecc0 2024-05-08T04:39:50.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20815b4f0 con 0x7fc2080cecc0 2024-05-08T04:39:50.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20815bab0 con 0x7fc2080cecc0 2024-05-08T04:39:50.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc208072d30 con 0x7fc2080cecc0 2024-05-08T04:39:50.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fc1f00b43f0 con 0x7fc2080cecc0 2024-05-08T04:39:50.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 0x7fc1e40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1f011f8b0 con 0x7fc2080cecc0 2024-05-08T04:39:50.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.009+0000 7fc206a00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 0x7fc1e40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.013+0000 7fc206a00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 0x7fc1e40694b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8000f70 tx=0x7fc1f8007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.013+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1f0016110 con 0x7fc2080cecc0 2024-05-08T04:39:50.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.169+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc20804f010 con 0x7fc2080cecc0 2024-05-08T04:39:50.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.169+0000 7fc1fea00640 1 -- 172.21.15.164:0/2422629955 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1f00f04e0 con 0x7fc2080cecc0 2024-05-08T04:39:50.176 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:39:50.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 msgr2=0x7fc1e40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 0x7fc1e40694b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8000f70 tx=0x7fc1f8007a10 comp rx=0 tx=0).stop 2024-05-08T04:39:50.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 msgr2=0x7fc2081f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0019070 tx=0x7fc1f001aca0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 shutdown_connections 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1e4066ff0 0x7fc1e40694b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20815ab10 0x7fc2080c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2080cecc0 0x7fc2081f8310 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 --2- 172.21.15.164:0/2422629955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc20806cce0 0x7fc208154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 >> 172.21.15.164:0/2422629955 conn(0x7fc2080cf770 msgr2=0x7fc2080ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 shutdown_connections 2024-05-08T04:39:50.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.173+0000 7fc20dc00640 1 -- 172.21.15.164:0/2422629955 wait complete. 2024-05-08T04:39:50.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:39:50.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 Processor -- start 2024-05-08T04:39:50.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 -- start start 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47f4161230 0x7f47f40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47f40ce640 0x7f47f40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40b8440 con 0x7f47f40d17a0 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40d1c70 con 0x7f47f40ce640 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47fae00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40d1ec0 con 0x7f47f4161230 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.329+0000 7f47f8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47f40ce640 0x7f47f40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47f40ce640 0x7f47f40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41596/0 (socket says 172.21.15.164:41596) 2024-05-08T04:39:50.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 -- 172.21.15.164:0/720035041 learned_addr learned my addr 172.21.15.164:0/720035041 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f3e00640 1 --2- 172.21.15.164:0/720035041 >> v2:172.21.15.172:3300/0 conn(0x7f47f4161230 0x7f47f40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 -- 172.21.15.164:0/720035041 >> v2:172.21.15.172:3300/0 conn(0x7f47f4161230 msgr2=0x7f47f40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 --2- 172.21.15.164:0/720035041 >> v2:172.21.15.172:3300/0 conn(0x7f47f4161230 0x7f47f40d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 -- 172.21.15.164:0/720035041 >> v1:172.21.15.172:6789/0 conn(0x7f47f40d17a0 legacy=0x7f47f40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f8e00640 1 -- 172.21.15.164:0/720035041 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40d2250 con 0x7f47f40ce640 2024-05-08T04:39:50.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.333+0000 7f47f3400640 1 -- 172.21.15.164:0/720035041 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173095711 0 0) 0x7f47dc001120 con 0x7f47f40d17a0 2024-05-08T04:39:50.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47f8e00640 1 --2- 172.21.15.164:0/720035041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f40ce640 0x7f47f40cea90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47f408e190 tx=0x7f47ec01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4219435188c17475 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47f3400640 1 -- 172.21.15.164:0/720035041 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5730 con 0x7f47f40ce640 2024-05-08T04:39:50.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47f3400640 1 -- 172.21.15.164:0/720035041 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec0a58d0 con 0x7f47f40ce640 2024-05-08T04:39:50.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47f3400640 1 -- 172.21.15.164:0/720035041 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5ca0 con 0x7f47f40ce640 2024-05-08T04:39:50.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 -- 172.21.15.164:0/720035041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f40ce640 msgr2=0x7f47f40cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 --2- 172.21.15.164:0/720035041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f40ce640 0x7f47f40cea90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47f408e190 tx=0x7f47ec01c960 comp rx=0 tx=0).stop 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 -- 172.21.15.164:0/720035041 shutdown_connections 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 --2- 172.21.15.164:0/720035041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f40ce640 0x7f47f40cea90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 --2- 172.21.15.164:0/720035041 >> v2:172.21.15.172:3300/0 conn(0x7f47f4161230 0x7f47f40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 -- 172.21.15.164:0/720035041 >> 172.21.15.164:0/720035041 conn(0x7f47f409cea0 msgr2=0x7f47f40d3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 -- 172.21.15.164:0/720035041 shutdown_connections 2024-05-08T04:39:50.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 -- 172.21.15.164:0/720035041 wait complete. 2024-05-08T04:39:50.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.337+0000 7f47fae00640 1 Processor -- start 2024-05-08T04:39:50.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- start start 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f40ce640 0x7f47f40b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f40d17a0 0x7f47f41552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f4161790 con 0x7f47f4161230 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f4161610 con 0x7f47f40ce640 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f416aa90 con 0x7f47f40d17a0 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41608/0 (socket says 172.21.15.164:41608) 2024-05-08T04:39:50.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 -- 172.21.15.164:0/3988482103 learned_addr learned my addr 172.21.15.164:0/3988482103 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47ebe00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f40d17a0 0x7f47f41552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f3e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f40ce640 0x7f47f40b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 -- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f40d17a0 msgr2=0x7f47f41552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f40d17a0 0x7f47f41552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 -- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f40ce640 msgr2=0x7f47f40b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f40ce640 0x7f47f40b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 -- 172.21.15.164:0/3988482103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f41fa320 con 0x7f47f4161230 2024-05-08T04:39:50.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f8e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0af040 tx=0x7f47ec019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0bc020 con 0x7f47f4161230 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec0b8670 con 0x7f47f4161230 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b8a60 con 0x7f47f4161230 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f41fa600 con 0x7f47f4161230 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f41fabc0 con 0x7f47f4161230 2024-05-08T04:39:50.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.341+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4069d60 con 0x7f47f4161230 2024-05-08T04:39:50.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.345+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f47ec0bf020 con 0x7f47f4161230 2024-05-08T04:39:50.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.345+0000 7f47f0c00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 0x7f47d00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.345+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f47ec11fca0 con 0x7f47f4161230 2024-05-08T04:39:50.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.345+0000 7f47f3e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 0x7f47d00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.345+0000 7f47f3e00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 0x7f47d00694b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009a50 tx=0x7f47e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.349+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47ec124070 con 0x7f47f4161230 2024-05-08T04:39:50.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.505+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f47f40ca820 con 0x7f47f4161230 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.505+0000 7f47f0c00640 1 -- 172.21.15.164:0/3988482103 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f47ec0c6030 con 0x7f47f4161230 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:39:50.512 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 msgr2=0x7f47d00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 0x7f47d00694b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009a50 tx=0x7f47e4008040 comp rx=0 tx=0).stop 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 msgr2=0x7f47f41557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0af040 tx=0x7f47ec019700 comp rx=0 tx=0).stop 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 shutdown_connections 2024-05-08T04:39:50.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47d0066ff0 0x7f47d00694b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f40d17a0 0x7f47f41552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4161230 0x7f47f41557f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 --2- 172.21.15.164:0/3988482103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f40ce640 0x7f47f40b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 >> 172.21.15.164:0/3988482103 conn(0x7f47f409cea0 msgr2=0x7f47f406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 shutdown_connections 2024-05-08T04:39:50.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.509+0000 7f47fae00640 1 -- 172.21.15.164:0/3988482103 wait complete. 2024-05-08T04:39:50.533 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:39:50.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:39:50.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.665+0000 7f4886000640 1 Processor -- start 2024-05-08T04:39:50.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 -- start start 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f488015dca0 0x7f48800b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48800b0f00 0x7f48800d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48800d1c20 0x7f488006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4880161880 con 0x7f48800d1c20 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800d20b0 con 0x7f48800b0f00 2024-05-08T04:39:50.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4886000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880161ad0 con 0x7f488015dca0 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48800d1c20 0x7f488006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48800d1c20 0x7f488006fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41622/0 (socket says 172.21.15.164:41622) 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 -- 172.21.15.164:0/298716628 learned_addr learned my addr 172.21.15.164:0/298716628 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f487fe00640 1 --2- 172.21.15.164:0/298716628 >> v2:172.21.15.172:3300/0 conn(0x7f48800b0f00 0x7f48800d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4884c00640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f488015dca0 0x7f48800b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 -- 172.21.15.164:0/298716628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f488015dca0 msgr2=0x7f48800b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f488015dca0 0x7f48800b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 -- 172.21.15.164:0/298716628 >> v2:172.21.15.172:3300/0 conn(0x7f48800b0f00 msgr2=0x7f48800d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 --2- 172.21.15.164:0/298716628 >> v2:172.21.15.172:3300/0 conn(0x7f48800b0f00 0x7f48800d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.669+0000 7f4885600640 1 -- 172.21.15.164:0/298716628 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880161db0 con 0x7f48800d1c20 2024-05-08T04:39:50.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4885600640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800d1c20 0x7f488006fc50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f488006bc60 tx=0x7f487801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8b26e7212481f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f487f400640 1 -- 172.21.15.164:0/298716628 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48780b0070 con 0x7f48800d1c20 2024-05-08T04:39:50.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f487f400640 1 -- 172.21.15.164:0/298716628 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48780ab600 con 0x7f48800d1c20 2024-05-08T04:39:50.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f487f400640 1 -- 172.21.15.164:0/298716628 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48780ab9d0 con 0x7f48800d1c20 2024-05-08T04:39:50.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 -- 172.21.15.164:0/298716628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800d1c20 msgr2=0x7f488006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800d1c20 0x7f488006fc50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f488006bc60 tx=0x7f487801c260 comp rx=0 tx=0).stop 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 -- 172.21.15.164:0/298716628 shutdown_connections 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800d1c20 0x7f488006fc50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 --2- 172.21.15.164:0/298716628 >> v2:172.21.15.172:3300/0 conn(0x7f48800b0f00 0x7f48800d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 --2- 172.21.15.164:0/298716628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f488015dca0 0x7f48800b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 -- 172.21.15.164:0/298716628 >> 172.21.15.164:0/298716628 conn(0x7f48800723e0 msgr2=0x7f48800727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 -- 172.21.15.164:0/298716628 shutdown_connections 2024-05-08T04:39:50.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 -- 172.21.15.164:0/298716628 wait complete. 2024-05-08T04:39:50.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.673+0000 7f4886000640 1 Processor -- start 2024-05-08T04:39:50.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- start start 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48800d1c20 0x7f4880157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f488015dca0 0x7f48801545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488006c6f0 con 0x7f48800b0f00 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800d0bf0 con 0x7f488015dca0 2024-05-08T04:39:50.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488006c870 con 0x7f48800d1c20 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41630/0 (socket says 172.21.15.164:41630) 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 -- 172.21.15.164:0/835157819 learned_addr learned my addr 172.21.15.164:0/835157819 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f487fe00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48800d1c20 0x7f4880157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4885600640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f488015dca0 0x7f48801545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 -- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48800d1c20 msgr2=0x7f4880157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48800d1c20 0x7f4880157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 -- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f488015dca0 msgr2=0x7f48801545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f488015dca0 0x7f48801545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 -- 172.21.15.164:0/835157819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488014dc90 con 0x7f48800b0f00 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4884c00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f487000efc0 tx=0x7f487000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870018020 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870014480 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870014850 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488014f660 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488014df70 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.677+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488004ef30 con 0x7f48800b0f00 2024-05-08T04:39:50.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.681+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f487001b020 con 0x7f48800b0f00 2024-05-08T04:39:50.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.681+0000 7f487cc00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 0x7f48600694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:50.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.681+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48700858b0 con 0x7f48800b0f00 2024-05-08T04:39:50.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.681+0000 7f487fe00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 0x7f48600694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:50.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.681+0000 7f487fe00640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 0x7f48600694b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48800b6ee0 tx=0x7f486c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:50.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.685+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4870022070 con 0x7f48800b0f00 2024-05-08T04:39:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:50.841+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f48800cf440 con 0x7f48800b0f00 2024-05-08T04:39:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.757+0000 7f487cc00640 1 -- 172.21.15.164:0/835157819 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4870010040 con 0x7f48800b0f00 2024-05-08T04:39:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:39:51.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 msgr2=0x7f48600694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 0x7f48600694b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48800b6ee0 tx=0x7f486c009190 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 msgr2=0x7f488014d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f487000efc0 tx=0x7f487000c490 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 shutdown_connections 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4860066ff0 0x7f48600694b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48800d1c20 0x7f4880157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f488015dca0 0x7f48801545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 --2- 172.21.15.164:0/835157819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48800b0f00 0x7f488014d310 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 >> 172.21.15.164:0/835157819 conn(0x7f48800723e0 msgr2=0x7f488015e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 shutdown_connections 2024-05-08T04:39:51.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.761+0000 7f4886000640 1 -- 172.21.15.164:0/835157819 wait complete. 2024-05-08T04:39:51.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:39:51.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 Processor -- start 2024-05-08T04:39:51.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 -- start start 2024-05-08T04:39:51.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22b415a8a0 0x7f22b40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 0x7f22b40cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b4071ef0 con 0x7f22b40ce5e0 2024-05-08T04:39:51.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b415ac80 con 0x7f22b415a8a0 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b40cfdc0 con 0x7f22b40d0d80 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22b415a8a0 0x7f22b40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22b415a8a0 0x7f22b40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41632/0 (socket says 172.21.15.164:41632) 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 -- 172.21.15.164:0/1671744543 learned_addr learned my addr 172.21.15.164:0/1671744543 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3400640 1 --2- 172.21.15.164:0/1671744543 >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 0x7f22b40cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 -- 172.21.15.164:0/1671744543 >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 msgr2=0x7f22b40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 --2- 172.21.15.164:0/1671744543 >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 0x7f22b40cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 -- 172.21.15.164:0/1671744543 >> v1:172.21.15.172:6789/0 conn(0x7f22b40ce5e0 legacy=0x7f22b40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b2a00640 1 -- 172.21.15.164:0/1671744543 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40d0060 con 0x7f22b415a8a0 2024-05-08T04:39:51.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.937+0000 7f22b3400640 1 --2- 172.21.15.164:0/1671744543 >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 0x7f22b40cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:51.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b2a00640 1 --2- 172.21.15.164:0/1671744543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b415a8a0 0x7f22b40b3630 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f22b40b5d20 tx=0x7f22a401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09b7feb3f6b90fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:51.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b1600640 1 -- 172.21.15.164:0/1671744543 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a40b0070 con 0x7f22b415a8a0 2024-05-08T04:39:51.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b1600640 1 -- 172.21.15.164:0/1671744543 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a40ab400 con 0x7f22b415a8a0 2024-05-08T04:39:51.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b1600640 1 -- 172.21.15.164:0/1671744543 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a40ab7d0 con 0x7f22b415a8a0 2024-05-08T04:39:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- 172.21.15.164:0/1671744543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b415a8a0 msgr2=0x7f22b40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 --2- 172.21.15.164:0/1671744543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b415a8a0 0x7f22b40b3630 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f22b40b5d20 tx=0x7f22a401c450 comp rx=0 tx=0).stop 2024-05-08T04:39:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- 172.21.15.164:0/1671744543 shutdown_connections 2024-05-08T04:39:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 --2- 172.21.15.164:0/1671744543 >> v2:172.21.15.172:3300/0 conn(0x7f22b40d0d80 0x7f22b40cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 --2- 172.21.15.164:0/1671744543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b415a8a0 0x7f22b40b3630 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- 172.21.15.164:0/1671744543 >> 172.21.15.164:0/1671744543 conn(0x7f22b4066fa0 msgr2=0x7f22b406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- 172.21.15.164:0/1671744543 shutdown_connections 2024-05-08T04:39:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- 172.21.15.164:0/1671744543 wait complete. 2024-05-08T04:39:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 Processor -- start 2024-05-08T04:39:51.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.941+0000 7f22b3e00640 1 -- start start 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b40ce5e0 0x7f22b406b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b415a8a0 0x7f22b40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b41655a0 con 0x7f22b40d0d80 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4165420 con 0x7f22b40ce5e0 2024-05-08T04:39:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4165720 con 0x7f22b415a8a0 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41636/0 (socket says 172.21.15.164:41636) 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 -- 172.21.15.164:0/2712866638 learned_addr learned my addr 172.21.15.164:0/2712866638 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2a00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b40ce5e0 0x7f22b406b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3400640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b415a8a0 0x7f22b40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 -- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b415a8a0 msgr2=0x7f22b40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b415a8a0 0x7f22b40b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 -- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b40ce5e0 msgr2=0x7f22b406b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b40ce5e0 0x7f22b406b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 -- 172.21.15.164:0/2712866638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40cc920 con 0x7f22b40d0d80 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b2000640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c450 tx=0x7f22a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0012830 con 0x7f22b40d0d80 2024-05-08T04:39:51.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22b40d0d80 2024-05-08T04:39:51.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b40b37c0 con 0x7f22b40d0d80 2024-05-08T04:39:51.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0012c30 con 0x7f22b40d0d80 2024-05-08T04:39:51.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b40b3d80 con 0x7f22b40d0d80 2024-05-08T04:39:51.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.945+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b404ef30 con 0x7f22b40d0d80 2024-05-08T04:39:51.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.949+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f22a001c460 con 0x7f22b40d0d80 2024-05-08T04:39:51.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.949+0000 7f22ab400640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 0x7f22940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:51.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.949+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f22a0084b50 con 0x7f22b40d0d80 2024-05-08T04:39:51.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.949+0000 7f22b2a00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 0x7f22940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:51.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.949+0000 7f22b2a00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 0x7f22940694b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22a40ad000 tx=0x7f22a40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:51.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:51.953+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22a0055600 con 0x7f22b40d0d80 2024-05-08T04:39:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.109+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f22b414fe00 con 0x7f22b40d0d80 2024-05-08T04:39:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22ab400640 1 -- 172.21.15.164:0/2712866638 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f22a00596a0 con 0x7f22b40d0d80 2024-05-08T04:39:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 msgr2=0x7f22940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 0x7f22940694b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22a40ad000 tx=0x7f22a40a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 msgr2=0x7f22b4167e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c450 tx=0x7f22a000c910 comp rx=0 tx=0).stop 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 shutdown_connections 2024-05-08T04:39:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294066ff0 0x7f22940694b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b415a8a0 0x7f22b40b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b40d0d80 0x7f22b4167e90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 --2- 172.21.15.164:0/2712866638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b40ce5e0 0x7f22b406b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 >> 172.21.15.164:0/2712866638 conn(0x7f22b4066fa0 msgr2=0x7f22b406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 shutdown_connections 2024-05-08T04:39:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.761+0000 7f22b3e00640 1 -- 172.21.15.164:0/2712866638 wait complete. 2024-05-08T04:39:52.802 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:39:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 Processor -- start 2024-05-08T04:39:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 -- start start 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f767415ba20 0x7f7674160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f767406de50 0x7f767406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76740c6600 con 0x7f76740ae6e0 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767415fbe0 con 0x7f767415ba20 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f767c400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76740c6040 con 0x7f767406de50 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f767415ba20 0x7f7674160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f767415ba20 0x7f7674160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41652/0 (socket says 172.21.15.164:41652) 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 -- 172.21.15.164:0/1543453477 learned_addr learned my addr 172.21.15.164:0/1543453477 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679000640 1 --2- 172.21.15.164:0/1543453477 >> v2:172.21.15.172:3300/0 conn(0x7f767406de50 0x7f767406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 -- 172.21.15.164:0/1543453477 >> v2:172.21.15.172:3300/0 conn(0x7f767406de50 msgr2=0x7f767406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:52.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 --2- 172.21.15.164:0/1543453477 >> v2:172.21.15.172:3300/0 conn(0x7f767406de50 0x7f767406e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 -- 172.21.15.164:0/1543453477 >> v1:172.21.15.172:6789/0 conn(0x7f76740ae6e0 legacy=0x7f76740cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:52.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.937+0000 7f7679a00640 1 -- 172.21.15.164:0/1543453477 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76740c6320 con 0x7f767415ba20 2024-05-08T04:39:52.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f7679a00640 1 --2- 172.21.15.164:0/1543453477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767415ba20 0x7f7674160070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f76740b5980 tx=0x7f766801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2a11d59551066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:52.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f7673e00640 1 -- 172.21.15.164:0/1543453477 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680a5820 con 0x7f767415ba20 2024-05-08T04:39:52.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f7673e00640 1 -- 172.21.15.164:0/1543453477 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76680a59c0 con 0x7f767415ba20 2024-05-08T04:39:52.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f7673e00640 1 -- 172.21.15.164:0/1543453477 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680a5d90 con 0x7f767415ba20 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 -- 172.21.15.164:0/1543453477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767415ba20 msgr2=0x7f7674160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 --2- 172.21.15.164:0/1543453477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767415ba20 0x7f7674160070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f76740b5980 tx=0x7f766801c9c0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 -- 172.21.15.164:0/1543453477 shutdown_connections 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 --2- 172.21.15.164:0/1543453477 >> v2:172.21.15.172:3300/0 conn(0x7f767406de50 0x7f767406e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 --2- 172.21.15.164:0/1543453477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767415ba20 0x7f7674160070 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 -- 172.21.15.164:0/1543453477 >> 172.21.15.164:0/1543453477 conn(0x7f76740cf770 msgr2=0x7f76740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 -- 172.21.15.164:0/1543453477 shutdown_connections 2024-05-08T04:39:52.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.941+0000 7f767c400640 1 -- 172.21.15.164:0/1543453477 wait complete. 2024-05-08T04:39:52.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 Processor -- start 2024-05-08T04:39:52.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- start start 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f767406de50 0x7f7674071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f767415ba20 0x7f767406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76740aec40 con 0x7f76740ae6e0 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76740aeac0 con 0x7f767415ba20 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76740aedc0 con 0x7f767406de50 2024-05-08T04:39:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41666/0 (socket says 172.21.15.164:41666) 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 -- 172.21.15.164:0/436271612 learned_addr learned my addr 172.21.15.164:0/436271612 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679a00640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f767406de50 0x7f7674071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767a400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f767415ba20 0x7f767406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 -- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f767406de50 msgr2=0x7f7674071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f767406de50 0x7f7674071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 -- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f767415ba20 msgr2=0x7f767406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f767415ba20 0x7f767406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:52.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 -- 172.21.15.164:0/436271612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674072020 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7679000640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f766000d800 tx=0x7f766000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766000b6c0 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660019070 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7674153410 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766000ba60 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76741539d0 con 0x7f76740ae6e0 2024-05-08T04:39:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.945+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7674072d30 con 0x7f76740ae6e0 2024-05-08T04:39:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.949+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f766000bc00 con 0x7f76740ae6e0 2024-05-08T04:39:52.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.949+0000 7f7671600640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 0x7f76540694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:52.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.949+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7660085ee0 con 0x7f76740ae6e0 2024-05-08T04:39:52.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.949+0000 7f7679a00640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 0x7f76540694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:52.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.949+0000 7f7679a00640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 0x7f76540694b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76680ae000 tx=0x7f76680a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:52.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:52.953+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7660056890 con 0x7f76740ae6e0 2024-05-08T04:39:53.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.141+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7674150ad0 con 0x7f76740ae6e0 2024-05-08T04:39:53.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.789+0000 7f7671600640 1 -- 172.21.15.164:0/436271612 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f766005a930 con 0x7f76740ae6e0 2024-05-08T04:39:53.805 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 6 and data pool 7 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 msgr2=0x7f76540694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 0x7f76540694b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76680ae000 tx=0x7f76680a7070 comp rx=0 tx=0).stop 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 msgr2=0x7f76740c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f766000d800 tx=0x7f766000dcc0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 shutdown_connections 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7654066ff0 0x7f76540694b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f767415ba20 0x7f767406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76740ae6e0 0x7f76740c7760 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f766000d800 tx=0x7f766000dcc0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 --2- 172.21.15.164:0/436271612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f767406de50 0x7f7674071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.801+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 >> 172.21.15.164:0/436271612 conn(0x7f76740cf770 msgr2=0x7f767415d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.805+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 shutdown_connections 2024-05-08T04:39:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.805+0000 7f767c400640 1 -- 172.21.15.164:0/436271612 wait complete. 2024-05-08T04:39:53.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:53.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 Processor -- start 2024-05-08T04:39:53.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 -- start start 2024-05-08T04:39:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc52c0cfda0 0x7fc52c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c06c770 con 0x7fc52c15ab60 2024-05-08T04:39:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c06ca50 con 0x7fc52c0b26c0 2024-05-08T04:39:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc531a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c06cd30 con 0x7fc52c0cfda0 2024-05-08T04:39:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc52aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc52c15ab60 0x7fc52c072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34080/0 (socket says 172.21.15.164:34080) 2024-05-08T04:39:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.969+0000 7fc52aa00640 1 -- 172.21.15.164:0/3784274673 learned_addr learned my addr 172.21.15.164:0/3784274673 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:53.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52b400640 1 --2- 172.21.15.164:0/3784274673 >> v2:172.21.15.172:3300/0 conn(0x7fc52c0cfda0 0x7fc52c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835628997 0 0) 0x7fc52c06c770 con 0x7fc52c15ab60 2024-05-08T04:39:53.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc510005740 con 0x7fc52c15ab60 2024-05-08T04:39:53.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924058096 0 0) 0x7fc52c06ca50 con 0x7fc52c0b26c0 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c06c770 con 0x7fc52c0b26c0 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999608407 0 0) 0x7fc510005740 con 0x7fc52c15ab60 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 >> v2:172.21.15.172:3300/0 conn(0x7fc52c0cfda0 msgr2=0x7fc52c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 --2- 172.21.15.164:0/3784274673 >> v2:172.21.15.172:3300/0 conn(0x7fc52c0cfda0 0x7fc52c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 >> v1:172.21.15.172:6789/0 conn(0x7fc52c0b26c0 legacy=0x7fc52c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c06d010 con 0x7fc52c15ab60 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc51c001680 con 0x7fc52c15ab60 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 >> v1:172.21.15.164:6789/0 conn(0x7fc52c15ab60 legacy=0x7fc52c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc510005d60 0x7fc510006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc510006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5100073e0 0x7fc51000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c06cd30 con 0x7fc510006a20 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51000e260 con 0x7fc5100073e0 2024-05-08T04:39:53.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51000e460 con 0x7fc510005d60 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc510006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52aa00640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc510005d60 0x7fc510006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52b400640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5100073e0 0x7fc51000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 -- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc510005d60 msgr2=0x7fc510006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc510005d60 0x7fc510006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 -- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5100073e0 msgr2=0x7fc51000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5100073e0 0x7fc51000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc522000640 1 -- 172.21.15.164:0/3784274673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510005740 con 0x7fc510006a20 2024-05-08T04:39:53.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.973+0000 7fc52b400640 1 -- 172.21.15.164:0/3784274673 reap_dead start 2024-05-08T04:39:53.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc522000640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc510006ea0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc5140018d0 tx=0x7fc51401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498fdd00e1aeb758 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:53.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514001520 con 0x7fc510006a20 2024-05-08T04:39:53.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5140052f0 con 0x7fc510006a20 2024-05-08T04:39:53.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc52a000640 1 -- 172.21.15.164:0/3784274673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140a88f0 con 0x7fc510006a20 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 msgr2=0x7fc510006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc510006ea0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc5140018d0 tx=0x7fc51401ec90 comp rx=0 tx=0).stop 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- 172.21.15.164:0/3784274673 shutdown_connections 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 --2- 172.21.15.164:0/3784274673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc510006ea0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- 172.21.15.164:0/3784274673 >> 172.21.15.164:0/3784274673 conn(0x7fc52c06ac90 msgr2=0x7fc52c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- 172.21.15.164:0/3784274673 shutdown_connections 2024-05-08T04:39:53.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- 172.21.15.164:0/3784274673 wait complete. 2024-05-08T04:39:53.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 Processor -- start 2024-05-08T04:39:53.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- start start 2024-05-08T04:39:53.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c15af40 0x7fc52c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc52c1469e0 0x7fc52c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c06c550 con 0x7fc510006a20 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c06c3d0 con 0x7fc52c15af40 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.977+0000 7fc531a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0aed30 con 0x7fc52c1469e0 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41680/0 (socket says 172.21.15.164:41680) 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 -- 172.21.15.164:0/3268097381 learned_addr learned my addr 172.21.15.164:0/3268097381 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52b400640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc52c1469e0 0x7fc52c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc522000640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c15af40 0x7fc52c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 -- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc52c1469e0 msgr2=0x7fc52c156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc52c1469e0 0x7fc52c156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 -- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c15af40 msgr2=0x7fc52c15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c15af40 0x7fc52c15d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 -- 172.21.15.164:0/3268097381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c154700 con 0x7fc510006a20 2024-05-08T04:39:53.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc52aa00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc51c004ed0 tx=0x7fc51c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:53.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00f7b0 con 0x7fc510006a20 2024-05-08T04:39:53.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c018070 con 0x7fc510006a20 2024-05-08T04:39:53.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00fbb0 con 0x7fc510006a20 2024-05-08T04:39:53.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1549e0 con 0x7fc510006a20 2024-05-08T04:39:53.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c154fa0 con 0x7fc510006a20 2024-05-08T04:39:53.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c15b320 con 0x7fc510006a20 2024-05-08T04:39:53.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fc51c00fd50 con 0x7fc510006a20 2024-05-08T04:39:53.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.981+0000 7fc523400640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 0x7fc5080694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:53.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.985+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc51c084fe0 con 0x7fc510006a20 2024-05-08T04:39:53.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.985+0000 7fc522000640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 0x7fc5080694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:53.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.985+0000 7fc522000640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 0x7fc5080694b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5140b0000 tx=0x7fc5140aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:53.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:53.989+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc51c055ff0 con 0x7fc510006a20 2024-05-08T04:39:54.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.141+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc52c069d60 con 0x7fc510006a20 2024-05-08T04:39:54.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.141+0000 7fc523400640 1 -- 172.21.15.164:0/3268097381 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16686 (secure 0 0 0) 0x7fc51c0558f0 con 0x7fc510006a20 2024-05-08T04:39:54.147 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:54.147 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:53.146891+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:54.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 msgr2=0x7fc5080694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 0x7fc5080694b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5140b0000 tx=0x7fc5140aa070 comp rx=0 tx=0).stop 2024-05-08T04:39:54.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 msgr2=0x7fc52c146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc51c004ed0 tx=0x7fc51c007430 comp rx=0 tx=0).stop 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 shutdown_connections 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc508066ff0 0x7fc5080694b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc52c1469e0 0x7fc52c156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc52c15af40 0x7fc52c15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 --2- 172.21.15.164:0/3268097381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510006a20 0x7fc52c146340 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 >> 172.21.15.164:0/3268097381 conn(0x7fc52c06ac90 msgr2=0x7fc52c0cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 shutdown_connections 2024-05-08T04:39:54.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.145+0000 7fc531a00640 1 -- 172.21.15.164:0/3268097381 wait complete. 2024-05-08T04:39:54.169 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:54.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 Processor -- start 2024-05-08T04:39:54.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 -- start start 2024-05-08T04:39:54.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe59c06b3a0 0x7fe59c0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c15f400 con 0x7fe59c15b9f0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c15f6e0 con 0x7fe59c06b3a0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c15f9c0 con 0x7fe59c06ef60 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a0a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe59c15b9f0 0x7fe59c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34094/0 (socket says 172.21.15.164:34094) 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a0a00640 1 -- 172.21.15.164:0/1197732852 learned_addr learned my addr 172.21.15.164:0/1197732852 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59be00640 1 --2- 172.21.15.164:0/1197732852 >> v2:172.21.15.172:3300/0 conn(0x7fe59c06b3a0 0x7fe59c0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1400640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59b400640 1 -- 172.21.15.164:0/1197732852 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494326208 0 0) 0x7fe59c15f400 con 0x7fe59c15b9f0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59b400640 1 -- 172.21.15.164:0/1197732852 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe584003530 con 0x7fe59c15b9f0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59be00640 1 -- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 msgr2=0x7fe59c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59be00640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59be00640 1 -- 172.21.15.164:0/1197732852 >> v1:172.21.15.164:6789/0 conn(0x7fe59c15b9f0 legacy=0x7fe59c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe59be00640 1 -- 172.21.15.164:0/1197732852 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c15fca0 con 0x7fe59c06b3a0 2024-05-08T04:39:54.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.301+0000 7fe5a1400640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:39:54.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe59be00640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c0cea10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe58801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=926ea2f3f1605847 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe59b400640 1 -- 172.21.15.164:0/1197732852 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880aa600 con 0x7fe59c06b3a0 2024-05-08T04:39:54.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe59b400640 1 -- 172.21.15.164:0/1197732852 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5880af070 con 0x7fe59c06b3a0 2024-05-08T04:39:54.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe59b400640 1 -- 172.21.15.164:0/1197732852 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880aa9c0 con 0x7fe59c06b3a0 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 -- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 msgr2=0x7fe59c0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c0cea10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe58801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 -- 172.21.15.164:0/1197732852 shutdown_connections 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/1197732852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c0cea10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 -- 172.21.15.164:0/1197732852 >> 172.21.15.164:0/1197732852 conn(0x7fe59c066fa0 msgr2=0x7fe59c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 -- 172.21.15.164:0/1197732852 shutdown_connections 2024-05-08T04:39:54.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.305+0000 7fe5a1e00640 1 -- 172.21.15.164:0/1197732852 wait complete. 2024-05-08T04:39:54.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 Processor -- start 2024-05-08T04:39:54.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- start start 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c168b30 con 0x7fe59c15b9f0 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1689b0 con 0x7fe59c06b3a0 2024-05-08T04:39:54.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c168cb0 con 0x7fe59c06ef60 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41682/0 (socket says 172.21.15.164:41682) 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 -- 172.21.15.164:0/4210273847 learned_addr learned my addr 172.21.15.164:0/4210273847 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe59be00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a0a00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 -- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 msgr2=0x7fe59c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 -- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 msgr2=0x7fe59c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 -- 172.21.15.164:0/4210273847 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0ae2e0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1400640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f00 tx=0x7fe59400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594018030 con 0x7fe59c15b9f0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59400ce50 con 0x7fe59c15b9f0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c0ae5c0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594003bc0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c0ae8a0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.309+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c04ef30 con 0x7fe59c15b9f0 2024-05-08T04:39:54.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.313+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fe59401b070 con 0x7fe59c15b9f0 2024-05-08T04:39:54.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.313+0000 7fe598c00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 0x7fe57c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.313+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe594085cc0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.313+0000 7fe5a0a00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 0x7fe57c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.313+0000 7fe5a0a00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 0x7fe57c0694b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe590005890 tx=0x7fe590009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.317+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe59408c080 con 0x7fe59c15b9f0 2024-05-08T04:39:54.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe59c0725c0 con 0x7fe59c15b9f0 2024-05-08T04:39:54.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe598c00640 1 -- 172.21.15.164:0/4210273847 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16686 (secure 0 0 0) 0x7fe594010040 con 0x7fe59c15b9f0 2024-05-08T04:39:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:54.478 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:53.146891+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 msgr2=0x7fe57c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 0x7fe57c0694b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe590005890 tx=0x7fe590009040 comp rx=0 tx=0).stop 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 msgr2=0x7fe59c0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f00 tx=0x7fe59400c490 comp rx=0 tx=0).stop 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 shutdown_connections 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe57c066ff0 0x7fe57c0694b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe59c06ef60 0x7fe59c0b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c15b9f0 0x7fe59c0ad890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 --2- 172.21.15.164:0/4210273847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe59c06b3a0 0x7fe59c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.473+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 >> 172.21.15.164:0/4210273847 conn(0x7fe59c066fa0 msgr2=0x7fe59c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:54.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.477+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 shutdown_connections 2024-05-08T04:39:54.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.477+0000 7fe5a1e00640 1 -- 172.21.15.164:0/4210273847 wait complete. 2024-05-08T04:39:54.500 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:39:54.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 Processor -- start 2024-05-08T04:39:54.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 -- start start 2024-05-08T04:39:54.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd14158090 0x7fdd140af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd1415ec00 0x7fdd140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd140d0a50 0x7fdd140d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140b5620 con 0x7fdd140d0a50 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1415e120 con 0x7fdd1415ec00 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1415e400 con 0x7fdd14158090 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd140d0a50 0x7fdd140d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd140d0a50 0x7fdd140d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41684/0 (socket says 172.21.15.164:41684) 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd13400640 1 -- 172.21.15.164:0/3781923097 learned_addr learned my addr 172.21.15.164:0/3781923097 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.633+0000 7fdd0a000640 1 --2- 172.21.15.164:0/3781923097 >> v2:172.21.15.172:3300/0 conn(0x7fdd1415ec00 0x7fdd140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd12a00640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd14158090 0x7fdd140af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 -- 172.21.15.164:0/3781923097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd14158090 msgr2=0x7fdd140af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:54.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd14158090 0x7fdd140af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 -- 172.21.15.164:0/3781923097 >> v2:172.21.15.172:3300/0 conn(0x7fdd1415ec00 msgr2=0x7fdd140b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:54.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 --2- 172.21.15.164:0/3781923097 >> v2:172.21.15.172:3300/0 conn(0x7fdd1415ec00 0x7fdd140b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 -- 172.21.15.164:0/3781923097 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1415e6e0 con 0x7fdd140d0a50 2024-05-08T04:39:54.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13400640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd140d0a50 0x7fdd140d0f00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd140d1600 tx=0x7fdd0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eae9300d3d49b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd12000640 1 -- 172.21.15.164:0/3781923097 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0b0070 con 0x7fdd140d0a50 2024-05-08T04:39:54.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd12000640 1 -- 172.21.15.164:0/3781923097 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0ab600 con 0x7fdd140d0a50 2024-05-08T04:39:54.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd12000640 1 -- 172.21.15.164:0/3781923097 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0ab9d0 con 0x7fdd140d0a50 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 -- 172.21.15.164:0/3781923097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd140d0a50 msgr2=0x7fdd140d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd140d0a50 0x7fdd140d0f00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd140d1600 tx=0x7fdd0c01c520 comp rx=0 tx=0).stop 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 -- 172.21.15.164:0/3781923097 shutdown_connections 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd140d0a50 0x7fdd140d0f00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 --2- 172.21.15.164:0/3781923097 >> v2:172.21.15.172:3300/0 conn(0x7fdd1415ec00 0x7fdd140b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 --2- 172.21.15.164:0/3781923097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd14158090 0x7fdd140af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 -- 172.21.15.164:0/3781923097 >> 172.21.15.164:0/3781923097 conn(0x7fdd1406c1a0 msgr2=0x7fdd1406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 -- 172.21.15.164:0/3781923097 shutdown_connections 2024-05-08T04:39:54.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.637+0000 7fdd13e00640 1 -- 172.21.15.164:0/3781923097 wait complete. 2024-05-08T04:39:54.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 Processor -- start 2024-05-08T04:39:54.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- start start 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd140d0a50 0x7fdd1406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd14158090 0x7fdd14153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1406b830 con 0x7fdd1415ec00 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1406b6b0 con 0x7fdd14158090 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1406b9b0 con 0x7fdd140d0a50 2024-05-08T04:39:54.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41688/0 (socket says 172.21.15.164:41688) 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 -- 172.21.15.164:0/339858690 learned_addr learned my addr 172.21.15.164:0/339858690 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd12a00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd140d0a50 0x7fdd1406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd0a000640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd14158090 0x7fdd14153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 -- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd140d0a50 msgr2=0x7fdd1406e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd140d0a50 0x7fdd1406e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 -- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd14158090 msgr2=0x7fdd14153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd14158090 0x7fdd14153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 -- 172.21.15.164:0/339858690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14154240 con 0x7fdd1415ec00 2024-05-08T04:39:54.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13400640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ae040 tx=0x7fdd0c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0b0040 con 0x7fdd1415ec00 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0b9070 con 0x7fdd1415ec00 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0b4840 con 0x7fdd1415ec00 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1406ea50 con 0x7fdd1415ec00 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.641+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1406f010 con 0x7fdd1415ec00 2024-05-08T04:39:54.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1404ef30 con 0x7fdd1415ec00 2024-05-08T04:39:54.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fdd0c0b49e0 con 0x7fdd1415ec00 2024-05-08T04:39:54.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd0b400640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 0x7fdcf00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:54.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd0c11ee60 con 0x7fdd1415ec00 2024-05-08T04:39:54.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd12a00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 0x7fdcf00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:54.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.645+0000 7fdd12a00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 0x7fdcf00694b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd140b4af0 tx=0x7fdd04009790 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:54.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.649+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd14154240 con 0x7fdd1415ec00 2024-05-08T04:39:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:54.841+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdd14160830 con 0x7fdd1415ec00 2024-05-08T04:39:55.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.797+0000 7fdd0b400640 1 -- 172.21.15.164:0/339858690 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdd0c016110 con 0x7fdd1415ec00 2024-05-08T04:39:55.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 msgr2=0x7fdcf00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:55.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 0x7fdcf00694b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd140b4af0 tx=0x7fdd04009790 comp rx=0 tx=0).stop 2024-05-08T04:39:55.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 msgr2=0x7fdd14157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ae040 tx=0x7fdd0c0192c0 comp rx=0 tx=0).stop 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 shutdown_connections 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdcf0066ff0 0x7fdcf00694b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd1415ec00 0x7fdd14157320 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd14158090 0x7fdd14153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 --2- 172.21.15.164:0/339858690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd140d0a50 0x7fdd1406e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 >> 172.21.15.164:0/339858690 conn(0x7fdd1406c1a0 msgr2=0x7fdd1415f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 shutdown_connections 2024-05-08T04:39:55.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:55.801+0000 7fdd13e00640 1 -- 172.21.15.164:0/339858690 wait complete. 2024-05-08T04:39:55.863 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:39:55.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:39:56.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 Processor -- start 2024-05-08T04:39:56.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 -- start start 2024-05-08T04:39:56.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b3015ab10 0x7f2b3006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b300ae6e0 0x7f2b300aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3015e870 con 0x7f2b300ce780 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3006bb80 con 0x7f2b3015ab10 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b37a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3015eb50 con 0x7f2b300ae6e0 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b3015ab10 0x7f2b3006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b3015ab10 0x7f2b3006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41702/0 (socket says 172.21.15.164:41702) 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 -- 172.21.15.164:0/1993760106 learned_addr learned my addr 172.21.15.164:0/1993760106 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b27e00640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b300ae6e0 0x7f2b300aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 -- 172.21.15.164:0/1993760106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b300ae6e0 msgr2=0x7f2b300aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b300ae6e0 0x7f2b300aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 -- 172.21.15.164:0/1993760106 >> v1:172.21.15.172:6789/0 conn(0x7f2b300ce780 legacy=0x7f2b300c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.001+0000 7f2b35000640 1 -- 172.21.15.164:0/1993760106 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3015ee30 con 0x7f2b3015ab10 2024-05-08T04:39:56.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b35000640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3015ab10 0x7f2b3006cce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009870 tx=0x7f2b2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bfa970111cfd0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b2fe00640 1 -- 172.21.15.164:0/1993760106 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200aa600 con 0x7f2b3015ab10 2024-05-08T04:39:56.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b2fe00640 1 -- 172.21.15.164:0/1993760106 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b200af070 con 0x7f2b3015ab10 2024-05-08T04:39:56.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b2fe00640 1 -- 172.21.15.164:0/1993760106 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200aa9c0 con 0x7f2b3015ab10 2024-05-08T04:39:56.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 -- 172.21.15.164:0/1993760106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3015ab10 msgr2=0x7f2b3006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3015ab10 0x7f2b3006cce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009870 tx=0x7f2b2001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 -- 172.21.15.164:0/1993760106 shutdown_connections 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b300ae6e0 0x7f2b300aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 --2- 172.21.15.164:0/1993760106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3015ab10 0x7f2b3006cce0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 -- 172.21.15.164:0/1993760106 >> 172.21.15.164:0/1993760106 conn(0x7f2b300cf770 msgr2=0x7f2b300cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 -- 172.21.15.164:0/1993760106 shutdown_connections 2024-05-08T04:39:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.005+0000 7f2b37a00640 1 -- 172.21.15.164:0/1993760106 wait complete. 2024-05-08T04:39:56.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 Processor -- start 2024-05-08T04:39:56.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- start start 2024-05-08T04:39:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b300ae6e0 0x7f2b300adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b3015ab10 0x7f2b30158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300cefa0 con 0x7f2b300ce780 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30165ad0 con 0x7f2b300ae6e0 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30157e90 con 0x7f2b3015ab10 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41712/0 (socket says 172.21.15.164:41712) 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 -- 172.21.15.164:0/3736839879 learned_addr learned my addr 172.21.15.164:0/3736839879 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b35a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b3015ab10 0x7f2b30158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b35000640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b300ae6e0 0x7f2b300adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 -- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b3015ab10 msgr2=0x7f2b30158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b3015ab10 0x7f2b30158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 -- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b300ae6e0 msgr2=0x7f2b300adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b300ae6e0 0x7f2b300adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 -- 172.21.15.164:0/3736839879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30158170 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b27e00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800d800 tx=0x7f2b1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800b6c0 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18019070 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800ba60 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b30158450 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.009+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b300ae060 con 0x7f2b300ce780 2024-05-08T04:39:56.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30072d30 con 0x7f2b300ce780 2024-05-08T04:39:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f2b1800bc00 con 0x7f2b300ce780 2024-05-08T04:39:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b2d600640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 0x7f2b0c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b18085490 con 0x7f2b300ce780 2024-05-08T04:39:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b35000640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 0x7f2b0c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.013+0000 7f2b35000640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 0x7f2b0c0694b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b200ae000 tx=0x7f2b200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.017+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b18055df0 con 0x7f2b300ce780 2024-05-08T04:39:56.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.205+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b3004f010 con 0x7f2b300ce780 2024-05-08T04:39:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.805+0000 7f2b2d600640 1 -- 172.21.15.164:0/3736839879 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b18059e90 con 0x7f2b300ce780 2024-05-08T04:39:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.805+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 msgr2=0x7f2b0c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.805+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 0x7f2b0c0694b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b200ae000 tx=0x7f2b200a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.805+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 msgr2=0x7f2b300b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.805+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800d800 tx=0x7f2b1800dcc0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 shutdown_connections 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2b0c066ff0 0x7f2b0c0694b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b3015ab10 0x7f2b30158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b300ce780 0x7f2b300b24b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 --2- 172.21.15.164:0/3736839879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b300ae6e0 0x7f2b300adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 >> 172.21.15.164:0/3736839879 conn(0x7f2b300cf770 msgr2=0x7f2b300cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 shutdown_connections 2024-05-08T04:39:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.809+0000 7f2b37a00640 1 -- 172.21.15.164:0/3736839879 wait complete. 2024-05-08T04:39:56.833 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:39:56.833 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:39:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 Processor -- start 2024-05-08T04:39:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 -- start start 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c15ab60 0x7f8f7c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7c072640 con 0x7f8f7c0cfbd0 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7c0d0060 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f83a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c06c770 con 0x7f8f7c15ab60 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41718/0 (socket says 172.21.15.164:41718) 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 -- 172.21.15.164:0/2945630056 learned_addr learned my addr 172.21.15.164:0/2945630056 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f81000640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c15ab60 0x7f8f7c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 -- 172.21.15.164:0/2945630056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c15ab60 msgr2=0x7f8f7c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c15ab60 0x7f8f7c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 -- 172.21.15.164:0/2945630056 >> v1:172.21.15.172:6789/0 conn(0x7f8f7c0cfbd0 legacy=0x7f8f7c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.965+0000 7f8f7be00640 1 -- 172.21.15.164:0/2945630056 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c06ca50 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f7be00640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f6801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91761c967cc7a7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f7b400640 1 -- 172.21.15.164:0/2945630056 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f680b2070 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f7b400640 1 -- 172.21.15.164:0/2945630056 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f680a9070 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f7b400640 1 -- 172.21.15.164:0/2945630056 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f680ad880 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- 172.21.15.164:0/2945630056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 msgr2=0x7f8f7c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f6801c560 comp rx=0 tx=0).stop 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- 172.21.15.164:0/2945630056 shutdown_connections 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c0cf690 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 --2- 172.21.15.164:0/2945630056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c15ab60 0x7f8f7c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- 172.21.15.164:0/2945630056 >> 172.21.15.164:0/2945630056 conn(0x7f8f7c06ac90 msgr2=0x7f8f7c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- 172.21.15.164:0/2945630056 shutdown_connections 2024-05-08T04:39:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- 172.21.15.164:0/2945630056 wait complete. 2024-05-08T04:39:56.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 Processor -- start 2024-05-08T04:39:56.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.969+0000 7f8f83a00640 1 -- start start 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c0cfbd0 0x7f8f7c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 0x7f8f7c0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c06c550 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c06c3d0 con 0x7f8f7c15ab60 2024-05-08T04:39:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c0aed30 con 0x7f8f7c0cfbd0 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41728/0 (socket says 172.21.15.164:41728) 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 -- 172.21.15.164:0/567875865 learned_addr learned my addr 172.21.15.164:0/567875865 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 0x7f8f7c0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f7be00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c0cfbd0 0x7f8f7c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 -- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c0cfbd0 msgr2=0x7f8f7c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c0cfbd0 0x7f8f7c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 -- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 msgr2=0x7f8f7c0b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 0x7f8f7c0b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 -- 172.21.15.164:0/567875865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c142b90 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 0x7f8f7c0b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f81000640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d5f0 tx=0x7f8f7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700125f0 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70019070 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700129b0 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c156bf0 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c154c80 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c069d60 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.973+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f8f70012b50 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.977+0000 7f8f78c00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 0x7f8f5c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:56.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.977+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f700854c0 con 0x7f8f7c0b26c0 2024-05-08T04:39:56.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.977+0000 7f8f7be00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 0x7f8f5c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:56.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.977+0000 7f8f7be00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 0x7f8f5c0694e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f680af000 tx=0x7f8f68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:56.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:56.981+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f70011340 con 0x7f8f7c0b26c0 2024-05-08T04:39:57.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.165+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f7c14c110 con 0x7f8f7c0b26c0 2024-05-08T04:39:57.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f78c00640 1 -- 172.21.15.164:0/567875865 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f8f700563e0 con 0x7f8f7c0b26c0 2024-05-08T04:39:57.172 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:57.172 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:39:57.173 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 25 2024-05-08T04:39:57.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 msgr2=0x7f8f5c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 0x7f8f5c0694e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f680af000 tx=0x7f8f68008040 comp rx=0 tx=0).stop 2024-05-08T04:39:57.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 msgr2=0x7f8f7c142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d5f0 tx=0x7f8f7000dab0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 shutdown_connections 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f5c067020 0x7f8f5c0694e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f7c0cfbd0 0x7f8f7c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f7c15ab60 0x7f8f7c0b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 --2- 172.21.15.164:0/567875865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7c0b26c0 0x7f8f7c142650 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 >> 172.21.15.164:0/567875865 conn(0x7f8f7c06ac90 msgr2=0x7f8f7c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 shutdown_connections 2024-05-08T04:39:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.169+0000 7f8f83a00640 1 -- 172.21.15.164:0/567875865 wait complete. 2024-05-08T04:39:57.194 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:39:57.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 Processor -- start 2024-05-08T04:39:57.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 -- start start 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1df015ab60 0x7f1df0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df0072640 con 0x7f1df00cfda0 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df006c770 con 0x7f1df00b26c0 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.325+0000 7f1df4c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df006ca50 con 0x7f1df015ab60 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dee800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1df00cfda0 0x7f1df00cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34102/0 (socket says 172.21.15.164:34102) 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dee800640 1 -- 172.21.15.164:0/1010457679 learned_addr learned my addr 172.21.15.164:0/1010457679 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dede00640 1 --2- 172.21.15.164:0/1010457679 >> v2:172.21.15.172:3300/0 conn(0x7f1df015ab60 0x7f1df0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1ded400640 1 -- 172.21.15.164:0/1010457679 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585813233 0 0) 0x7f1df0072640 con 0x7f1df00cfda0 2024-05-08T04:39:57.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1ded400640 1 -- 172.21.15.164:0/1010457679 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd4003530 con 0x7f1df00cfda0 2024-05-08T04:39:57.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dede00640 1 -- 172.21.15.164:0/1010457679 >> v1:172.21.15.172:6789/0 conn(0x7f1df00b26c0 legacy=0x7f1df00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dede00640 1 -- 172.21.15.164:0/1010457679 >> v1:172.21.15.164:6789/0 conn(0x7f1df00cfda0 legacy=0x7f1df00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dede00640 1 -- 172.21.15.164:0/1010457679 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df006cd30 con 0x7f1df015ab60 2024-05-08T04:39:57.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1dede00640 1 --2- 172.21.15.164:0/1010457679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df015ab60 0x7f1df0072100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1de00097c0 tx=0x7f1de001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628280aec1942b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1ded400640 1 -- 172.21.15.164:0/1010457679 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00b0070 con 0x7f1df015ab60 2024-05-08T04:39:57.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1ded400640 1 -- 172.21.15.164:0/1010457679 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de00ab600 con 0x7f1df015ab60 2024-05-08T04:39:57.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.329+0000 7f1ded400640 1 -- 172.21.15.164:0/1010457679 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00ab9d0 con 0x7f1df015ab60 2024-05-08T04:39:57.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- 172.21.15.164:0/1010457679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df015ab60 msgr2=0x7f1df0072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1010457679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df015ab60 0x7f1df0072100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1de00097c0 tx=0x7f1de001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- 172.21.15.164:0/1010457679 shutdown_connections 2024-05-08T04:39:57.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1010457679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df015ab60 0x7f1df0072100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- 172.21.15.164:0/1010457679 >> 172.21.15.164:0/1010457679 conn(0x7f1df006ac90 msgr2=0x7f1df006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:57.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- 172.21.15.164:0/1010457679 shutdown_connections 2024-05-08T04:39:57.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- 172.21.15.164:0/1010457679 wait complete. 2024-05-08T04:39:57.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 Processor -- start 2024-05-08T04:39:57.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- start start 2024-05-08T04:39:57.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1df00b26c0 0x7f1df014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df00cfda0 0x7f1df01519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00aeeb0 con 0x7f1df015ab60 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00aed30 con 0x7f1df00cfda0 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1df4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0168980 con 0x7f1df00b26c0 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41734/0 (socket says 172.21.15.164:41734) 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 -- 172.21.15.164:0/1131221276 learned_addr learned my addr 172.21.15.164:0/1131221276 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dede00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1df00b26c0 0x7f1df014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1de5400640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df00cfda0 0x7f1df01519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 -- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1df00b26c0 msgr2=0x7f1df014c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1df00b26c0 0x7f1df014c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 -- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df00cfda0 msgr2=0x7f1df01519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df00cfda0 0x7f1df01519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.333+0000 7f1dee800640 1 -- 172.21.15.164:0/1131221276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df01522c0 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1dee800640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1de800cbb0 tx=0x7f1de800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8017070 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800ace0 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8012600 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df015c190 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df015c750 con 0x7f1df015ab60 2024-05-08T04:39:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df014cc50 con 0x7f1df015ab60 2024-05-08T04:39:57.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f1de801a070 con 0x7f1df015ab60 2024-05-08T04:39:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 0x7f1dcc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de8084da0 con 0x7f1df015ab60 2024-05-08T04:39:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.337+0000 7f1dede00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 0x7f1dcc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.341+0000 7f1dede00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 0x7f1dcc0694b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1de00ad000 tx=0x7f1de00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.341+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1df014cc50 con 0x7f1df015ab60 2024-05-08T04:39:57.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.501+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df0069d60 con 0x7f1df015ab60 2024-05-08T04:39:57.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.501+0000 7f1de6a00640 1 -- 172.21.15.164:0/1131221276 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f1de8055700 con 0x7f1df015ab60 2024-05-08T04:39:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:57.507 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":45,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:39:54.782264+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 msgr2=0x7f1dcc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 0x7f1dcc0694b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1de00ad000 tx=0x7f1de00a6000 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 msgr2=0x7f1df014ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1de800cbb0 tx=0x7f1de800a430 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 shutdown_connections 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1dcc066ff0 0x7f1dcc0694b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1df015ab60 0x7f1df014ec30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1df00cfda0 0x7f1df01519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 --2- 172.21.15.164:0/1131221276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1df00b26c0 0x7f1df014c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 >> 172.21.15.164:0/1131221276 conn(0x7f1df006ac90 msgr2=0x7f1df00afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:57.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 shutdown_connections 2024-05-08T04:39:57.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.505+0000 7f1df4c00640 1 -- 172.21.15.164:0/1131221276 wait complete. 2024-05-08T04:39:57.527 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:39:57.527 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:57.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 Processor -- start 2024-05-08T04:39:57.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 -- start start 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a680b4e60 0x7f3a680b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a68161640 0x7f3a68161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680ae280 con 0x7f3a6815d9d0 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680b57f0 con 0x7f3a680b4e60 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.657+0000 7f3a67e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680b3b80 con 0x7f3a68161640 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a680b4e60 0x7f3a680b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a680b4e60 0x7f3a680b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41738/0 (socket says 172.21.15.164:41738) 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 -- 172.21.15.164:0/4130592523 learned_addr learned my addr 172.21.15.164:0/4130592523 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a67400640 1 --2- 172.21.15.164:0/4130592523 >> v2:172.21.15.172:3300/0 conn(0x7f3a68161640 0x7f3a68161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 -- 172.21.15.164:0/4130592523 >> v2:172.21.15.172:3300/0 conn(0x7f3a68161640 msgr2=0x7f3a68161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 --2- 172.21.15.164:0/4130592523 >> v2:172.21.15.172:3300/0 conn(0x7f3a68161640 0x7f3a68161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 -- 172.21.15.164:0/4130592523 >> v1:172.21.15.172:6789/0 conn(0x7f3a6815d9d0 legacy=0x7f3a680ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 -- 172.21.15.164:0/4130592523 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680b3dd0 con 0x7f3a680b4e60 2024-05-08T04:39:57.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a5e000640 1 --2- 172.21.15.164:0/4130592523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a680b4e60 0x7f3a680b52b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a6806f5e0 tx=0x7f3a5001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e88c56731fe1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a66000640 1 -- 172.21.15.164:0/4130592523 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500ab660 con 0x7f3a680b4e60 2024-05-08T04:39:57.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a66000640 1 -- 172.21.15.164:0/4130592523 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500b0070 con 0x7f3a680b4e60 2024-05-08T04:39:57.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.661+0000 7f3a66000640 1 -- 172.21.15.164:0/4130592523 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500aba20 con 0x7f3a680b4e60 2024-05-08T04:39:57.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- 172.21.15.164:0/4130592523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a680b4e60 msgr2=0x7f3a680b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- 172.21.15.164:0/4130592523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a680b4e60 0x7f3a680b52b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a6806f5e0 tx=0x7f3a5001c460 comp rx=0 tx=0).stop 2024-05-08T04:39:57.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- 172.21.15.164:0/4130592523 shutdown_connections 2024-05-08T04:39:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- 172.21.15.164:0/4130592523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a680b4e60 0x7f3a680b52b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- 172.21.15.164:0/4130592523 >> v2:172.21.15.172:3300/0 conn(0x7f3a68161640 0x7f3a68161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- 172.21.15.164:0/4130592523 >> 172.21.15.164:0/4130592523 conn(0x7f3a680b2330 msgr2=0x7f3a680b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- 172.21.15.164:0/4130592523 shutdown_connections 2024-05-08T04:39:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- 172.21.15.164:0/4130592523 wait complete. 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 Processor -- start 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- start start 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a680b4e60 0x7f3a6813fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 0x7f3a6813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815ddb0 con 0x7f3a6815d9d0 2024-05-08T04:39:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680b48b0 con 0x7f3a680b4e60 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680b1640 con 0x7f3a68161640 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41748/0 (socket says 172.21.15.164:41748) 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 -- 172.21.15.164:0/1999294889 learned_addr learned my addr 172.21.15.164:0/1999294889 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a66a00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a680b4e60 0x7f3a6813fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67400640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 0x7f3a6813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 -- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 msgr2=0x7f3a6813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 0x7f3a6813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 -- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a680b4e60 msgr2=0x7f3a6813fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a680b4e60 0x7f3a6813fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a5e000640 1 -- 172.21.15.164:0/1999294889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6813fc10 con 0x7f3a6815d9d0 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.665+0000 7f3a67400640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 0x7f3a6813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5e000640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3a50019070 tx=0x7f3a5001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500be5f0 con 0x7f3a6815d9d0 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500be790 con 0x7f3a6815d9d0 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500beb80 con 0x7f3a6815d9d0 2024-05-08T04:39:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680b0d10 con 0x7f3a6815d9d0 2024-05-08T04:39:57.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680b12d0 con 0x7f3a6815d9d0 2024-05-08T04:39:57.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6804ef30 con 0x7f3a6815d9d0 2024-05-08T04:39:57.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f3a500b43f0 con 0x7f3a6815d9d0 2024-05-08T04:39:57.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 0x7f3a440694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:57.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a5011fc40 con 0x7f3a6815d9d0 2024-05-08T04:39:57.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.669+0000 7f3a66a00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 0x7f3a440694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:57.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.673+0000 7f3a66a00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 0x7f3a440694b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3a58001810 tx=0x7f3a58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:57.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.673+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a50016110 con 0x7f3a6815d9d0 2024-05-08T04:39:57.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.829+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a6814f5d0 con 0x7f3a44066ff0 2024-05-08T04:39:57.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.837+0000 7f3a5f400640 1 -- 172.21.15.164:0/1999294889 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239854 (secure 0 0 0) 0x7f3a6814f5d0 con 0x7f3a44066ff0 2024-05-08T04:39:57.845 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:57.847 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 msgr2=0x7f3a440694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 0x7f3a440694b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3a58001810 tx=0x7f3a58009040 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 msgr2=0x7f3a6813e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3a50019070 tx=0x7f3a5001aca0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 shutdown_connections 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a44066ff0 0x7f3a440694b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a68161640 0x7f3a6813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a6815d9d0 0x7f3a6813e090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 --2- 172.21.15.164:0/1999294889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a680b4e60 0x7f3a6813fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 >> 172.21.15.164:0/1999294889 conn(0x7f3a680b2330 msgr2=0x7f3a68071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:57.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 shutdown_connections 2024-05-08T04:39:57.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:57.841+0000 7f3a67e00640 1 -- 172.21.15.164:0/1999294889 wait complete. 2024-05-08T04:39:57.865 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-08T04:39:57.448475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225308,"kb_used_data":4976,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749412,"statfs":{"total":773094113280,"available":772863397888,"internally_reserved":0,"allocated":5095424,"data_stored":2735350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.685844"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770101+0000","last_change":"2024-05-08T04:39:51.770101+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770101+0000","last_undegraded":"2024-05-08T04:39:51.770101+0000","last_fullsized":"2024-05-08T04:39:51.770101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766509+0000","last_change":"2024-05-08T04:39:51.766509+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766509+0000","last_undegraded":"2024-05-08T04:39:51.766509+0000","last_fullsized":"2024-05-08T04:39:51.766509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768808+0000","last_change":"2024-05-08T04:39:51.768808+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768808+0000","last_undegraded":"2024-05-08T04:39:51.768808+0000","last_fullsized":"2024-05-08T04:39:51.768808+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771440+0000","last_change":"2024-05-08T04:39:51.771440+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771440+0000","last_undegraded":"2024-05-08T04:39:51.771440+0000","last_fullsized":"2024-05-08T04:39:51.771440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769699+0000","last_change":"2024-05-08T04:39:51.769699+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769699+0000","last_undegraded":"2024-05-08T04:39:51.769699+0000","last_fullsized":"2024-05-08T04:39:51.769699+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766525+0000","last_change":"2024-05-08T04:39:51.766525+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766525+0000","last_undegraded":"2024-05-08T04:39:51.766525+0000","last_fullsized":"2024-05-08T04:39:51.766525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.776627+0000","last_change":"2024-05-08T04:39:51.776627+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.776627+0000","last_undegraded":"2024-05-08T04:39:51.776627+0000","last_fullsized":"2024-05-08T04:39:51.776627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765314+0000","last_change":"2024-05-08T04:39:51.765314+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765314+0000","last_undegraded":"2024-05-08T04:39:51.765314+0000","last_fullsized":"2024-05-08T04:39:51.765314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775276+0000","last_change":"2024-05-08T04:39:51.775276+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775276+0000","last_undegraded":"2024-05-08T04:39:51.775276+0000","last_fullsized":"2024-05-08T04:39:51.775276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.772036+0000","last_change":"2024-05-08T04:39:51.772036+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.772036+0000","last_undegraded":"2024-05-08T04:39:51.772036+0000","last_fullsized":"2024-05-08T04:39:51.772036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770629+0000","last_change":"2024-05-08T04:39:51.770629+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770629+0000","last_undegraded":"2024-05-08T04:39:51.770629+0000","last_fullsized":"2024-05-08T04:39:51.770629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769202+0000","last_change":"2024-05-08T04:39:51.769202+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769202+0000","last_undegraded":"2024-05-08T04:39:51.769202+0000","last_fullsized":"2024-05-08T04:39:51.769202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771439+0000","last_change":"2024-05-08T04:39:51.771439+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771439+0000","last_undegraded":"2024-05-08T04:39:51.771439+0000","last_fullsized":"2024-05-08T04:39:51.771439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769227+0000","last_change":"2024-05-08T04:39:51.769227+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769227+0000","last_undegraded":"2024-05-08T04:39:51.769227+0000","last_fullsized":"2024-05-08T04:39:51.769227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775721+0000","last_change":"2024-05-08T04:39:51.775721+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775721+0000","last_undegraded":"2024-05-08T04:39:51.775721+0000","last_fullsized":"2024-05-08T04:39:51.775721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770130+0000","last_change":"2024-05-08T04:39:51.770130+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770130+0000","last_undegraded":"2024-05-08T04:39:51.770130+0000","last_fullsized":"2024-05-08T04:39:51.770130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765341+0000","last_change":"2024-05-08T04:39:51.765341+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765341+0000","last_undegraded":"2024-05-08T04:39:51.765341+0000","last_fullsized":"2024-05-08T04:39:51.765341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765391+0000","last_change":"2024-05-08T04:39:51.765391+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765391+0000","last_undegraded":"2024-05-08T04:39:51.765391+0000","last_fullsized":"2024-05-08T04:39:51.765391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768747+0000","last_change":"2024-05-08T04:39:51.768747+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768747+0000","last_undegraded":"2024-05-08T04:39:51.768747+0000","last_fullsized":"2024-05-08T04:39:51.768747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768712+0000","last_change":"2024-05-08T04:39:51.768712+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768712+0000","last_undegraded":"2024-05-08T04:39:51.768712+0000","last_fullsized":"2024-05-08T04:39:51.768712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775196+0000","last_change":"2024-05-08T04:39:51.775196+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775196+0000","last_undegraded":"2024-05-08T04:39:51.775196+0000","last_fullsized":"2024-05-08T04:39:51.775196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768659+0000","last_change":"2024-05-08T04:39:51.768659+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768659+0000","last_undegraded":"2024-05-08T04:39:51.768659+0000","last_fullsized":"2024-05-08T04:39:51.768659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768571+0000","last_change":"2024-05-08T04:39:51.768571+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768571+0000","last_undegraded":"2024-05-08T04:39:51.768571+0000","last_fullsized":"2024-05-08T04:39:51.768571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771403+0000","last_change":"2024-05-08T04:39:51.771403+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771403+0000","last_undegraded":"2024-05-08T04:39:51.771403+0000","last_fullsized":"2024-05-08T04:39:51.771403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765304+0000","last_change":"2024-05-08T04:39:51.765304+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765304+0000","last_undegraded":"2024-05-08T04:39:51.765304+0000","last_fullsized":"2024-05-08T04:39:51.765304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768489+0000","last_change":"2024-05-08T04:39:51.768489+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768489+0000","last_undegraded":"2024-05-08T04:39:51.768489+0000","last_fullsized":"2024-05-08T04:39:51.768489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775215+0000","last_change":"2024-05-08T04:39:51.775215+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775215+0000","last_undegraded":"2024-05-08T04:39:51.775215+0000","last_fullsized":"2024-05-08T04:39:51.775215+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765755+0000","last_change":"2024-05-08T04:39:51.765755+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765755+0000","last_undegraded":"2024-05-08T04:39:51.765755+0000","last_fullsized":"2024-05-08T04:39:51.765755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T04:39:51.772482+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:51.772482+0000","last_peered":"2024-05-08T04:39:51.772482+0000","last_clean":"2024-05-08T04:39:51.772482+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:51.772482+0000","last_undegraded":"2024-05-08T04:39:51.772482+0000","last_fullsized":"2024-05-08T04:39:51.772482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:39:57.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:57.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:39:57.963 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:39:58.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.041+0000 7f34f4200640 1 Processor -- start 2024-05-08T04:39:58.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.041+0000 7f34f4200640 1 -- start start 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f4200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34ec144200 0x7f34ec161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f4200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec161fe0 con 0x7f34ec15ea20 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f4200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec1622c0 con 0x7f34ec0cfd60 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f4200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec162880 con 0x7f34ec144200 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f1800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34ec144200 0x7f34ec161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f2200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f34ec15ea20 0x7f34ec0ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34108/0 (socket says 172.21.15.164:34108) 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f2200640 1 -- 172.21.15.164:0/2933922432 learned_addr learned my addr 172.21.15.164:0/2933922432 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:58.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f1800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34ec144200 0x7f34ec161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41750/0 (socket says 172.21.15.164:41750) 2024-05-08T04:39:58.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f1800640 1 -- 172.21.15.164:0/2933922432 >> v1:172.21.15.172:6789/0 conn(0x7f34ec0cfd60 legacy=0x7f34ec0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f1800640 1 -- 172.21.15.164:0/2933922432 >> v1:172.21.15.164:6789/0 conn(0x7f34ec15ea20 legacy=0x7f34ec0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34f1800640 1 -- 172.21.15.164:0/2933922432 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec1625a0 con 0x7f34ec144200 2024-05-08T04:39:58.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.045+0000 7f34ebe00640 1 -- 172.21.15.164:0/2933922432 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668159191 0 0) 0x7f34ec161fe0 con 0x7f34ec15ea20 2024-05-08T04:39:58.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f1800640 1 --2- 172.21.15.164:0/2933922432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec144200 0x7f34ec161950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097c0 tx=0x7f34e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabeb2dbb0a96667 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34ebe00640 1 -- 172.21.15.164:0/2933922432 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00b0070 con 0x7f34ec144200 2024-05-08T04:39:58.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34ebe00640 1 -- 172.21.15.164:0/2933922432 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00ab600 con 0x7f34ec144200 2024-05-08T04:39:58.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34ebe00640 1 -- 172.21.15.164:0/2933922432 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ab9d0 con 0x7f34ec144200 2024-05-08T04:39:58.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- 172.21.15.164:0/2933922432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec144200 msgr2=0x7f34ec161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 --2- 172.21.15.164:0/2933922432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec144200 0x7f34ec161950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097c0 tx=0x7f34e001c410 comp rx=0 tx=0).stop 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- 172.21.15.164:0/2933922432 shutdown_connections 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 --2- 172.21.15.164:0/2933922432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec144200 0x7f34ec161950 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- 172.21.15.164:0/2933922432 >> 172.21.15.164:0/2933922432 conn(0x7f34ec06ac90 msgr2=0x7f34ec06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- 172.21.15.164:0/2933922432 shutdown_connections 2024-05-08T04:39:58.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- 172.21.15.164:0/2933922432 wait complete. 2024-05-08T04:39:58.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 Processor -- start 2024-05-08T04:39:58.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- start start 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 0x7f34ec1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 0x7f34ec15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0cc7c0 con 0x7f34ec0cfd60 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0d0190 con 0x7f34ec15ea20 2024-05-08T04:39:58.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.049+0000 7f34f4200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0cc940 con 0x7f34ec144200 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41758/0 (socket says 172.21.15.164:41758) 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 -- 172.21.15.164:0/986407101 learned_addr learned my addr 172.21.15.164:0/986407101 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f2200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 0x7f34ec15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f0e00640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 0x7f34ec1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 -- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 msgr2=0x7f34ec1576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 0x7f34ec1576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 -- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 msgr2=0x7f34ec15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 0x7f34ec15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 -- 172.21.15.164:0/986407101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec15b990 con 0x7f34ec0cfd60 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f0e00640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 0x7f34ec1576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:58.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f2200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 0x7f34ec15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f1800640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ae040 tx=0x7f34e001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00b0050 con 0x7f34ec0cfd60 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00bc070 con 0x7f34ec0cfd60 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00b5a50 con 0x7f34ec0cfd60 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec1fa7f0 con 0x7f34ec0cfd60 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec1fadb0 con 0x7f34ec0cfd60 2024-05-08T04:39:58.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec069d60 con 0x7f34ec0cfd60 2024-05-08T04:39:58.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.053+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f34e00ab4a0 con 0x7f34ec0cfd60 2024-05-08T04:39:58.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.057+0000 7f34e9600640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 0x7f34cc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.057+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34e011fa80 con 0x7f34ec0cfd60 2024-05-08T04:39:58.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.057+0000 7f34f0e00640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 0x7f34cc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.057+0000 7f34f0e00640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 0x7f34cc0694b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34ec1608e0 tx=0x7f34d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.057+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34e0016090 con 0x7f34ec0cfd60 2024-05-08T04:39:58.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.209+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34ec161b10 con 0x7f34cc066ff0 2024-05-08T04:39:58.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.217+0000 7f34e9600640 1 -- 172.21.15.164:0/986407101 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239854 (secure 0 0 0) 0x7f34ec161b10 con 0x7f34cc066ff0 2024-05-08T04:39:58.224 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:58.226 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 msgr2=0x7f34cc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 0x7f34cc0694b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34ec1608e0 tx=0x7f34d8009040 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 msgr2=0x7f34ec15a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ae040 tx=0x7f34e001ac50 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 shutdown_connections 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f34cc066ff0 0x7f34cc0694b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34ec144200 0x7f34ec1576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34ec15ea20 0x7f34ec15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 --2- 172.21.15.164:0/986407101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34ec0cfd60 0x7f34ec15a9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 >> 172.21.15.164:0/986407101 conn(0x7f34ec06ac90 msgr2=0x7f34ec0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 shutdown_connections 2024-05-08T04:39:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.221+0000 7f34f4200640 1 -- 172.21.15.164:0/986407101 wait complete. 2024-05-08T04:39:58.245 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-08T04:39:57.448475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225308,"kb_used_data":4976,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749412,"statfs":{"total":773094113280,"available":772863397888,"internally_reserved":0,"allocated":5095424,"data_stored":2735350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.685844"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770101+0000","last_change":"2024-05-08T04:39:51.770101+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770101+0000","last_undegraded":"2024-05-08T04:39:51.770101+0000","last_fullsized":"2024-05-08T04:39:51.770101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766509+0000","last_change":"2024-05-08T04:39:51.766509+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766509+0000","last_undegraded":"2024-05-08T04:39:51.766509+0000","last_fullsized":"2024-05-08T04:39:51.766509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768808+0000","last_change":"2024-05-08T04:39:51.768808+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768808+0000","last_undegraded":"2024-05-08T04:39:51.768808+0000","last_fullsized":"2024-05-08T04:39:51.768808+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771440+0000","last_change":"2024-05-08T04:39:51.771440+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771440+0000","last_undegraded":"2024-05-08T04:39:51.771440+0000","last_fullsized":"2024-05-08T04:39:51.771440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769699+0000","last_change":"2024-05-08T04:39:51.769699+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769699+0000","last_undegraded":"2024-05-08T04:39:51.769699+0000","last_fullsized":"2024-05-08T04:39:51.769699+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766525+0000","last_change":"2024-05-08T04:39:51.766525+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766525+0000","last_undegraded":"2024-05-08T04:39:51.766525+0000","last_fullsized":"2024-05-08T04:39:51.766525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.776627+0000","last_change":"2024-05-08T04:39:51.776627+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.776627+0000","last_undegraded":"2024-05-08T04:39:51.776627+0000","last_fullsized":"2024-05-08T04:39:51.776627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765314+0000","last_change":"2024-05-08T04:39:51.765314+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765314+0000","last_undegraded":"2024-05-08T04:39:51.765314+0000","last_fullsized":"2024-05-08T04:39:51.765314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775276+0000","last_change":"2024-05-08T04:39:51.775276+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775276+0000","last_undegraded":"2024-05-08T04:39:51.775276+0000","last_fullsized":"2024-05-08T04:39:51.775276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.772036+0000","last_change":"2024-05-08T04:39:51.772036+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.772036+0000","last_undegraded":"2024-05-08T04:39:51.772036+0000","last_fullsized":"2024-05-08T04:39:51.772036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770629+0000","last_change":"2024-05-08T04:39:51.770629+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770629+0000","last_undegraded":"2024-05-08T04:39:51.770629+0000","last_fullsized":"2024-05-08T04:39:51.770629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769202+0000","last_change":"2024-05-08T04:39:51.769202+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769202+0000","last_undegraded":"2024-05-08T04:39:51.769202+0000","last_fullsized":"2024-05-08T04:39:51.769202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771439+0000","last_change":"2024-05-08T04:39:51.771439+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771439+0000","last_undegraded":"2024-05-08T04:39:51.771439+0000","last_fullsized":"2024-05-08T04:39:51.771439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769227+0000","last_change":"2024-05-08T04:39:51.769227+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769227+0000","last_undegraded":"2024-05-08T04:39:51.769227+0000","last_fullsized":"2024-05-08T04:39:51.769227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775721+0000","last_change":"2024-05-08T04:39:51.775721+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775721+0000","last_undegraded":"2024-05-08T04:39:51.775721+0000","last_fullsized":"2024-05-08T04:39:51.775721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770130+0000","last_change":"2024-05-08T04:39:51.770130+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770130+0000","last_undegraded":"2024-05-08T04:39:51.770130+0000","last_fullsized":"2024-05-08T04:39:51.770130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765341+0000","last_change":"2024-05-08T04:39:51.765341+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765341+0000","last_undegraded":"2024-05-08T04:39:51.765341+0000","last_fullsized":"2024-05-08T04:39:51.765341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765391+0000","last_change":"2024-05-08T04:39:51.765391+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765391+0000","last_undegraded":"2024-05-08T04:39:51.765391+0000","last_fullsized":"2024-05-08T04:39:51.765391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768747+0000","last_change":"2024-05-08T04:39:51.768747+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768747+0000","last_undegraded":"2024-05-08T04:39:51.768747+0000","last_fullsized":"2024-05-08T04:39:51.768747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768712+0000","last_change":"2024-05-08T04:39:51.768712+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768712+0000","last_undegraded":"2024-05-08T04:39:51.768712+0000","last_fullsized":"2024-05-08T04:39:51.768712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775196+0000","last_change":"2024-05-08T04:39:51.775196+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775196+0000","last_undegraded":"2024-05-08T04:39:51.775196+0000","last_fullsized":"2024-05-08T04:39:51.775196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768659+0000","last_change":"2024-05-08T04:39:51.768659+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768659+0000","last_undegraded":"2024-05-08T04:39:51.768659+0000","last_fullsized":"2024-05-08T04:39:51.768659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768571+0000","last_change":"2024-05-08T04:39:51.768571+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768571+0000","last_undegraded":"2024-05-08T04:39:51.768571+0000","last_fullsized":"2024-05-08T04:39:51.768571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771403+0000","last_change":"2024-05-08T04:39:51.771403+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771403+0000","last_undegraded":"2024-05-08T04:39:51.771403+0000","last_fullsized":"2024-05-08T04:39:51.771403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765304+0000","last_change":"2024-05-08T04:39:51.765304+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765304+0000","last_undegraded":"2024-05-08T04:39:51.765304+0000","last_fullsized":"2024-05-08T04:39:51.765304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768489+0000","last_change":"2024-05-08T04:39:51.768489+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768489+0000","last_undegraded":"2024-05-08T04:39:51.768489+0000","last_fullsized":"2024-05-08T04:39:51.768489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775215+0000","last_change":"2024-05-08T04:39:51.775215+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775215+0000","last_undegraded":"2024-05-08T04:39:51.775215+0000","last_fullsized":"2024-05-08T04:39:51.775215+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765755+0000","last_change":"2024-05-08T04:39:51.765755+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765755+0000","last_undegraded":"2024-05-08T04:39:51.765755+0000","last_fullsized":"2024-05-08T04:39:51.765755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T04:39:51.772482+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:51.772482+0000","last_peered":"2024-05-08T04:39:51.772482+0000","last_clean":"2024-05-08T04:39:51.772482+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:51.772482+0000","last_undegraded":"2024-05-08T04:39:51.772482+0000","last_fullsized":"2024-05-08T04:39:51.772482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:39:58.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:58.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 Processor -- start 2024-05-08T04:39:58.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 -- start start 2024-05-08T04:39:58.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b9006c710 0x7f1b9006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9015b8c0 con 0x7f1b900708c0 2024-05-08T04:39:58.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9006dbb0 con 0x7f1b9006d0e0 2024-05-08T04:39:58.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8f200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9015a8b0 con 0x7f1b9006c710 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b9006c710 0x7f1b9006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1b900708c0 0x7f1b90070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34120/0 (socket says 172.21.15.164:34120) 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8de00640 1 -- 172.21.15.164:0/3052014388 learned_addr learned my addr 172.21.15.164:0/3052014388 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8d400640 1 -- 172.21.15.164:0/3052014388 >> v1:172.21.15.172:6789/0 conn(0x7f1b9006d0e0 legacy=0x7f1b90164e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8d400640 1 -- 172.21.15.164:0/3052014388 >> v1:172.21.15.164:6789/0 conn(0x7f1b900708c0 legacy=0x7f1b90070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8d400640 1 -- 172.21.15.164:0/3052014388 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9006de90 con 0x7f1b9006c710 2024-05-08T04:39:58.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.425+0000 7f1b8ca00640 1 -- 172.21.15.164:0/3052014388 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258249935 0 0) 0x7f1b9015b8c0 con 0x7f1b900708c0 2024-05-08T04:39:58.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8d400640 1 --2- 172.21.15.164:0/3052014388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9006c710 0x7f1b9006cba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0097c0 tx=0x7f1b7c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f30ea3ccc02b758 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8ca00640 1 -- 172.21.15.164:0/3052014388 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0b1070 con 0x7f1b9006c710 2024-05-08T04:39:58.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8ca00640 1 -- 172.21.15.164:0/3052014388 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0ac620 con 0x7f1b9006c710 2024-05-08T04:39:58.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8ca00640 1 -- 172.21.15.164:0/3052014388 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0ac9f0 con 0x7f1b9006c710 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- 172.21.15.164:0/3052014388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9006c710 msgr2=0x7f1b9006cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3052014388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9006c710 0x7f1b9006cba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0097c0 tx=0x7f1b7c01c3a0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- 172.21.15.164:0/3052014388 shutdown_connections 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3052014388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9006c710 0x7f1b9006cba0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- 172.21.15.164:0/3052014388 >> 172.21.15.164:0/3052014388 conn(0x7f1b900cfaf0 msgr2=0x7f1b900cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:58.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- 172.21.15.164:0/3052014388 shutdown_connections 2024-05-08T04:39:58.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- 172.21.15.164:0/3052014388 wait complete. 2024-05-08T04:39:58.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 Processor -- start 2024-05-08T04:39:58.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.429+0000 7f1b8f200640 1 -- start start 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9006c710 0x7f1b90159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 0x7f1b9013d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90143ba0 con 0x7f1b900708c0 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90143a20 con 0x7f1b9006d0e0 2024-05-08T04:39:58.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9015ceb0 con 0x7f1b9006c710 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41778/0 (socket says 172.21.15.164:41778) 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 -- 172.21.15.164:0/3649390545 learned_addr learned my addr 172.21.15.164:0/3649390545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8de00640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9006c710 0x7f1b90159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8d400640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 0x7f1b9013d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 -- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9006c710 msgr2=0x7f1b90159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9006c710 0x7f1b90159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 -- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 msgr2=0x7f1b9013d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 0x7f1b9013d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 -- 172.21.15.164:0/3649390545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90159b80 con 0x7f1b900708c0 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8d400640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 0x7f1b9013d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:39:58.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8e800640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c780 tx=0x7f1b8800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88004b70 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88004d10 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800aa90 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9013dc00 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900cc0e0 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.433+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b900cbef0 con 0x7f1b900708c0 2024-05-08T04:39:58.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.437+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f1b8800ac30 con 0x7f1b900708c0 2024-05-08T04:39:58.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.437+0000 7f1b86000640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 0x7f1b700694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.437+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b88085340 con 0x7f1b900708c0 2024-05-08T04:39:58.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.437+0000 7f1b8de00640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 0x7f1b700694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.437+0000 7f1b8de00640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 0x7f1b700694b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f70 tx=0x7f1b80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.441+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b88020070 con 0x7f1b900708c0 2024-05-08T04:39:58.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.589+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b900ced10 con 0x7f1b70066ff0 2024-05-08T04:39:58.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.593+0000 7f1b86000640 1 -- 172.21.15.164:0/3649390545 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239854 (secure 0 0 0) 0x7f1b900ced10 con 0x7f1b70066ff0 2024-05-08T04:39:58.602 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:58.603 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:58.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 msgr2=0x7f1b700694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 0x7f1b700694b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f70 tx=0x7f1b80009040 comp rx=0 tx=0).stop 2024-05-08T04:39:58.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 msgr2=0x7f1b9013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c780 tx=0x7f1b8800cc40 comp rx=0 tx=0).stop 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 shutdown_connections 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b70066ff0 0x7f1b700694b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b900708c0 0x7f1b9013d580 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9006d0e0 0x7f1b9013d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 --2- 172.21.15.164:0/3649390545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9006c710 0x7f1b90159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 >> 172.21.15.164:0/3649390545 conn(0x7f1b900cfaf0 msgr2=0x7f1b9015bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 shutdown_connections 2024-05-08T04:39:58.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.597+0000 7f1b8f200640 1 -- 172.21.15.164:0/3649390545 wait complete. 2024-05-08T04:39:58.623 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-08T04:39:57.448475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225308,"kb_used_data":4976,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749412,"statfs":{"total":773094113280,"available":772863397888,"internally_reserved":0,"allocated":5095424,"data_stored":2735350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.685844"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770101+0000","last_change":"2024-05-08T04:39:51.770101+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770101+0000","last_undegraded":"2024-05-08T04:39:51.770101+0000","last_fullsized":"2024-05-08T04:39:51.770101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766509+0000","last_change":"2024-05-08T04:39:51.766509+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766509+0000","last_undegraded":"2024-05-08T04:39:51.766509+0000","last_fullsized":"2024-05-08T04:39:51.766509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768808+0000","last_change":"2024-05-08T04:39:51.768808+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768808+0000","last_undegraded":"2024-05-08T04:39:51.768808+0000","last_fullsized":"2024-05-08T04:39:51.768808+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771440+0000","last_change":"2024-05-08T04:39:51.771440+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771440+0000","last_undegraded":"2024-05-08T04:39:51.771440+0000","last_fullsized":"2024-05-08T04:39:51.771440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769699+0000","last_change":"2024-05-08T04:39:51.769699+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769699+0000","last_undegraded":"2024-05-08T04:39:51.769699+0000","last_fullsized":"2024-05-08T04:39:51.769699+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766525+0000","last_change":"2024-05-08T04:39:51.766525+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766525+0000","last_undegraded":"2024-05-08T04:39:51.766525+0000","last_fullsized":"2024-05-08T04:39:51.766525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.776627+0000","last_change":"2024-05-08T04:39:51.776627+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.776627+0000","last_undegraded":"2024-05-08T04:39:51.776627+0000","last_fullsized":"2024-05-08T04:39:51.776627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765314+0000","last_change":"2024-05-08T04:39:51.765314+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765314+0000","last_undegraded":"2024-05-08T04:39:51.765314+0000","last_fullsized":"2024-05-08T04:39:51.765314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775276+0000","last_change":"2024-05-08T04:39:51.775276+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775276+0000","last_undegraded":"2024-05-08T04:39:51.775276+0000","last_fullsized":"2024-05-08T04:39:51.775276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.772036+0000","last_change":"2024-05-08T04:39:51.772036+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.772036+0000","last_undegraded":"2024-05-08T04:39:51.772036+0000","last_fullsized":"2024-05-08T04:39:51.772036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770629+0000","last_change":"2024-05-08T04:39:51.770629+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770629+0000","last_undegraded":"2024-05-08T04:39:51.770629+0000","last_fullsized":"2024-05-08T04:39:51.770629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769202+0000","last_change":"2024-05-08T04:39:51.769202+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769202+0000","last_undegraded":"2024-05-08T04:39:51.769202+0000","last_fullsized":"2024-05-08T04:39:51.769202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771439+0000","last_change":"2024-05-08T04:39:51.771439+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771439+0000","last_undegraded":"2024-05-08T04:39:51.771439+0000","last_fullsized":"2024-05-08T04:39:51.771439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769227+0000","last_change":"2024-05-08T04:39:51.769227+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769227+0000","last_undegraded":"2024-05-08T04:39:51.769227+0000","last_fullsized":"2024-05-08T04:39:51.769227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775721+0000","last_change":"2024-05-08T04:39:51.775721+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775721+0000","last_undegraded":"2024-05-08T04:39:51.775721+0000","last_fullsized":"2024-05-08T04:39:51.775721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770130+0000","last_change":"2024-05-08T04:39:51.770130+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770130+0000","last_undegraded":"2024-05-08T04:39:51.770130+0000","last_fullsized":"2024-05-08T04:39:51.770130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765341+0000","last_change":"2024-05-08T04:39:51.765341+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765341+0000","last_undegraded":"2024-05-08T04:39:51.765341+0000","last_fullsized":"2024-05-08T04:39:51.765341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765391+0000","last_change":"2024-05-08T04:39:51.765391+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765391+0000","last_undegraded":"2024-05-08T04:39:51.765391+0000","last_fullsized":"2024-05-08T04:39:51.765391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768747+0000","last_change":"2024-05-08T04:39:51.768747+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768747+0000","last_undegraded":"2024-05-08T04:39:51.768747+0000","last_fullsized":"2024-05-08T04:39:51.768747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768712+0000","last_change":"2024-05-08T04:39:51.768712+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768712+0000","last_undegraded":"2024-05-08T04:39:51.768712+0000","last_fullsized":"2024-05-08T04:39:51.768712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775196+0000","last_change":"2024-05-08T04:39:51.775196+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775196+0000","last_undegraded":"2024-05-08T04:39:51.775196+0000","last_fullsized":"2024-05-08T04:39:51.775196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768659+0000","last_change":"2024-05-08T04:39:51.768659+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768659+0000","last_undegraded":"2024-05-08T04:39:51.768659+0000","last_fullsized":"2024-05-08T04:39:51.768659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768571+0000","last_change":"2024-05-08T04:39:51.768571+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768571+0000","last_undegraded":"2024-05-08T04:39:51.768571+0000","last_fullsized":"2024-05-08T04:39:51.768571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771403+0000","last_change":"2024-05-08T04:39:51.771403+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771403+0000","last_undegraded":"2024-05-08T04:39:51.771403+0000","last_fullsized":"2024-05-08T04:39:51.771403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765304+0000","last_change":"2024-05-08T04:39:51.765304+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765304+0000","last_undegraded":"2024-05-08T04:39:51.765304+0000","last_fullsized":"2024-05-08T04:39:51.765304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768489+0000","last_change":"2024-05-08T04:39:51.768489+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768489+0000","last_undegraded":"2024-05-08T04:39:51.768489+0000","last_fullsized":"2024-05-08T04:39:51.768489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775215+0000","last_change":"2024-05-08T04:39:51.775215+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775215+0000","last_undegraded":"2024-05-08T04:39:51.775215+0000","last_fullsized":"2024-05-08T04:39:51.775215+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765755+0000","last_change":"2024-05-08T04:39:51.765755+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765755+0000","last_undegraded":"2024-05-08T04:39:51.765755+0000","last_fullsized":"2024-05-08T04:39:51.765755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T04:39:51.772482+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:51.772482+0000","last_peered":"2024-05-08T04:39:51.772482+0000","last_clean":"2024-05-08T04:39:51.772482+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:51.772482+0000","last_undegraded":"2024-05-08T04:39:51.772482+0000","last_fullsized":"2024-05-08T04:39:51.772482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:39:58.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-05-08T04:39:58.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-05-08T04:39:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-05-08T04:39:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-05-08T04:39:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-05-08T04:39:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.770101+0000', 'last_change': '2024-05-08T04:39:51.770101+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.770101+0000', 'last_undegraded': '2024-05-08T04:39:51.770101+0000', 'last_fullsized': '2024-05-08T04:39:51.770101+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.766509+0000', 'last_change': '2024-05-08T04:39:51.766509+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.766509+0000', 'last_undegraded': '2024-05-08T04:39:51.766509+0000', 'last_fullsized': '2024-05-08T04:39:51.766509+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768808+0000', 'last_change': '2024-05-08T04:39:51.768808+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768808+0000', 'last_undegraded': '2024-05-08T04:39:51.768808+0000', 'last_fullsized': '2024-05-08T04:39:51.768808+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.771440+0000', 'last_change': '2024-05-08T04:39:51.771440+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.771440+0000', 'last_undegraded': '2024-05-08T04:39:51.771440+0000', 'last_fullsized': '2024-05-08T04:39:51.771440+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-05-08T04:39:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.769699+0000', 'last_change': '2024-05-08T04:39:51.769699+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.769699+0000', 'last_undegraded': '2024-05-08T04:39:51.769699+0000', 'last_fullsized': '2024-05-08T04:39:51.769699+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.766525+0000', 'last_change': '2024-05-08T04:39:51.766525+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.766525+0000', 'last_undegraded': '2024-05-08T04:39:51.766525+0000', 'last_fullsized': '2024-05-08T04:39:51.766525+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765398+0000', 'last_change': '2024-05-08T04:39:51.765398+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765398+0000', 'last_undegraded': '2024-05-08T04:39:51.765398+0000', 'last_fullsized': '2024-05-08T04:39:51.765398+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-05-08T04:39:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.776627+0000', 'last_change': '2024-05-08T04:39:51.776627+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.776627+0000', 'last_undegraded': '2024-05-08T04:39:51.776627+0000', 'last_fullsized': '2024-05-08T04:39:51.776627+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765314+0000', 'last_change': '2024-05-08T04:39:51.765314+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765314+0000', 'last_undegraded': '2024-05-08T04:39:51.765314+0000', 'last_fullsized': '2024-05-08T04:39:51.765314+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-05-08T04:39:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.775276+0000', 'last_change': '2024-05-08T04:39:51.775276+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.775276+0000', 'last_undegraded': '2024-05-08T04:39:51.775276+0000', 'last_fullsized': '2024-05-08T04:39:51.775276+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.772036+0000', 'last_change': '2024-05-08T04:39:51.772036+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.772036+0000', 'last_undegraded': '2024-05-08T04:39:51.772036+0000', 'last_fullsized': '2024-05-08T04:39:51.772036+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.770629+0000', 'last_change': '2024-05-08T04:39:51.770629+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.770629+0000', 'last_undegraded': '2024-05-08T04:39:51.770629+0000', 'last_fullsized': '2024-05-08T04:39:51.770629+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.769202+0000', 'last_change': '2024-05-08T04:39:51.769202+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.769202+0000', 'last_undegraded': '2024-05-08T04:39:51.769202+0000', 'last_fullsized': '2024-05-08T04:39:51.769202+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.771439+0000', 'last_change': '2024-05-08T04:39:51.771439+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.771439+0000', 'last_undegraded': '2024-05-08T04:39:51.771439+0000', 'last_fullsized': '2024-05-08T04:39:51.771439+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.769227+0000', 'last_change': '2024-05-08T04:39:51.769227+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.769227+0000', 'last_undegraded': '2024-05-08T04:39:51.769227+0000', 'last_fullsized': '2024-05-08T04:39:51.769227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-05-08T04:39:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.775721+0000', 'last_change': '2024-05-08T04:39:51.775721+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.775721+0000', 'last_undegraded': '2024-05-08T04:39:51.775721+0000', 'last_fullsized': '2024-05-08T04:39:51.775721+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.770130+0000', 'last_change': '2024-05-08T04:39:51.770130+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.770130+0000', 'last_undegraded': '2024-05-08T04:39:51.770130+0000', 'last_fullsized': '2024-05-08T04:39:51.770130+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765341+0000', 'last_change': '2024-05-08T04:39:51.765341+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765341+0000', 'last_undegraded': '2024-05-08T04:39:51.765341+0000', 'last_fullsized': '2024-05-08T04:39:51.765341+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:39:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765391+0000', 'last_change': '2024-05-08T04:39:51.765391+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765391+0000', 'last_undegraded': '2024-05-08T04:39:51.765391+0000', 'last_fullsized': '2024-05-08T04:39:51.765391+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768747+0000', 'last_change': '2024-05-08T04:39:51.768747+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768747+0000', 'last_undegraded': '2024-05-08T04:39:51.768747+0000', 'last_fullsized': '2024-05-08T04:39:51.768747+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-05-08T04:39:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768712+0000', 'last_change': '2024-05-08T04:39:51.768712+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768712+0000', 'last_undegraded': '2024-05-08T04:39:51.768712+0000', 'last_fullsized': '2024-05-08T04:39:51.768712+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765398+0000', 'last_change': '2024-05-08T04:39:51.765398+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765398+0000', 'last_undegraded': '2024-05-08T04:39:51.765398+0000', 'last_fullsized': '2024-05-08T04:39:51.765398+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.775196+0000', 'last_change': '2024-05-08T04:39:51.775196+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.775196+0000', 'last_undegraded': '2024-05-08T04:39:51.775196+0000', 'last_fullsized': '2024-05-08T04:39:51.775196+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.771445+0000', 'last_change': '2024-05-08T04:39:51.771445+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.771445+0000', 'last_undegraded': '2024-05-08T04:39:51.771445+0000', 'last_fullsized': '2024-05-08T04:39:51.771445+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768659+0000', 'last_change': '2024-05-08T04:39:51.768659+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768659+0000', 'last_undegraded': '2024-05-08T04:39:51.768659+0000', 'last_fullsized': '2024-05-08T04:39:51.768659+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.771445+0000', 'last_change': '2024-05-08T04:39:51.771445+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.771445+0000', 'last_undegraded': '2024-05-08T04:39:51.771445+0000', 'last_fullsized': '2024-05-08T04:39:51.771445+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-05-08T04:39:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768571+0000', 'last_change': '2024-05-08T04:39:51.768571+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768571+0000', 'last_undegraded': '2024-05-08T04:39:51.768571+0000', 'last_fullsized': '2024-05-08T04:39:51.768571+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.771403+0000', 'last_change': '2024-05-08T04:39:51.771403+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.771403+0000', 'last_undegraded': '2024-05-08T04:39:51.771403+0000', 'last_fullsized': '2024-05-08T04:39:51.771403+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-05-08T04:39:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765304+0000', 'last_change': '2024-05-08T04:39:51.765304+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765304+0000', 'last_undegraded': '2024-05-08T04:39:51.765304+0000', 'last_fullsized': '2024-05-08T04:39:51.765304+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.768489+0000', 'last_change': '2024-05-08T04:39:51.768489+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.768489+0000', 'last_undegraded': '2024-05-08T04:39:51.768489+0000', 'last_fullsized': '2024-05-08T04:39:51.768489+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.775215+0000', 'last_change': '2024-05-08T04:39:51.775215+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.775215+0000', 'last_undegraded': '2024-05-08T04:39:51.775215+0000', 'last_fullsized': '2024-05-08T04:39:51.775215+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-05-08T04:39:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:39:52.761450+0000', 'last_change': '2024-05-08T04:39:52.761450+0000', 'last_active': '2024-05-08T04:39:52.761450+0000', 'last_peered': '2024-05-08T04:39:52.761450+0000', 'last_clean': '2024-05-08T04:39:52.761450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:52.761450+0000', 'last_undegraded': '2024-05-08T04:39:52.761450+0000', 'last_fullsized': '2024-05-08T04:39:52.761450+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:52.761450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:39:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-05-08T04:39:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:39:51.765755+0000', 'last_change': '2024-05-08T04:39:51.765755+0000', 'last_active': '2024-05-08T04:39:51.752781+0000', 'last_peered': '2024-05-08T04:39:51.752781+0000', 'last_clean': '2024-05-08T04:39:51.752781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:39:51.765755+0000', 'last_undegraded': '2024-05-08T04:39:51.765755+0000', 'last_fullsized': '2024-05-08T04:39:51.765755+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'last_clean_scrub_stamp': '2024-05-08T04:39:51.752781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:39:58.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T04:39:58.754 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-05-08T04:39:58.749+0000 7fe970400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:39:58.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 Processor -- start 2024-05-08T04:39:58.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 -- start start 2024-05-08T04:39:58.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09f415bc50 0x7f09f40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40b42a0 con 0x7f09f40d0780 2024-05-08T04:39:58.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40cd4e0 con 0x7f09f40ccfe0 2024-05-08T04:39:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.797+0000 7f09fa000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f40cd7c0 con 0x7f09f415bc50 2024-05-08T04:39:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f09f40d0780 0x7f09f415f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:34130/0 (socket says 172.21.15.164:34130) 2024-05-08T04:39:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3e00640 1 -- 172.21.15.164:0/2007387684 learned_addr learned my addr 172.21.15.164:0/2007387684 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:39:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3400640 1 --2- 172.21.15.164:0/2007387684 >> v2:172.21.15.172:3300/0 conn(0x7f09f415bc50 0x7f09f40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191773761 0 0) 0x7f09f40b42a0 con 0x7f09f40d0780 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d8003530 con 0x7f09f40d0780 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756092232 0 0) 0x7f09d8003530 con 0x7f09f40d0780 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 >> v2:172.21.15.172:3300/0 conn(0x7f09f415bc50 msgr2=0x7f09f40b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 --2- 172.21.15.164:0/2007387684 >> v2:172.21.15.172:3300/0 conn(0x7f09f415bc50 0x7f09f40b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 >> v1:172.21.15.172:6789/0 conn(0x7f09f40ccfe0 legacy=0x7f09f40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40cf590 con 0x7f09f40d0780 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f09ec001a20 con 0x7f09f40d0780 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 >> v1:172.21.15.164:6789/0 conn(0x7f09f40d0780 legacy=0x7f09f415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80039c0 0x7f09d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 0x7f09d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09d80051d0 0x7f09d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40cd7c0 con 0x7f09d8004810 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d800c170 con 0x7f09d80039c0 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d800c370 con 0x7f09d80051d0 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 0x7f09d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3400640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80039c0 0x7f09d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3e00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09d80051d0 0x7f09d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 -- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09d80051d0 msgr2=0x7f09d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09d80051d0 0x7f09d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 -- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80039c0 msgr2=0x7f09d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80039c0 0x7f09d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09ebe00640 1 -- 172.21.15.164:0/2007387684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d8003530 con 0x7f09d8004810 2024-05-08T04:39:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.801+0000 7f09f3e00640 1 -- 172.21.15.164:0/2007387684 reap_dead start 2024-05-08T04:39:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09ebe00640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 0x7f09d8004c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f09dc001780 tx=0x7f09dc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dc0f4fb976fed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0013a0 con 0x7f09d8004810 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc0052f0 con 0x7f09d8004810 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09f2a00640 1 -- 172.21.15.164:0/2007387684 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003840 con 0x7f09d8004810 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 msgr2=0x7f09d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 0x7f09d8004c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f09dc001780 tx=0x7f09dc01c3d0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- 172.21.15.164:0/2007387684 shutdown_connections 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 --2- 172.21.15.164:0/2007387684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09d8004810 0x7f09d8004c90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- 172.21.15.164:0/2007387684 >> 172.21.15.164:0/2007387684 conn(0x7f09f40b34f0 msgr2=0x7f09f40b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- 172.21.15.164:0/2007387684 shutdown_connections 2024-05-08T04:39:58.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- 172.21.15.164:0/2007387684 wait complete. 2024-05-08T04:39:58.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 Processor -- start 2024-05-08T04:39:58.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.805+0000 7f09fa000640 1 -- start start 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d8004810 0x7f09f40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09f40c8110 0x7f09f41f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40d0480 con 0x7f09f40c7c90 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f406ddb0 con 0x7f09d8004810 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40d0600 con 0x7f09f40c8110 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41792/0 (socket says 172.21.15.164:41792) 2024-05-08T04:39:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 -- 172.21.15.164:0/345072790 learned_addr learned my addr 172.21.15.164:0/345072790 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09f3400640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d8004810 0x7f09f40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09f3e00640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09f40c8110 0x7f09f41f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 -- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09f40c8110 msgr2=0x7f09f41f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09f40c8110 0x7f09f41f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 -- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d8004810 msgr2=0x7f09f40c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d8004810 0x7f09f40c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 -- 172.21.15.164:0/345072790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f415a4a0 con 0x7f09f40c7c90 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09ebe00640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005040 tx=0x7f09d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0013a0 con 0x7f09f40c7c90 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc0ba070 con 0x7f09f40c7c90 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f40c6510 con 0x7f09f40c7c90 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc007a40 con 0x7f09f40c7c90 2024-05-08T04:39:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f40c54f0 con 0x7f09f40c7c90 2024-05-08T04:39:58.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4069d60 con 0x7f09f40c7c90 2024-05-08T04:39:58.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.809+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f09dc005270 con 0x7f09f40c7c90 2024-05-08T04:39:58.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.813+0000 7f09eb400640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 0x7f09d00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:58.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.813+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09dc11f100 con 0x7f09f40c7c90 2024-05-08T04:39:58.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.813+0000 7f09f3400640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 0x7f09d00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:58.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.813+0000 7f09f3400640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 0x7f09d00694b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009be0 tx=0x7f09e4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:58.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:58.817+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09dc016110 con 0x7f09f40c7c90 2024-05-08T04:39:59.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f09f40b7940 con 0x7f09f40c7c90 2024-05-08T04:39:59.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09eb400640 1 -- 172.21.15.164:0/345072790 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1528 (secure 0 0 0) 0x7f09dc0eff10 con 0x7f09f40c7c90 2024-05-08T04:39:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:59.042 INFO:teuthology.orchestra.run.smithi164.stdout:{"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1715143096,"num_in_osds":8,"osd_in_since":1715143086,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":230715392,"bytes_avail":772863397888,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5523}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-08T04:39:41.445335+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 msgr2=0x7f09d00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 0x7f09d00694b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009be0 tx=0x7f09e4009190 comp rx=0 tx=0).stop 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 msgr2=0x7f09f41f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005040 tx=0x7f09d8005840 comp rx=0 tx=0).stop 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 shutdown_connections 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09d0066ff0 0x7f09d00694b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09f40c8110 0x7f09f41f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09f40c7c90 0x7f09f41f6060 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 --2- 172.21.15.164:0/345072790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d8004810 0x7f09f40c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 >> 172.21.15.164:0/345072790 conn(0x7f09f40b34f0 msgr2=0x7f09f40b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:59.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.037+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 shutdown_connections 2024-05-08T04:39:59.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.041+0000 7f09fa000640 1 -- 172.21.15.164:0/345072790 wait complete. 2024-05-08T04:39:59.063 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T04:39:59.063 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:39:59.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.193+0000 7f3357c00640 1 Processor -- start 2024-05-08T04:39:59.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.193+0000 7f3357c00640 1 -- start start 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3357c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f335015ba70 0x7f3350160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3357c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 0x7f33500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3357c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33500cd9f0 con 0x7f33500aeda0 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3357c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335015fe70 con 0x7f335015ba70 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3357c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33500cdcd0 con 0x7f335015be50 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f335015ba70 0x7f3350160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f335015ba70 0x7f3350160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37938/0 (socket says 172.21.15.164:37938) 2024-05-08T04:39:59.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 -- 172.21.15.164:0/986628306 learned_addr learned my addr 172.21.15.164:0/986628306 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 -- 172.21.15.164:0/986628306 >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 msgr2=0x7f33500cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f334fe00640 1 --2- 172.21.15.164:0/986628306 >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 0x7f33500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 --2- 172.21.15.164:0/986628306 >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 0x7f33500cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 -- 172.21.15.164:0/986628306 >> v1:172.21.15.172:6789/0 conn(0x7f33500aeda0 legacy=0x7f33500b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f3355200640 1 -- 172.21.15.164:0/986628306 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335015f820 con 0x7f335015ba70 2024-05-08T04:39:59.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.197+0000 7f334fe00640 1 --2- 172.21.15.164:0/986628306 >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 0x7f33500cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:39:59.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3355200640 1 --2- 172.21.15.164:0/986628306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015ba70 0x7f3350160300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f335015f780 tx=0x7f333801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3726b29b3e95ef52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:59.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f334f400640 1 -- 172.21.15.164:0/986628306 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380b1070 con 0x7f335015ba70 2024-05-08T04:39:59.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f334f400640 1 -- 172.21.15.164:0/986628306 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33380ac620 con 0x7f335015ba70 2024-05-08T04:39:59.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f334f400640 1 -- 172.21.15.164:0/986628306 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33380ac9f0 con 0x7f335015ba70 2024-05-08T04:39:59.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- 172.21.15.164:0/986628306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015ba70 msgr2=0x7f3350160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 --2- 172.21.15.164:0/986628306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015ba70 0x7f3350160300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f335015f780 tx=0x7f333801c4b0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- 172.21.15.164:0/986628306 shutdown_connections 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 --2- 172.21.15.164:0/986628306 >> v2:172.21.15.172:3300/0 conn(0x7f335015be50 0x7f33500cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 --2- 172.21.15.164:0/986628306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015ba70 0x7f3350160300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- 172.21.15.164:0/986628306 >> 172.21.15.164:0/986628306 conn(0x7f335006ac90 msgr2=0x7f335006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- 172.21.15.164:0/986628306 shutdown_connections 2024-05-08T04:39:59.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- 172.21.15.164:0/986628306 wait complete. 2024-05-08T04:39:59.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 Processor -- start 2024-05-08T04:39:59.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.201+0000 7f3357c00640 1 -- start start 2024-05-08T04:39:59.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33500aeda0 0x7f335014f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f335015ba70 0x7f3350070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350168d40 con 0x7f335015be50 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350168bc0 con 0x7f335015ba70 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3350168ec0 con 0x7f33500aeda0 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37946/0 (socket says 172.21.15.164:37946) 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 -- 172.21.15.164:0/3287065594 learned_addr learned my addr 172.21.15.164:0/3287065594 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:39:59.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355200640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33500aeda0 0x7f335014f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f334fe00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f335015ba70 0x7f3350070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 -- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33500aeda0 msgr2=0x7f335014f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33500aeda0 0x7f335014f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 -- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f335015ba70 msgr2=0x7f3350070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f335015ba70 0x7f3350070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 -- 172.21.15.164:0/3287065594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350071e70 con 0x7f335015be50 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3355c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f334800c780 tx=0x7f334800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348004b70 con 0x7f335015be50 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3348004d10 con 0x7f335015be50 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334800aa90 con 0x7f335015be50 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3350148770 con 0x7f335015be50 2024-05-08T04:39:59.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33500718b0 con 0x7f335015be50 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3350069d60 con 0x7f335015be50 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.205+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f334800ac30 con 0x7f335015be50 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.209+0000 7f334cc00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 0x7f332c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.209+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3348085090 con 0x7f335015be50 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.209+0000 7f3355200640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 0x7f332c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:39:59.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.209+0000 7f3355200640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 0x7f332c0694e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33380ae000 tx=0x7f33380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:39:59.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.213+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3348020070 con 0x7f335015be50 2024-05-08T04:39:59.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.365+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3350148940 con 0x7f332c067020 2024-05-08T04:39:59.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.369+0000 7f334cc00640 1 -- 172.21.15.164:0/3287065594 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239854 (secure 0 0 0) 0x7f3350148940 con 0x7f332c067020 2024-05-08T04:39:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:39:59.379 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:39:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 msgr2=0x7f332c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 0x7f332c0694e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33380ae000 tx=0x7f33380a7000 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 msgr2=0x7f3350147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f334800c780 tx=0x7f334800cc40 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 shutdown_connections 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f332c067020 0x7f332c0694e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f335015be50 0x7f3350147e00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f335015ba70 0x7f3350070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 --2- 172.21.15.164:0/3287065594 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33500aeda0 0x7f335014f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 >> 172.21.15.164:0/3287065594 conn(0x7f335006ac90 msgr2=0x7f335006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 shutdown_connections 2024-05-08T04:39:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:39:59.373+0000 7f3357c00640 1 -- 172.21.15.164:0/3287065594 wait complete. 2024-05-08T04:39:59.398 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-08T04:39:57.448475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225308,"kb_used_data":4976,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749412,"statfs":{"total":773094113280,"available":772863397888,"internally_reserved":0,"allocated":5095424,"data_stored":2735350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.685844"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770101+0000","last_change":"2024-05-08T04:39:51.770101+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770101+0000","last_undegraded":"2024-05-08T04:39:51.770101+0000","last_fullsized":"2024-05-08T04:39:51.770101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766509+0000","last_change":"2024-05-08T04:39:51.766509+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766509+0000","last_undegraded":"2024-05-08T04:39:51.766509+0000","last_fullsized":"2024-05-08T04:39:51.766509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768808+0000","last_change":"2024-05-08T04:39:51.768808+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768808+0000","last_undegraded":"2024-05-08T04:39:51.768808+0000","last_fullsized":"2024-05-08T04:39:51.768808+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771440+0000","last_change":"2024-05-08T04:39:51.771440+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771440+0000","last_undegraded":"2024-05-08T04:39:51.771440+0000","last_fullsized":"2024-05-08T04:39:51.771440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769699+0000","last_change":"2024-05-08T04:39:51.769699+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769699+0000","last_undegraded":"2024-05-08T04:39:51.769699+0000","last_fullsized":"2024-05-08T04:39:51.769699+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.766525+0000","last_change":"2024-05-08T04:39:51.766525+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.766525+0000","last_undegraded":"2024-05-08T04:39:51.766525+0000","last_fullsized":"2024-05-08T04:39:51.766525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.776627+0000","last_change":"2024-05-08T04:39:51.776627+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.776627+0000","last_undegraded":"2024-05-08T04:39:51.776627+0000","last_fullsized":"2024-05-08T04:39:51.776627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765314+0000","last_change":"2024-05-08T04:39:51.765314+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765314+0000","last_undegraded":"2024-05-08T04:39:51.765314+0000","last_fullsized":"2024-05-08T04:39:51.765314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775276+0000","last_change":"2024-05-08T04:39:51.775276+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775276+0000","last_undegraded":"2024-05-08T04:39:51.775276+0000","last_fullsized":"2024-05-08T04:39:51.775276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.772036+0000","last_change":"2024-05-08T04:39:51.772036+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.772036+0000","last_undegraded":"2024-05-08T04:39:51.772036+0000","last_fullsized":"2024-05-08T04:39:51.772036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770629+0000","last_change":"2024-05-08T04:39:51.770629+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770629+0000","last_undegraded":"2024-05-08T04:39:51.770629+0000","last_fullsized":"2024-05-08T04:39:51.770629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769202+0000","last_change":"2024-05-08T04:39:51.769202+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769202+0000","last_undegraded":"2024-05-08T04:39:51.769202+0000","last_fullsized":"2024-05-08T04:39:51.769202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771439+0000","last_change":"2024-05-08T04:39:51.771439+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771439+0000","last_undegraded":"2024-05-08T04:39:51.771439+0000","last_fullsized":"2024-05-08T04:39:51.771439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.769227+0000","last_change":"2024-05-08T04:39:51.769227+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.769227+0000","last_undegraded":"2024-05-08T04:39:51.769227+0000","last_fullsized":"2024-05-08T04:39:51.769227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775721+0000","last_change":"2024-05-08T04:39:51.775721+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775721+0000","last_undegraded":"2024-05-08T04:39:51.775721+0000","last_fullsized":"2024-05-08T04:39:51.775721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.770130+0000","last_change":"2024-05-08T04:39:51.770130+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.770130+0000","last_undegraded":"2024-05-08T04:39:51.770130+0000","last_fullsized":"2024-05-08T04:39:51.770130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765341+0000","last_change":"2024-05-08T04:39:51.765341+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765341+0000","last_undegraded":"2024-05-08T04:39:51.765341+0000","last_fullsized":"2024-05-08T04:39:51.765341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765391+0000","last_change":"2024-05-08T04:39:51.765391+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765391+0000","last_undegraded":"2024-05-08T04:39:51.765391+0000","last_fullsized":"2024-05-08T04:39:51.765391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768747+0000","last_change":"2024-05-08T04:39:51.768747+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768747+0000","last_undegraded":"2024-05-08T04:39:51.768747+0000","last_fullsized":"2024-05-08T04:39:51.768747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768712+0000","last_change":"2024-05-08T04:39:51.768712+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768712+0000","last_undegraded":"2024-05-08T04:39:51.768712+0000","last_fullsized":"2024-05-08T04:39:51.768712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765398+0000","last_change":"2024-05-08T04:39:51.765398+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765398+0000","last_undegraded":"2024-05-08T04:39:51.765398+0000","last_fullsized":"2024-05-08T04:39:51.765398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775196+0000","last_change":"2024-05-08T04:39:51.775196+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775196+0000","last_undegraded":"2024-05-08T04:39:51.775196+0000","last_fullsized":"2024-05-08T04:39:51.775196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768659+0000","last_change":"2024-05-08T04:39:51.768659+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768659+0000","last_undegraded":"2024-05-08T04:39:51.768659+0000","last_fullsized":"2024-05-08T04:39:51.768659+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771445+0000","last_change":"2024-05-08T04:39:51.771445+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771445+0000","last_undegraded":"2024-05-08T04:39:51.771445+0000","last_fullsized":"2024-05-08T04:39:51.771445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768571+0000","last_change":"2024-05-08T04:39:51.768571+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768571+0000","last_undegraded":"2024-05-08T04:39:51.768571+0000","last_fullsized":"2024-05-08T04:39:51.768571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.771403+0000","last_change":"2024-05-08T04:39:51.771403+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.771403+0000","last_undegraded":"2024-05-08T04:39:51.771403+0000","last_fullsized":"2024-05-08T04:39:51.771403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765304+0000","last_change":"2024-05-08T04:39:51.765304+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765304+0000","last_undegraded":"2024-05-08T04:39:51.765304+0000","last_fullsized":"2024-05-08T04:39:51.765304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.768489+0000","last_change":"2024-05-08T04:39:51.768489+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.768489+0000","last_undegraded":"2024-05-08T04:39:51.768489+0000","last_fullsized":"2024-05-08T04:39:51.768489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.775215+0000","last_change":"2024-05-08T04:39:51.775215+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.775215+0000","last_undegraded":"2024-05-08T04:39:51.775215+0000","last_fullsized":"2024-05-08T04:39:51.775215+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:39:52.761450+0000","last_change":"2024-05-08T04:39:52.761450+0000","last_active":"2024-05-08T04:39:52.761450+0000","last_peered":"2024-05-08T04:39:52.761450+0000","last_clean":"2024-05-08T04:39:52.761450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:52.761450+0000","last_undegraded":"2024-05-08T04:39:52.761450+0000","last_fullsized":"2024-05-08T04:39:52.761450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-05-08T04:39:51.765755+0000","last_change":"2024-05-08T04:39:51.765755+0000","last_active":"2024-05-08T04:39:51.752781+0000","last_peered":"2024-05-08T04:39:51.752781+0000","last_clean":"2024-05-08T04:39:51.752781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:39:51.765755+0000","last_undegraded":"2024-05-08T04:39:51.765755+0000","last_fullsized":"2024-05-08T04:39:51.765755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-05-08T04:39:51.772482+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:51.772482+0000","last_peered":"2024-05-08T04:39:51.772482+0000","last_clean":"2024-05-08T04:39:51.772482+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:51.772482+0000","last_undegraded":"2024-05-08T04:39:51.772482+0000","last_fullsized":"2024-05-08T04:39:51.772482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":479232,"data_stored":189964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:40:02.403 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:40:02.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 Processor -- start 2024-05-08T04:40:02.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 -- start start 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c15ba70 0x7f461c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cd9f0 con 0x7f461c0aeda0 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c15fe70 con 0x7f461c15ba70 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4624600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cdcd0 con 0x7f461c15be50 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.545+0000 7f4621c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c15ba70 0x7f461c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f461c15ba70 0x7f461c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37950/0 (socket says 172.21.15.164:37950) 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 -- 172.21.15.164:0/1025476973 learned_addr learned my addr 172.21.15.164:0/1025476973 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4622600640 1 --1- 172.21.15.164:0/1025476973 >> v1:172.21.15.164:6789/0 conn(0x7f461c0aeda0 0x7f461c0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60438/0 (socket says 172.21.15.164:60438) 2024-05-08T04:40:02.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621200640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 -- 172.21.15.164:0/1025476973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 msgr2=0x7f461c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 -- 172.21.15.164:0/1025476973 >> v1:172.21.15.164:6789/0 conn(0x7f461c0aeda0 legacy=0x7f461c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 -- 172.21.15.164:0/1025476973 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c15f820 con 0x7f461c15ba70 2024-05-08T04:40:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f461be00640 1 -- 172.21.15.164:0/1025476973 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613547687 0 0) 0x7f461c0cd9f0 con 0x7f461c0aeda0 2024-05-08T04:40:02.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f4621c00640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c160300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4ed61fa48173a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f461be00640 1 -- 172.21.15.164:0/1025476973 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100ab640 con 0x7f461c15ba70 2024-05-08T04:40:02.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f461be00640 1 -- 172.21.15.164:0/1025476973 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100b0070 con 0x7f461c15ba70 2024-05-08T04:40:02.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.549+0000 7f461be00640 1 -- 172.21.15.164:0/1025476973 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100aba00 con 0x7f461c15ba70 2024-05-08T04:40:02.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- 172.21.15.164:0/1025476973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 msgr2=0x7f461c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c160300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461001c530 comp rx=0 tx=0).stop 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- 172.21.15.164:0/1025476973 shutdown_connections 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- 172.21.15.164:0/1025476973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c160300 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- 172.21.15.164:0/1025476973 >> 172.21.15.164:0/1025476973 conn(0x7f461c06ac90 msgr2=0x7f461c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- 172.21.15.164:0/1025476973 shutdown_connections 2024-05-08T04:40:02.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- 172.21.15.164:0/1025476973 wait complete. 2024-05-08T04:40:02.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 Processor -- start 2024-05-08T04:40:02.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- start start 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 0x7f461c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c168d40 con 0x7f461c15ba70 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c168bc0 con 0x7f461c0aeda0 2024-05-08T04:40:02.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4624600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c168ec0 con 0x7f461c15be50 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37962/0 (socket says 172.21.15.164:37962) 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 -- 172.21.15.164:0/2310782054 learned_addr learned my addr 172.21.15.164:0/2310782054 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621c00640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 0x7f461c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4622600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 -- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 msgr2=0x7f461c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 -- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 msgr2=0x7f461c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 0x7f461c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621200640 1 -- 172.21.15.164:0/2310782054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c154610 con 0x7f461c15ba70 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.553+0000 7f4621c00640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 0x7f461c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4621200640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f460800d800 tx=0x7f460800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460800b6c0 con 0x7f461c15ba70 2024-05-08T04:40:02.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608019070 con 0x7f461c15ba70 2024-05-08T04:40:02.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460800ba80 con 0x7f461c15ba70 2024-05-08T04:40:02.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c150ff0 con 0x7f461c15ba70 2024-05-08T04:40:02.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c1548b0 con 0x7f461c15ba70 2024-05-08T04:40:02.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c1530f0 con 0x7f461c15ba70 2024-05-08T04:40:02.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f460800bc20 con 0x7f461c15ba70 2024-05-08T04:40:02.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 0x7f45fc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46080855b0 con 0x7f461c15ba70 2024-05-08T04:40:02.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.557+0000 7f4621c00640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 0x7f45fc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.561+0000 7f4621c00640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 0x7f45fc0694b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f46100a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.561+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4608056440 con 0x7f461c15ba70 2024-05-08T04:40:02.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.713+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f461c069d60 con 0x7f45fc066ff0 2024-05-08T04:40:02.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.721+0000 7f4619600640 1 -- 172.21.15.164:0/2310782054 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251958 (secure 0 0 0) 0x7f461c069d60 con 0x7f45fc066ff0 2024-05-08T04:40:02.728 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:02.729 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:40:02.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 msgr2=0x7f45fc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 0x7f45fc0694b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f46100a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:02.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 msgr2=0x7f461c13ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f460800d800 tx=0x7f460800dcc0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 shutdown_connections 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f45fc066ff0 0x7f45fc0694b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15be50 0x7f461c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f461c15ba70 0x7f461c13ff90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 --2- 172.21.15.164:0/2310782054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c0aeda0 0x7f461c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 >> 172.21.15.164:0/2310782054 conn(0x7f461c06ac90 msgr2=0x7f461c0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 shutdown_connections 2024-05-08T04:40:02.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.725+0000 7f4624600640 1 -- 172.21.15.164:0/2310782054 wait complete. 2024-05-08T04:40:02.748 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-08T04:40:01.449362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225676,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749044,"statfs":{"total":773094113280,"available":772863021056,"internally_reserved":0,"allocated":5480448,"data_stored":2906540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001874"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917286+0000","last_change":"2024-05-08T04:39:53.811877+0000","last_active":"2024-05-08T04:39:54.917286+0000","last_peered":"2024-05-08T04:39:54.917286+0000","last_clean":"2024-05-08T04:39:54.917286+0000","last_became_active":"2024-05-08T04:39:53.811608+0000","last_became_peered":"2024-05-08T04:39:53.811608+0000","last_unstale":"2024-05-08T04:39:54.917286+0000","last_undegraded":"2024-05-08T04:39:54.917286+0000","last_fullsized":"2024-05-08T04:39:54.917286+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:47.273031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792502+0000","last_change":"2024-05-08T04:39:53.816247+0000","last_active":"2024-05-08T04:39:54.792502+0000","last_peered":"2024-05-08T04:39:54.792502+0000","last_clean":"2024-05-08T04:39:54.792502+0000","last_became_active":"2024-05-08T04:39:53.815940+0000","last_became_peered":"2024-05-08T04:39:53.815940+0000","last_unstale":"2024-05-08T04:39:54.792502+0000","last_undegraded":"2024-05-08T04:39:54.792502+0000","last_fullsized":"2024-05-08T04:39:54.792502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:18.896636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792127+0000","last_change":"2024-05-08T04:39:53.809956+0000","last_active":"2024-05-08T04:39:54.792127+0000","last_peered":"2024-05-08T04:39:54.792127+0000","last_clean":"2024-05-08T04:39:54.792127+0000","last_became_active":"2024-05-08T04:39:53.809685+0000","last_became_peered":"2024-05-08T04:39:53.809685+0000","last_unstale":"2024-05-08T04:39:54.792127+0000","last_undegraded":"2024-05-08T04:39:54.792127+0000","last_fullsized":"2024-05-08T04:39:54.792127+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:16.943540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391044+0000","last_change":"2024-05-08T04:39:53.817994+0000","last_active":"2024-05-08T04:39:55.391044+0000","last_peered":"2024-05-08T04:39:55.391044+0000","last_clean":"2024-05-08T04:39:55.391044+0000","last_became_active":"2024-05-08T04:39:53.817615+0000","last_became_peered":"2024-05-08T04:39:53.817615+0000","last_unstale":"2024-05-08T04:39:55.391044+0000","last_undegraded":"2024-05-08T04:39:55.391044+0000","last_fullsized":"2024-05-08T04:39:55.391044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:40.263015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792291+0000","last_change":"2024-05-08T04:39:53.814866+0000","last_active":"2024-05-08T04:39:54.792291+0000","last_peered":"2024-05-08T04:39:54.792291+0000","last_clean":"2024-05-08T04:39:54.792291+0000","last_became_active":"2024-05-08T04:39:53.814572+0000","last_became_peered":"2024-05-08T04:39:53.814572+0000","last_unstale":"2024-05-08T04:39:54.792291+0000","last_undegraded":"2024-05-08T04:39:54.792291+0000","last_fullsized":"2024-05-08T04:39:54.792291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:38.839149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078609+0000","last_change":"2024-05-08T04:39:53.796753+0000","last_active":"2024-05-08T04:39:55.078609+0000","last_peered":"2024-05-08T04:39:55.078609+0000","last_clean":"2024-05-08T04:39:55.078609+0000","last_became_active":"2024-05-08T04:39:53.796514+0000","last_became_peered":"2024-05-08T04:39:53.796514+0000","last_unstale":"2024-05-08T04:39:55.078609+0000","last_undegraded":"2024-05-08T04:39:55.078609+0000","last_fullsized":"2024-05-08T04:39:55.078609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:37.031324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917009+0000","last_change":"2024-05-08T04:39:53.806178+0000","last_active":"2024-05-08T04:39:54.917009+0000","last_peered":"2024-05-08T04:39:54.917009+0000","last_clean":"2024-05-08T04:39:54.917009+0000","last_became_active":"2024-05-08T04:39:53.805933+0000","last_became_peered":"2024-05-08T04:39:53.805933+0000","last_unstale":"2024-05-08T04:39:54.917009+0000","last_undegraded":"2024-05-08T04:39:54.917009+0000","last_fullsized":"2024-05-08T04:39:54.917009+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:18.896636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390658+0000","last_change":"2024-05-08T04:39:53.809123+0000","last_active":"2024-05-08T04:39:55.390658+0000","last_peered":"2024-05-08T04:39:55.390658+0000","last_clean":"2024-05-08T04:39:55.390658+0000","last_became_active":"2024-05-08T04:39:53.808162+0000","last_became_peered":"2024-05-08T04:39:53.808162+0000","last_unstale":"2024-05-08T04:39:55.390658+0000","last_undegraded":"2024-05-08T04:39:55.390658+0000","last_fullsized":"2024-05-08T04:39:55.390658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:19.142238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391522+0000","last_change":"2024-05-08T04:39:53.815875+0000","last_active":"2024-05-08T04:39:55.391522+0000","last_peered":"2024-05-08T04:39:55.391522+0000","last_clean":"2024-05-08T04:39:55.391522+0000","last_became_active":"2024-05-08T04:39:53.815633+0000","last_became_peered":"2024-05-08T04:39:53.815633+0000","last_unstale":"2024-05-08T04:39:55.391522+0000","last_undegraded":"2024-05-08T04:39:55.391522+0000","last_fullsized":"2024-05-08T04:39:55.391522+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:18.896636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391515+0000","last_change":"2024-05-08T04:39:53.817096+0000","last_active":"2024-05-08T04:39:55.391515+0000","last_peered":"2024-05-08T04:39:55.391515+0000","last_clean":"2024-05-08T04:39:55.391515+0000","last_became_active":"2024-05-08T04:39:53.816803+0000","last_became_peered":"2024-05-08T04:39:53.816803+0000","last_unstale":"2024-05-08T04:39:55.391515+0000","last_undegraded":"2024-05-08T04:39:55.391515+0000","last_fullsized":"2024-05-08T04:39:55.391515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:19.104705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391251+0000","last_change":"2024-05-08T04:39:53.814819+0000","last_active":"2024-05-08T04:39:55.391251+0000","last_peered":"2024-05-08T04:39:55.391251+0000","last_clean":"2024-05-08T04:39:55.391251+0000","last_became_active":"2024-05-08T04:39:53.814561+0000","last_became_peered":"2024-05-08T04:39:53.814561+0000","last_unstale":"2024-05-08T04:39:55.391251+0000","last_undegraded":"2024-05-08T04:39:55.391251+0000","last_fullsized":"2024-05-08T04:39:55.391251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:29.198870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792349+0000","last_change":"2024-05-08T04:39:53.802350+0000","last_active":"2024-05-08T04:39:54.792349+0000","last_peered":"2024-05-08T04:39:54.792349+0000","last_clean":"2024-05-08T04:39:54.792349+0000","last_became_active":"2024-05-08T04:39:53.802041+0000","last_became_peered":"2024-05-08T04:39:53.802041+0000","last_unstale":"2024-05-08T04:39:54.792349+0000","last_undegraded":"2024-05-08T04:39:54.792349+0000","last_fullsized":"2024-05-08T04:39:54.792349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:29.198870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786147+0000","last_change":"2024-05-08T04:39:53.804378+0000","last_active":"2024-05-08T04:39:54.786147+0000","last_peered":"2024-05-08T04:39:54.786147+0000","last_clean":"2024-05-08T04:39:54.786147+0000","last_became_active":"2024-05-08T04:39:53.803116+0000","last_became_peered":"2024-05-08T04:39:53.803116+0000","last_unstale":"2024-05-08T04:39:54.786147+0000","last_undegraded":"2024-05-08T04:39:54.786147+0000","last_fullsized":"2024-05-08T04:39:54.786147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:04.643305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390756+0000","last_change":"2024-05-08T04:39:53.810920+0000","last_active":"2024-05-08T04:39:55.390756+0000","last_peered":"2024-05-08T04:39:55.390756+0000","last_clean":"2024-05-08T04:39:55.390756+0000","last_became_active":"2024-05-08T04:39:53.810642+0000","last_became_peered":"2024-05-08T04:39:53.810642+0000","last_unstale":"2024-05-08T04:39:55.390756+0000","last_undegraded":"2024-05-08T04:39:55.390756+0000","last_fullsized":"2024-05-08T04:39:55.390756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:48.511337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078578+0000","last_change":"2024-05-08T04:39:53.814396+0000","last_active":"2024-05-08T04:39:55.078578+0000","last_peered":"2024-05-08T04:39:55.078578+0000","last_clean":"2024-05-08T04:39:55.078578+0000","last_became_active":"2024-05-08T04:39:53.814112+0000","last_became_peered":"2024-05-08T04:39:53.814112+0000","last_unstale":"2024-05-08T04:39:55.078578+0000","last_undegraded":"2024-05-08T04:39:55.078578+0000","last_fullsized":"2024-05-08T04:39:55.078578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:22:50.761312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391080+0000","last_change":"2024-05-08T04:39:53.820228+0000","last_active":"2024-05-08T04:39:55.391080+0000","last_peered":"2024-05-08T04:39:55.391080+0000","last_clean":"2024-05-08T04:39:55.391080+0000","last_became_active":"2024-05-08T04:39:53.819933+0000","last_became_peered":"2024-05-08T04:39:53.819933+0000","last_unstale":"2024-05-08T04:39:55.391080+0000","last_undegraded":"2024-05-08T04:39:55.391080+0000","last_fullsized":"2024-05-08T04:39:55.391080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:31.764193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792290+0000","last_change":"2024-05-08T04:39:53.809303+0000","last_active":"2024-05-08T04:39:54.792290+0000","last_peered":"2024-05-08T04:39:54.792290+0000","last_clean":"2024-05-08T04:39:54.792290+0000","last_became_active":"2024-05-08T04:39:53.808979+0000","last_became_peered":"2024-05-08T04:39:53.808979+0000","last_unstale":"2024-05-08T04:39:54.792290+0000","last_undegraded":"2024-05-08T04:39:54.792290+0000","last_fullsized":"2024-05-08T04:39:54.792290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:53.174229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792749+0000","last_change":"2024-05-08T04:39:53.818350+0000","last_active":"2024-05-08T04:39:54.792749+0000","last_peered":"2024-05-08T04:39:54.792749+0000","last_clean":"2024-05-08T04:39:54.792749+0000","last_became_active":"2024-05-08T04:39:53.818093+0000","last_became_peered":"2024-05-08T04:39:53.818093+0000","last_unstale":"2024-05-08T04:39:54.792749+0000","last_undegraded":"2024-05-08T04:39:54.792749+0000","last_fullsized":"2024-05-08T04:39:54.792749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:48.511337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391707+0000","last_change":"2024-05-08T04:39:53.821771+0000","last_active":"2024-05-08T04:39:55.391707+0000","last_peered":"2024-05-08T04:39:55.391707+0000","last_clean":"2024-05-08T04:39:55.391707+0000","last_became_active":"2024-05-08T04:39:53.821480+0000","last_became_peered":"2024-05-08T04:39:53.821480+0000","last_unstale":"2024-05-08T04:39:55.391707+0000","last_undegraded":"2024-05-08T04:39:55.391707+0000","last_fullsized":"2024-05-08T04:39:55.391707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:00.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786630+0000","last_change":"2024-05-08T04:39:53.814434+0000","last_active":"2024-05-08T04:39:54.786630+0000","last_peered":"2024-05-08T04:39:54.786630+0000","last_clean":"2024-05-08T04:39:54.786630+0000","last_became_active":"2024-05-08T04:39:53.814197+0000","last_became_peered":"2024-05-08T04:39:53.814197+0000","last_unstale":"2024-05-08T04:39:54.786630+0000","last_undegraded":"2024-05-08T04:39:54.786630+0000","last_fullsized":"2024-05-08T04:39:54.786630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:15:51.584091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078757+0000","last_change":"2024-05-08T04:39:53.799691+0000","last_active":"2024-05-08T04:39:55.078757+0000","last_peered":"2024-05-08T04:39:55.078757+0000","last_clean":"2024-05-08T04:39:55.078757+0000","last_became_active":"2024-05-08T04:39:53.799431+0000","last_became_peered":"2024-05-08T04:39:53.799431+0000","last_unstale":"2024-05-08T04:39:55.078757+0000","last_undegraded":"2024-05-08T04:39:55.078757+0000","last_fullsized":"2024-05-08T04:39:55.078757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:00.067759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391214+0000","last_change":"2024-05-08T04:39:53.817133+0000","last_active":"2024-05-08T04:39:55.391214+0000","last_peered":"2024-05-08T04:39:55.391214+0000","last_clean":"2024-05-08T04:39:55.391214+0000","last_became_active":"2024-05-08T04:39:53.816842+0000","last_became_peered":"2024-05-08T04:39:53.816842+0000","last_unstale":"2024-05-08T04:39:55.391214+0000","last_undegraded":"2024-05-08T04:39:55.391214+0000","last_fullsized":"2024-05-08T04:39:55.391214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:38.839149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391461+0000","last_change":"2024-05-08T04:39:53.806676+0000","last_active":"2024-05-08T04:39:55.391461+0000","last_peered":"2024-05-08T04:39:55.391461+0000","last_clean":"2024-05-08T04:39:55.391461+0000","last_became_active":"2024-05-08T04:39:53.806381+0000","last_became_peered":"2024-05-08T04:39:53.806381+0000","last_unstale":"2024-05-08T04:39:55.391461+0000","last_undegraded":"2024-05-08T04:39:55.391461+0000","last_fullsized":"2024-05-08T04:39:55.391461+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:31:06.848907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391156+0000","last_change":"2024-05-08T04:39:53.801011+0000","last_active":"2024-05-08T04:39:55.391156+0000","last_peered":"2024-05-08T04:39:55.391156+0000","last_clean":"2024-05-08T04:39:55.391156+0000","last_became_active":"2024-05-08T04:39:53.800705+0000","last_became_peered":"2024-05-08T04:39:53.800705+0000","last_unstale":"2024-05-08T04:39:55.391156+0000","last_undegraded":"2024-05-08T04:39:55.391156+0000","last_fullsized":"2024-05-08T04:39:55.391156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:25.658846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391591+0000","last_change":"2024-05-08T04:39:53.810435+0000","last_active":"2024-05-08T04:39:55.391591+0000","last_peered":"2024-05-08T04:39:55.391591+0000","last_clean":"2024-05-08T04:39:55.391591+0000","last_became_active":"2024-05-08T04:39:53.810118+0000","last_became_peered":"2024-05-08T04:39:53.810118+0000","last_unstale":"2024-05-08T04:39:55.391591+0000","last_undegraded":"2024-05-08T04:39:55.391591+0000","last_fullsized":"2024-05-08T04:39:55.391591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:38.866308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786392+0000","last_change":"2024-05-08T04:39:53.796202+0000","last_active":"2024-05-08T04:39:54.786392+0000","last_peered":"2024-05-08T04:39:54.786392+0000","last_clean":"2024-05-08T04:39:54.786392+0000","last_became_active":"2024-05-08T04:39:53.795878+0000","last_became_peered":"2024-05-08T04:39:53.795878+0000","last_unstale":"2024-05-08T04:39:54.786392+0000","last_undegraded":"2024-05-08T04:39:54.786392+0000","last_fullsized":"2024-05-08T04:39:54.786392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:34.097529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792299+0000","last_change":"2024-05-08T04:39:53.809389+0000","last_active":"2024-05-08T04:39:54.792299+0000","last_peered":"2024-05-08T04:39:54.792299+0000","last_clean":"2024-05-08T04:39:54.792299+0000","last_became_active":"2024-05-08T04:39:53.809088+0000","last_became_peered":"2024-05-08T04:39:53.809088+0000","last_unstale":"2024-05-08T04:39:54.792299+0000","last_undegraded":"2024-05-08T04:39:54.792299+0000","last_fullsized":"2024-05-08T04:39:54.792299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:19.142238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792496+0000","last_change":"2024-05-08T04:39:53.795366+0000","last_active":"2024-05-08T04:39:54.792496+0000","last_peered":"2024-05-08T04:39:54.792496+0000","last_clean":"2024-05-08T04:39:54.792496+0000","last_became_active":"2024-05-08T04:39:53.795035+0000","last_became_peered":"2024-05-08T04:39:53.795035+0000","last_unstale":"2024-05-08T04:39:54.792496+0000","last_undegraded":"2024-05-08T04:39:54.792496+0000","last_fullsized":"2024-05-08T04:39:54.792496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:19.104705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391429+0000","last_change":"2024-05-08T04:39:53.804485+0000","last_active":"2024-05-08T04:39:55.391429+0000","last_peered":"2024-05-08T04:39:55.391429+0000","last_clean":"2024-05-08T04:39:55.391429+0000","last_became_active":"2024-05-08T04:39:53.804006+0000","last_became_peered":"2024-05-08T04:39:53.804006+0000","last_unstale":"2024-05-08T04:39:55.391429+0000","last_undegraded":"2024-05-08T04:39:55.391429+0000","last_fullsized":"2024-05-08T04:39:55.391429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:31:39.819404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.791881+0000","last_change":"2024-05-08T04:39:53.809539+0000","last_active":"2024-05-08T04:39:54.791881+0000","last_peered":"2024-05-08T04:39:54.791881+0000","last_clean":"2024-05-08T04:39:54.791881+0000","last_became_active":"2024-05-08T04:39:53.809239+0000","last_became_peered":"2024-05-08T04:39:53.809239+0000","last_unstale":"2024-05-08T04:39:54.791881+0000","last_undegraded":"2024-05-08T04:39:54.791881+0000","last_fullsized":"2024-05-08T04:39:54.791881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:15.797885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078720+0000","last_change":"2024-05-08T04:39:53.802718+0000","last_active":"2024-05-08T04:39:55.078720+0000","last_peered":"2024-05-08T04:39:55.078720+0000","last_clean":"2024-05-08T04:39:55.078720+0000","last_became_active":"2024-05-08T04:39:53.802453+0000","last_became_peered":"2024-05-08T04:39:53.802453+0000","last_unstale":"2024-05-08T04:39:55.078720+0000","last_undegraded":"2024-05-08T04:39:55.078720+0000","last_fullsized":"2024-05-08T04:39:55.078720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:03.654254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786272+0000","last_change":"2024-05-08T04:39:53.801122+0000","last_active":"2024-05-08T04:39:54.786272+0000","last_peered":"2024-05-08T04:39:54.786272+0000","last_clean":"2024-05-08T04:39:54.786272+0000","last_became_active":"2024-05-08T04:39:53.800830+0000","last_became_peered":"2024-05-08T04:39:53.800830+0000","last_unstale":"2024-05-08T04:39:54.786272+0000","last_undegraded":"2024-05-08T04:39:54.786272+0000","last_fullsized":"2024-05-08T04:39:54.786272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:22.825964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.890040+0000","last_change":"2024-05-08T04:39:53.807352+0000","last_active":"2024-05-08T04:39:54.890040+0000","last_peered":"2024-05-08T04:39:54.890040+0000","last_clean":"2024-05-08T04:39:54.890040+0000","last_became_active":"2024-05-08T04:39:53.805815+0000","last_became_peered":"2024-05-08T04:39:53.805815+0000","last_unstale":"2024-05-08T04:39:54.890040+0000","last_undegraded":"2024-05-08T04:39:54.890040+0000","last_fullsized":"2024-05-08T04:39:54.890040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:00.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.079012+0000","last_change":"2024-05-08T04:39:52.793663+0000","last_active":"2024-05-08T04:39:55.079012+0000","last_peered":"2024-05-08T04:39:55.079012+0000","last_clean":"2024-05-08T04:39:55.079012+0000","last_became_active":"2024-05-08T04:39:52.793418+0000","last_became_peered":"2024-05-08T04:39:52.793418+0000","last_unstale":"2024-05-08T04:39:55.079012+0000","last_undegraded":"2024-05-08T04:39:55.079012+0000","last_fullsized":"2024-05-08T04:39:55.079012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:56.494420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391575+0000","last_change":"2024-05-08T04:39:53.813724+0000","last_active":"2024-05-08T04:39:55.391575+0000","last_peered":"2024-05-08T04:39:55.391575+0000","last_clean":"2024-05-08T04:39:55.391575+0000","last_became_active":"2024-05-08T04:39:53.813460+0000","last_became_peered":"2024-05-08T04:39:53.813460+0000","last_unstale":"2024-05-08T04:39:55.391575+0000","last_undegraded":"2024-05-08T04:39:55.391575+0000","last_fullsized":"2024-05-08T04:39:55.391575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:15.797885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792868+0000","last_change":"2024-05-08T04:39:52.784560+0000","last_active":"2024-05-08T04:39:54.792868+0000","last_peered":"2024-05-08T04:39:54.792868+0000","last_clean":"2024-05-08T04:39:54.792868+0000","last_became_active":"2024-05-08T04:39:52.784278+0000","last_became_peered":"2024-05-08T04:39:52.784278+0000","last_unstale":"2024-05-08T04:39:54.792868+0000","last_undegraded":"2024-05-08T04:39:54.792868+0000","last_fullsized":"2024-05-08T04:39:54.792868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:20.129176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792561+0000","last_change":"2024-05-08T04:39:53.801684+0000","last_active":"2024-05-08T04:39:54.792561+0000","last_peered":"2024-05-08T04:39:54.792561+0000","last_clean":"2024-05-08T04:39:54.792561+0000","last_became_active":"2024-05-08T04:39:53.801412+0000","last_became_peered":"2024-05-08T04:39:53.801412+0000","last_unstale":"2024-05-08T04:39:54.792561+0000","last_undegraded":"2024-05-08T04:39:54.792561+0000","last_fullsized":"2024-05-08T04:39:54.792561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:58.578402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.889968+0000","last_change":"2024-05-08T04:39:52.784200+0000","last_active":"2024-05-08T04:39:54.889968+0000","last_peered":"2024-05-08T04:39:54.889968+0000","last_clean":"2024-05-08T04:39:54.889968+0000","last_became_active":"2024-05-08T04:39:52.783808+0000","last_became_peered":"2024-05-08T04:39:52.783808+0000","last_unstale":"2024-05-08T04:39:54.889968+0000","last_undegraded":"2024-05-08T04:39:54.889968+0000","last_fullsized":"2024-05-08T04:39:54.889968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:21.432766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.890051+0000","last_change":"2024-05-08T04:39:53.810698+0000","last_active":"2024-05-08T04:39:54.890051+0000","last_peered":"2024-05-08T04:39:54.890051+0000","last_clean":"2024-05-08T04:39:54.890051+0000","last_became_active":"2024-05-08T04:39:53.810426+0000","last_became_peered":"2024-05-08T04:39:53.810426+0000","last_unstale":"2024-05-08T04:39:54.890051+0000","last_undegraded":"2024-05-08T04:39:54.890051+0000","last_fullsized":"2024-05-08T04:39:54.890051+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:16.943540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917018+0000","last_change":"2024-05-08T04:39:52.789662+0000","last_active":"2024-05-08T04:39:54.917018+0000","last_peered":"2024-05-08T04:39:54.917018+0000","last_clean":"2024-05-08T04:39:54.917018+0000","last_became_active":"2024-05-08T04:39:52.788436+0000","last_became_peered":"2024-05-08T04:39:52.788436+0000","last_unstale":"2024-05-08T04:39:54.917018+0000","last_undegraded":"2024-05-08T04:39:54.917018+0000","last_fullsized":"2024-05-08T04:39:54.917018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:18.133568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792070+0000","last_change":"2024-05-08T04:39:53.811549+0000","last_active":"2024-05-08T04:39:54.792070+0000","last_peered":"2024-05-08T04:39:54.792070+0000","last_clean":"2024-05-08T04:39:54.792070+0000","last_became_active":"2024-05-08T04:39:53.811197+0000","last_became_peered":"2024-05-08T04:39:53.811197+0000","last_unstale":"2024-05-08T04:39:54.792070+0000","last_undegraded":"2024-05-08T04:39:54.792070+0000","last_fullsized":"2024-05-08T04:39:54.792070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:00.379935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078839+0000","last_change":"2024-05-08T04:39:52.790536+0000","last_active":"2024-05-08T04:39:55.078839+0000","last_peered":"2024-05-08T04:39:55.078839+0000","last_clean":"2024-05-08T04:39:55.078839+0000","last_became_active":"2024-05-08T04:39:52.790330+0000","last_became_peered":"2024-05-08T04:39:52.790330+0000","last_unstale":"2024-05-08T04:39:55.078839+0000","last_undegraded":"2024-05-08T04:39:55.078839+0000","last_fullsized":"2024-05-08T04:39:55.078839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:28.190200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390880+0000","last_change":"2024-05-08T04:39:53.812799+0000","last_active":"2024-05-08T04:39:55.390880+0000","last_peered":"2024-05-08T04:39:55.390880+0000","last_clean":"2024-05-08T04:39:55.390880+0000","last_became_active":"2024-05-08T04:39:53.812361+0000","last_became_peered":"2024-05-08T04:39:53.812361+0000","last_unstale":"2024-05-08T04:39:55.390880+0000","last_undegraded":"2024-05-08T04:39:55.390880+0000","last_fullsized":"2024-05-08T04:39:55.390880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:50.217570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792133+0000","last_change":"2024-05-08T04:39:52.777727+0000","last_active":"2024-05-08T04:39:54.792133+0000","last_peered":"2024-05-08T04:39:54.792133+0000","last_clean":"2024-05-08T04:39:54.792133+0000","last_became_active":"2024-05-08T04:39:52.777435+0000","last_became_peered":"2024-05-08T04:39:52.777435+0000","last_unstale":"2024-05-08T04:39:54.792133+0000","last_undegraded":"2024-05-08T04:39:54.792133+0000","last_fullsized":"2024-05-08T04:39:54.792133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:32:21.132313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390850+0000","last_change":"2024-05-08T04:39:53.803730+0000","last_active":"2024-05-08T04:39:55.390850+0000","last_peered":"2024-05-08T04:39:55.390850+0000","last_clean":"2024-05-08T04:39:55.390850+0000","last_became_active":"2024-05-08T04:39:53.803483+0000","last_became_peered":"2024-05-08T04:39:53.803483+0000","last_unstale":"2024-05-08T04:39:55.390850+0000","last_undegraded":"2024-05-08T04:39:55.390850+0000","last_fullsized":"2024-05-08T04:39:55.390850+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:38.707114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.791845+0000","last_change":"2024-05-08T04:39:52.786592+0000","last_active":"2024-05-08T04:39:54.791845+0000","last_peered":"2024-05-08T04:39:54.791845+0000","last_clean":"2024-05-08T04:39:54.791845+0000","last_became_active":"2024-05-08T04:39:52.786290+0000","last_became_peered":"2024-05-08T04:39:52.786290+0000","last_unstale":"2024-05-08T04:39:54.791845+0000","last_undegraded":"2024-05-08T04:39:54.791845+0000","last_fullsized":"2024-05-08T04:39:54.791845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:15.934871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391119+0000","last_change":"2024-05-08T04:39:53.806736+0000","last_active":"2024-05-08T04:39:55.391119+0000","last_peered":"2024-05-08T04:39:55.391119+0000","last_clean":"2024-05-08T04:39:55.391119+0000","last_became_active":"2024-05-08T04:39:53.806432+0000","last_became_peered":"2024-05-08T04:39:53.806432+0000","last_unstale":"2024-05-08T04:39:55.391119+0000","last_undegraded":"2024-05-08T04:39:55.391119+0000","last_fullsized":"2024-05-08T04:39:55.391119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:14.979877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390735+0000","last_change":"2024-05-08T04:39:52.792289+0000","last_active":"2024-05-08T04:39:55.390735+0000","last_peered":"2024-05-08T04:39:55.390735+0000","last_clean":"2024-05-08T04:39:55.390735+0000","last_became_active":"2024-05-08T04:39:52.792017+0000","last_became_peered":"2024-05-08T04:39:52.792017+0000","last_unstale":"2024-05-08T04:39:55.390735+0000","last_undegraded":"2024-05-08T04:39:55.390735+0000","last_fullsized":"2024-05-08T04:39:55.390735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:31.810287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792837+0000","last_change":"2024-05-08T04:39:53.818296+0000","last_active":"2024-05-08T04:39:54.792837+0000","last_peered":"2024-05-08T04:39:54.792837+0000","last_clean":"2024-05-08T04:39:54.792837+0000","last_became_active":"2024-05-08T04:39:53.818026+0000","last_became_peered":"2024-05-08T04:39:53.818026+0000","last_unstale":"2024-05-08T04:39:54.792837+0000","last_undegraded":"2024-05-08T04:39:54.792837+0000","last_fullsized":"2024-05-08T04:39:54.792837+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:45.621255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792576+0000","last_change":"2024-05-08T04:39:52.783223+0000","last_active":"2024-05-08T04:39:54.792576+0000","last_peered":"2024-05-08T04:39:54.792576+0000","last_clean":"2024-05-08T04:39:54.792576+0000","last_became_active":"2024-05-08T04:39:52.782776+0000","last_became_peered":"2024-05-08T04:39:52.782776+0000","last_unstale":"2024-05-08T04:39:54.792576+0000","last_undegraded":"2024-05-08T04:39:54.792576+0000","last_fullsized":"2024-05-08T04:39:54.792576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:04.412071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917322+0000","last_change":"2024-05-08T04:39:53.801526+0000","last_active":"2024-05-08T04:39:54.917322+0000","last_peered":"2024-05-08T04:39:54.917322+0000","last_clean":"2024-05-08T04:39:54.917322+0000","last_became_active":"2024-05-08T04:39:53.801226+0000","last_became_peered":"2024-05-08T04:39:53.801226+0000","last_unstale":"2024-05-08T04:39:54.917322+0000","last_undegraded":"2024-05-08T04:39:54.917322+0000","last_fullsized":"2024-05-08T04:39:54.917322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:46.099431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391574+0000","last_change":"2024-05-08T04:39:52.791688+0000","last_active":"2024-05-08T04:39:55.391574+0000","last_peered":"2024-05-08T04:39:55.391574+0000","last_clean":"2024-05-08T04:39:55.391574+0000","last_became_active":"2024-05-08T04:39:52.791442+0000","last_became_peered":"2024-05-08T04:39:52.791442+0000","last_unstale":"2024-05-08T04:39:55.391574+0000","last_undegraded":"2024-05-08T04:39:55.391574+0000","last_fullsized":"2024-05-08T04:39:55.391574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:04.412071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078932+0000","last_change":"2024-05-08T04:39:53.809724+0000","last_active":"2024-05-08T04:39:55.078932+0000","last_peered":"2024-05-08T04:39:55.078932+0000","last_clean":"2024-05-08T04:39:55.078932+0000","last_became_active":"2024-05-08T04:39:53.806586+0000","last_became_peered":"2024-05-08T04:39:53.806586+0000","last_unstale":"2024-05-08T04:39:55.078932+0000","last_undegraded":"2024-05-08T04:39:55.078932+0000","last_fullsized":"2024-05-08T04:39:55.078932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:24.249849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917324+0000","last_change":"2024-05-08T04:39:52.792557+0000","last_active":"2024-05-08T04:39:54.917324+0000","last_peered":"2024-05-08T04:39:54.917324+0000","last_clean":"2024-05-08T04:39:54.917324+0000","last_became_active":"2024-05-08T04:39:52.792294+0000","last_became_peered":"2024-05-08T04:39:52.792294+0000","last_unstale":"2024-05-08T04:39:54.917324+0000","last_undegraded":"2024-05-08T04:39:54.917324+0000","last_fullsized":"2024-05-08T04:39:54.917324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:44.612585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917245+0000","last_change":"2024-05-08T04:39:53.803549+0000","last_active":"2024-05-08T04:39:54.917245+0000","last_peered":"2024-05-08T04:39:54.917245+0000","last_clean":"2024-05-08T04:39:54.917245+0000","last_became_active":"2024-05-08T04:39:53.803232+0000","last_became_peered":"2024-05-08T04:39:53.803232+0000","last_unstale":"2024-05-08T04:39:54.917245+0000","last_undegraded":"2024-05-08T04:39:54.917245+0000","last_fullsized":"2024-05-08T04:39:54.917245+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:33.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786231+0000","last_change":"2024-05-08T04:39:52.781227+0000","last_active":"2024-05-08T04:39:54.786231+0000","last_peered":"2024-05-08T04:39:54.786231+0000","last_clean":"2024-05-08T04:39:54.786231+0000","last_became_active":"2024-05-08T04:39:52.780729+0000","last_became_peered":"2024-05-08T04:39:52.780729+0000","last_unstale":"2024-05-08T04:39:54.786231+0000","last_undegraded":"2024-05-08T04:39:54.786231+0000","last_fullsized":"2024-05-08T04:39:54.786231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:52.898436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391276+0000","last_change":"2024-05-08T04:39:53.821179+0000","last_active":"2024-05-08T04:39:55.391276+0000","last_peered":"2024-05-08T04:39:55.391276+0000","last_clean":"2024-05-08T04:39:55.391276+0000","last_became_active":"2024-05-08T04:39:53.818386+0000","last_became_peered":"2024-05-08T04:39:53.818386+0000","last_unstale":"2024-05-08T04:39:55.391276+0000","last_undegraded":"2024-05-08T04:39:55.391276+0000","last_fullsized":"2024-05-08T04:39:55.391276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:51.379243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078845+0000","last_change":"2024-05-08T04:39:52.795529+0000","last_active":"2024-05-08T04:39:55.078845+0000","last_peered":"2024-05-08T04:39:55.078845+0000","last_clean":"2024-05-08T04:39:55.078845+0000","last_became_active":"2024-05-08T04:39:52.795257+0000","last_became_peered":"2024-05-08T04:39:52.795257+0000","last_unstale":"2024-05-08T04:39:55.078845+0000","last_undegraded":"2024-05-08T04:39:55.078845+0000","last_fullsized":"2024-05-08T04:39:55.078845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:18.096035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391002+0000","last_change":"2024-05-08T04:39:53.812599+0000","last_active":"2024-05-08T04:39:55.391002+0000","last_peered":"2024-05-08T04:39:55.391002+0000","last_clean":"2024-05-08T04:39:55.391002+0000","last_became_active":"2024-05-08T04:39:53.812047+0000","last_became_peered":"2024-05-08T04:39:53.812047+0000","last_unstale":"2024-05-08T04:39:55.391002+0000","last_undegraded":"2024-05-08T04:39:55.391002+0000","last_fullsized":"2024-05-08T04:39:55.391002+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:57.503090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.916938+0000","last_change":"2024-05-08T04:39:52.791194+0000","last_active":"2024-05-08T04:39:54.916938+0000","last_peered":"2024-05-08T04:39:54.916938+0000","last_clean":"2024-05-08T04:39:54.916938+0000","last_became_active":"2024-05-08T04:39:52.790236+0000","last_became_peered":"2024-05-08T04:39:52.790236+0000","last_unstale":"2024-05-08T04:39:54.916938+0000","last_undegraded":"2024-05-08T04:39:54.916938+0000","last_fullsized":"2024-05-08T04:39:54.916938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:15.934871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078592+0000","last_change":"2024-05-08T04:39:53.806389+0000","last_active":"2024-05-08T04:39:55.078592+0000","last_peered":"2024-05-08T04:39:55.078592+0000","last_clean":"2024-05-08T04:39:55.078592+0000","last_became_active":"2024-05-08T04:39:53.806097+0000","last_became_peered":"2024-05-08T04:39:53.806097+0000","last_unstale":"2024-05-08T04:39:55.078592+0000","last_undegraded":"2024-05-08T04:39:55.078592+0000","last_fullsized":"2024-05-08T04:39:55.078592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:44:37.997823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078640+0000","last_change":"2024-05-08T04:39:52.794897+0000","last_active":"2024-05-08T04:39:55.078640+0000","last_peered":"2024-05-08T04:39:55.078640+0000","last_clean":"2024-05-08T04:39:55.078640+0000","last_became_active":"2024-05-08T04:39:52.794552+0000","last_became_peered":"2024-05-08T04:39:52.794552+0000","last_unstale":"2024-05-08T04:39:55.078640+0000","last_undegraded":"2024-05-08T04:39:55.078640+0000","last_fullsized":"2024-05-08T04:39:55.078640+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:39.254346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917039+0000","last_change":"2024-05-08T04:39:53.800058+0000","last_active":"2024-05-08T04:39:54.917039+0000","last_peered":"2024-05-08T04:39:54.917039+0000","last_clean":"2024-05-08T04:39:54.917039+0000","last_became_active":"2024-05-08T04:39:53.799788+0000","last_became_peered":"2024-05-08T04:39:53.799788+0000","last_unstale":"2024-05-08T04:39:54.917039+0000","last_undegraded":"2024-05-08T04:39:54.917039+0000","last_fullsized":"2024-05-08T04:39:54.917039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:58.578402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390912+0000","last_change":"2024-05-08T04:39:52.791942+0000","last_active":"2024-05-08T04:39:55.390912+0000","last_peered":"2024-05-08T04:39:55.390912+0000","last_clean":"2024-05-08T04:39:55.390912+0000","last_became_active":"2024-05-08T04:39:52.791202+0000","last_became_peered":"2024-05-08T04:39:52.791202+0000","last_unstale":"2024-05-08T04:39:55.390912+0000","last_undegraded":"2024-05-08T04:39:55.390912+0000","last_fullsized":"2024-05-08T04:39:55.390912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:32:21.132313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786478+0000","last_change":"2024-05-08T04:39:53.792272+0000","last_active":"2024-05-08T04:39:54.786478+0000","last_peered":"2024-05-08T04:39:54.786478+0000","last_clean":"2024-05-08T04:39:54.786478+0000","last_became_active":"2024-05-08T04:39:53.791856+0000","last_became_peered":"2024-05-08T04:39:53.791856+0000","last_unstale":"2024-05-08T04:39:54.786478+0000","last_undegraded":"2024-05-08T04:39:54.786478+0000","last_fullsized":"2024-05-08T04:39:54.786478+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:13:41.697325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786524+0000","last_change":"2024-05-08T04:39:52.787694+0000","last_active":"2024-05-08T04:39:54.786524+0000","last_peered":"2024-05-08T04:39:54.786524+0000","last_clean":"2024-05-08T04:39:54.786524+0000","last_became_active":"2024-05-08T04:39:52.787311+0000","last_became_peered":"2024-05-08T04:39:52.787311+0000","last_unstale":"2024-05-08T04:39:54.786524+0000","last_undegraded":"2024-05-08T04:39:54.786524+0000","last_fullsized":"2024-05-08T04:39:54.786524+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:33.714875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391556+0000","last_change":"2024-05-08T04:39:53.818136+0000","last_active":"2024-05-08T04:39:55.391556+0000","last_peered":"2024-05-08T04:39:55.391556+0000","last_clean":"2024-05-08T04:39:55.391556+0000","last_became_active":"2024-05-08T04:39:53.817843+0000","last_became_peered":"2024-05-08T04:39:53.817843+0000","last_unstale":"2024-05-08T04:39:55.391556+0000","last_undegraded":"2024-05-08T04:39:55.391556+0000","last_fullsized":"2024-05-08T04:39:55.391556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:33.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792929+0000","last_change":"2024-05-08T04:39:52.789120+0000","last_active":"2024-05-08T04:39:54.792929+0000","last_peered":"2024-05-08T04:39:54.792929+0000","last_clean":"2024-05-08T04:39:54.792929+0000","last_became_active":"2024-05-08T04:39:52.788836+0000","last_became_peered":"2024-05-08T04:39:52.788836+0000","last_unstale":"2024-05-08T04:39:54.792929+0000","last_undegraded":"2024-05-08T04:39:54.792929+0000","last_fullsized":"2024-05-08T04:39:54.792929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:06.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391404+0000","last_change":"2024-05-08T04:39:53.800081+0000","last_active":"2024-05-08T04:39:55.391404+0000","last_peered":"2024-05-08T04:39:55.391404+0000","last_clean":"2024-05-08T04:39:55.391404+0000","last_became_active":"2024-05-08T04:39:53.799772+0000","last_became_peered":"2024-05-08T04:39:53.799772+0000","last_unstale":"2024-05-08T04:39:55.391404+0000","last_undegraded":"2024-05-08T04:39:55.391404+0000","last_fullsized":"2024-05-08T04:39:55.391404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:22.441436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792083+0000","last_change":"2024-05-08T04:39:52.787362+0000","last_active":"2024-05-08T04:39:54.792083+0000","last_peered":"2024-05-08T04:39:54.792083+0000","last_clean":"2024-05-08T04:39:54.792083+0000","last_became_active":"2024-05-08T04:39:52.786827+0000","last_became_peered":"2024-05-08T04:39:52.786827+0000","last_unstale":"2024-05-08T04:39:54.792083+0000","last_undegraded":"2024-05-08T04:39:54.792083+0000","last_fullsized":"2024-05-08T04:39:54.792083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:59.371266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391566+0000","last_change":"2024-05-08T04:39:53.812212+0000","last_active":"2024-05-08T04:39:55.391566+0000","last_peered":"2024-05-08T04:39:55.391566+0000","last_clean":"2024-05-08T04:39:55.391566+0000","last_became_active":"2024-05-08T04:39:53.811752+0000","last_became_peered":"2024-05-08T04:39:53.811752+0000","last_unstale":"2024-05-08T04:39:55.391566+0000","last_undegraded":"2024-05-08T04:39:55.391566+0000","last_fullsized":"2024-05-08T04:39:55.391566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:46:18.417861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.079023+0000","last_change":"2024-05-08T04:39:52.791567+0000","last_active":"2024-05-08T04:39:55.079023+0000","last_peered":"2024-05-08T04:39:55.079023+0000","last_clean":"2024-05-08T04:39:55.079023+0000","last_became_active":"2024-05-08T04:39:52.791255+0000","last_became_peered":"2024-05-08T04:39:52.791255+0000","last_unstale":"2024-05-08T04:39:55.079023+0000","last_undegraded":"2024-05-08T04:39:55.079023+0000","last_fullsized":"2024-05-08T04:39:55.079023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:19.273415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078620+0000","last_change":"2024-05-08T04:39:53.806861+0000","last_active":"2024-05-08T04:39:55.078620+0000","last_peered":"2024-05-08T04:39:55.078620+0000","last_clean":"2024-05-08T04:39:55.078620+0000","last_became_active":"2024-05-08T04:39:53.806561+0000","last_became_peered":"2024-05-08T04:39:53.806561+0000","last_unstale":"2024-05-08T04:39:55.078620+0000","last_undegraded":"2024-05-08T04:39:55.078620+0000","last_fullsized":"2024-05-08T04:39:55.078620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:04.848748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078989+0000","last_change":"2024-05-08T04:39:52.790744+0000","last_active":"2024-05-08T04:39:55.078989+0000","last_peered":"2024-05-08T04:39:55.078989+0000","last_clean":"2024-05-08T04:39:55.078989+0000","last_became_active":"2024-05-08T04:39:52.790454+0000","last_became_peered":"2024-05-08T04:39:52.790454+0000","last_unstale":"2024-05-08T04:39:55.078989+0000","last_undegraded":"2024-05-08T04:39:55.078989+0000","last_fullsized":"2024-05-08T04:39:55.078989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:17.887966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.889927+0000","last_change":"2024-05-08T04:39:53.801690+0000","last_active":"2024-05-08T04:39:54.889927+0000","last_peered":"2024-05-08T04:39:54.889927+0000","last_clean":"2024-05-08T04:39:54.889927+0000","last_became_active":"2024-05-08T04:39:53.801369+0000","last_became_peered":"2024-05-08T04:39:53.801369+0000","last_unstale":"2024-05-08T04:39:54.889927+0000","last_undegraded":"2024-05-08T04:39:54.889927+0000","last_fullsized":"2024-05-08T04:39:54.889927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:38.866308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.791858+0000","last_change":"2024-05-08T04:39:52.786376+0000","last_active":"2024-05-08T04:39:54.791858+0000","last_peered":"2024-05-08T04:39:54.791858+0000","last_clean":"2024-05-08T04:39:54.791858+0000","last_became_active":"2024-05-08T04:39:52.786038+0000","last_became_peered":"2024-05-08T04:39:52.786038+0000","last_unstale":"2024-05-08T04:39:54.791858+0000","last_undegraded":"2024-05-08T04:39:54.791858+0000","last_fullsized":"2024-05-08T04:39:54.791858+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:37.857638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.786187+0000","last_change":"2024-05-08T04:39:53.791489+0000","last_active":"2024-05-08T04:39:54.786187+0000","last_peered":"2024-05-08T04:39:54.786187+0000","last_clean":"2024-05-08T04:39:54.786187+0000","last_became_active":"2024-05-08T04:39:53.791222+0000","last_became_peered":"2024-05-08T04:39:53.791222+0000","last_unstale":"2024-05-08T04:39:54.786187+0000","last_undegraded":"2024-05-08T04:39:54.786187+0000","last_fullsized":"2024-05-08T04:39:54.786187+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:54.317372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390698+0000","last_change":"2024-05-08T04:39:52.792462+0000","last_active":"2024-05-08T04:39:55.390698+0000","last_peered":"2024-05-08T04:39:55.390698+0000","last_clean":"2024-05-08T04:39:55.390698+0000","last_became_active":"2024-05-08T04:39:52.792153+0000","last_became_peered":"2024-05-08T04:39:52.792153+0000","last_unstale":"2024-05-08T04:39:55.390698+0000","last_undegraded":"2024-05-08T04:39:55.390698+0000","last_fullsized":"2024-05-08T04:39:55.390698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:20.129176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.390832+0000","last_change":"2024-05-08T04:39:53.813591+0000","last_active":"2024-05-08T04:39:55.390832+0000","last_peered":"2024-05-08T04:39:55.390832+0000","last_clean":"2024-05-08T04:39:55.390832+0000","last_became_active":"2024-05-08T04:39:53.813265+0000","last_became_peered":"2024-05-08T04:39:53.813265+0000","last_unstale":"2024-05-08T04:39:55.390832+0000","last_undegraded":"2024-05-08T04:39:55.390832+0000","last_fullsized":"2024-05-08T04:39:55.390832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:45.621255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.916684+0000","last_change":"2024-05-08T04:39:52.797075+0000","last_active":"2024-05-08T04:39:54.916684+0000","last_peered":"2024-05-08T04:39:54.916684+0000","last_clean":"2024-05-08T04:39:54.916684+0000","last_became_active":"2024-05-08T04:39:52.796866+0000","last_became_peered":"2024-05-08T04:39:52.796866+0000","last_unstale":"2024-05-08T04:39:54.916684+0000","last_undegraded":"2024-05-08T04:39:54.916684+0000","last_fullsized":"2024-05-08T04:39:54.916684+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:47.502667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391707+0000","last_change":"2024-05-08T04:39:53.819654+0000","last_active":"2024-05-08T04:39:55.391707+0000","last_peered":"2024-05-08T04:39:55.391707+0000","last_clean":"2024-05-08T04:39:55.391707+0000","last_became_active":"2024-05-08T04:39:53.819310+0000","last_became_peered":"2024-05-08T04:39:53.819310+0000","last_unstale":"2024-05-08T04:39:55.391707+0000","last_undegraded":"2024-05-08T04:39:55.391707+0000","last_fullsized":"2024-05-08T04:39:55.391707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:16.943540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078924+0000","last_change":"2024-05-08T04:39:52.789659+0000","last_active":"2024-05-08T04:39:55.078924+0000","last_peered":"2024-05-08T04:39:55.078924+0000","last_clean":"2024-05-08T04:39:55.078924+0000","last_became_active":"2024-05-08T04:39:52.789348+0000","last_became_peered":"2024-05-08T04:39:52.789348+0000","last_unstale":"2024-05-08T04:39:55.078924+0000","last_undegraded":"2024-05-08T04:39:55.078924+0000","last_fullsized":"2024-05-08T04:39:55.078924+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:37.830479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391134+0000","last_change":"2024-05-08T04:39:53.811237+0000","last_active":"2024-05-08T04:39:55.391134+0000","last_peered":"2024-05-08T04:39:55.391134+0000","last_clean":"2024-05-08T04:39:55.391134+0000","last_became_active":"2024-05-08T04:39:53.809637+0000","last_became_peered":"2024-05-08T04:39:53.809637+0000","last_unstale":"2024-05-08T04:39:55.391134+0000","last_undegraded":"2024-05-08T04:39:55.391134+0000","last_fullsized":"2024-05-08T04:39:55.391134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:20.282084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917355+0000","last_change":"2024-05-08T04:39:52.793162+0000","last_active":"2024-05-08T04:39:54.917355+0000","last_peered":"2024-05-08T04:39:54.917355+0000","last_clean":"2024-05-08T04:39:54.917355+0000","last_became_active":"2024-05-08T04:39:52.792881+0000","last_became_peered":"2024-05-08T04:39:52.792881+0000","last_unstale":"2024-05-08T04:39:54.917355+0000","last_undegraded":"2024-05-08T04:39:54.917355+0000","last_fullsized":"2024-05-08T04:39:54.917355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:37.698445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917313+0000","last_change":"2024-05-08T04:39:53.803497+0000","last_active":"2024-05-08T04:39:54.917313+0000","last_peered":"2024-05-08T04:39:54.917313+0000","last_clean":"2024-05-08T04:39:54.917313+0000","last_became_active":"2024-05-08T04:39:53.803197+0000","last_became_peered":"2024-05-08T04:39:53.803197+0000","last_unstale":"2024-05-08T04:39:54.917313+0000","last_undegraded":"2024-05-08T04:39:54.917313+0000","last_fullsized":"2024-05-08T04:39:54.917313+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:13.140775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.889789+0000","last_change":"2024-05-08T04:39:52.785377+0000","last_active":"2024-05-08T04:39:54.889789+0000","last_peered":"2024-05-08T04:39:54.889789+0000","last_clean":"2024-05-08T04:39:54.889789+0000","last_became_active":"2024-05-08T04:39:52.785121+0000","last_became_peered":"2024-05-08T04:39:52.785121+0000","last_unstale":"2024-05-08T04:39:54.889789+0000","last_undegraded":"2024-05-08T04:39:54.889789+0000","last_fullsized":"2024-05-08T04:39:54.889789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:24.650176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078814+0000","last_change":"2024-05-08T04:39:53.806258+0000","last_active":"2024-05-08T04:39:55.078814+0000","last_peered":"2024-05-08T04:39:55.078814+0000","last_clean":"2024-05-08T04:39:55.078814+0000","last_became_active":"2024-05-08T04:39:53.805447+0000","last_became_peered":"2024-05-08T04:39:53.805447+0000","last_unstale":"2024-05-08T04:39:55.078814+0000","last_undegraded":"2024-05-08T04:39:55.078814+0000","last_fullsized":"2024-05-08T04:39:55.078814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:30:11.272719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.917055+0000","last_change":"2024-05-08T04:39:52.793413+0000","last_active":"2024-05-08T04:39:54.917055+0000","last_peered":"2024-05-08T04:39:54.917055+0000","last_clean":"2024-05-08T04:39:54.917055+0000","last_became_active":"2024-05-08T04:39:52.792948+0000","last_became_peered":"2024-05-08T04:39:52.792948+0000","last_unstale":"2024-05-08T04:39:54.917055+0000","last_undegraded":"2024-05-08T04:39:54.917055+0000","last_fullsized":"2024-05-08T04:39:54.917055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:52.165559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.078764+0000","last_change":"2024-05-08T04:39:53.806851+0000","last_active":"2024-05-08T04:39:55.078764+0000","last_peered":"2024-05-08T04:39:55.078764+0000","last_clean":"2024-05-08T04:39:55.078764+0000","last_became_active":"2024-05-08T04:39:53.806534+0000","last_became_peered":"2024-05-08T04:39:53.806534+0000","last_unstale":"2024-05-08T04:39:55.078764+0000","last_undegraded":"2024-05-08T04:39:55.078764+0000","last_fullsized":"2024-05-08T04:39:55.078764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:50.953605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792257+0000","last_change":"2024-05-08T04:39:52.790536+0000","last_active":"2024-05-08T04:39:54.792257+0000","last_peered":"2024-05-08T04:39:54.792257+0000","last_clean":"2024-05-08T04:39:54.792257+0000","last_became_active":"2024-05-08T04:39:52.790213+0000","last_became_peered":"2024-05-08T04:39:52.790213+0000","last_unstale":"2024-05-08T04:39:54.792257+0000","last_undegraded":"2024-05-08T04:39:54.792257+0000","last_fullsized":"2024-05-08T04:39:54.792257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:06.088237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.791980+0000","last_change":"2024-05-08T04:39:53.802710+0000","last_active":"2024-05-08T04:39:54.791980+0000","last_peered":"2024-05-08T04:39:54.791980+0000","last_clean":"2024-05-08T04:39:54.791980+0000","last_became_active":"2024-05-08T04:39:53.802435+0000","last_became_peered":"2024-05-08T04:39:53.802435+0000","last_unstale":"2024-05-08T04:39:54.791980+0000","last_undegraded":"2024-05-08T04:39:54.791980+0000","last_fullsized":"2024-05-08T04:39:54.791980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:20.282084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.889762+0000","last_change":"2024-05-08T04:39:52.793284+0000","last_active":"2024-05-08T04:39:54.889762+0000","last_peered":"2024-05-08T04:39:54.889762+0000","last_clean":"2024-05-08T04:39:54.889762+0000","last_became_active":"2024-05-08T04:39:52.792992+0000","last_became_peered":"2024-05-08T04:39:52.792992+0000","last_unstale":"2024-05-08T04:39:54.889762+0000","last_undegraded":"2024-05-08T04:39:54.889762+0000","last_fullsized":"2024-05-08T04:39:54.889762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:50.370574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.791958+0000","last_change":"2024-05-08T04:39:53.807399+0000","last_active":"2024-05-08T04:39:54.791958+0000","last_peered":"2024-05-08T04:39:54.791958+0000","last_clean":"2024-05-08T04:39:54.791958+0000","last_became_active":"2024-05-08T04:39:53.806906+0000","last_became_peered":"2024-05-08T04:39:53.806906+0000","last_unstale":"2024-05-08T04:39:54.791958+0000","last_undegraded":"2024-05-08T04:39:54.791958+0000","last_fullsized":"2024-05-08T04:39:54.791958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:38.866308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391106+0000","last_change":"2024-05-08T04:39:52.791765+0000","last_active":"2024-05-08T04:39:55.391106+0000","last_peered":"2024-05-08T04:39:55.391106+0000","last_clean":"2024-05-08T04:39:55.391106+0000","last_became_active":"2024-05-08T04:39:52.791512+0000","last_became_peered":"2024-05-08T04:39:52.791512+0000","last_unstale":"2024-05-08T04:39:55.391106+0000","last_undegraded":"2024-05-08T04:39:55.391106+0000","last_fullsized":"2024-05-08T04:39:55.391106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:06.088237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:55.391533+0000","last_change":"2024-05-08T04:39:53.815551+0000","last_active":"2024-05-08T04:39:55.391533+0000","last_peered":"2024-05-08T04:39:55.391533+0000","last_clean":"2024-05-08T04:39:55.391533+0000","last_became_active":"2024-05-08T04:39:53.812950+0000","last_became_peered":"2024-05-08T04:39:53.812950+0000","last_unstale":"2024-05-08T04:39:55.391533+0000","last_undegraded":"2024-05-08T04:39:55.391533+0000","last_fullsized":"2024-05-08T04:39:55.391533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:52.761450+0000","last_clean_scrub_stamp":"2024-05-08T04:39:52.761450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:58.578402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.792396+0000","last_change":"2024-05-08T04:39:52.787496+0000","last_active":"2024-05-08T04:39:54.792396+0000","last_peered":"2024-05-08T04:39:54.792396+0000","last_clean":"2024-05-08T04:39:54.792396+0000","last_became_active":"2024-05-08T04:39:52.787208+0000","last_became_peered":"2024-05-08T04:39:52.787208+0000","last_unstale":"2024-05-08T04:39:54.792396+0000","last_undegraded":"2024-05-08T04:39:54.792396+0000","last_fullsized":"2024-05-08T04:39:54.792396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:39:51.752781+0000","last_clean_scrub_stamp":"2024-05-08T04:39:51.752781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:52.165559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T04:39:54.916705+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:39:54.916705+0000","last_peered":"2024-05-08T04:39:54.916705+0000","last_clean":"2024-05-08T04:39:54.916705+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:39:54.916705+0000","last_undegraded":"2024-05-08T04:39:54.916705+0000","last_fullsized":"2024-05-08T04:39:54.916705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":215761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:40:02.752 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:40:02.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:40:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 Processor -- start 2024-05-08T04:40:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 -- start start 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 0x7f493006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49300ae6e0 0x7f49300aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493015e870 con 0x7f49300ce780 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006bb80 con 0x7f49300ae6e0 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4936800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493015eb50 con 0x7f493015ab10 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49300ae6e0 0x7f49300aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49300ae6e0 0x7f49300aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37974/0 (socket says 172.21.15.164:37974) 2024-05-08T04:40:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 -- 172.21.15.164:0/1234631501 learned_addr learned my addr 172.21.15.164:0/1234631501 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f492f400640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 0x7f493006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 -- 172.21.15.164:0/1234631501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 msgr2=0x7f493006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 0x7f493006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 -- 172.21.15.164:0/1234631501 >> v1:172.21.15.172:6789/0 conn(0x7f49300ce780 legacy=0x7f49300c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f4926a00640 1 -- 172.21.15.164:0/1234631501 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493015ee30 con 0x7f49300ae6e0 2024-05-08T04:40:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.933+0000 7f492f400640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 0x7f493006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4926a00640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ae6e0 0x7f49300aeb40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f49180098a0 tx=0x7f491801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63242e942c4ce0b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f492ea00640 1 -- 172.21.15.164:0/1234631501 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180ab600 con 0x7f49300ae6e0 2024-05-08T04:40:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f492ea00640 1 -- 172.21.15.164:0/1234631501 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180b0070 con 0x7f49300ae6e0 2024-05-08T04:40:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f492ea00640 1 -- 172.21.15.164:0/1234631501 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180ab9c0 con 0x7f49300ae6e0 2024-05-08T04:40:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 -- 172.21.15.164:0/1234631501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ae6e0 msgr2=0x7f49300aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ae6e0 0x7f49300aeb40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f49180098a0 tx=0x7f491801c440 comp rx=0 tx=0).stop 2024-05-08T04:40:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 -- 172.21.15.164:0/1234631501 shutdown_connections 2024-05-08T04:40:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ae6e0 0x7f49300aeb40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 --2- 172.21.15.164:0/1234631501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f493015ab10 0x7f493006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 -- 172.21.15.164:0/1234631501 >> 172.21.15.164:0/1234631501 conn(0x7f49300cf770 msgr2=0x7f49300cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 -- 172.21.15.164:0/1234631501 shutdown_connections 2024-05-08T04:40:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 -- 172.21.15.164:0/1234631501 wait complete. 2024-05-08T04:40:02.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.937+0000 7f4936800640 1 Processor -- start 2024-05-08T04:40:02.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- start start 2024-05-08T04:40:02.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49300ae6e0 0x7f4930153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f493015ab10 0x7f4930147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930165ad0 con 0x7f49300ce780 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493015aef0 con 0x7f493015ab10 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4930165c50 con 0x7f49300ae6e0 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37984/0 (socket says 172.21.15.164:37984) 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 -- 172.21.15.164:0/3565711454 learned_addr learned my addr 172.21.15.164:0/3565711454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f492f400640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49300ae6e0 0x7f4930153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f492fe00640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f493015ab10 0x7f4930147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 -- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49300ae6e0 msgr2=0x7f4930153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49300ae6e0 0x7f4930153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 -- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f493015ab10 msgr2=0x7f4930147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:02.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f493015ab10 0x7f4930147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:02.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 -- 172.21.15.164:0/3565711454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930156c10 con 0x7f49300ce780 2024-05-08T04:40:02.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4926a00640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f491801aaf0 tx=0x7f491801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180b4800 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180abe50 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180b4c00 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4930156930 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493014f620 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.941+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493014f1a0 con 0x7f49300ce780 2024-05-08T04:40:02.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.945+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f49180aa070 con 0x7f49300ce780 2024-05-08T04:40:02.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.945+0000 7f4927e00640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 0x7f490c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:02.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.945+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f491811f5a0 con 0x7f49300ce780 2024-05-08T04:40:02.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.945+0000 7f492f400640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 0x7f490c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.945+0000 7f492f400640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 0x7f490c0694b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4920006fd0 tx=0x7f4920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:02.949+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49180c4060 con 0x7f49300ce780 2024-05-08T04:40:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.157+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f493014f1a0 con 0x7f49300ce780 2024-05-08T04:40:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.161+0000 7f4927e00640 1 -- 172.21.15.164:0/3565711454 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f49180160f0 con 0x7f49300ce780 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:40:03.167 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:40:03.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 msgr2=0x7f490c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 0x7f490c0694b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4920006fd0 tx=0x7f4920008040 comp rx=0 tx=0).stop 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 msgr2=0x7f4930166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f491801aaf0 tx=0x7f491801ac80 comp rx=0 tx=0).stop 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 shutdown_connections 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f490c066ff0 0x7f490c0694b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f493015ab10 0x7f4930147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49300ce780 0x7f4930166ae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 --2- 172.21.15.164:0/3565711454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49300ae6e0 0x7f4930153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 >> 172.21.15.164:0/3565711454 conn(0x7f49300cf770 msgr2=0x7f49300cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:03.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 shutdown_connections 2024-05-08T04:40:03.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.165+0000 7f4936800640 1 -- 172.21.15.164:0/3565711454 wait complete. 2024-05-08T04:40:03.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:03.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 Processor -- start 2024-05-08T04:40:03.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 -- start start 2024-05-08T04:40:03.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa80815ab60 0x7fa808072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa808072640 con 0x7fa8080cfda0 2024-05-08T04:40:03.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80806c770 con 0x7fa8080b26c0 2024-05-08T04:40:03.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa80e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80806ca50 con 0x7fa80815ab60 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa8080cfda0 0x7fa8080cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60448/0 (socket says 172.21.15.164:60448) 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807e00640 1 -- 172.21.15.164:0/2798867857 learned_addr learned my addr 172.21.15.164:0/2798867857 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807400640 1 --2- 172.21.15.164:0/2798867857 >> v2:172.21.15.172:3300/0 conn(0x7fa80815ab60 0x7fa808072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839747253 0 0) 0x7fa808072640 con 0x7fa8080cfda0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec005740 con 0x7fa8080cfda0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496787169 0 0) 0x7fa7ec005740 con 0x7fa8080cfda0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 >> v2:172.21.15.172:3300/0 conn(0x7fa80815ab60 msgr2=0x7fa808072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 --2- 172.21.15.164:0/2798867857 >> v2:172.21.15.172:3300/0 conn(0x7fa80815ab60 0x7fa808072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 >> v1:172.21.15.172:6789/0 conn(0x7fa8080b26c0 legacy=0x7fa8080cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80806cd30 con 0x7fa8080cfda0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa800001a00 con 0x7fa8080cfda0 2024-05-08T04:40:03.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 >> v1:172.21.15.164:6789/0 conn(0x7fa8080cfda0 legacy=0x7fa8080cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7ec005bd0 0x7fa7ec005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa7ec006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7ec0073e0 0x7fa7ec00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80806ca50 con 0x7fa7ec006a20 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00e230 con 0x7fa7ec005bd0 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00e430 con 0x7fa7ec0073e0 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa7ec006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807400640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7ec005bd0 0x7fa7ec005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807e00640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7ec0073e0 0x7fa7ec00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 -- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7ec0073e0 msgr2=0x7fa7ec00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7ec0073e0 0x7fa7ec00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 -- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7ec005bd0 msgr2=0x7fa7ec005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7ec005bd0 0x7fa7ec005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa806a00640 1 -- 172.21.15.164:0/2798867857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec005740 con 0x7fa7ec006a20 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807e00640 1 -- 172.21.15.164:0/2798867857 reap_dead start 2024-05-08T04:40:03.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.321+0000 7fa807400640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7ec005bd0 0x7fa7ec005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa806a00640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa7ec006ea0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001860 tx=0x7fa7f401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c536d9f39365f5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40b0070 con 0x7fa7ec006a20 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40ab600 con 0x7fa7ec006a20 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa806000640 1 -- 172.21.15.164:0/2798867857 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4001460 con 0x7fa7ec006a20 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 -- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 msgr2=0x7fa7ec006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa7ec006ea0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001860 tx=0x7fa7f401c4e0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 -- 172.21.15.164:0/2798867857 shutdown_connections 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 --2- 172.21.15.164:0/2798867857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa7ec006ea0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 -- 172.21.15.164:0/2798867857 >> 172.21.15.164:0/2798867857 conn(0x7fa80806ac90 msgr2=0x7fa80806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:03.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 -- 172.21.15.164:0/2798867857 shutdown_connections 2024-05-08T04:40:03.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.325+0000 7fa80e000640 1 -- 172.21.15.164:0/2798867857 wait complete. 2024-05-08T04:40:03.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 Processor -- start 2024-05-08T04:40:03.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- start start 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa80814fcd0 0x7fa808152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 0x7fa8081fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080cfc20 con 0x7fa7ec006a20 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080cfaa0 con 0x7fa80814fcd0 2024-05-08T04:40:03.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080aed30 con 0x7fa808151200 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38000/0 (socket says 172.21.15.164:38000) 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 -- 172.21.15.164:0/3442397441 learned_addr learned my addr 172.21.15.164:0/3442397441 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa806a00640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa80814fcd0 0x7fa808152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807e00640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 0x7fa8081fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 -- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 msgr2=0x7fa8081fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 0x7fa8081fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 -- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa80814fcd0 msgr2=0x7fa808152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa80814fcd0 0x7fa808152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 -- 172.21.15.164:0/3442397441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808150160 con 0x7fa7ec006a20 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807e00640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 0x7fa8081fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:03.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa807400640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8011bd0 tx=0x7fa7f800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80096b0 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f8011cc0 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa808150440 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8004240 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.329+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa808150a00 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa808069d60 con 0x7fa7ec006a20 2024-05-08T04:40:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fa7f8012040 con 0x7fa7ec006a20 2024-05-08T04:40:03.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa7ff400640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 0x7fa7e80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:03.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7f8085430 con 0x7fa7ec006a20 2024-05-08T04:40:03.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa806a00640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 0x7fa7e80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:03.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.333+0000 7fa806a00640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 0x7fa7e80694b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ad000 tx=0x7fa7f40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:03.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.337+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7f8019030 con 0x7fa7ec006a20 2024-05-08T04:40:03.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.525+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa80806d9a0 con 0x7fa7ec006a20 2024-05-08T04:40:03.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.525+0000 7fa7ff400640 1 -- 172.21.15.164:0/3442397441 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa7f8055d00 con 0x7fa7ec006a20 2024-05-08T04:40:03.531 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:03.531 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:40:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 25 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 msgr2=0x7fa7e80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 0x7fa7e80694b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ad000 tx=0x7fa7f40a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 msgr2=0x7fa80814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8011bd0 tx=0x7fa7f800f610 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 shutdown_connections 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7e8066ff0 0x7fa7e80694b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa808151200 0x7fa8081fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa80814fcd0 0x7fa808152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 --2- 172.21.15.164:0/3442397441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7ec006a20 0x7fa80814ef80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 >> 172.21.15.164:0/3442397441 conn(0x7fa80806ac90 msgr2=0x7fa8080ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 shutdown_connections 2024-05-08T04:40:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:03.529+0000 7fa80e000640 1 -- 172.21.15.164:0/3442397441 wait complete. 2024-05-08T04:40:03.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:39:53.147237+0000', 'modified': '2024-05-08T04:39:56.208727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5523': {'gid': 5523, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6835/4206585796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 4206585796}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 4206585796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:40:03.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:40:03.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T04:40:04.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 Processor -- start 2024-05-08T04:40:04.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 -- start start 2024-05-08T04:40:04.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc13c0cfda0 0x7fc13c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c0cd9f0 con 0x7fc13c15ba70 2024-05-08T04:40:04.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c0cdcd0 con 0x7fc13c15be50 2024-05-08T04:40:04.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc143e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13c15fe70 con 0x7fc13c0cfda0 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc140a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc13c0cfda0 0x7fc13c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc141400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc13c15ba70 0x7fc13c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60458/0 (socket says 172.21.15.164:60458) 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc141400640 1 -- 172.21.15.164:0/3625657307 learned_addr learned my addr 172.21.15.164:0/3625657307 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc140a00640 1 -- 172.21.15.164:0/3625657307 >> v1:172.21.15.172:6789/0 conn(0x7fc13c15be50 legacy=0x7fc13c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc140a00640 1 -- 172.21.15.164:0/3625657307 >> v1:172.21.15.164:6789/0 conn(0x7fc13c15ba70 legacy=0x7fc13c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.089+0000 7fc140a00640 1 -- 172.21.15.164:0/3625657307 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c15f820 con 0x7fc13c0cfda0 2024-05-08T04:40:04.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc140a00640 1 --2- 172.21.15.164:0/3625657307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c160300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1280097c0 tx=0x7fc12801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11927def0aca07c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc13be00640 1 -- 172.21.15.164:0/3625657307 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b0070 con 0x7fc13c0cfda0 2024-05-08T04:40:04.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc13be00640 1 -- 172.21.15.164:0/3625657307 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280ab600 con 0x7fc13c0cfda0 2024-05-08T04:40:04.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc13be00640 1 -- 172.21.15.164:0/3625657307 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280ab9d0 con 0x7fc13c0cfda0 2024-05-08T04:40:04.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 -- 172.21.15.164:0/3625657307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 msgr2=0x7fc13c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 --2- 172.21.15.164:0/3625657307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c160300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1280097c0 tx=0x7fc12801c430 comp rx=0 tx=0).stop 2024-05-08T04:40:04.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 -- 172.21.15.164:0/3625657307 shutdown_connections 2024-05-08T04:40:04.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 --2- 172.21.15.164:0/3625657307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c160300 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 -- 172.21.15.164:0/3625657307 >> 172.21.15.164:0/3625657307 conn(0x7fc13c06ac90 msgr2=0x7fc13c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 -- 172.21.15.164:0/3625657307 shutdown_connections 2024-05-08T04:40:04.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.093+0000 7fc143e00640 1 -- 172.21.15.164:0/3625657307 wait complete. 2024-05-08T04:40:04.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 Processor -- start 2024-05-08T04:40:04.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- start start 2024-05-08T04:40:04.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc13c15ba70 0x7fc13c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 0x7fc13c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c169f60 con 0x7fc13c0cfda0 2024-05-08T04:40:04.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c169de0 con 0x7fc13c15be50 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c16a0e0 con 0x7fc13c15ba70 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38022/0 (socket says 172.21.15.164:38022) 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 -- 172.21.15.164:0/3392313181 learned_addr learned my addr 172.21.15.164:0/3392313181 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc140a00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc13c15ba70 0x7fc13c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 0x7fc13c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 -- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc13c15ba70 msgr2=0x7fc13c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc13c15ba70 0x7fc13c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 -- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 msgr2=0x7fc13c14f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 0x7fc13c14f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 -- 172.21.15.164:0/3392313181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c15cc80 con 0x7fc13c0cfda0 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 0x7fc13c14f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc141400640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c420 tx=0x7fc13000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc139600640 1 -- 172.21.15.164:0/3392313181 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130017070 con 0x7fc13c0cfda0 2024-05-08T04:40:04.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc139600640 1 -- 172.21.15.164:0/3392313181 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130005cf0 con 0x7fc13c0cfda0 2024-05-08T04:40:04.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc139600640 1 -- 172.21.15.164:0/3392313181 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300072f0 con 0x7fc13c0cfda0 2024-05-08T04:40:04.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c15cf60 con 0x7fc13c0cfda0 2024-05-08T04:40:04.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.097+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c150820 con 0x7fc13c0cfda0 2024-05-08T04:40:04.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc139600640 1 -- 172.21.15.164:0/3392313181 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fc13001a070 con 0x7fc13c0cfda0 2024-05-08T04:40:04.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc139600640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 0x7fc11c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc139600640 1 -- 172.21.15.164:0/3392313181 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1300843d0 con 0x7fc13c0cfda0 2024-05-08T04:40:04.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc140a00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 0x7fc11c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 Processor -- start 2024-05-08T04:40:04.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 -- start start 2024-05-08T04:40:04.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc140a00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 0x7fc11c0694b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc13c06bc70 tx=0x7fc128008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093fa4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093e8b5f0 con 0x55d093fed920 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093df2da0 con 0x55d093fee240 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d093d7b150 con 0x55d093fa3f30 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38024/0 (socket says 172.21.15.164:38024) 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc141400640 1 -- 172.21.15.164:0/2249426866 learned_addr learned my addr 172.21.15.164:0/2249426866 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc141e00640 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093fa4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.101+0000 7fc140a00640 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 -- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 msgr2=0x55d093fa4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093fa4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 -- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 msgr2=0x55d093fa39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 -- 172.21.15.164:0/2249426866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d093d94ca0 con 0x55d093fed920 2024-05-08T04:40:04.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc13005ce60 tx=0x7fc130054eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976ee324044b7b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc12e000640 1 -- 172.21.15.164:0/2249426866 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130082960 con 0x55d093fed920 2024-05-08T04:40:04.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc12e000640 1 -- 172.21.15.164:0/2249426866 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1300840c0 con 0x55d093fed920 2024-05-08T04:40:04.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc12e000640 1 -- 172.21.15.164:0/2249426866 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130007490 con 0x55d093fed920 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- 172.21.15.164:0/2249426866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 msgr2=0x55d093fedd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc13005ce60 tx=0x7fc130054eb0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- 172.21.15.164:0/2249426866 shutdown_connections 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093fa4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- 172.21.15.164:0/2249426866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093fedd00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- 172.21.15.164:0/2249426866 >> 172.21.15.164:0/2249426866 conn(0x55d093faa2a0 msgr2=0x55d093fa22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- 172.21.15.164:0/2249426866 shutdown_connections 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- 172.21.15.164:0/2249426866 wait complete. 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- start start 2024-05-08T04:40:04.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 Processor -- start 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093ff4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093dad990 con 0x55d093fed920 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093deee40 con 0x55d093fee240 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d093d7e900 con 0x55d093fa3f30 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc1472f8000 10 client.0 _ll_register_callbacks cb 0x55d093ea61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38034/0 (socket says 172.21.15.164:38034) 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 -- 172.21.15.164:0/3326144179 learned_addr learned my addr 172.21.15.164:0/3326144179 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 -- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 msgr2=0x55d093ff4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141400640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093ff4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093ff4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc140a00640 1 -- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 msgr2=0x55d093fa7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.105+0000 7fc141e00640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc140a00640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc140a00640 1 -- 172.21.15.164:0/3326144179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d093dab630 con 0x55d093fed920 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc141400640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093ff4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc141e00640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc140a00640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c10 tx=0x7fc12801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b495a324c870aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b0050 con 0x55d093fed920 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1280b87b0 con 0x55d093fed920 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1300840c0 con 0x55d093fed920 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1280d1050 con 0x55d093fed920 2024-05-08T04:40:04.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 10 client.5694 fetch_fsmap learned FSMap version 25 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55d093d7b3a0 con 0x55d093fed920 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1300840c0 con 0x55d093fed920 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1280d4020 con 0x55d093fed920 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 10 client.5694 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 10 client.5694 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 20 client.5694 populate_metadata read hostname 'smithi164' 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d093fa9160 0x55d093ffedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 4 client.5694 mds_command: new command op to 5523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 --> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d093d8ea50 con 0x55d093fa9160 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc141400640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d093fa9160 0x55d093ffedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 10 client.5694 ms_handle_connect on v2:172.21.15.164:6834/4206585796 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc141400640 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d093fa9160 0x55d093ffedc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81460ddc3759e5bd server_cookie=234e94e7eaea28ee in_seq=0 out_seq=0 2024-05-08T04:40:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 1 -- 172.21.15.164:0/3326144179 <== mds.0 v2:172.21.15.164:6834/4206585796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d093d8ea50 con 0x55d093fa9160 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 10 client.5694 handle_command_reply: tid=0 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 client.5694 shutdown 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 msgr2=0x55d093ff52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c10 tx=0x7fc12801ac70 comp rx=0 tx=0).stop 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 shutdown_connections 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d093fa9160 0x55d093ffedc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093fa7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d093ff52a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 --2- 172.21.15.164:0/3326144179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fa3f30 0x55d093ff4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc122a00640 0 client.5694 ms_handle_reset on v2:172.21.15.164:6834/4206585796 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 >> 172.21.15.164:0/3326144179 conn(0x55d093faa2a0 msgr2=0x55d093fa29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 shutdown_connections 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 1 -- 172.21.15.164:0/3326144179 wait complete. 2024-05-08T04:40:04.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.109+0000 7fc1472f8000 20 client.5694 trim_cache size 0 max 16384 2024-05-08T04:40:04.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 Processor -- start 2024-05-08T04:40:04.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 -- start start 2024-05-08T04:40:04.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fa3f30 0x55d09404d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fee240 0x55d09404e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c200510 con 0x55d093fed920 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1ff970 con 0x55d093fa3f30 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1fff40 con 0x55d093fee240 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc140a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc140a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38050/0 (socket says 172.21.15.164:38050) 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc140a00640 1 -- 172.21.15.164:0/1613172573 learned_addr learned my addr 172.21.15.164:0/1613172573 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc141400640 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fa3f30 0x55d09404d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.125+0000 7fc141e00640 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fee240 0x55d09404e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 -- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fee240 msgr2=0x55d09404e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fee240 0x55d09404e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 -- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fa3f30 msgr2=0x55d09404d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fa3f30 0x55d09404d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 -- 172.21.15.164:0/1613172573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d093fee070 con 0x55d093fed920 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc12801a060 tx=0x7fc1280ba070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177f086d4c08d1e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc12d600640 1 -- 172.21.15.164:0/1613172573 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280d9070 con 0x55d093fed920 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc12d600640 1 -- 172.21.15.164:0/1613172573 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280c9070 con 0x55d093fed920 2024-05-08T04:40:04.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc12d600640 1 -- 172.21.15.164:0/1613172573 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b0070 con 0x55d093fed920 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- 172.21.15.164:0/1613172573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 msgr2=0x55d09404dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc12801a060 tx=0x7fc1280ba070 comp rx=0 tx=0).stop 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- 172.21.15.164:0/1613172573 shutdown_connections 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fee240 0x55d09404e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fed920 0x55d09404dbc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1613172573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fa3f30 0x55d09404d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- 172.21.15.164:0/1613172573 >> 172.21.15.164:0/1613172573 conn(0x55d093faa2a0 msgr2=0x55d093fa7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- 172.21.15.164:0/1613172573 shutdown_connections 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- 172.21.15.164:0/1613172573 wait complete. 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- start start 2024-05-08T04:40:04.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 Processor -- start 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fed920 0x55d093ff9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093ffa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093df73d0 con 0x55d093fa3f30 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d093eaed50 con 0x55d093fee240 2024-05-08T04:40:04.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d093d8d980 con 0x55d093fed920 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc1472f8000 10 client.0 _ll_register_callbacks cb 0x7fc130050fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38062/0 (socket says 172.21.15.164:38062) 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 -- 172.21.15.164:0/1097449603 learned_addr learned my addr 172.21.15.164:0/1097449603 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141e00640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093ffa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc140a00640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fed920 0x55d093ff9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 -- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fed920 msgr2=0x55d093ff9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fed920 0x55d093ff9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 -- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 msgr2=0x55d093ffa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093ffa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141400640 1 -- 172.21.15.164:0/1097449603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d093e74600 con 0x55d093fa3f30 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.129+0000 7fc141e00640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093ffa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc141400640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc130054cc0 tx=0x7fc13000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe6d0525555e581 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13006c040 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1300513a0 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1280c9070 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1300840c0 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 10 client.5700 fetch_fsmap learned FSMap version 25 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55d093d879b0 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280c9070 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc130081040 con 0x55d093fa3f30 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 10 client.5700 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 10 client.5700 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 20 client.5700 populate_metadata read hostname 'smithi164' 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d094036510 0x55d0940368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 4 client.5700 mds_command: new command op to 5523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 --> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d093e002e0 con 0x55d094036510 2024-05-08T04:40:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc140a00640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d094036510 0x55d0940368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc140a00640 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d094036510 0x55d0940368f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88254de0f1a4fd3c server_cookie=768988c624c09d11 in_seq=0 out_seq=0 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 10 client.5700 ms_handle_connect on v2:172.21.15.164:6834/4206585796 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 1 -- 172.21.15.164:0/1097449603 <== mds.0 v2:172.21.15.164:6834/4206585796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d093e002e0 con 0x55d094036510 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 10 client.5700 handle_command_reply: tid=0 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 client.5700 shutdown 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 msgr2=0x55d094035dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc130054cc0 tx=0x7fc13000cf90 comp rx=0 tx=0).stop 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 shutdown_connections 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:6834/4206585796,v1:172.21.15.164:6835/4206585796] conn(0x55d094036510 0x55d0940368f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d093fed920 0x55d093ff9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d093fee240 0x55d093ffa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 --2- 172.21.15.164:0/1097449603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d093fa3f30 0x55d094035dc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc122a00640 0 client.5700 ms_handle_reset on v2:172.21.15.164:6834/4206585796 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 >> 172.21.15.164:0/1097449603 conn(0x55d093faa2a0 msgr2=0x55d093ff5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 shutdown_connections 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 1 -- 172.21.15.164:0/1097449603 wait complete. 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc1472f8000 20 client.5700 trim_cache size 0 max 16384 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 5523, 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:40:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 10.343575389, 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 25, 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 45, 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 45, 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 16.221533269999998 2024-05-08T04:40:04.144 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 msgr2=0x7fc11c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 0x7fc11c0694b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc13c06bc70 tx=0x7fc128008040 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 msgr2=0x7fc13c149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c420 tx=0x7fc13000c8e0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 shutdown_connections 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc11c066ff0 0x7fc11c0694b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc13c15ba70 0x7fc13c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc13c15be50 0x7fc13c14f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 --2- 172.21.15.164:0/3392313181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc13c0cfda0 0x7fc13c149370 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 >> 172.21.15.164:0/3392313181 conn(0x7fc13c06ac90 msgr2=0x7fc13c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 shutdown_connections 2024-05-08T04:40:04.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.133+0000 7fc143e00640 1 -- 172.21.15.164:0/3392313181 wait complete. 2024-05-08T04:40:04.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:40:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.289+0000 7fa922e00640 1 Processor -- start 2024-05-08T04:40:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.289+0000 7fa922e00640 1 -- start start 2024-05-08T04:40:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa91c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c15df70 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c0d07e0 con 0x7fa91c15db90 2024-05-08T04:40:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0ceb80 con 0x7fa91c0d0380 2024-05-08T04:40:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa91c0ce6f0 0x7fa91c06e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60460/0 (socket says 172.21.15.164:60460) 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922400640 1 -- 172.21.15.164:0/779152068 learned_addr learned my addr 172.21.15.164:0/779152068 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa91c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089254731 0 0) 0x7fa91c15df70 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa904003530 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277558975 0 0) 0x7fa904003530 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 msgr2=0x7fa91c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa91c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 >> v1:172.21.15.172:6789/0 conn(0x7fa91c15db90 legacy=0x7fa91c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0cf5c0 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa914001a00 con 0x7fa91c0ce6f0 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa91c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 >> v1:172.21.15.164:6789/0 conn(0x7fa91c0ce6f0 legacy=0x7fa91c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9040039f0 0x7fa904003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa904004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa904005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0ceb80 con 0x7fa9040043a0 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90400b9d0 con 0x7fa9040039f0 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90400bbd0 con 0x7fa91c0d0380 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa904004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921a00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9040039f0 0x7fa904003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa922400640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa904005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 -- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 msgr2=0x7fa904005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c0d0380 0x7fa904005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 -- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9040039f0 msgr2=0x7fa904003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9040039f0 0x7fa904003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.293+0000 7fa921000640 1 -- 172.21.15.164:0/779152068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904003530 con 0x7fa9040043a0 2024-05-08T04:40:04.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa921000640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa904004870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa908009cb0 tx=0x7fa90801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cac467dd1e02fab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080b2070 con 0x7fa9040043a0 2024-05-08T04:40:04.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9080a58d0 con 0x7fa9040043a0 2024-05-08T04:40:04.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa91be00640 1 -- 172.21.15.164:0/779152068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908009880 con 0x7fa9040043a0 2024-05-08T04:40:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 -- 172.21.15.164:0/779152068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 msgr2=0x7fa904004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 --2- 172.21.15.164:0/779152068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa904004870 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa908009cb0 tx=0x7fa90801cc10 comp rx=0 tx=0).stop 2024-05-08T04:40:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922400640 1 -- 172.21.15.164:0/779152068 reap_dead start 2024-05-08T04:40:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 -- 172.21.15.164:0/779152068 shutdown_connections 2024-05-08T04:40:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 -- 172.21.15.164:0/779152068 >> 172.21.15.164:0/779152068 conn(0x7fa91c09aa20 msgr2=0x7fa91c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 -- 172.21.15.164:0/779152068 shutdown_connections 2024-05-08T04:40:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 -- 172.21.15.164:0/779152068 wait complete. 2024-05-08T04:40:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.297+0000 7fa922e00640 1 Processor -- start 2024-05-08T04:40:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- start start 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 0x7fa91c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa91c0c6380 0x7fa91c0af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0cfe60 con 0x7fa9040043a0 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0cfb80 con 0x7fa91c0c6380 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0cd580 con 0x7fa91c15e170 2024-05-08T04:40:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38068/0 (socket says 172.21.15.164:38068) 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 -- 172.21.15.164:0/1865853612 learned_addr learned my addr 172.21.15.164:0/1865853612 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 -- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 msgr2=0x7fa91c0c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921000640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 0x7fa91c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 0x7fa91c0c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 -- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa91c0c6380 msgr2=0x7fa91c0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa91c0c6380 0x7fa91c0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 -- 172.21.15.164:0/1865853612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0c50e0 con 0x7fa9040043a0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921000640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 0x7fa91c0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa921a00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa910011c10 tx=0x7fa91000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910019020 con 0x7fa9040043a0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91000fc30 con 0x7fa9040043a0 2024-05-08T04:40:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100050d0 con 0x7fa9040043a0 2024-05-08T04:40:04.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c0cce70 con 0x7fa9040043a0 2024-05-08T04:40:04.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c0c9d70 con 0x7fa9040043a0 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.301+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c04ef30 con 0x7fa9040043a0 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.305+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fa91001c020 con 0x7fa9040043a0 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.305+0000 7fa919600640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 0x7fa8fc0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.305+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9100887f0 con 0x7fa9040043a0 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.305+0000 7fa921000640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 0x7fa8fc0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:04.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.305+0000 7fa921000640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 0x7fa8fc0694e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9080af000 tx=0x7fa9080046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:04.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.309+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa910023020 con 0x7fa9040043a0 2024-05-08T04:40:04.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa91c09af00 con 0x7fa9040043a0 2024-05-08T04:40:04.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa919600640 1 -- 172.21.15.164:0/1865853612 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa910059710 con 0x7fa9040043a0 2024-05-08T04:40:04.501 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:40:04.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 msgr2=0x7fa8fc0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 0x7fa8fc0694e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9080af000 tx=0x7fa9080046d0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 msgr2=0x7fa91c15a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:04.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa910011c10 tx=0x7fa91000f5a0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 shutdown_connections 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa8fc067020 0x7fa8fc0694e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa91c15e170 0x7fa91c0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa91c0c6380 0x7fa91c0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 --2- 172.21.15.164:0/1865853612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9040043a0 0x7fa91c15a8b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 >> 172.21.15.164:0/1865853612 conn(0x7fa91c09aa20 msgr2=0x7fa91c0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 shutdown_connections 2024-05-08T04:40:04.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:04.497+0000 7fa922e00640 1 -- 172.21.15.164:0/1865853612 wait complete. 2024-05-08T04:40:04.522 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:40:04.522 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:40:04.522 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:40:04.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:04.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.523 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:40:04.532 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:40:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:40:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.533 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42262sec preferred_lft 42262sec 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:04.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.535 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:40:04.593 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:40:04.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.595 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:40:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:40:04.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.647 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:40:04.647 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:40:04.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:40:04.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:40:04.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:40:04.648 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:40:04.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.740 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:40:04.748 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:40:04.749 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:40:04.749 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:04.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:40:04.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:04.815 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:04.863 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:04.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:04.864 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:04.915 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:40:04.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:04.915 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:05.002 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:05.002 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:05.003 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:05.017 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:05.017 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:05.017 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:05.017 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:05.018 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:40:05.028 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:40:05.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:40:05.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:05.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.052 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:40:05.111 INFO:teuthology.orchestra.run.smithi164.stdout:client5709 2024-05-08T04:40:05.111 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:05.111 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:05.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:40:05.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 Processor -- start 2024-05-08T04:40:05.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 -- start start 2024-05-08T04:40:05.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f83ec0c5530 0x7f83ec0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 0x7f83ec0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec15ae90 con 0x7f83ec0c5530 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0c5ed0 con 0x7f83ec0cbed0 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ebe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0724b0 con 0x7f83ec15aab0 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f83ec0c5530 0x7f83ec0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f83ec0c5530 0x7f83ec0c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38076/0 (socket says 172.21.15.164:38076) 2024-05-08T04:40:05.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 -- 172.21.15.164:0/1448495062 learned_addr learned my addr 172.21.15.164:0/1448495062 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83eb400640 1 --2- 172.21.15.164:0/1448495062 >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 0x7f83ec0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83eaa00640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 -- 172.21.15.164:0/1448495062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 msgr2=0x7f83ec0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 -- 172.21.15.164:0/1448495062 >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 msgr2=0x7f83ec0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 --2- 172.21.15.164:0/1448495062 >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 0x7f83ec0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83ea000640 1 -- 172.21.15.164:0/1448495062 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0ccc40 con 0x7f83ec0c5530 2024-05-08T04:40:05.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.337+0000 7f83eb400640 1 --2- 172.21.15.164:0/1448495062 >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 0x7f83ec0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:05.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ea000640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0c5990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f83ec06ff30 tx=0x7f83d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fa9d9e011c35a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:05.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83e9600640 1 -- 172.21.15.164:0/1448495062 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80b0070 con 0x7f83ec0c5530 2024-05-08T04:40:05.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83e9600640 1 -- 172.21.15.164:0/1448495062 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80ab600 con 0x7f83ec0c5530 2024-05-08T04:40:05.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83e9600640 1 -- 172.21.15.164:0/1448495062 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80ab9d0 con 0x7f83ec0c5530 2024-05-08T04:40:05.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 -- 172.21.15.164:0/1448495062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 msgr2=0x7f83ec0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:05.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0c5990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f83ec06ff30 tx=0x7f83d801c520 comp rx=0 tx=0).stop 2024-05-08T04:40:05.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 -- 172.21.15.164:0/1448495062 shutdown_connections 2024-05-08T04:40:05.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 --2- 172.21.15.164:0/1448495062 >> v2:172.21.15.172:3300/0 conn(0x7f83ec0cbed0 0x7f83ec0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0c5990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 --2- 172.21.15.164:0/1448495062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 -- 172.21.15.164:0/1448495062 >> 172.21.15.164:0/1448495062 conn(0x7f83ec06b670 msgr2=0x7f83ec0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 -- 172.21.15.164:0/1448495062 shutdown_connections 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 -- 172.21.15.164:0/1448495062 wait complete. 2024-05-08T04:40:05.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.341+0000 7f83ebe00640 1 Processor -- start 2024-05-08T04:40:05.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- start start 2024-05-08T04:40:05.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec0cbed0 0x7f83ec145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0cf310 con 0x7f83ec0c5530 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0cf190 con 0x7f83ec0cbed0 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0711e0 con 0x7f83ec15aab0 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38090/0 (socket says 172.21.15.164:38090) 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 -- 172.21.15.164:0/3096564309 learned_addr learned my addr 172.21.15.164:0/3096564309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ea000640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec0cbed0 0x7f83ec145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eb400640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 -- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 msgr2=0x7f83ec146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 -- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec0cbed0 msgr2=0x7f83ec145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec0cbed0 0x7f83ec145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 -- 172.21.15.164:0/3096564309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec15ca40 con 0x7f83ec0c5530 2024-05-08T04:40:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83eaa00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00d5f0 tx=0x7f83dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0125f0 con 0x7f83ec0c5530 2024-05-08T04:40:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc019070 con 0x7f83ec0c5530 2024-05-08T04:40:05.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec15cd20 con 0x7f83ec0c5530 2024-05-08T04:40:05.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0129d0 con 0x7f83ec0c5530 2024-05-08T04:40:05.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec15d2e0 con 0x7f83ec0c5530 2024-05-08T04:40:05.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.345+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec04ef30 con 0x7f83ec0c5530 2024-05-08T04:40:05.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.349+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f83dc012b70 con 0x7f83ec0c5530 2024-05-08T04:40:05.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.349+0000 7f83e2a00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 0x7f83cc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:05.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.349+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83dc085330 con 0x7f83ec0c5530 2024-05-08T04:40:05.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.349+0000 7f83ea000640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 0x7f83cc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:05.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.349+0000 7f83ea000640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 0x7f83cc0694b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83d80ad000 tx=0x7f83d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:05.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.353+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83dc055d00 con 0x7f83ec0c5530 2024-05-08T04:40:05.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83ec068170 con 0x7f83ec0c5530 2024-05-08T04:40:05.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83e2a00640 1 -- 172.21.15.164:0/3096564309 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f83dc059da0 con 0x7f83ec0c5530 2024-05-08T04:40:05.546 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 msgr2=0x7f83cc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 0x7f83cc0694b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83d80ad000 tx=0x7f83d8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 msgr2=0x7f83ec0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00d5f0 tx=0x7f83dc00dab0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 shutdown_connections 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83cc066ff0 0x7f83cc0694b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ec15aab0 0x7f83ec146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec0cbed0 0x7f83ec145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 --2- 172.21.15.164:0/3096564309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec0c5530 0x7f83ec0ca540 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 >> 172.21.15.164:0/3096564309 conn(0x7f83ec06b670 msgr2=0x7f83ec1574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 shutdown_connections 2024-05-08T04:40:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:05.541+0000 7f83ebe00640 1 -- 172.21.15.164:0/3096564309 wait complete. 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:40:05.566 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:05.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.567 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:40:05.577 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:40:05.577 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:40:05.577 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:40:05.577 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.577 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.578 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:40:05.579 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:40:05.580 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:05.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.581 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:40:05.641 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:40:05.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.644 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:05.700 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:40:05.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:40:05.701 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:40:05.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.797 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:40:05.806 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:40:05.806 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:40:05.806 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:05.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.806 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:40:05.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:05.871 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:05.926 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:05.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:05.927 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:05.982 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:40:05.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:05.983 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:06.081 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:06.082 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:06.082 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:06.097 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:06.098 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:06.098 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:06.098 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:06.098 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:40:06.108 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:40:06.113 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:40:06.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:06.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:06.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:06.132 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:40:06.189 INFO:teuthology.orchestra.run.smithi172.stdout:client5718 2024-05-08T04:40:06.190 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:06.190 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:06.239 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:40:06.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 Processor -- start 2024-05-08T04:40:06.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 -- start start 2024-05-08T04:40:06.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f436815ce10 0x7f436806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f43680ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680c6600 con 0x7f436815ba20 2024-05-08T04:40:06.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436815fbe0 con 0x7f436815ce10 2024-05-08T04:40:06.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f436e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680c6040 con 0x7f436806e390 2024-05-08T04:40:06.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f436815ba20 0x7f4368160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60470/0 (socket says 172.21.15.164:60470) 2024-05-08T04:40:06.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 -- 172.21.15.164:0/2818562519 learned_addr learned my addr 172.21.15.164:0/2818562519 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217908912 0 0) 0x7f43680c6600 con 0x7f436815ba20 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367e00640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f43680ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f434c003530 con 0x7f436815ba20 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366a00640 1 --2- 172.21.15.164:0/2818562519 >> v2:172.21.15.172:3300/0 conn(0x7f436815ce10 0x7f436806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719022906 0 0) 0x7f434c003530 con 0x7f436815ba20 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 msgr2=0x7f43680ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f43680ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 >> v2:172.21.15.172:3300/0 conn(0x7f436815ce10 msgr2=0x7f436806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:06.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 --2- 172.21.15.164:0/2818562519 >> v2:172.21.15.172:3300/0 conn(0x7f436815ce10 0x7f436806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680c6320 con 0x7f436815ba20 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f4358001680 con 0x7f436815ba20 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 >> v1:172.21.15.164:6789/0 conn(0x7f436815ba20 legacy=0x7f4368160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f434c0039c0 0x7f434c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f434c004810 0x7f434c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f434c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436815fbe0 con 0x7f434c0039c0 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680c6040 con 0x7f434c004810 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c00b9d0 con 0x7f436806e390 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f434c0039c0 0x7f434c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4366a00640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f434c004810 0x7f434c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367e00640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f434c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 -- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 msgr2=0x7f434c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436806e390 0x7f434c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 -- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f434c004810 msgr2=0x7f434c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f434c004810 0x7f434c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.377+0000 7f4367400640 1 -- 172.21.15.164:0/2818562519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c003530 con 0x7f434c0039c0 2024-05-08T04:40:06.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f4367400640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f434c0039c0 0x7f434c003de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f435800c2c0 tx=0x7f435801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bd85da724af7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:06.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358007d60 con 0x7f434c0039c0 2024-05-08T04:40:06.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43580a9020 con 0x7f434c0039c0 2024-05-08T04:40:06.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f4366000640 1 -- 172.21.15.164:0/2818562519 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358005b20 con 0x7f434c0039c0 2024-05-08T04:40:06.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 -- 172.21.15.164:0/2818562519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f434c0039c0 msgr2=0x7f434c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 --2- 172.21.15.164:0/2818562519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f434c0039c0 0x7f434c003de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f435800c2c0 tx=0x7f435801cd40 comp rx=0 tx=0).stop 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f4367e00640 1 -- 172.21.15.164:0/2818562519 reap_dead start 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 -- 172.21.15.164:0/2818562519 shutdown_connections 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 -- 172.21.15.164:0/2818562519 >> 172.21.15.164:0/2818562519 conn(0x7f43680cf770 msgr2=0x7f43680cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 -- 172.21.15.164:0/2818562519 shutdown_connections 2024-05-08T04:40:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.381+0000 7f436e200640 1 -- 172.21.15.164:0/2818562519 wait complete. 2024-05-08T04:40:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 Processor -- start 2024-05-08T04:40:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- start start 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f434c0039c0 0x7f43680c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f436814dbf0 0x7f43680b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680cda00 con 0x7f43680caa50 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680cd880 con 0x7f436814dbf0 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680cdb80 con 0x7f434c0039c0 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38110/0 (socket says 172.21.15.164:38110) 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 -- 172.21.15.164:0/14178136 learned_addr learned my addr 172.21.15.164:0/14178136 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367400640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f434c0039c0 0x7f43680c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4366a00640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f436814dbf0 0x7f43680b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 -- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f434c0039c0 msgr2=0x7f43680c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:06.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f434c0039c0 0x7f43680c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 -- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f436814dbf0 msgr2=0x7f43680b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:06.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f436814dbf0 0x7f43680b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:06.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 -- 172.21.15.164:0/14178136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680b1040 con 0x7f43680caa50 2024-05-08T04:40:06.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f4367e00640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4360011bf0 tx=0x7f436000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000fcb0 con 0x7f43680caa50 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436001d070 con 0x7f43680caa50 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360015670 con 0x7f43680caa50 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436814e070 con 0x7f43680caa50 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.385+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43680c8090 con 0x7f43680caa50 2024-05-08T04:40:06.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368072d30 con 0x7f43680caa50 2024-05-08T04:40:06.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f4360015810 con 0x7f43680caa50 2024-05-08T04:40:06.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f435f400640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 0x7f43480694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:06.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4360085a00 con 0x7f43680caa50 2024-05-08T04:40:06.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f4367400640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 0x7f43480694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:06.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.389+0000 7f4367400640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 0x7f43480694b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43580aa000 tx=0x7f43580b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:06.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.393+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4360010040 con 0x7f43680caa50 2024-05-08T04:40:06.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:06.581+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f436804f010 con 0x7f43680caa50 2024-05-08T04:40:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.173+0000 7f435f400640 1 -- 172.21.15.164:0/14178136 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7f4360056810 con 0x7f43680caa50 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 msgr2=0x7f43480694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 0x7f43480694b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43580aa000 tx=0x7f43580b1040 comp rx=0 tx=0).stop 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 msgr2=0x7f43680cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4360011bf0 tx=0x7f436000f1b0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 shutdown_connections 2024-05-08T04:40:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4348066ff0 0x7f43480694b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43680caa50 0x7f43680cb340 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f436814dbf0 0x7f43680b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 --2- 172.21.15.164:0/14178136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f434c0039c0 0x7f43680c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 >> 172.21.15.164:0/14178136 conn(0x7f43680cf770 msgr2=0x7f436815d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 shutdown_connections 2024-05-08T04:40:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.177+0000 7f436e200640 1 -- 172.21.15.164:0/14178136 wait complete. 2024-05-08T04:40:07.203 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:40:07.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 Processor -- start 2024-05-08T04:40:07.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 -- start start 2024-05-08T04:40:07.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff32015ba70 0x7ff320160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3200cfda0 0x7ff3200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200cd9f0 con 0x7ff32015be50 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32015fe70 con 0x7ff32015ba70 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff325800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200cdcd0 con 0x7ff3200cfda0 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff32015ba70 0x7ff320160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff316000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff32015be50 0x7ff3200cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60484/0 (socket says 172.21.15.164:60484) 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff316000640 1 -- 172.21.15.164:0/2536008690 learned_addr learned my addr 172.21.15.164:0/2536008690 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:07.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31f400640 1 --2- 172.21.15.164:0/2536008690 >> v2:172.21.15.172:3300/0 conn(0x7ff3200cfda0 0x7ff3200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31ea00640 1 -- 172.21.15.164:0/2536008690 >> v2:172.21.15.172:3300/0 conn(0x7ff3200cfda0 msgr2=0x7ff3200b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:07.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31ea00640 1 --2- 172.21.15.164:0/2536008690 >> v2:172.21.15.172:3300/0 conn(0x7ff3200cfda0 0x7ff3200b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31ea00640 1 -- 172.21.15.164:0/2536008690 >> v1:172.21.15.164:6789/0 conn(0x7ff32015be50 legacy=0x7ff3200cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31ea00640 1 -- 172.21.15.164:0/2536008690 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32015f820 con 0x7ff32015ba70 2024-05-08T04:40:07.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.337+0000 7ff31e000640 1 -- 172.21.15.164:0/2536008690 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981428117 0 0) 0x7ff3200cd9f0 con 0x7ff32015be50 2024-05-08T04:40:07.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff31ea00640 1 --2- 172.21.15.164:0/2536008690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015ba70 0x7ff320160300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff32015fde0 tx=0x7ff31001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373d4937d42f8e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff31e000640 1 -- 172.21.15.164:0/2536008690 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100b2070 con 0x7ff32015ba70 2024-05-08T04:40:07.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff31e000640 1 -- 172.21.15.164:0/2536008690 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100a58d0 con 0x7ff32015ba70 2024-05-08T04:40:07.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff31e000640 1 -- 172.21.15.164:0/2536008690 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100a5ca0 con 0x7ff32015ba70 2024-05-08T04:40:07.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 -- 172.21.15.164:0/2536008690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015ba70 msgr2=0x7ff320160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 --2- 172.21.15.164:0/2536008690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015ba70 0x7ff320160300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff32015fde0 tx=0x7ff31001c900 comp rx=0 tx=0).stop 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 -- 172.21.15.164:0/2536008690 shutdown_connections 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 --2- 172.21.15.164:0/2536008690 >> v2:172.21.15.172:3300/0 conn(0x7ff3200cfda0 0x7ff3200b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 --2- 172.21.15.164:0/2536008690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015ba70 0x7ff320160300 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 -- 172.21.15.164:0/2536008690 >> 172.21.15.164:0/2536008690 conn(0x7ff32006ac90 msgr2=0x7ff32006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 -- 172.21.15.164:0/2536008690 shutdown_connections 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 -- 172.21.15.164:0/2536008690 wait complete. 2024-05-08T04:40:07.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.341+0000 7ff325800640 1 Processor -- start 2024-05-08T04:40:07.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- start start 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3200cfda0 0x7ff320149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff32015ba70 0x7ff32014bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320168d40 con 0x7ff32015be50 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320168bc0 con 0x7ff3200cfda0 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320168ec0 con 0x7ff32015ba70 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38120/0 (socket says 172.21.15.164:38120) 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 -- 172.21.15.164:0/1590857870 learned_addr learned my addr 172.21.15.164:0/1590857870 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:07.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31ea00640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3200cfda0 0x7ff320149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff316000640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff32015ba70 0x7ff32014bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 -- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff32015ba70 msgr2=0x7ff32014bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff32015ba70 0x7ff32014bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 -- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3200cfda0 msgr2=0x7ff320149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3200cfda0 0x7ff320149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 -- 172.21.15.164:0/1590857870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200b1de0 con 0x7ff32015be50 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff31f400640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e960 tx=0x7ff31800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318017070 con 0x7ff32015be50 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800c9a0 con 0x7ff32015be50 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32015ee80 con 0x7ff32015be50 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800cd70 con 0x7ff32015be50 2024-05-08T04:40:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3200707f0 con 0x7ff32015be50 2024-05-08T04:40:07.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.345+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff320069d60 con 0x7ff32015be50 2024-05-08T04:40:07.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.349+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7ff31801a070 con 0x7ff32015be50 2024-05-08T04:40:07.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.349+0000 7ff317400640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 0x7ff2fc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.349+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3180850b0 con 0x7ff32015be50 2024-05-08T04:40:07.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.349+0000 7ff31ea00640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 0x7ff2fc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.349+0000 7ff31ea00640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 0x7ff2fc0694b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3100af000 tx=0x7ff3100a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.353+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff318056060 con 0x7ff32015be50 2024-05-08T04:40:07.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.533+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff32006e210 con 0x7ff2fc066ff0 2024-05-08T04:40:07.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.533+0000 7ff317400640 1 -- 172.21.15.164:0/1590857870 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff32006e210 con 0x7ff2fc066ff0 2024-05-08T04:40:07.539 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:40:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:40:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:07.539 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 msgr2=0x7ff2fc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 0x7ff2fc0694b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3100af000 tx=0x7ff3100a7070 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 msgr2=0x7ff3200ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e960 tx=0x7ff31800ee20 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 shutdown_connections 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff2fc066ff0 0x7ff2fc0694b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff32015ba70 0x7ff32014bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32015be50 0x7ff3200ad890 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 --2- 172.21.15.164:0/1590857870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3200cfda0 0x7ff320149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 >> 172.21.15.164:0/1590857870 conn(0x7ff32006ac90 msgr2=0x7ff32006d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:07.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 shutdown_connections 2024-05-08T04:40:07.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.537+0000 7ff325800640 1 -- 172.21.15.164:0/1590857870 wait complete. 2024-05-08T04:40:07.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:40:07.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 Processor -- start 2024-05-08T04:40:07.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 -- start start 2024-05-08T04:40:07.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdab815ba20 0x7fdab8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab80c6600 con 0x7fdab806de50 2024-05-08T04:40:07.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab815fbe0 con 0x7fdab815ba20 2024-05-08T04:40:07.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.693+0000 7fdabec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80c6040 con 0x7fdab80aedf0 2024-05-08T04:40:07.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:40358/0 (socket says 172.21.15.164:40358) 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 -- 172.21.15.164:0/2312787307 learned_addr learned my addr 172.21.15.164:0/2312787307 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdab7e00640 1 --2- 172.21.15.164:0/2312787307 >> v2:172.21.15.172:3300/0 conn(0x7fdab815ba20 0x7fdab8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdaaf400640 1 --1- 172.21.15.164:0/2312787307 >> v1:172.21.15.172:6789/0 conn(0x7fdab806de50 0x7fdab80ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:53380/0 (socket says 172.21.15.164:53380) 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 -- 172.21.15.164:0/2312787307 >> v2:172.21.15.172:3300/0 conn(0x7fdab815ba20 msgr2=0x7fdab8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 --2- 172.21.15.164:0/2312787307 >> v2:172.21.15.172:3300/0 conn(0x7fdab815ba20 0x7fdab8160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 -- 172.21.15.164:0/2312787307 >> v1:172.21.15.172:6789/0 conn(0x7fdab806de50 legacy=0x7fdab80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 -- 172.21.15.164:0/2312787307 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab80c6320 con 0x7fdab80aedf0 2024-05-08T04:40:07.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.697+0000 7fdabcc00640 1 --2- 172.21.15.164:0/2312787307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009e50 tx=0x7fdab001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e975e044cdfe905 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdab7400640 1 -- 172.21.15.164:0/2312787307 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a5870 con 0x7fdab80aedf0 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdab7400640 1 -- 172.21.15.164:0/2312787307 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab00a5a10 con 0x7fdab80aedf0 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdab7400640 1 -- 172.21.15.164:0/2312787307 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a5de0 con 0x7fdab80aedf0 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- 172.21.15.164:0/2312787307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 msgr2=0x7fdab806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- 172.21.15.164:0/2312787307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009e50 tx=0x7fdab001ca00 comp rx=0 tx=0).stop 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- 172.21.15.164:0/2312787307 shutdown_connections 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- 172.21.15.164:0/2312787307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab80aedf0 0x7fdab806e2c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- 172.21.15.164:0/2312787307 >> v2:172.21.15.172:3300/0 conn(0x7fdab815ba20 0x7fdab8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- 172.21.15.164:0/2312787307 >> 172.21.15.164:0/2312787307 conn(0x7fdab80cf770 msgr2=0x7fdab80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:07.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- 172.21.15.164:0/2312787307 shutdown_connections 2024-05-08T04:40:07.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- 172.21.15.164:0/2312787307 wait complete. 2024-05-08T04:40:07.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 Processor -- start 2024-05-08T04:40:07.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- start start 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 0x7fdab814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdab80aedf0 0x7fdab8153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80ce000 con 0x7fdab815ba20 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80cde80 con 0x7fdab80aedf0 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab8168b70 con 0x7fdab806de50 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38122/0 (socket says 172.21.15.164:38122) 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdabcc00640 1 -- 172.21.15.164:0/1747147992 learned_addr learned my addr 172.21.15.164:0/1747147992 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdaaf400640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdab80aedf0 0x7fdab8153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.701+0000 7fdab7e00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 0x7fdab814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 -- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 msgr2=0x7fdab814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 0x7fdab814ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 -- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdab80aedf0 msgr2=0x7fdab8153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdab80aedf0 0x7fdab8153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 -- 172.21.15.164:0/1747147992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8154290 con 0x7fdab815ba20 2024-05-08T04:40:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab7e00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 0x7fdab814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabcc00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdab00af040 tx=0x7fdab00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00bc020 con 0x7fdab815ba20 2024-05-08T04:40:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab00b86b0 con 0x7fdab815ba20 2024-05-08T04:40:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab81f5f00 con 0x7fdab815ba20 2024-05-08T04:40:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00b8aa0 con 0x7fdab815ba20 2024-05-08T04:40:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab814f770 con 0x7fdab815ba20 2024-05-08T04:40:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab8072d30 con 0x7fdab815ba20 2024-05-08T04:40:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fdab00bf020 con 0x7fdab815ba20 2024-05-08T04:40:07.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 0x7fda940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:07.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.705+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdab011ffb0 con 0x7fdab815ba20 2024-05-08T04:40:07.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.709+0000 7fdab7e00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 0x7fda940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:07.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.709+0000 7fdab7e00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 0x7fda940694b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdab814e9c0 tx=0x7fdaa8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:07.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.709+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdab8154290 con 0x7fdab815ba20 2024-05-08T04:40:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdab8166940 con 0x7fdab815ba20 2024-05-08T04:40:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdab4c00640 1 -- 172.21.15.164:0/1747147992 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdab0016620 con 0x7fdab815ba20 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 msgr2=0x7fda940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 0x7fda940694b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdab814e9c0 tx=0x7fdaa8009190 comp rx=0 tx=0).stop 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 msgr2=0x7fdab81f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.865+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdab00af040 tx=0x7fdab00197a0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 shutdown_connections 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fda94066ff0 0x7fda940694b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab815ba20 0x7fdab81f5180 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdab80aedf0 0x7fdab8153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 --2- 172.21.15.164:0/1747147992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdab806de50 0x7fdab814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:07.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 >> 172.21.15.164:0/1747147992 conn(0x7fdab80cf770 msgr2=0x7fdab806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:07.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 shutdown_connections 2024-05-08T04:40:07.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:07.869+0000 7fdabec00640 1 -- 172.21.15.164:0/1747147992 wait complete. 2024-05-08T04:40:07.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp71350 2024-05-08T04:40:08.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.025+0000 7f3e5a200640 1 Processor -- start 2024-05-08T04:40:08.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.025+0000 7f3e5a200640 1 -- start start 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e5a200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e5415db10 0x7f3e540b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e5a200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e540b40a0 0x7f3e54161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e5a200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5406eed0 con 0x7f3e54161cc0 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e5a200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e54162110 con 0x7f3e5415db10 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e5a200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5406f1b0 con 0x7f3e540b40a0 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e58e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e5415db10 0x7f3e540b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e59800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3e54161cc0 0x7f3e540cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60498/0 (socket says 172.21.15.164:60498) 2024-05-08T04:40:08.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e59800640 1 -- 172.21.15.164:0/2143365796 learned_addr learned my addr 172.21.15.164:0/2143365796 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e4be00640 1 --2- 172.21.15.164:0/2143365796 >> v2:172.21.15.172:3300/0 conn(0x7f3e540b40a0 0x7f3e54161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e58e00640 1 -- 172.21.15.164:0/2143365796 >> v2:172.21.15.172:3300/0 conn(0x7f3e540b40a0 msgr2=0x7f3e54161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e58e00640 1 --2- 172.21.15.164:0/2143365796 >> v2:172.21.15.172:3300/0 conn(0x7f3e540b40a0 0x7f3e54161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e58e00640 1 -- 172.21.15.164:0/2143365796 >> v1:172.21.15.164:6789/0 conn(0x7f3e54161cc0 legacy=0x7f3e540cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e58e00640 1 -- 172.21.15.164:0/2143365796 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5406f490 con 0x7f3e5415db10 2024-05-08T04:40:08.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.029+0000 7f3e53e00640 1 -- 172.21.15.164:0/2143365796 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69353246 0 0) 0x7f3e5406eed0 con 0x7f3e54161cc0 2024-05-08T04:40:08.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e58e00640 1 --2- 172.21.15.164:0/2143365796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e5415db10 0x7f3e540b3b60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e540730d0 tx=0x7f3e4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5326595928cfba9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e53e00640 1 -- 172.21.15.164:0/2143365796 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a7070 con 0x7f3e5415db10 2024-05-08T04:40:08.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e53e00640 1 -- 172.21.15.164:0/2143365796 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440b0070 con 0x7f3e5415db10 2024-05-08T04:40:08.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e53e00640 1 -- 172.21.15.164:0/2143365796 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440ab890 con 0x7f3e5415db10 2024-05-08T04:40:08.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- 172.21.15.164:0/2143365796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e5415db10 msgr2=0x7f3e540b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 --2- 172.21.15.164:0/2143365796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e5415db10 0x7f3e540b3b60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e540730d0 tx=0x7f3e4401c450 comp rx=0 tx=0).stop 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- 172.21.15.164:0/2143365796 shutdown_connections 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 --2- 172.21.15.164:0/2143365796 >> v2:172.21.15.172:3300/0 conn(0x7f3e540b40a0 0x7f3e54161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 --2- 172.21.15.164:0/2143365796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e5415db10 0x7f3e540b3b60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- 172.21.15.164:0/2143365796 >> 172.21.15.164:0/2143365796 conn(0x7f3e5409aa20 msgr2=0x7f3e540b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- 172.21.15.164:0/2143365796 shutdown_connections 2024-05-08T04:40:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- 172.21.15.164:0/2143365796 wait complete. 2024-05-08T04:40:08.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 Processor -- start 2024-05-08T04:40:08.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.033+0000 7f3e5a200640 1 -- start start 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e540b40a0 0x7f3e540b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e5415db10 0x7f3e5414ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e541623d0 con 0x7f3e54161cc0 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415e100 con 0x7f3e5415db10 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54162550 con 0x7f3e540b40a0 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38136/0 (socket says 172.21.15.164:38136) 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 -- 172.21.15.164:0/1195579997 learned_addr learned my addr 172.21.15.164:0/1195579997 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:08.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e58e00640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e540b40a0 0x7f3e540b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e4be00640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e5415db10 0x7f3e5414ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 -- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e540b40a0 msgr2=0x7f3e540b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e540b40a0 0x7f3e540b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 -- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e5415db10 msgr2=0x7f3e5414ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e5415db10 0x7f3e5414ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 -- 172.21.15.164:0/1195579997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540b6920 con 0x7f3e54161cc0 2024-05-08T04:40:08.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e59800640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c730 tx=0x7f3e4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c018070 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00ce40 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00a860 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e540b6c00 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e540b71c0 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5404ef30 con 0x7f3e54161cc0 2024-05-08T04:40:08.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.037+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f3e4c01b070 con 0x7f3e54161cc0 2024-05-08T04:40:08.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.041+0000 7f3e51600640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 0x7f3e300694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.041+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e4c084640 con 0x7f3e54161cc0 2024-05-08T04:40:08.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.041+0000 7f3e58e00640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 0x7f3e300694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.041+0000 7f3e58e00640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 0x7f3e300694b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e440af000 tx=0x7f3e440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.045+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e4c054f10 con 0x7f3e54161cc0 2024-05-08T04:40:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.221+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e5406d730 con 0x7f3e30066ff0 2024-05-08T04:40:08.232 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:08.225+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:08.232 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:08.225+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:08.232 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:08.225+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:08.232 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:08.225+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:08.232 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:08.225+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:08.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.241+0000 7f3e51600640 1 -- 172.21.15.164:0/1195579997 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3e5406d730 con 0x7f3e30066ff0 2024-05-08T04:40:08.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 msgr2=0x7f3e300694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 0x7f3e300694b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e440af000 tx=0x7f3e440a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 msgr2=0x7f3e5414e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c730 tx=0x7f3e4c00cbf0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 shutdown_connections 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3e30066ff0 0x7f3e300694b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e54161cc0 0x7f3e5414e400 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e5415db10 0x7f3e5414ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 --2- 172.21.15.164:0/1195579997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e540b40a0 0x7f3e540b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 >> 172.21.15.164:0/1195579997 conn(0x7f3e5409aa20 msgr2=0x7f3e540d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 shutdown_connections 2024-05-08T04:40:08.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.245+0000 7f3e5a200640 1 -- 172.21.15.164:0/1195579997 wait complete. 2024-05-08T04:40:08.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol93330 subvol_grp71350 2024-05-08T04:40:08.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 Processor -- start 2024-05-08T04:40:08.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 -- start start 2024-05-08T04:40:08.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f978815ab10 0x7f978806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97880cee90 0x7f97880aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978815e870 con 0x7f97880ae6e0 2024-05-08T04:40:08.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978806bb80 con 0x7f978815ab10 2024-05-08T04:40:08.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978f000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978815eb50 con 0x7f97880cee90 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f9787400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f97880ae6e0 0x7f97880ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60500/0 (socket says 172.21.15.164:60500) 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f9787e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f978815ab10 0x7f978806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f9787400640 1 -- 172.21.15.164:0/3531849723 learned_addr learned my addr 172.21.15.164:0/3531849723 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f978d000640 1 --2- 172.21.15.164:0/3531849723 >> v2:172.21.15.172:3300/0 conn(0x7f97880cee90 0x7f97880aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f9787e00640 1 -- 172.21.15.164:0/3531849723 >> v2:172.21.15.172:3300/0 conn(0x7f97880cee90 msgr2=0x7f97880aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:08.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.405+0000 7f9787e00640 1 --2- 172.21.15.164:0/3531849723 >> v2:172.21.15.172:3300/0 conn(0x7f97880cee90 0x7f97880aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9787e00640 1 -- 172.21.15.164:0/3531849723 >> v1:172.21.15.164:6789/0 conn(0x7f97880ae6e0 legacy=0x7f97880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9787e00640 1 -- 172.21.15.164:0/3531849723 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978815ee30 con 0x7f978815ab10 2024-05-08T04:40:08.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9786a00640 1 -- 172.21.15.164:0/3531849723 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414375147 0 0) 0x7f978815e870 con 0x7f97880ae6e0 2024-05-08T04:40:08.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9787e00640 1 --2- 172.21.15.164:0/3531849723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f978815ab10 0x7f978806cce0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f97880ca1f0 tx=0x7f977801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a78a154817439b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9786a00640 1 -- 172.21.15.164:0/3531849723 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780b1070 con 0x7f978815ab10 2024-05-08T04:40:08.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9786a00640 1 -- 172.21.15.164:0/3531849723 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97780ac620 con 0x7f978815ab10 2024-05-08T04:40:08.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f9786a00640 1 -- 172.21.15.164:0/3531849723 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780ac9f0 con 0x7f978815ab10 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f978f000640 1 -- 172.21.15.164:0/3531849723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f978815ab10 msgr2=0x7f978806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f978f000640 1 --2- 172.21.15.164:0/3531849723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f978815ab10 0x7f978806cce0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f97880ca1f0 tx=0x7f977801c3c0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.409+0000 7f978f000640 1 -- 172.21.15.164:0/3531849723 shutdown_connections 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 --2- 172.21.15.164:0/3531849723 >> v2:172.21.15.172:3300/0 conn(0x7f97880cee90 0x7f97880aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 --2- 172.21.15.164:0/3531849723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f978815ab10 0x7f978806cce0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- 172.21.15.164:0/3531849723 >> 172.21.15.164:0/3531849723 conn(0x7f97880cf770 msgr2=0x7f97880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:08.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- 172.21.15.164:0/3531849723 shutdown_connections 2024-05-08T04:40:08.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- 172.21.15.164:0/3531849723 wait complete. 2024-05-08T04:40:08.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 Processor -- start 2024-05-08T04:40:08.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- start start 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97880ae6e0 0x7f978814ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 0x7f97881514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cb070 con 0x7f97880cee90 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880caef0 con 0x7f978815ab10 2024-05-08T04:40:08.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cb1f0 con 0x7f97880ae6e0 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38178/0 (socket says 172.21.15.164:38178) 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 -- 172.21.15.164:0/1722644169 learned_addr learned my addr 172.21.15.164:0/1722644169 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978d000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 0x7f97881514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787e00640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97880ae6e0 0x7f978814ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 -- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97880ae6e0 msgr2=0x7f978814ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97880ae6e0 0x7f978814ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 -- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 msgr2=0x7f97881514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 0x7f97881514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 -- 172.21.15.164:0/1722644169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880c9ba0 con 0x7f97880cee90 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978d000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 0x7f97881514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:08.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f9787400640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f977400c550 tx=0x7f977400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774017070 con 0x7f97880cee90 2024-05-08T04:40:08.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774005d10 con 0x7f97880cee90 2024-05-08T04:40:08.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774007330 con 0x7f97880cee90 2024-05-08T04:40:08.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.413+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97880b2a10 con 0x7f97880cee90 2024-05-08T04:40:08.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97880b2fd0 con 0x7f97880cee90 2024-05-08T04:40:08.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9788072d30 con 0x7f97880cee90 2024-05-08T04:40:08.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f977401a070 con 0x7f97880cee90 2024-05-08T04:40:08.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f977fe00640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 0x7f97680694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9774084450 con 0x7f97880cee90 2024-05-08T04:40:08.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f9787e00640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 0x7f97680694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.417+0000 7f9787e00640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 0x7f97680694e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97780ae000 tx=0x7f9778008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.421+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9774054db0 con 0x7f97880cee90 2024-05-08T04:40:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.601+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol93330", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7f978806c600 con 0x7f9768067020 2024-05-08T04:40:08.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.729+0000 7f977fe00640 1 -- 172.21.15.164:0/1722644169 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f978806c600 con 0x7f9768067020 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 msgr2=0x7f97680694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 0x7f97680694e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97780ae000 tx=0x7f9778008040 comp rx=0 tx=0).stop 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 msgr2=0x7f9788150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f977400c550 tx=0x7f977400ca10 comp rx=0 tx=0).stop 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 shutdown_connections 2024-05-08T04:40:08.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9768067020 0x7f97680694e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f978815ab10 0x7f97881514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97880cee90 0x7f9788150f60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 --2- 172.21.15.164:0/1722644169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97880ae6e0 0x7f978814ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 >> 172.21.15.164:0/1722644169 conn(0x7f97880cf770 msgr2=0x7f978806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 shutdown_connections 2024-05-08T04:40:08.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.733+0000 7f978f000640 1 -- 172.21.15.164:0/1722644169 wait complete. 2024-05-08T04:40:08.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol93330 subvol_snap54010 subvol_grp71350 2024-05-08T04:40:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 Processor -- start 2024-05-08T04:40:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 -- start start 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcab40adfa0 0x7fcab40b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab415e300 con 0x7fcab40b4dc0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab415e570 con 0x7fcab40af6d0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab8e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab415e850 con 0x7fcab40adfa0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab2800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcab40b4dc0 0x7fcab406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60514/0 (socket says 172.21.15.164:60514) 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab2800640 1 -- 172.21.15.164:0/3365185256 learned_addr learned my addr 172.21.15.164:0/3365185256 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcaa9400640 1 --2- 172.21.15.164:0/3365185256 >> v2:172.21.15.172:3300/0 conn(0x7fcab40adfa0 0x7fcab40b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622247430 0 0) 0x7fcab415e300 con 0x7fcab40b4dc0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca98003530 con 0x7fcab40b4dc0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701090921 0 0) 0x7fcab415e570 con 0x7fcab40af6d0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab415e300 con 0x7fcab40af6d0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099705537 0 0) 0x7fca98003530 con 0x7fcab40b4dc0 2024-05-08T04:40:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 >> v2:172.21.15.172:3300/0 conn(0x7fcab40adfa0 msgr2=0x7fcab40b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 --2- 172.21.15.164:0/3365185256 >> v2:172.21.15.172:3300/0 conn(0x7fcab40adfa0 0x7fcab40b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 >> v1:172.21.15.172:6789/0 conn(0x7fcab40af6d0 legacy=0x7fcab40ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40b5250 con 0x7fcab40b4dc0 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fcaac001a00 con 0x7fcab40b4dc0 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 >> v1:172.21.15.164:6789/0 conn(0x7fcab40b4dc0 legacy=0x7fcab406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca98003b50 0x7fca98003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 0x7fca98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca980051d0 0x7fca9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab415e850 con 0x7fca98004810 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9800c100 con 0x7fca98003b50 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.897+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9800c300 con 0x7fca980051d0 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 0x7fca98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab1e00640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca98003b50 0x7fca98003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab2800640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca980051d0 0x7fca9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 -- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca980051d0 msgr2=0x7fca9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca980051d0 0x7fca9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 -- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca98003b50 msgr2=0x7fca98003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca98003b50 0x7fca98003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 -- 172.21.15.164:0/3365185256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98003530 con 0x7fca98004810 2024-05-08T04:40:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab2800640 1 -- 172.21.15.164:0/3365185256 reap_dead start 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcaa9400640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 0x7fca98004c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ae0 tx=0x7fca9c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6936a52ee05480c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0096b0 con 0x7fca98004810 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c0a6420 con 0x7fca98004810 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab1400640 1 -- 172.21.15.164:0/3365185256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0a6810 con 0x7fca98004810 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab8e00640 1 -- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 msgr2=0x7fca98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.901+0000 7fcab8e00640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 0x7fca98004c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ae0 tx=0x7fca9c01ca70 comp rx=0 tx=0).stop 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- 172.21.15.164:0/3365185256 shutdown_connections 2024-05-08T04:40:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 --2- 172.21.15.164:0/3365185256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca98004810 0x7fca98004c90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- 172.21.15.164:0/3365185256 >> 172.21.15.164:0/3365185256 conn(0x7fcab406cca0 msgr2=0x7fcab406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- 172.21.15.164:0/3365185256 shutdown_connections 2024-05-08T04:40:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- 172.21.15.164:0/3365185256 wait complete. 2024-05-08T04:40:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 Processor -- start 2024-05-08T04:40:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- start start 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca98004810 0x7fcab4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 0x7fcab415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4161ff0 con 0x7fcab415add0 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b3400 con 0x7fcab415d350 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab8e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4162170 con 0x7fca98004810 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38200/0 (socket says 172.21.15.164:38200) 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 -- 172.21.15.164:0/583633536 learned_addr learned my addr 172.21.15.164:0/583633536 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab1e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca98004810 0x7fcab4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcaa9400640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 0x7fcab415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 -- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca98004810 msgr2=0x7fcab4153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca98004810 0x7fcab4153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 -- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 msgr2=0x7fcab415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 0x7fcab415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 -- 172.21.15.164:0/583633536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab415b790 con 0x7fcab415add0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcaa9400640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 0x7fcab415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.905+0000 7fcab2800640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcaac001360 tx=0x7fcaac007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00eaa0 con 0x7fcab415add0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac00ec60 con 0x7fcab415add0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00ee50 con 0x7fcab415add0 2024-05-08T04:40:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab41565b0 con 0x7fcab415add0 2024-05-08T04:40:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab4156b70 con 0x7fcab415add0 2024-05-08T04:40:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab406ddf0 con 0x7fcab415add0 2024-05-08T04:40:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7fcaac01a020 con 0x7fcab415add0 2024-05-08T04:40:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 0x7fca900694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaac084860 con 0x7fcab415add0 2024-05-08T04:40:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcab1e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 0x7fca900694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.909+0000 7fcab1e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 0x7fca900694e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4007fd0 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:08.913+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcaac021020 con 0x7fcab415add0 2024-05-08T04:40:09.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.093+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol93330", "snap_name": "subvol_snap54010", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7fcab4156360 con 0x7fca90067020 2024-05-08T04:40:09.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.105+0000 7fcaaaa00640 1 -- 172.21.15.164:0/583633536 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcab4156360 con 0x7fca90067020 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 msgr2=0x7fca900694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 0x7fca900694e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4007fd0 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 msgr2=0x7fcab415b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcaac001360 tx=0x7fcaac007b70 comp rx=0 tx=0).stop 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 shutdown_connections 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca90067020 0x7fca900694e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcab415add0 0x7fcab415b250 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab415d350 0x7fcab415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 --2- 172.21.15.164:0/583633536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca98004810 0x7fcab4153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 >> 172.21.15.164:0/583633536 conn(0x7fcab406cca0 msgr2=0x7fcab4072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 shutdown_connections 2024-05-08T04:40:09.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.109+0000 7fcab8e00640 1 -- 172.21.15.164:0/583633536 wait complete. 2024-05-08T04:40:09.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol93330 subvol_snap54010 key value subvol_grp71350 2024-05-08T04:40:09.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.273+0000 7f9a43e00640 1 Processor -- start 2024-05-08T04:40:09.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a43e00640 1 -- start start 2024-05-08T04:40:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a3c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a43e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c0c6600 con 0x7f9a3c15ce10 2024-05-08T04:40:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a43e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c0c6040 con 0x7f9a3c15ba20 2024-05-08T04:40:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a43e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c15fbe0 con 0x7f9a3c0ae6e0 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a40a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9a3c15ce10 0x7f9a3c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38634/0 (socket says 172.21.15.164:38634) 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a40a00640 1 -- 172.21.15.164:0/1906987760 learned_addr learned my addr 172.21.15.164:0/1906987760 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a3c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885392854 0 0) 0x7f9a3c0c6600 con 0x7f9a3c15ce10 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a24003530 con 0x7f9a3c15ce10 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107323929 0 0) 0x7f9a24003530 con 0x7f9a3c15ce10 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 msgr2=0x7f9a3c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a3c0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 >> v1:172.21.15.172:6789/0 conn(0x7f9a3c15ba20 legacy=0x7f9a3c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0c6320 con 0x7f9a3c15ce10 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f9a28001680 con 0x7f9a3c15ce10 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 >> v1:172.21.15.164:6789/0 conn(0x7f9a3c15ce10 legacy=0x7f9a3c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a240039f0 0x7f9a24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a24004d50 0x7f9a240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c15fbe0 con 0x7f9a24004d50 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2400bae0 con 0x7f9a240039f0 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2400bce0 con 0x7f9a3c0ae6e0 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a24004d50 0x7f9a240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a40a00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41400640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a240039f0 0x7f9a24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 -- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 msgr2=0x7f9a24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0ae6e0 0x7f9a24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 -- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a240039f0 msgr2=0x7f9a24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a240039f0 0x7f9a24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41e00640 1 -- 172.21.15.164:0/1906987760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24003530 con 0x7f9a24004d50 2024-05-08T04:40:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.277+0000 7f9a41400640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a240039f0 0x7f9a24003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a41e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a24004d50 0x7f9a240051d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009f90 tx=0x7f9a3401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4bdd138a11b17f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a340b2070 con 0x7f9a24004d50 2024-05-08T04:40:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a340a5c50 con 0x7f9a24004d50 2024-05-08T04:40:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a3be00640 1 -- 172.21.15.164:0/1906987760 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34009c10 con 0x7f9a24004d50 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 -- 172.21.15.164:0/1906987760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a24004d50 msgr2=0x7f9a240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 --2- 172.21.15.164:0/1906987760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a24004d50 0x7f9a240051d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009f90 tx=0x7f9a3401cc40 comp rx=0 tx=0).stop 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a41e00640 1 -- 172.21.15.164:0/1906987760 reap_dead start 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 -- 172.21.15.164:0/1906987760 shutdown_connections 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 -- 172.21.15.164:0/1906987760 >> 172.21.15.164:0/1906987760 conn(0x7f9a3c0cf770 msgr2=0x7f9a3c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 -- 172.21.15.164:0/1906987760 shutdown_connections 2024-05-08T04:40:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.281+0000 7f9a43e00640 1 -- 172.21.15.164:0/1906987760 wait complete. 2024-05-08T04:40:09.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 Processor -- start 2024-05-08T04:40:09.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- start start 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c14fe40 0x7f9a3c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c154510 0x7f9a3c0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c06e560 con 0x7f9a3c0c8390 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0cf410 con 0x7f9a3c14fe40 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c06e6e0 con 0x7f9a3c154510 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34600/0 (socket says 172.21.15.164:34600) 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 -- 172.21.15.164:0/162605466 learned_addr learned my addr 172.21.15.164:0/162605466 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a41400640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c14fe40 0x7f9a3c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a41e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c154510 0x7f9a3c0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 -- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c154510 msgr2=0x7f9a3c0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c154510 0x7f9a3c0ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 -- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c14fe40 msgr2=0x7f9a3c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c14fe40 0x7f9a3c0cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 -- 172.21.15.164:0/162605466 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0c8900 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a40a00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9a28007340 tx=0x7f9a28004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2801a070 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2800f960 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c0c8be0 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800fd30 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c0ca0a0 con 0x7f9a3c0c8390 2024-05-08T04:40:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.285+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c072d30 con 0x7f9a3c0c8390 2024-05-08T04:40:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.289+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 6) v1 ==== 86778+0+0 (secure 0 0 0) 0x7f9a2801d070 con 0x7f9a3c0c8390 2024-05-08T04:40:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.289+0000 7f9a39600640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 0x7f9a1c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.289+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a28084a20 con 0x7f9a3c0c8390 2024-05-08T04:40:09.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.289+0000 7f9a41400640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 0x7f9a1c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:09.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.289+0000 7f9a41400640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 0x7f9a1c0694e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000c10 tx=0x7f9a30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:09.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.293+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a280552f0 con 0x7f9a3c0c8390 2024-05-08T04:40:09.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.481+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol93330", "snap_name": "subvol_snap54010", "key_name": "key", "value": "value", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a3c04f010 con 0x7f9a1c067020 2024-05-08T04:40:09.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a39600640 1 -- 172.21.15.164:0/162605466 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9a3c04f010 con 0x7f9a1c067020 2024-05-08T04:40:09.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 msgr2=0x7f9a1c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 0x7f9a1c0694e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000c10 tx=0x7f9a30009040 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 msgr2=0x7f9a3c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9a28007340 tx=0x7f9a28004f90 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 shutdown_connections 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a1c067020 0x7f9a1c0694e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c154510 0x7f9a3c0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3c0c8390 0x7f9a3c153fd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 --2- 172.21.15.164:0/162605466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c14fe40 0x7f9a3c0cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 >> 172.21.15.164:0/162605466 conn(0x7f9a3c0cf770 msgr2=0x7f9a3c15d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 shutdown_connections 2024-05-08T04:40:09.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:09.497+0000 7f9a43e00640 1 -- 172.21.15.164:0/162605466 wait complete. 2024-05-08T04:40:09.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:09.521 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp71350/subvol93330/.meta ./volumes/subvol_grp71350/subvol93330/.meta.stale_snap_section) 2024-05-08T04:40:12.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol93330 subvol_snap54010 subvol_grp71350 2024-05-08T04:40:13.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.081+0000 7fdfee400640 1 Processor -- start 2024-05-08T04:40:13.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.081+0000 7fdfee400640 1 -- start start 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfee400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdfe806b3a0 0x7fdfe80c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfee400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfee400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe80adc50 con 0x7fdfe8144360 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfee400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe80ae860 con 0x7fdfe806b3a0 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfee400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8144740 con 0x7fdfe80ca280 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdfe806b3a0 0x7fdfe80c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdfe806b3a0 0x7fdfe80c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34606/0 (socket says 172.21.15.164:34606) 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 -- 172.21.15.164:0/1322924611 learned_addr learned my addr 172.21.15.164:0/1322924611 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe7e00640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 -- 172.21.15.164:0/1322924611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 msgr2=0x7fdfe80ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe80ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 -- 172.21.15.164:0/1322924611 >> v1:172.21.15.172:6789/0 conn(0x7fdfe8144360 legacy=0x7fdfe806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 -- 172.21.15.164:0/1322924611 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80adf30 con 0x7fdfe806b3a0 2024-05-08T04:40:13.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe7e00640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:13.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.085+0000 7fdfe6a00640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe80c5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25943ba7b796baa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfe6000640 1 -- 172.21.15.164:0/1322924611 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40ab650 con 0x7fdfe806b3a0 2024-05-08T04:40:13.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfe6000640 1 -- 172.21.15.164:0/1322924611 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd40b0070 con 0x7fdfe806b3a0 2024-05-08T04:40:13.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfe6000640 1 -- 172.21.15.164:0/1322924611 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40aba10 con 0x7fdfe806b3a0 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- 172.21.15.164:0/1322924611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 msgr2=0x7fdfe80c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe80c5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd401c560 comp rx=0 tx=0).stop 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- 172.21.15.164:0/1322924611 shutdown_connections 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- 172.21.15.164:0/1322924611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe80c5040 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- 172.21.15.164:0/1322924611 >> 172.21.15.164:0/1322924611 conn(0x7fdfe80cf770 msgr2=0x7fdfe80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:13.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- 172.21.15.164:0/1322924611 shutdown_connections 2024-05-08T04:40:13.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- 172.21.15.164:0/1322924611 wait complete. 2024-05-08T04:40:13.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 Processor -- start 2024-05-08T04:40:13.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- start start 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe8140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe8144360 0x7fdfe8143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c4bc0 con 0x7fdfe806b3a0 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c4a40 con 0x7fdfe8144360 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfee400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe816ad70 con 0x7fdfe80ca280 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.089+0000 7fdfe7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34618/0 (socket says 172.21.15.164:34618) 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 -- 172.21.15.164:0/812480631 learned_addr learned my addr 172.21.15.164:0/812480631 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe6a00640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe8140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7e00640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe8144360 0x7fdfe8143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 -- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 msgr2=0x7fdfe8140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe8140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 -- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe8144360 msgr2=0x7fdfe8143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe8144360 0x7fdfe8143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 -- 172.21.15.164:0/812480631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8140960 con 0x7fdfe806b3a0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfe7400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c730 tx=0x7fdfd800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8016020 con 0x7fdfe806b3a0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8006d60 con 0x7fdfe806b3a0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8003bc0 con 0x7fdfe806b3a0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe8140c40 con 0x7fdfe806b3a0 2024-05-08T04:40:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe8141200 con 0x7fdfe806b3a0 2024-05-08T04:40:13.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe8072d30 con 0x7fdfe806b3a0 2024-05-08T04:40:13.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7fdfd800a850 con 0x7fdfe806b3a0 2024-05-08T04:40:13.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 0x7fdfc8069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.093+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdfd8084d40 con 0x7fdfe806b3a0 2024-05-08T04:40:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.097+0000 7fdfe6a00640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 0x7fdfc8069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.097+0000 7fdfe6a00640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 0x7fdfc8069540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40af000 tx=0x7fdfd40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.097+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfd8020020 con 0x7fdfe806b3a0 2024-05-08T04:40:13.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.281+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol93330", "snap_name": "subvol_snap54010", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfe804f010 con 0x7fdfc8067080 2024-05-08T04:40:13.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.313+0000 7fdfdf400640 1 -- 172.21.15.164:0/812480631 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdfe804f010 con 0x7fdfc8067080 2024-05-08T04:40:13.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 msgr2=0x7fdfc8069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 0x7fdfc8069540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40af000 tx=0x7fdfd40a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 msgr2=0x7fdfe815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c730 tx=0x7fdfd800cbf0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 shutdown_connections 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdfc8067080 0x7fdfc8069540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ca280 0x7fdfe8140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe8144360 0x7fdfe8143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 --2- 172.21.15.164:0/812480631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe806b3a0 0x7fdfe815b4d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 >> 172.21.15.164:0/812480631 conn(0x7fdfe80cf770 msgr2=0x7fdfe806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 shutdown_connections 2024-05-08T04:40:13.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.317+0000 7fdfee400640 1 -- 172.21.15.164:0/812480631 wait complete. 2024-05-08T04:40:13.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:13.340 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp71350/subvol93330/.meta.stale_snap_section ./volumes/subvol_grp71350/subvol93330/.meta) 2024-05-08T04:40:13.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:13.372 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap54010 ./volumes/subvol_grp71350/subvol93330/.meta) 2024-05-08T04:40:13.434 INFO:teuthology.orchestra.run.smithi164.stdout:[SNAP_METADATA_subvol_snap54010] 2024-05-08T04:40:13.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol93330 subvol_grp71350 2024-05-08T04:40:13.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.617+0000 7ff40da00640 1 Processor -- start 2024-05-08T04:40:13.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.617+0000 7ff40da00640 1 -- start start 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff40da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff4080ae6e0 0x7ff4080aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff40da00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080c6600 con 0x7ff40815ba20 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff40da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080c6040 con 0x7ff40815ce10 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff40da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40815fbe0 con 0x7ff4080ae6e0 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff40815ba20 0x7ff408160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38644/0 (socket says 172.21.15.164:38644) 2024-05-08T04:40:13.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406a00640 1 -- 172.21.15.164:0/3826945133 learned_addr learned my addr 172.21.15.164:0/3826945133 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:13.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 --2- 172.21.15.164:0/3826945133 >> v2:172.21.15.172:3300/0 conn(0x7ff4080ae6e0 0x7ff4080aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126501861 0 0) 0x7ff4080c6600 con 0x7ff40815ba20 2024-05-08T04:40:13.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec003530 con 0x7ff40815ba20 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679950042 0 0) 0x7ff3ec003530 con 0x7ff40815ba20 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 >> v2:172.21.15.172:3300/0 conn(0x7ff4080ae6e0 msgr2=0x7ff4080aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 --2- 172.21.15.164:0/3826945133 >> v2:172.21.15.172:3300/0 conn(0x7ff4080ae6e0 0x7ff4080aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 >> v1:172.21.15.172:6789/0 conn(0x7ff40815ce10 legacy=0x7ff40806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080c6320 con 0x7ff40815ba20 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7ff3f8001a00 con 0x7ff40815ba20 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 >> v1:172.21.15.164:6789/0 conn(0x7ff40815ba20 legacy=0x7ff408160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0039c0 0x7ff3ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec004810 0x7ff3ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 0x7ff3ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40815fbe0 con 0x7ff3ec0051d0 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00c070 con 0x7ff3ec0039c0 2024-05-08T04:40:13.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00c270 con 0x7ff3ec004810 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 0x7ff3ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff406a00640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0039c0 0x7ff3ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff3fe000640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec004810 0x7ff3ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 -- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec004810 msgr2=0x7ff3ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec004810 0x7ff3ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 -- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0039c0 msgr2=0x7ff3ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0039c0 0x7ff3ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.621+0000 7ff407400640 1 -- 172.21.15.164:0/3826945133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec003530 con 0x7ff3ec0051d0 2024-05-08T04:40:13.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff407400640 1 -- 172.21.15.164:0/3826945133 reap_dead start 2024-05-08T04:40:13.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff407400640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 0x7ff3ec00b9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff400009ae0 tx=0x7ff40001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a310d9c44250f31c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000096b0 con 0x7ff3ec0051d0 2024-05-08T04:40:13.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff400004020 con 0x7ff3ec0051d0 2024-05-08T04:40:13.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff406000640 1 -- 172.21.15.164:0/3826945133 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400004410 con 0x7ff3ec0051d0 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 msgr2=0x7ff3ec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 0x7ff3ec00b9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff400009ae0 tx=0x7ff40001cab0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- 172.21.15.164:0/3826945133 shutdown_connections 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 --2- 172.21.15.164:0/3826945133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ec0051d0 0x7ff3ec00b9d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- 172.21.15.164:0/3826945133 >> 172.21.15.164:0/3826945133 conn(0x7ff4080cf770 msgr2=0x7ff4080cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- 172.21.15.164:0/3826945133 shutdown_connections 2024-05-08T04:40:13.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- 172.21.15.164:0/3826945133 wait complete. 2024-05-08T04:40:13.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 Processor -- start 2024-05-08T04:40:13.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.625+0000 7ff40da00640 1 -- start start 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0051d0 0x7ff4080c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40806ac90 0x7ff40806b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080cf410 con 0x7ff4080cdf70 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080c5ec0 con 0x7ff3ec0051d0 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080aeb20 con 0x7ff40806ac90 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34636/0 (socket says 172.21.15.164:34636) 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 -- 172.21.15.164:0/2096724264 learned_addr learned my addr 172.21.15.164:0/2096724264 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff407400640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40806ac90 0x7ff40806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff406a00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0051d0 0x7ff4080c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 -- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40806ac90 msgr2=0x7ff40806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40806ac90 0x7ff40806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 -- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0051d0 msgr2=0x7ff4080c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0051d0 0x7ff4080c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 -- 172.21.15.164:0/2096724264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080b4370 con 0x7ff4080cdf70 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3fe000640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000f5f0 tx=0x7ff3f000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00052e0 con 0x7ff4080cdf70 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0018070 con 0x7ff4080cdf70 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4080b4650 con 0x7ff4080cdf70 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003820 con 0x7ff4080cdf70 2024-05-08T04:40:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080b4c10 con 0x7ff4080cdf70 2024-05-08T04:40:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408072d30 con 0x7ff4080cdf70 2024-05-08T04:40:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.629+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7ff3f00039c0 con 0x7ff4080cdf70 2024-05-08T04:40:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.633+0000 7ff3ff400640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 0x7ff3e4069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.633+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff3f0084fe0 con 0x7ff4080cdf70 2024-05-08T04:40:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.633+0000 7ff406a00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 0x7ff3e4069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.633+0000 7ff406a00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 0x7ff3e4069540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8004c30 tx=0x7ff3f8006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:13.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.637+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3f0055e40 con 0x7ff4080cdf70 2024-05-08T04:40:13.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.817+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol93330", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7ff40804f010 con 0x7ff3e4067080 2024-05-08T04:40:13.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.857+0000 7ff3ff400640 1 -- 172.21.15.164:0/2096724264 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+724 (secure 0 0 0) 0x7ff40804f010 con 0x7ff3e4067080 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "atime": "2024-05-08 04:40:08", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_pcent": "undefined", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_quota": "infinite", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_used": 0, 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "created_at": "2024-05-08 04:40:08", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "ctime": "2024-05-08 04:40:08", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "data_pool": "cephfs_data", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "features": [ 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-clone", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-autoprotect", 2024-05-08T04:40:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-retention" 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "flavor": 2, 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "gid": 0, 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": 16877, 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "mon_addrs": [ 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.164:6789", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6789", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6790" 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "mtime": "2024-05-08 04:40:08", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/volumes/subvol_grp71350/subvol93330/1dfdc6d8-fe91-4eab-b5cf-d481600069fb", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_namespace": "", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "complete", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "subvolume", 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "uid": 0 2024-05-08T04:40:13.864 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:40:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 msgr2=0x7ff3e4069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 0x7ff3e4069540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8004c30 tx=0x7ff3f8006040 comp rx=0 tx=0).stop 2024-05-08T04:40:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 msgr2=0x7ff4080ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000f5f0 tx=0x7ff3f000fab0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 shutdown_connections 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff3e4067080 0x7ff3e4069540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff40806ac90 0x7ff40806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4080cdf70 0x7ff4080ce3f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 --2- 172.21.15.164:0/2096724264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0051d0 0x7ff4080c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 >> 172.21.15.164:0/2096724264 conn(0x7ff4080cf770 msgr2=0x7ff4080c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 shutdown_connections 2024-05-08T04:40:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:13.861+0000 7ff40da00640 1 -- 172.21.15.164:0/2096724264 wait complete. 2024-05-08T04:40:13.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:13.884 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap54010 ./volumes/subvol_grp71350/subvol93330/.meta) 2024-05-08T04:40:13.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:40:13.910 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol93330 subvol_grp71350 2024-05-08T04:40:14.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 Processor -- start 2024-05-08T04:40:14.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 -- start start 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1e80aee90 0x7ff1e80d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1e80d0a90 0x7ff1e80b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80b5ab0 0x7ff1e815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e80af750 con 0x7ff1e80d0a90 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e80af9a0 con 0x7ff1e80aee90 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.085+0000 7ff1ed600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80afc80 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1e80d0a90 0x7ff1e80b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1e80d0a90 0x7ff1e80b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34644/0 (socket says 172.21.15.164:34644) 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 -- 172.21.15.164:0/1265534301 learned_addr learned my addr 172.21.15.164:0/1265534301 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6a00640 1 --2- 172.21.15.164:0/1265534301 >> v2:172.21.15.172:3300/0 conn(0x7ff1e80aee90 0x7ff1e80d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e7400640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80b5ab0 0x7ff1e815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 -- 172.21.15.164:0/1265534301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80b5ab0 msgr2=0x7ff1e815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80b5ab0 0x7ff1e815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 -- 172.21.15.164:0/1265534301 >> v2:172.21.15.172:3300/0 conn(0x7ff1e80aee90 msgr2=0x7ff1e80d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 --2- 172.21.15.164:0/1265534301 >> v2:172.21.15.172:3300/0 conn(0x7ff1e80aee90 0x7ff1e80d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 -- 172.21.15.164:0/1265534301 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80aff60 con 0x7ff1e80d0a90 2024-05-08T04:40:14.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e6000640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80d0a90 0x7ff1e80b5570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1e806cac0 tx=0x7ff1d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49625f3f8d2e58ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e5600640 1 -- 172.21.15.164:0/1265534301 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00b1070 con 0x7ff1e80d0a90 2024-05-08T04:40:14.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e5600640 1 -- 172.21.15.164:0/1265534301 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00ac620 con 0x7ff1e80d0a90 2024-05-08T04:40:14.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.089+0000 7ff1e5600640 1 -- 172.21.15.164:0/1265534301 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00ac9f0 con 0x7ff1e80d0a90 2024-05-08T04:40:14.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- 172.21.15.164:0/1265534301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80d0a90 msgr2=0x7ff1e80b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80d0a90 0x7ff1e80b5570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1e806cac0 tx=0x7ff1d001c4f0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- 172.21.15.164:0/1265534301 shutdown_connections 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80b5ab0 0x7ff1e815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- 172.21.15.164:0/1265534301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80d0a90 0x7ff1e80b5570 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- 172.21.15.164:0/1265534301 >> v2:172.21.15.172:3300/0 conn(0x7ff1e80aee90 0x7ff1e80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- 172.21.15.164:0/1265534301 >> 172.21.15.164:0/1265534301 conn(0x7ff1e8067050 msgr2=0x7ff1e806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- 172.21.15.164:0/1265534301 shutdown_connections 2024-05-08T04:40:14.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- 172.21.15.164:0/1265534301 wait complete. 2024-05-08T04:40:14.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 Processor -- start 2024-05-08T04:40:14.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- start start 2024-05-08T04:40:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 0x7ff1e8153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80d0a90 0x7ff1e81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8072270 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80720f0 con 0x7ff1e80aee90 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1ed600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e81618c0 con 0x7ff1e80d0a90 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34658/0 (socket says 172.21.15.164:34658) 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 -- 172.21.15.164:0/3602396639 learned_addr learned my addr 172.21.15.164:0/3602396639 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6a00640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 0x7ff1e8153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e7400640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80d0a90 0x7ff1e81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 -- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80d0a90 msgr2=0x7ff1e81599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80d0a90 0x7ff1e81599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 -- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 msgr2=0x7ff1e8153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 0x7ff1e8153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 -- 172.21.15.164:0/3602396639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e815b550 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6a00640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 0x7ff1e8153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:14.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.093+0000 7ff1e6000640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff1d001aba0 tx=0x7ff1d0019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00bb070 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00ace50 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00b1040 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e815b830 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e8161060 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e8072e80 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7ff1d00b5640 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 0x7ff1c4069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff1d011e520 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1e6a00640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 0x7ff1c4069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.097+0000 7ff1e6a00640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 0x7ff1c4069570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009fd0 tx=0x7ff1d800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.101+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1d0016110 con 0x7ff1e80b5ab0 2024-05-08T04:40:14.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.285+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol93330", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e804f010 con 0x7ff1c40670b0 2024-05-08T04:40:14.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.301+0000 7ff1dea00640 1 -- 172.21.15.164:0/3602396639 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff1e804f010 con 0x7ff1c40670b0 2024-05-08T04:40:14.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 msgr2=0x7ff1c4069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 0x7ff1c4069570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009fd0 tx=0x7ff1d800d040 comp rx=0 tx=0).stop 2024-05-08T04:40:14.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 msgr2=0x7ff1e80ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff1d001aba0 tx=0x7ff1d0019290 comp rx=0 tx=0).stop 2024-05-08T04:40:14.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 shutdown_connections 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff1c40670b0 0x7ff1c4069570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1e80d0a90 0x7ff1e81599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1e80b5ab0 0x7ff1e80ae080 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 --2- 172.21.15.164:0/3602396639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1e80aee90 0x7ff1e8153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 >> 172.21.15.164:0/3602396639 conn(0x7ff1e8067050 msgr2=0x7ff1e806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 shutdown_connections 2024-05-08T04:40:14.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ff1ed600640 1 -- 172.21.15.164:0/3602396639 wait complete. 2024-05-08T04:40:14.311 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.311 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.311 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.311 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.311 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.305+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.329 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.325+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.329 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.325+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.329 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.325+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.329 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.325+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.329 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:14.325+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:14.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp71350 2024-05-08T04:40:14.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.469+0000 7f14c0e00640 1 Processor -- start 2024-05-08T04:40:14.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.469+0000 7f14c0e00640 1 -- start start 2024-05-08T04:40:14.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14c0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 0x7f14bc0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14c0e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc15e300 con 0x7f14bc0af6d0 2024-05-08T04:40:14.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14c0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc15e570 con 0x7f14bc0b4dc0 2024-05-08T04:40:14.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14c0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc15e850 con 0x7f14bc0adfa0 2024-05-08T04:40:14.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f14bc0af6d0 0x7f14bc0ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38660/0 (socket says 172.21.15.164:38660) 2024-05-08T04:40:14.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9e00640 1 -- 172.21.15.164:0/4072574814 learned_addr learned my addr 172.21.15.164:0/4072574814 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9400640 1 --2- 172.21.15.164:0/4072574814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 0x7f14bc0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283626137 0 0) 0x7f14bc15e300 con 0x7f14bc0af6d0 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a0003530 con 0x7f14bc0af6d0 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103910737 0 0) 0x7f14bc15e570 con 0x7f14bc0b4dc0 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14bc15e300 con 0x7f14bc0b4dc0 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9400640 1 -- 172.21.15.164:0/4072574814 >> v1:172.21.15.172:6789/0 conn(0x7f14bc0b4dc0 legacy=0x7f14bc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9400640 1 -- 172.21.15.164:0/4072574814 >> v1:172.21.15.164:6789/0 conn(0x7f14bc0af6d0 legacy=0x7f14bc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.473+0000 7f14b9400640 1 -- 172.21.15.164:0/4072574814 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0b5250 con 0x7f14bc0adfa0 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14b9400640 1 --2- 172.21.15.164:0/4072574814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 0x7f14bc0b2ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14a80097f0 tx=0x7f14a801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c575d74c6316d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b2070 con 0x7f14bc0adfa0 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80a9070 con 0x7f14bc0adfa0 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14b8a00640 1 -- 172.21.15.164:0/4072574814 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80ad880 con 0x7f14bc0adfa0 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- 172.21.15.164:0/4072574814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 msgr2=0x7f14bc0b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 --2- 172.21.15.164:0/4072574814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 0x7f14bc0b2ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14a80097f0 tx=0x7f14a801c450 comp rx=0 tx=0).stop 2024-05-08T04:40:14.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- 172.21.15.164:0/4072574814 shutdown_connections 2024-05-08T04:40:14.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 --2- 172.21.15.164:0/4072574814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0adfa0 0x7f14bc0b2ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- 172.21.15.164:0/4072574814 >> 172.21.15.164:0/4072574814 conn(0x7f14bc06cca0 msgr2=0x7f14bc06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:14.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- 172.21.15.164:0/4072574814 shutdown_connections 2024-05-08T04:40:14.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- 172.21.15.164:0/4072574814 wait complete. 2024-05-08T04:40:14.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 Processor -- start 2024-05-08T04:40:14.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.477+0000 7f14c0e00640 1 -- start start 2024-05-08T04:40:14.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 0x7f14bc072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0b4dc0 0x7f14bc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc16bcf0 con 0x7f14bc0adfa0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc16bb70 con 0x7f14bc0af6d0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc16be70 con 0x7f14bc0b4dc0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34682/0 (socket says 172.21.15.164:34682) 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 -- 172.21.15.164:0/2600296031 learned_addr learned my addr 172.21.15.164:0/2600296031 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9400640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 0x7f14bc072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14ba800640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0b4dc0 0x7f14bc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 -- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0b4dc0 msgr2=0x7f14bc15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0b4dc0 0x7f14bc15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 -- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 msgr2=0x7f14bc072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 0x7f14bc072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 -- 172.21.15.164:0/2600296031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc160240 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9400640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 0x7f14bc072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b9e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cbf0 tx=0x7f14ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac017070 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00ad50 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac012620 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc160520 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc160ae0 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14bc06ddf0 con 0x7f14bc0adfa0 2024-05-08T04:40:14.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.481+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7f14ac01a070 con 0x7f14bc0adfa0 2024-05-08T04:40:14.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.485+0000 7f14b2000640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 0x7f149c069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:14.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.485+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14ac085070 con 0x7f14bc0adfa0 2024-05-08T04:40:14.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.485+0000 7f14b9400640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 0x7f149c069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:14.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.485+0000 7f14b9400640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 0x7f149c069540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af000 tx=0x7f14a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:14.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.489+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14ac055840 con 0x7f14bc0adfa0 2024-05-08T04:40:14.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.661+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp71350", "target": ["mon-mgr", ""]}) v1 -- 0x7f14bc04f010 con 0x7f149c067080 2024-05-08T04:40:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14b2000640 1 -- 172.21.15.164:0/2600296031 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f14bc04f010 con 0x7f149c067080 2024-05-08T04:40:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 msgr2=0x7f149c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 0x7f149c069540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af000 tx=0x7f14a8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 msgr2=0x7f14bc071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cbf0 tx=0x7f14ac00a430 comp rx=0 tx=0).stop 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 shutdown_connections 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f149c067080 0x7f149c069540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14bc0b4dc0 0x7f14bc15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14bc0af6d0 0x7f14bc072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 --2- 172.21.15.164:0/2600296031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14bc0adfa0 0x7f14bc071d50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 >> 172.21.15.164:0/2600296031 conn(0x7f14bc06cca0 msgr2=0x7f14bc15d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 shutdown_connections 2024-05-08T04:40:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:14.669+0000 7f14c0e00640 1 -- 172.21.15.164:0/2600296031 wait complete. 2024-05-08T04:40:14.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:14.693 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:40:14.708 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:40:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:14.709 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp71350/subvol93330/.meta.stale_snap_section) 2024-05-08T04:40:14.775 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:40:14.776 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:14.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:14.989 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:40:14.989 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:14.989 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:14.999 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:14.999 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:15.055 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:15.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.233+0000 7fad0d200640 1 Processor -- start 2024-05-08T04:40:15.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.233+0000 7fad0d200640 1 -- start start 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad0d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 0x7fad080af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad0d200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0806d580 con 0x7fad081561f0 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad0d200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0806d820 con 0x7fad080ce210 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad0d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0815d040 con 0x7fad0806b3a0 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad05e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fad081561f0 0x7fad0815c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38664/0 (socket says 172.21.15.164:38664) 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad05e00640 1 -- 172.21.15.164:0/1630910996 learned_addr learned my addr 172.21.15.164:0/1630910996 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:15.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad06800640 1 --2- 172.21.15.164:0/1630910996 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 0x7fad080af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289096512 0 0) 0x7fad0806d580 con 0x7fad081561f0 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facec003530 con 0x7fad081561f0 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110903834 0 0) 0x7fad0806d820 con 0x7fad080ce210 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad0806d580 con 0x7fad080ce210 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad06800640 1 -- 172.21.15.164:0/1630910996 >> v1:172.21.15.172:6789/0 conn(0x7fad080ce210 legacy=0x7fad0806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad06800640 1 -- 172.21.15.164:0/1630910996 >> v1:172.21.15.164:6789/0 conn(0x7fad081561f0 legacy=0x7fad0815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.237+0000 7fad06800640 1 -- 172.21.15.164:0/1630910996 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0806db00 con 0x7fad0806b3a0 2024-05-08T04:40:15.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad06800640 1 --2- 172.21.15.164:0/1630910996 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 0x7fad080af4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad000097f0 tx=0x7fad0001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9081c0fd2b459fe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000b0070 con 0x7fad0806b3a0 2024-05-08T04:40:15.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000ab600 con 0x7fad0806b3a0 2024-05-08T04:40:15.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad04a00640 1 -- 172.21.15.164:0/1630910996 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000ab9d0 con 0x7fad0806b3a0 2024-05-08T04:40:15.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- 172.21.15.164:0/1630910996 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 msgr2=0x7fad080af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 --2- 172.21.15.164:0/1630910996 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 0x7fad080af4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad000097f0 tx=0x7fad0001c4e0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- 172.21.15.164:0/1630910996 shutdown_connections 2024-05-08T04:40:15.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 --2- 172.21.15.164:0/1630910996 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad0806b3a0 0x7fad080af4e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- 172.21.15.164:0/1630910996 >> 172.21.15.164:0/1630910996 conn(0x7fad08066fa0 msgr2=0x7fad0806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:15.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- 172.21.15.164:0/1630910996 shutdown_connections 2024-05-08T04:40:15.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- 172.21.15.164:0/1630910996 wait complete. 2024-05-08T04:40:15.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 Processor -- start 2024-05-08T04:40:15.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- start start 2024-05-08T04:40:15.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad0806b3a0 0x7fad0814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad081561f0 0x7fad081507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08169a40 con 0x7fad080ce210 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad081698c0 con 0x7fad0806b3a0 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.241+0000 7fad0d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad08169bc0 con 0x7fad081561f0 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34694/0 (socket says 172.21.15.164:34694) 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 -- 172.21.15.164:0/2202304694 learned_addr learned my addr 172.21.15.164:0/2202304694 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad06800640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad081561f0 0x7fad081507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05e00640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad0806b3a0 0x7fad0814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 -- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad081561f0 msgr2=0x7fad081507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad081561f0 0x7fad081507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 -- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad0806b3a0 msgr2=0x7fad0814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad0806b3a0 0x7fad0814eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 -- 172.21.15.164:0/2202304694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080cf980 con 0x7fad080ce210 2024-05-08T04:40:15.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad05400640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facf400c410 tx=0x7facf400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4017070 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf40052f0 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4003840 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0814ce70 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad08149df0 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad08069d60 con 0x7fad080ce210 2024-05-08T04:40:15.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.245+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7facf401a070 con 0x7fad080ce210 2024-05-08T04:40:15.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.249+0000 7facfe000640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 0x7face8069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.249+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7facf4084650 con 0x7fad080ce210 2024-05-08T04:40:15.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.249+0000 7fad05e00640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 0x7face8069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.249+0000 7fad05e00640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 0x7face8069540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facf80012f0 tx=0x7facf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.253+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7facf4054e20 con 0x7fad080ce210 2024-05-08T04:40:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.405+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad08167040 con 0x7fad080ce210 2024-05-08T04:40:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7facfe000640 1 -- 172.21.15.164:0/2202304694 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16825 (secure 0 0 0) 0x7facf4058ec0 con 0x7fad080ce210 2024-05-08T04:40:15.412 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:15.413 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":47,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:11.199483+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:15.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 msgr2=0x7face8069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 0x7face8069540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facf80012f0 tx=0x7facf8009040 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 msgr2=0x7fad08150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facf400c410 tx=0x7facf400c8d0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 shutdown_connections 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7face8067080 0x7face8069540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad081561f0 0x7fad081507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad080ce210 0x7fad08150270 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 --2- 172.21.15.164:0/2202304694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad0806b3a0 0x7fad0814eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 >> 172.21.15.164:0/2202304694 conn(0x7fad08066fa0 msgr2=0x7fad080ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 shutdown_connections 2024-05-08T04:40:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.409+0000 7fad0d200640 1 -- 172.21.15.164:0/2202304694 wait complete. 2024-05-08T04:40:15.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:50.888294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:52.156282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:15.435 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:15.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 Processor -- start 2024-05-08T04:40:15.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 -- start start 2024-05-08T04:40:15.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff9a006cce0 0x7ff9a00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a00cecc0 0x7ff9a00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a015e870 con 0x7ff9a015ab10 2024-05-08T04:40:15.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00cf130 con 0x7ff9a006cce0 2024-05-08T04:40:15.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.569+0000 7ff9a5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006bb80 con 0x7ff9a00cecc0 2024-05-08T04:40:15.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff99e800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff9a015ab10 0x7ff9a00ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:37944/0 (socket says 172.21.15.164:37944) 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff9a006cce0 0x7ff9a00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff99e800640 1 -- 172.21.15.164:0/3579802789 learned_addr learned my addr 172.21.15.164:0/3579802789 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff99f200640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a00cecc0 0x7ff9a00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 -- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a00cecc0 msgr2=0x7ff9a00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a00cecc0 0x7ff9a00ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 -- 172.21.15.164:0/3579802789 >> v1:172.21.15.172:6789/0 conn(0x7ff9a015ab10 legacy=0x7ff9a00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 -- 172.21.15.164:0/3579802789 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a015eb50 con 0x7ff9a006cce0 2024-05-08T04:40:15.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff995e00640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a00ce780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9880097c0 tx=0x7ff98801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a201d0fbc43c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.573+0000 7ff99de00640 1 -- 172.21.15.164:0/3579802789 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880b1070 con 0x7ff9a006cce0 2024-05-08T04:40:15.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff99de00640 1 -- 172.21.15.164:0/3579802789 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880a8050 con 0x7ff9a006cce0 2024-05-08T04:40:15.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff99de00640 1 -- 172.21.15.164:0/3579802789 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880ac860 con 0x7ff9a006cce0 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 msgr2=0x7ff9a00ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a00ce780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9880097c0 tx=0x7ff98801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- 172.21.15.164:0/3579802789 shutdown_connections 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a00cecc0 0x7ff9a00ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- 172.21.15.164:0/3579802789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a00ce780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- 172.21.15.164:0/3579802789 >> 172.21.15.164:0/3579802789 conn(0x7ff9a00cf770 msgr2=0x7ff9a00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- 172.21.15.164:0/3579802789 shutdown_connections 2024-05-08T04:40:15.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- 172.21.15.164:0/3579802789 wait complete. 2024-05-08T04:40:15.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 Processor -- start 2024-05-08T04:40:15.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- start start 2024-05-08T04:40:15.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 0x7ff9a0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00aedf0 con 0x7ff9a00cecc0 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015aef0 con 0x7ff9a006cce0 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff9a5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00aef70 con 0x7ff9a015ab10 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff995e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff995e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34702/0 (socket says 172.21.15.164:34702) 2024-05-08T04:40:15.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.577+0000 7ff995e00640 1 -- 172.21.15.164:0/1524197313 learned_addr learned my addr 172.21.15.164:0/1524197313 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff99f200640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 0x7ff9a0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 -- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 msgr2=0x7ff9a0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff99e800640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 0x7ff9a0158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 -- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 msgr2=0x7ff9a014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 -- 172.21.15.164:0/1524197313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a014b7f0 con 0x7ff9a00cecc0 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff99f200640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 0x7ff9a0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff99e800640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff995e00640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff98801a010 tx=0x7ff98801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880bc070 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880b7650 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a014fd60 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880b1040 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a0150320 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0072d30 con 0x7ff9a00cecc0 2024-05-08T04:40:15.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7ff9880bf070 con 0x7ff9a00cecc0 2024-05-08T04:40:15.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 0x7ff97c069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff98811eea0 con 0x7ff9a00cecc0 2024-05-08T04:40:15.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.581+0000 7ff99e800640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 0x7ff97c069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.585+0000 7ff99e800640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 0x7ff97c069540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff9900012f0 tx=0x7ff990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.585+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff988016090 con 0x7ff9a00cecc0 2024-05-08T04:40:15.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.741+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9a004f010 con 0x7ff9a00cecc0 2024-05-08T04:40:15.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.741+0000 7ff997400640 1 -- 172.21.15.164:0/1524197313 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9880efcc0 con 0x7ff9a00cecc0 2024-05-08T04:40:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:15.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 msgr2=0x7ff97c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 0x7ff97c069540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff9900012f0 tx=0x7ff990009040 comp rx=0 tx=0).stop 2024-05-08T04:40:15.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 msgr2=0x7ff9a0158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff98801a010 tx=0x7ff98801ac20 comp rx=0 tx=0).stop 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 shutdown_connections 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff97c067080 0x7ff97c069540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9a015ab10 0x7ff9a0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a00cecc0 0x7ff9a0158690 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 --2- 172.21.15.164:0/1524197313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a006cce0 0x7ff9a014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 >> 172.21.15.164:0/1524197313 conn(0x7ff9a00cf770 msgr2=0x7ff9a01ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 shutdown_connections 2024-05-08T04:40:15.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.745+0000 7ff9a5400640 1 -- 172.21.15.164:0/1524197313 wait complete. 2024-05-08T04:40:15.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:15.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.901+0000 7fab8de00640 1 Processor -- start 2024-05-08T04:40:15.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.901+0000 7fab8de00640 1 -- start start 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab880cec40 0x7fab8815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8de00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8815c900 con 0x7fab880b1d40 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8de00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8815cec0 con 0x7fab880cec40 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8815cbe0 con 0x7fab880b1960 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab880cec40 0x7fab8815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8d400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fab880b1d40 0x7fab880ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38672/0 (socket says 172.21.15.164:38672) 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8d400640 1 -- 172.21.15.164:0/1708108883 learned_addr learned my addr 172.21.15.164:0/1708108883 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab880cec40 0x7fab8815bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34710/0 (socket says 172.21.15.164:34710) 2024-05-08T04:40:15.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab8ca00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87400640 1 -- 172.21.15.164:0/1708108883 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272956345 0 0) 0x7fab8815c900 con 0x7fab880b1d40 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87400640 1 -- 172.21.15.164:0/1708108883 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6c003530 con 0x7fab880b1d40 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 -- 172.21.15.164:0/1708108883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 msgr2=0x7fab88071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 -- 172.21.15.164:0/1708108883 >> v1:172.21.15.164:6789/0 conn(0x7fab880b1d40 legacy=0x7fab880ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.905+0000 7fab87e00640 1 -- 172.21.15.164:0/1708108883 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880712a0 con 0x7fab880cec40 2024-05-08T04:40:15.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab87e00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815bf90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fab740097c0 tx=0x7fab7401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8790970a2d50c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab87400640 1 -- 172.21.15.164:0/1708108883 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab740b0070 con 0x7fab880cec40 2024-05-08T04:40:15.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab87400640 1 -- 172.21.15.164:0/1708108883 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab740ab600 con 0x7fab880cec40 2024-05-08T04:40:15.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab87400640 1 -- 172.21.15.164:0/1708108883 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab740ab9d0 con 0x7fab880cec40 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- 172.21.15.164:0/1708108883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 msgr2=0x7fab8815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815bf90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fab740097c0 tx=0x7fab7401c350 comp rx=0 tx=0).stop 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- 172.21.15.164:0/1708108883 shutdown_connections 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815bf90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- 172.21.15.164:0/1708108883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- 172.21.15.164:0/1708108883 >> 172.21.15.164:0/1708108883 conn(0x7fab8806ac90 msgr2=0x7fab8806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:15.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- 172.21.15.164:0/1708108883 shutdown_connections 2024-05-08T04:40:15.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- 172.21.15.164:0/1708108883 wait complete. 2024-05-08T04:40:15.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 Processor -- start 2024-05-08T04:40:15.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- start start 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab880b1d40 0x7fab880afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b3bf0 con 0x7fab880cec40 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8815c6e0 con 0x7fab880b1d40 2024-05-08T04:40:15.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.909+0000 7fab8de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b3d70 con 0x7fab880b1960 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34716/0 (socket says 172.21.15.164:34716) 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 -- 172.21.15.164:0/3471616382 learned_addr learned my addr 172.21.15.164:0/3471616382 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8ca00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab87e00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab880b1d40 0x7fab880afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 -- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 msgr2=0x7fab88147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 -- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab880b1d40 msgr2=0x7fab880afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab880b1d40 0x7fab880afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 -- 172.21.15.164:0/3471616382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88148140 con 0x7fab880cec40 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8d400640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c450 tx=0x7fab8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80017070 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab80005cf0 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab800072b0 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8814dc60 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab880af590 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8804ef30 con 0x7fab880cec40 2024-05-08T04:40:15.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.913+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 7) v1 ==== 86829+0+0 (secure 0 0 0) 0x7fab80007450 con 0x7fab880cec40 2024-05-08T04:40:15.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.917+0000 7fab84c00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 0x7fab68069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:15.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.917+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fab800844f0 con 0x7fab880cec40 2024-05-08T04:40:15.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.917+0000 7fab8ca00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 0x7fab68069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:15.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.917+0000 7fab8ca00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 0x7fab68069540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab780113e0 tx=0x7fab7800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:15.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:15.921+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab80007740 con 0x7fab880cec40 2024-05-08T04:40:16.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab880b2ef0 con 0x7fab880cec40 2024-05-08T04:40:16.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab84c00640 1 -- 172.21.15.164:0/3471616382 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab80054cc0 con 0x7fab880cec40 2024-05-08T04:40:16.078 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 msgr2=0x7fab68069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 0x7fab68069540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab780113e0 tx=0x7fab7800b040 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 msgr2=0x7fab8815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c450 tx=0x7fab8000c910 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 shutdown_connections 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab68067080 0x7fab68069540 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab880cec40 0x7fab8815b4b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab880b1d40 0x7fab880afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 --2- 172.21.15.164:0/3471616382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab880b1960 0x7fab88147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 >> 172.21.15.164:0/3471616382 conn(0x7fab8806ac90 msgr2=0x7fab880b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 shutdown_connections 2024-05-08T04:40:16.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.073+0000 7fab8de00640 1 -- 172.21.15.164:0/3471616382 wait complete. 2024-05-08T04:40:16.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:16.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 Processor -- start 2024-05-08T04:40:16.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 -- start start 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdad40cdb10 0x7fdad406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 0x7fdad40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40cfe60 con 0x7fdad40d0c20 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40d0790 con 0x7fdad40cdb10 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdadc400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40d0140 con 0x7fdad40ce930 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdada400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 0x7fdad40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdad40cdb10 0x7fdad406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdad40d0c20 0x7fdad40ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:37952/0 (socket says 172.21.15.164:37952) 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdada400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 0x7fdad40cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:56460/0 (socket says 172.21.15.164:56460) 2024-05-08T04:40:16.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9000640 1 -- 172.21.15.164:0/3930182832 learned_addr learned my addr 172.21.15.164:0/3930182832 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:16.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9a00640 1 -- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 msgr2=0x7fdad40cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:16.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9a00640 1 --2- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 0x7fdad40cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9a00640 1 -- 172.21.15.164:0/3930182832 >> v1:172.21.15.172:6789/0 conn(0x7fdad40d0c20 legacy=0x7fdad40ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad9a00640 1 -- 172.21.15.164:0/3930182832 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40d0420 con 0x7fdad40cdb10 2024-05-08T04:40:16.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.237+0000 7fdad3e00640 1 -- 172.21.15.164:0/3930182832 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224309477 0 0) 0x7fdad40cfe60 con 0x7fdad40d0c20 2024-05-08T04:40:16.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdad9a00640 1 --2- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40cdb10 0x7fdad406efe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdac80097c0 tx=0x7fdac801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1277d6c4ac273f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdad3e00640 1 -- 172.21.15.164:0/3930182832 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80ab430 con 0x7fdad40cdb10 2024-05-08T04:40:16.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdad3e00640 1 -- 172.21.15.164:0/3930182832 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac80b0070 con 0x7fdad40cdb10 2024-05-08T04:40:16.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdad3e00640 1 -- 172.21.15.164:0/3930182832 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80ab7f0 con 0x7fdad40cdb10 2024-05-08T04:40:16.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 -- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40cdb10 msgr2=0x7fdad406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 --2- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40cdb10 0x7fdad406efe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdac80097c0 tx=0x7fdac801c410 comp rx=0 tx=0).stop 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 -- 172.21.15.164:0/3930182832 shutdown_connections 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 --2- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40ce930 0x7fdad40cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 --2- 172.21.15.164:0/3930182832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40cdb10 0x7fdad406efe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 -- 172.21.15.164:0/3930182832 >> 172.21.15.164:0/3930182832 conn(0x7fdad40b34f0 msgr2=0x7fdad40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 -- 172.21.15.164:0/3930182832 shutdown_connections 2024-05-08T04:40:16.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.241+0000 7fdadc400640 1 -- 172.21.15.164:0/3930182832 wait complete. 2024-05-08T04:40:16.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 Processor -- start 2024-05-08T04:40:16.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- start start 2024-05-08T04:40:16.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40cdb10 0x7fdad40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40d0c20 0x7fdad4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad416b260 con 0x7fdad40ce930 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad416b0e0 con 0x7fdad40d0c20 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad416b3e0 con 0x7fdad40cdb10 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34722/0 (socket says 172.21.15.164:34722) 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 -- 172.21.15.164:0/2576990942 learned_addr learned my addr 172.21.15.164:0/2576990942 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:16.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9a00640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40cdb10 0x7fdad40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdada400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40d0c20 0x7fdad4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 -- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40cdb10 msgr2=0x7fdad40c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40cdb10 0x7fdad40c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 -- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40d0c20 msgr2=0x7fdad4156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40d0c20 0x7fdad4156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 -- 172.21.15.164:0/2576990942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40c5e00 con 0x7fdad40ce930 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad9000640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c480 tx=0x7fdac000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0018070 con 0x7fdad40ce930 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac00074e0 con 0x7fdad40ce930 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac00049f0 con 0x7fdad40ce930 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad40c60e0 con 0x7fdad40ce930 2024-05-08T04:40:16.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.245+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad40c6980 con 0x7fdad40ce930 2024-05-08T04:40:16.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad40c6b80 con 0x7fdad40ce930 2024-05-08T04:40:16.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fdac001b070 con 0x7fdad40ce930 2024-05-08T04:40:16.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdad1600640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 0x7fdab4069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdac0084800 con 0x7fdad40ce930 2024-05-08T04:40:16.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdad9a00640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 0x7fdab4069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.249+0000 7fdad9a00640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 0x7fdab4069660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdac80af000 tx=0x7fdac8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.253+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdad40c6b80 con 0x7fdad40ce930 2024-05-08T04:40:16.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdad40cc310 con 0x7fdad40ce930 2024-05-08T04:40:16.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdad1600640 1 -- 172.21.15.164:0/2576990942 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdac0054e10 con 0x7fdad40ce930 2024-05-08T04:40:16.414 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 msgr2=0x7fdab4069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 0x7fdab4069660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdac80af000 tx=0x7fdac8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 msgr2=0x7fdad40c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.409+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c480 tx=0x7fdac000c940 comp rx=0 tx=0).stop 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 shutdown_connections 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab40671a0 0x7fdab4069660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad40d0c20 0x7fdad4156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad40ce930 0x7fdad40c54d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 --2- 172.21.15.164:0/2576990942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad40cdb10 0x7fdad40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 >> 172.21.15.164:0/2576990942 conn(0x7fdad40b34f0 msgr2=0x7fdad406d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 shutdown_connections 2024-05-08T04:40:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.413+0000 7fdadc400640 1 -- 172.21.15.164:0/2576990942 wait complete. 2024-05-08T04:40:16.435 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:40:16.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 Processor -- start 2024-05-08T04:40:16.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 -- start start 2024-05-08T04:40:16.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7d015dcb0 0x7fb7d0071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d00ce160 con 0x7fb7d00cf160 2024-05-08T04:40:16.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d0162290 con 0x7fb7d015dcb0 2024-05-08T04:40:16.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0162570 con 0x7fb7d00723d0 2024-05-08T04:40:16.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:56476/0 (socket says 172.21.15.164:56476) 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 -- 172.21.15.164:0/4228841113 learned_addr learned my addr 172.21.15.164:0/4228841113 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7cde00640 1 --2- 172.21.15.164:0/4228841113 >> v2:172.21.15.172:3300/0 conn(0x7fb7d015dcb0 0x7fb7d0071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 -- 172.21.15.164:0/4228841113 >> v2:172.21.15.172:3300/0 conn(0x7fb7d015dcb0 msgr2=0x7fb7d0071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 --2- 172.21.15.164:0/4228841113 >> v2:172.21.15.172:3300/0 conn(0x7fb7d015dcb0 0x7fb7d0071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 -- 172.21.15.164:0/4228841113 >> v1:172.21.15.172:6789/0 conn(0x7fb7d00cf160 legacy=0x7fb7d01616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.569+0000 7fb7c7e00640 1 -- 172.21.15.164:0/4228841113 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00713b0 con 0x7fb7d00723d0 2024-05-08T04:40:16.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7c7e00640 1 --2- 172.21.15.164:0/4228841113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009870 tx=0x7fb7b801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a8b7aec6c615ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cd400640 1 -- 172.21.15.164:0/4228841113 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80a7070 con 0x7fb7d00723d0 2024-05-08T04:40:16.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cd400640 1 -- 172.21.15.164:0/4228841113 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b80b0070 con 0x7fb7d00723d0 2024-05-08T04:40:16.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cd400640 1 -- 172.21.15.164:0/4228841113 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80ab890 con 0x7fb7d00723d0 2024-05-08T04:40:16.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 -- 172.21.15.164:0/4228841113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 msgr2=0x7fb7d00cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 --2- 172.21.15.164:0/4228841113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009870 tx=0x7fb7b801c560 comp rx=0 tx=0).stop 2024-05-08T04:40:16.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 -- 172.21.15.164:0/4228841113 shutdown_connections 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 --2- 172.21.15.164:0/4228841113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d00cec20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 --2- 172.21.15.164:0/4228841113 >> v2:172.21.15.172:3300/0 conn(0x7fb7d015dcb0 0x7fb7d0071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 -- 172.21.15.164:0/4228841113 >> 172.21.15.164:0/4228841113 conn(0x7fb7d006bda0 msgr2=0x7fb7d006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 -- 172.21.15.164:0/4228841113 shutdown_connections 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.573+0000 7fb7cf200640 1 -- 172.21.15.164:0/4228841113 wait complete. 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 Processor -- start 2024-05-08T04:40:16.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- start start 2024-05-08T04:40:16.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7d015dcb0 0x7fb7d0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d016aa50 con 0x7fb7d00cf160 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d006e120 con 0x7fb7d015dcb0 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d016abd0 con 0x7fb7d00723d0 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34726/0 (socket says 172.21.15.164:34726) 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 -- 172.21.15.164:0/3029521026 learned_addr learned my addr 172.21.15.164:0/3029521026 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cde00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7ce800640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7d015dcb0 0x7fb7d0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 -- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 msgr2=0x7fb7d014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 -- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7d015dcb0 msgr2=0x7fb7d0153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7d015dcb0 0x7fb7d0153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 -- 172.21.15.164:0/3029521026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d015ed90 con 0x7fb7d00cf160 2024-05-08T04:40:16.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c7e00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7b801a190 tx=0x7fb7b801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80be5f0 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b80be7b0 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d015f070 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80beba0 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.577+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d015f630 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d00b38f0 con 0x7fb7d00cf160 2024-05-08T04:40:16.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fb7b80b43f0 con 0x7fb7d00cf160 2024-05-08T04:40:16.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7c6000640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 0x7fb7ac069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:16.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb7b811fff0 con 0x7fb7d00cf160 2024-05-08T04:40:16.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7cde00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 0x7fb7ac069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:16.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.581+0000 7fb7cde00640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 0x7fb7ac069660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ac0 tx=0x7fb7c0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:16.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.585+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7d00b38f0 con 0x7fb7d00cf160 2024-05-08T04:40:16.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.749+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7d004ef30 con 0x7fb7d00cf160 2024-05-08T04:40:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.749+0000 7fb7c6000640 1 -- 172.21.15.164:0/3029521026 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb7b80c4060 con 0x7fb7d00cf160 2024-05-08T04:40:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:40:16.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 msgr2=0x7fb7ac069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 0x7fb7ac069660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ac0 tx=0x7fb7c0009190 comp rx=0 tx=0).stop 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 msgr2=0x7fb7d0153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7b801a190 tx=0x7fb7b801ada0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 shutdown_connections 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb7ac0671a0 0x7fb7ac069660 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7d015dcb0 0x7fb7d0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7d00cf160 0x7fb7d0153a30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 --2- 172.21.15.164:0/3029521026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7d00723d0 0x7fb7d014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 >> 172.21.15.164:0/3029521026 conn(0x7fb7d006bda0 msgr2=0x7fb7d00cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 shutdown_connections 2024-05-08T04:40:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:16.753+0000 7fb7cf200640 1 -- 172.21.15.164:0/3029521026 wait complete. 2024-05-08T04:40:16.777 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:40:16.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:16.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:16.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:40:16.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:16.807 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:40:16.860 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:16.860 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:16.869 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:16.869 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:16.918 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:17.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 Processor -- start 2024-05-08T04:40:17.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 -- start start 2024-05-08T04:40:17.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7cc0cfda0 0x7ff7cc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7cc15be50 0x7ff7cc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0cd9f0 con 0x7ff7cc15ba70 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc15fe70 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7d0a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cdcd0 con 0x7ff7cc15be50 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7c9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7cc0cfda0 0x7ff7cc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7c9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7cc0cfda0 0x7ff7cc160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34736/0 (socket says 172.21.15.164:34736) 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.049+0000 7ff7c9400640 1 -- 172.21.15.164:0/3814518568 learned_addr learned my addr 172.21.15.164:0/3814518568 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:17.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7ca800640 1 --2- 172.21.15.164:0/3814518568 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc15be50 0x7ff7cc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c9400640 1 -- 172.21.15.164:0/3814518568 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc15be50 msgr2=0x7ff7cc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:17.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c9400640 1 --2- 172.21.15.164:0/3814518568 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc15be50 0x7ff7cc0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c9400640 1 -- 172.21.15.164:0/3814518568 >> v1:172.21.15.172:6789/0 conn(0x7ff7cc15ba70 legacy=0x7ff7cc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c9400640 1 -- 172.21.15.164:0/3814518568 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc15f820 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c9400640 1 --2- 172.21.15.164:0/3814518568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc0cfda0 0x7ff7cc160300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc15fde0 tx=0x7ff7b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e441e94bb572ce98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c8a00640 1 -- 172.21.15.164:0/3814518568 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80ab600 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c8a00640 1 -- 172.21.15.164:0/3814518568 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b80b0070 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.053+0000 7ff7c8a00640 1 -- 172.21.15.164:0/3814518568 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80ab9c0 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- 172.21.15.164:0/3814518568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc0cfda0 msgr2=0x7ff7cc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/3814518568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc0cfda0 0x7ff7cc160300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc15fde0 tx=0x7ff7b801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- 172.21.15.164:0/3814518568 shutdown_connections 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/3814518568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc0cfda0 0x7ff7cc160300 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/3814518568 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc15be50 0x7ff7cc0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- 172.21.15.164:0/3814518568 >> 172.21.15.164:0/3814518568 conn(0x7ff7cc06ac90 msgr2=0x7ff7cc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- 172.21.15.164:0/3814518568 shutdown_connections 2024-05-08T04:40:17.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- 172.21.15.164:0/3814518568 wait complete. 2024-05-08T04:40:17.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 Processor -- start 2024-05-08T04:40:17.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- start start 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0cfda0 0x7ff7cc0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba70 0x7ff7cc0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc168d40 con 0x7ff7cc15be50 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc168bc0 con 0x7ff7cc15ba70 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7d0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc168ec0 con 0x7ff7cc0cfda0 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34748/0 (socket says 172.21.15.164:34748) 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 -- 172.21.15.164:0/2274066719 learned_addr learned my addr 172.21.15.164:0/2274066719 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7c9400640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba70 0x7ff7cc0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7c9e00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0cfda0 0x7ff7cc0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 -- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0cfda0 msgr2=0x7ff7cc0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0cfda0 0x7ff7cc0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 -- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba70 msgr2=0x7ff7cc0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba70 0x7ff7cc0b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 -- 172.21.15.164:0/2274066719 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0b1170 con 0x7ff7cc15be50 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.057+0000 7ff7ca800640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400d560 tx=0x7ff7c400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40125f0 con 0x7ff7cc15be50 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4019070 con 0x7ff7cc15be50 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc1506f0 con 0x7ff7cc15be50 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40129f0 con 0x7ff7cc15be50 2024-05-08T04:40:17.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc150ca0 con 0x7ff7cc15be50 2024-05-08T04:40:17.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc069d60 con 0x7ff7cc15be50 2024-05-08T04:40:17.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7ff7c4012b90 con 0x7ff7cc15be50 2024-05-08T04:40:17.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 0x7ff7ac069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff7c4085800 con 0x7ff7cc15be50 2024-05-08T04:40:17.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c9e00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 0x7ff7ac069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.061+0000 7ff7c9e00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 0x7ff7ac069660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc001ea0 tx=0x7ff7bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.065+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7cc0b1170 con 0x7ff7cc15be50 2024-05-08T04:40:17.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.221+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7cc06e210 con 0x7ff7cc15be50 2024-05-08T04:40:17.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.221+0000 7ff7c2000640 1 -- 172.21.15.164:0/2274066719 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16825 (secure 0 0 0) 0x7ff7c4055d80 con 0x7ff7cc15be50 2024-05-08T04:40:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:17.228 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":47,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:11.199483+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 msgr2=0x7ff7ac069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 0x7ff7ac069660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc001ea0 tx=0x7ff7bc009040 comp rx=0 tx=0).stop 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 msgr2=0x7ff7cc1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400d560 tx=0x7ff7c400da20 comp rx=0 tx=0).stop 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 shutdown_connections 2024-05-08T04:40:17.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7ac0671a0 0x7ff7ac069660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7cc15be50 0x7ff7cc1501b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba70 0x7ff7cc0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 --2- 172.21.15.164:0/2274066719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0cfda0 0x7ff7cc0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 >> 172.21.15.164:0/2274066719 conn(0x7ff7cc06ac90 msgr2=0x7ff7cc1552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 shutdown_connections 2024-05-08T04:40:17.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.225+0000 7ff7d0a00640 1 -- 172.21.15.164:0/2274066719 wait complete. 2024-05-08T04:40:17.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:50.888294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:52.156282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:17.251 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:17.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 Processor -- start 2024-05-08T04:40:17.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 -- start start 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50400c5040 0x7f50400ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400c54b0 con 0x7f5040144360 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400adc50 con 0x7f504006b3a0 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f5044a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50400ae860 con 0x7f50400c5040 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f503de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5040144360 0x7f504006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38688/0 (socket says 172.21.15.164:38688) 2024-05-08T04:40:17.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f503de00640 1 -- 172.21.15.164:0/2823972841 learned_addr learned my addr 172.21.15.164:0/2823972841 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:17.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792416741 0 0) 0x7f50400c54b0 con 0x7f5040144360 2024-05-08T04:40:17.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.385+0000 7f503e800640 1 --2- 172.21.15.164:0/2823972841 >> v2:172.21.15.172:3300/0 conn(0x7f50400c5040 0x7f50400ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5024003530 con 0x7f5040144360 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2369728831 0 0) 0x7f5024003530 con 0x7f5040144360 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 >> v2:172.21.15.172:3300/0 conn(0x7f50400c5040 msgr2=0x7f50400ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 --2- 172.21.15.164:0/2823972841 >> v2:172.21.15.172:3300/0 conn(0x7f50400c5040 0x7f50400ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 >> v1:172.21.15.172:6789/0 conn(0x7f504006b3a0 legacy=0x7f50400ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400adf30 con 0x7f5040144360 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5030001680 con 0x7f5040144360 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 >> v1:172.21.15.164:6789/0 conn(0x7f5040144360 legacy=0x7f504006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50240039c0 0x7f5024003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5024004810 0x7f5024004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 0x7f502400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ae860 con 0x7f50240051d0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c070 con 0x7f50240039c0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c270 con 0x7f5024004810 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 0x7f502400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503de00640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50240039c0 0x7f5024003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f5035400640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5024004810 0x7f5024004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 -- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5024004810 msgr2=0x7f5024004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5024004810 0x7f5024004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 -- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50240039c0 msgr2=0x7f5024003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50240039c0 0x7f5024003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 -- 172.21.15.164:0/2823972841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024003530 con 0x7f50240051d0 2024-05-08T04:40:17.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503de00640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50240039c0 0x7f5024003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:17.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.389+0000 7f503e800640 1 -- 172.21.15.164:0/2823972841 reap_dead start 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f503e800640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 0x7f502400b9d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5038009fc0 tx=0x7f503801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608c037e0e4427b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030007d00 con 0x7f50240051d0 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50380a5d10 con 0x7f50240051d0 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f503d400640 1 -- 172.21.15.164:0/2823972841 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038009c10 con 0x7f50240051d0 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 msgr2=0x7f502400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 0x7f502400b9d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5038009fc0 tx=0x7f503801ccc0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- 172.21.15.164:0/2823972841 shutdown_connections 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 --2- 172.21.15.164:0/2823972841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50240051d0 0x7f502400b9d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- 172.21.15.164:0/2823972841 >> 172.21.15.164:0/2823972841 conn(0x7f50400cf770 msgr2=0x7f50400cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- 172.21.15.164:0/2823972841 shutdown_connections 2024-05-08T04:40:17.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- 172.21.15.164:0/2823972841 wait complete. 2024-05-08T04:40:17.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 Processor -- start 2024-05-08T04:40:17.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- start start 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50240051d0 0x7f504006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f504015c020 0x7f504015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ae4f0 con 0x7f504006c3d0 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040144740 con 0x7f504015c020 2024-05-08T04:40:17.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5044a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c4a60 con 0x7f50240051d0 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5035400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5035400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34758/0 (socket says 172.21.15.164:34758) 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.393+0000 7f5035400640 1 -- 172.21.15.164:0/1895407071 learned_addr learned my addr 172.21.15.164:0/1895407071 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f503de00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50240051d0 0x7f504006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f503e800640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f504015c020 0x7f504015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 -- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50240051d0 msgr2=0x7f504006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50240051d0 0x7f504006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 -- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f504015c020 msgr2=0x7f504015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f504015c020 0x7f504015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 -- 172.21.15.164:0/1895407071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504006d970 con 0x7f504006c3d0 2024-05-08T04:40:17.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5035400640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f502800f540 tx=0x7f502800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50280052e0 con 0x7f504006c3d0 2024-05-08T04:40:17.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5028018070 con 0x7f504006c3d0 2024-05-08T04:40:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028005480 con 0x7f504006c3d0 2024-05-08T04:40:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504006c8d0 con 0x7f504006c3d0 2024-05-08T04:40:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504006ce90 con 0x7f504006c3d0 2024-05-08T04:40:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040072d30 con 0x7f504006c3d0 2024-05-08T04:40:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f5028003800 con 0x7f504006c3d0 2024-05-08T04:40:17.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 0x7f501c069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5028085130 con 0x7f504006c3d0 2024-05-08T04:40:17.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.397+0000 7f503de00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 0x7f501c069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.401+0000 7f503de00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 0x7f501c069690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c10 tx=0x7f5030007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.401+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50280556b0 con 0x7f504006c3d0 2024-05-08T04:40:17.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.557+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f504004f010 con 0x7f504006c3d0 2024-05-08T04:40:17.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5036a00640 1 -- 172.21.15.164:0/1895407071 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5028059750 con 0x7f504006c3d0 2024-05-08T04:40:17.564 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 msgr2=0x7f501c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 0x7f501c069690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c10 tx=0x7f5030007900 comp rx=0 tx=0).stop 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 msgr2=0x7f504015b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f502800f540 tx=0x7f502800fa00 comp rx=0 tx=0).stop 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 shutdown_connections 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f501c0671d0 0x7f501c069690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f504015c020 0x7f504015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f504006c3d0 0x7f504015b390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 --2- 172.21.15.164:0/1895407071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50240051d0 0x7f504006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 >> 172.21.15.164:0/1895407071 conn(0x7f50400cf770 msgr2=0x7f50400cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 shutdown_connections 2024-05-08T04:40:17.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.561+0000 7f5044a00640 1 -- 172.21.15.164:0/1895407071 wait complete. 2024-05-08T04:40:17.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:17.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.717+0000 7fa52b400640 1 Processor -- start 2024-05-08T04:40:17.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.717+0000 7fa52b400640 1 -- start start 2024-05-08T04:40:17.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa52b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa52415ab10 0x7fa52406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa52b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa5240ae6e0 0x7fa5240aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa52b400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52415e870 con 0x7fa5240ce780 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa52b400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52406bb80 con 0x7fa5240ae6e0 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa52b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52415eb50 con 0x7fa52415ab10 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa5240ae6e0 0x7fa5240aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa5240ae6e0 0x7fa5240aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34768/0 (socket says 172.21.15.164:34768) 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 -- 172.21.15.164:0/3957994217 learned_addr learned my addr 172.21.15.164:0/3957994217 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa528a00640 1 --2- 172.21.15.164:0/3957994217 >> v2:172.21.15.172:3300/0 conn(0x7fa52415ab10 0x7fa52406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 -- 172.21.15.164:0/3957994217 >> v2:172.21.15.172:3300/0 conn(0x7fa52415ab10 msgr2=0x7fa52406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 --2- 172.21.15.164:0/3957994217 >> v2:172.21.15.172:3300/0 conn(0x7fa52415ab10 0x7fa52406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 -- 172.21.15.164:0/3957994217 >> v1:172.21.15.172:6789/0 conn(0x7fa5240ce780 legacy=0x7fa5240c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 -- 172.21.15.164:0/3957994217 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52415ee30 con 0x7fa5240ae6e0 2024-05-08T04:40:17.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523e00640 1 --2- 172.21.15.164:0/3957994217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa5240aeb70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa5240c4920 tx=0x7fa51001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca212aa96690095 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.721+0000 7fa523400640 1 -- 172.21.15.164:0/3957994217 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100ab660 con 0x7fa5240ae6e0 2024-05-08T04:40:17.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa523400640 1 -- 172.21.15.164:0/3957994217 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5100b0070 con 0x7fa5240ae6e0 2024-05-08T04:40:17.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa523400640 1 -- 172.21.15.164:0/3957994217 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100aba20 con 0x7fa5240ae6e0 2024-05-08T04:40:17.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- 172.21.15.164:0/3957994217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 msgr2=0x7fa5240aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- 172.21.15.164:0/3957994217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa5240aeb70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa5240c4920 tx=0x7fa51001c460 comp rx=0 tx=0).stop 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- 172.21.15.164:0/3957994217 shutdown_connections 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- 172.21.15.164:0/3957994217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa5240aeb70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- 172.21.15.164:0/3957994217 >> v2:172.21.15.172:3300/0 conn(0x7fa52415ab10 0x7fa52406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- 172.21.15.164:0/3957994217 >> 172.21.15.164:0/3957994217 conn(0x7fa5240cf770 msgr2=0x7fa5240cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- 172.21.15.164:0/3957994217 shutdown_connections 2024-05-08T04:40:17.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- 172.21.15.164:0/3957994217 wait complete. 2024-05-08T04:40:17.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 Processor -- start 2024-05-08T04:40:17.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- start start 2024-05-08T04:40:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5240ce780 0x7fa5240b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa52415ab10 0x7fa5240cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240cefa0 con 0x7fa5240ae6e0 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524167ae0 con 0x7fa52415ab10 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa52b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52413d0b0 con 0x7fa5240ce780 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa528a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa528a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34782/0 (socket says 172.21.15.164:34782) 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa528a00640 1 -- 172.21.15.164:0/3349949878 learned_addr learned my addr 172.21.15.164:0/3349949878 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa523e00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5240ce780 0x7fa5240b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.725+0000 7fa529400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa52415ab10 0x7fa5240cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 -- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5240ce780 msgr2=0x7fa5240b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5240ce780 0x7fa5240b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 -- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa52415ab10 msgr2=0x7fa5240cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa52415ab10 0x7fa5240cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 -- 172.21.15.164:0/3349949878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52413d350 con 0x7fa5240ae6e0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa528a00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa51400efc0 tx=0x7fa51400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514018020 con 0x7fa5240ae6e0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514014480 con 0x7fa5240ae6e0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514014850 con 0x7fa5240ae6e0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52413d630 con 0x7fa5240ae6e0 2024-05-08T04:40:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52413dbf0 con 0x7fa5240ae6e0 2024-05-08T04:40:17.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524072d30 con 0x7fa5240ae6e0 2024-05-08T04:40:17.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fa51401b020 con 0x7fa5240ae6e0 2024-05-08T04:40:17.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 0x7fa504069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:17.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa5140862c0 con 0x7fa5240ae6e0 2024-05-08T04:40:17.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.729+0000 7fa523e00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 0x7fa504069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:17.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.733+0000 7fa523e00640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 0x7fa504069690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa5100af000 tx=0x7fa5100a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:17.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.733+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa514022030 con 0x7fa5240ae6e0 2024-05-08T04:40:17.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.893+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa52404f010 con 0x7fa5240ae6e0 2024-05-08T04:40:17.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.893+0000 7fa520c00640 1 -- 172.21.15.164:0/3349949878 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa514010040 con 0x7fa5240ae6e0 2024-05-08T04:40:17.899 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 msgr2=0x7fa504069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 0x7fa504069690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa5100af000 tx=0x7fa5100a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 msgr2=0x7fa52415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa51400efc0 tx=0x7fa51400c490 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 shutdown_connections 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa5040671d0 0x7fa504069690 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5240ce780 0x7fa5240b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa52415ab10 0x7fa5240cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 --2- 172.21.15.164:0/3349949878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5240ae6e0 0x7fa52415c010 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 >> 172.21.15.164:0/3349949878 conn(0x7fa5240cf770 msgr2=0x7fa5240c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:17.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 shutdown_connections 2024-05-08T04:40:17.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:17.897+0000 7fa52b400640 1 -- 172.21.15.164:0/3349949878 wait complete. 2024-05-08T04:40:17.919 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:18.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 Processor -- start 2024-05-08T04:40:18.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 -- start start 2024-05-08T04:40:18.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f481468f0 0x7f0f480ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0f4806b3a0 0x7f0f480b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f48146cd0 con 0x7f0f480addd0 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f480b4500 con 0x7f0f4806b3a0 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.053+0000 7f0f4da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b3c10 con 0x7f0f481468f0 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0f4806b3a0 0x7f0f480b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f46a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f481468f0 0x7f0f480ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f46000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0f480addd0 0x7f0f4806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:37992/0 (socket says 172.21.15.164:37992) 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f46000640 1 -- 172.21.15.164:0/4184797239 learned_addr learned my addr 172.21.15.164:0/4184797239 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 -- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f481468f0 msgr2=0x7f0f480ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 --2- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f481468f0 0x7f0f480ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 -- 172.21.15.164:0/4184797239 >> v1:172.21.15.172:6789/0 conn(0x7f0f480addd0 legacy=0x7f0f4806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 -- 172.21.15.164:0/4184797239 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f480afd10 con 0x7f0f4806b3a0 2024-05-08T04:40:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f47400640 1 --2- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f4806b3a0 0x7f0f480b4990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009d90 tx=0x7f0f4001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5602b866415d3bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.057+0000 7f0f45600640 1 -- 172.21.15.164:0/4184797239 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400b2070 con 0x7f0f4806b3a0 2024-05-08T04:40:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f45600640 1 -- 172.21.15.164:0/4184797239 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400a5920 con 0x7f0f4806b3a0 2024-05-08T04:40:18.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f45600640 1 -- 172.21.15.164:0/4184797239 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a5cf0 con 0x7f0f4806b3a0 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f4806b3a0 msgr2=0x7f0f480b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f4806b3a0 0x7f0f480b4990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009d90 tx=0x7f0f4001c970 comp rx=0 tx=0).stop 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- 172.21.15.164:0/4184797239 shutdown_connections 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f4806b3a0 0x7f0f480b4990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- 172.21.15.164:0/4184797239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f481468f0 0x7f0f480ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- 172.21.15.164:0/4184797239 >> 172.21.15.164:0/4184797239 conn(0x7f0f480b2d00 msgr2=0x7f0f480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- 172.21.15.164:0/4184797239 shutdown_connections 2024-05-08T04:40:18.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- 172.21.15.164:0/4184797239 wait complete. 2024-05-08T04:40:18.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 Processor -- start 2024-05-08T04:40:18.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- start start 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f480addd0 0x7f0f48159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 0x7f0f4815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b5860 con 0x7f0f4806b3a0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b56e0 con 0x7f0f481468f0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f4da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b59e0 con 0x7f0f480addd0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f46a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f46a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34796/0 (socket says 172.21.15.164:34796) 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f46a00640 1 -- 172.21.15.164:0/1683244775 learned_addr learned my addr 172.21.15.164:0/1683244775 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f47400640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 0x7f0f4815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.061+0000 7f0f46000640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f480addd0 0x7f0f48159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 -- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f480addd0 msgr2=0x7f0f48159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f480addd0 0x7f0f48159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 -- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 msgr2=0x7f0f4815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 0x7f0f4815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 -- 172.21.15.164:0/1683244775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4806df40 con 0x7f0f4806b3a0 2024-05-08T04:40:18.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f47400640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 0x7f0f4815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:18.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f46a00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e870 tx=0x7f0f3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38017070 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3800c940 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4806e220 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ccf0 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4806e7e0 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f48072e80 con 0x7f0f4806b3a0 2024-05-08T04:40:18.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f0f3801a070 con 0x7f0f4806b3a0 2024-05-08T04:40:18.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.065+0000 7f0f3ea00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 0x7f0f28069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.069+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0f38085690 con 0x7f0f4806b3a0 2024-05-08T04:40:18.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.069+0000 7f0f46000640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 0x7f0f28069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.069+0000 7f0f46000640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 0x7f0f28069660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f4806c410 tx=0x7f0f34007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.073+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f38055ca0 con 0x7f0f4806b3a0 2024-05-08T04:40:18.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f4804f010 con 0x7f0f4806b3a0 2024-05-08T04:40:18.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f3ea00640 1 -- 172.21.15.164:0/1683244775 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f38059d40 con 0x7f0f4806b3a0 2024-05-08T04:40:18.229 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:18.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 msgr2=0x7f0f28069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 0x7f0f28069660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f4806c410 tx=0x7f0f34007e90 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 msgr2=0x7f0f4813cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e870 tx=0x7f0f3800ec70 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 shutdown_connections 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f280671a0 0x7f0f28069660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f480addd0 0x7f0f48159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f481468f0 0x7f0f4815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 --2- 172.21.15.164:0/1683244775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f4806b3a0 0x7f0f4813cff0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 >> 172.21.15.164:0/1683244775 conn(0x7f0f480b2d00 msgr2=0x7f0f481436b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 shutdown_connections 2024-05-08T04:40:18.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.225+0000 7f0f4da00640 1 -- 172.21.15.164:0/1683244775 wait complete. 2024-05-08T04:40:18.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:40:18.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 Processor -- start 2024-05-08T04:40:18.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 -- start start 2024-05-08T04:40:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86c415ab60 0x7f86c4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c40cfda0 0x7f86c40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c4072640 con 0x7f86c40b26c0 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c406c770 con 0x7f86c415ab60 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86cac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c406ca50 con 0x7f86c40cfda0 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86c415ab60 0x7f86c4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86bbe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f86c40b26c0 0x7f86c40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38696/0 (socket says 172.21.15.164:38696) 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86bbe00640 1 -- 172.21.15.164:0/404936981 learned_addr learned my addr 172.21.15.164:0/404936981 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c8c00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c40cfda0 0x7f86c40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3e00640 1 -- 172.21.15.164:0/404936981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c40cfda0 msgr2=0x7f86c40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:18.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3e00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c40cfda0 0x7f86c40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3e00640 1 -- 172.21.15.164:0/404936981 >> v1:172.21.15.164:6789/0 conn(0x7f86c40b26c0 legacy=0x7f86c40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3e00640 1 -- 172.21.15.164:0/404936981 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c406cd30 con 0x7f86c415ab60 2024-05-08T04:40:18.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.389+0000 7f86c3400640 1 -- 172.21.15.164:0/404936981 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984934690 0 0) 0x7f86c4072640 con 0x7f86c40b26c0 2024-05-08T04:40:18.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86c3e00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c415ab60 0x7f86c4072100 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f86b40097f0 tx=0x7f86b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aa3732d374cda8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86c3400640 1 -- 172.21.15.164:0/404936981 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40b0070 con 0x7f86c415ab60 2024-05-08T04:40:18.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86c3400640 1 -- 172.21.15.164:0/404936981 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b40ab600 con 0x7f86c415ab60 2024-05-08T04:40:18.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86c3400640 1 -- 172.21.15.164:0/404936981 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40ab9d0 con 0x7f86c415ab60 2024-05-08T04:40:18.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- 172.21.15.164:0/404936981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c415ab60 msgr2=0x7f86c4072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c415ab60 0x7f86c4072100 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f86b40097f0 tx=0x7f86b401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- 172.21.15.164:0/404936981 shutdown_connections 2024-05-08T04:40:18.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c40cfda0 0x7f86c40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 --2- 172.21.15.164:0/404936981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c415ab60 0x7f86c4072100 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- 172.21.15.164:0/404936981 >> 172.21.15.164:0/404936981 conn(0x7f86c406ac90 msgr2=0x7f86c406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- 172.21.15.164:0/404936981 shutdown_connections 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- 172.21.15.164:0/404936981 wait complete. 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 Processor -- start 2024-05-08T04:40:18.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.393+0000 7f86cac00640 1 -- start start 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c40b26c0 0x7f86c40b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c415ab60 0x7f86c415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40aeeb0 con 0x7f86c40cfda0 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40aed30 con 0x7f86c40b26c0 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4165960 con 0x7f86c415ab60 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34806/0 (socket says 172.21.15.164:34806) 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 -- 172.21.15.164:0/3089791129 learned_addr learned my addr 172.21.15.164:0/3089791129 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c8c00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c415ab60 0x7f86c415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c3e00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c40b26c0 0x7f86c40b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 -- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c415ab60 msgr2=0x7f86c415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c415ab60 0x7f86c415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 -- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c40b26c0 msgr2=0x7f86c40b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c40b26c0 0x7f86c40b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 -- 172.21.15.164:0/3089791129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40b3340 con 0x7f86c40cfda0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86bbe00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c520 tx=0x7f86ac00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac018070 con 0x7f86c40cfda0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac0074e0 con 0x7f86c40cfda0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40b3620 con 0x7f86c40cfda0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0049f0 con 0x7f86c40cfda0 2024-05-08T04:40:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c40b3ec0 con 0x7f86c40cfda0 2024-05-08T04:40:18.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c40b37f0 con 0x7f86c40cfda0 2024-05-08T04:40:18.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.397+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f86ac01b070 con 0x7f86c40cfda0 2024-05-08T04:40:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.401+0000 7f86c0c00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 0x7f86a0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.401+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f86ac0848a0 con 0x7f86c40cfda0 2024-05-08T04:40:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.401+0000 7f86c3e00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 0x7f86a0069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.401+0000 7f86c3e00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 0x7f86a0069660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ad000 tx=0x7f86b40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.405+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86ac054f00 con 0x7f86c40cfda0 2024-05-08T04:40:18.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f86c4069d60 con 0x7f86c40cfda0 2024-05-08T04:40:18.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86c0c00640 1 -- 172.21.15.164:0/3089791129 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f86ac058fa0 con 0x7f86c40cfda0 2024-05-08T04:40:18.578 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 msgr2=0x7f86a0069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 0x7f86a0069660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ad000 tx=0x7f86b40a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 msgr2=0x7f86c40b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c520 tx=0x7f86ac00c9e0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 shutdown_connections 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86a00671a0 0x7f86a0069660 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c415ab60 0x7f86c415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c40cfda0 0x7f86c40b43c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 --2- 172.21.15.164:0/3089791129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c40b26c0 0x7f86c40b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.573+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 >> 172.21.15.164:0/3089791129 conn(0x7f86c406ac90 msgr2=0x7f86c4159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:18.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.577+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 shutdown_connections 2024-05-08T04:40:18.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.577+0000 7f86cac00640 1 -- 172.21.15.164:0/3089791129 wait complete. 2024-05-08T04:40:18.599 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:40:18.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:18.599 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:18.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:40:18.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:18.631 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:40:18.687 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:18.687 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:18.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:40:18.697 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:18.697 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:18.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:40:18.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 Processor -- start 2024-05-08T04:40:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 -- start start 2024-05-08T04:40:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fabd0161230 0x7fabd00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 0x7fabd00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd00b8440 con 0x7fabd00d17a0 2024-05-08T04:40:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00d1c70 con 0x7fabd0161230 2024-05-08T04:40:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd8200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00d1ec0 con 0x7fabd00ce640 2024-05-08T04:40:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fabd0161230 0x7fabd00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fabd0161230 0x7fabd00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34808/0 (socket says 172.21.15.164:34808) 2024-05-08T04:40:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 -- 172.21.15.164:0/2260855532 learned_addr learned my addr 172.21.15.164:0/2260855532 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd4e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fabd00d17a0 0x7fabd00d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38708/0 (socket says 172.21.15.164:38708) 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd6200640 1 --2- 172.21.15.164:0/2260855532 >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 0x7fabd00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 -- 172.21.15.164:0/2260855532 >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 msgr2=0x7fabd00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 --2- 172.21.15.164:0/2260855532 >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 0x7fabd00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 -- 172.21.15.164:0/2260855532 >> v1:172.21.15.164:6789/0 conn(0x7fabd00d17a0 legacy=0x7fabd00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabcfe00640 1 -- 172.21.15.164:0/2260855532 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750465005 0 0) 0x7fabbc001120 con 0x7fabd00d17a0 2024-05-08T04:40:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd5800640 1 -- 172.21.15.164:0/2260855532 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00d2250 con 0x7fabd0161230 2024-05-08T04:40:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.877+0000 7fabd6200640 1 --2- 172.21.15.164:0/2260855532 >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 0x7fabd00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd5800640 1 --2- 172.21.15.164:0/2260855532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd0161230 0x7fabd00d0370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fabd00d37d0 tx=0x7fabc401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15548e256f2e9edf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabcfe00640 1 -- 172.21.15.164:0/2260855532 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40b1070 con 0x7fabd0161230 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabcfe00640 1 -- 172.21.15.164:0/2260855532 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc40ac600 con 0x7fabd0161230 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabcfe00640 1 -- 172.21.15.164:0/2260855532 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40ac9d0 con 0x7fabd0161230 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 -- 172.21.15.164:0/2260855532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd0161230 msgr2=0x7fabd00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 --2- 172.21.15.164:0/2260855532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd0161230 0x7fabd00d0370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fabd00d37d0 tx=0x7fabc401c490 comp rx=0 tx=0).stop 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 -- 172.21.15.164:0/2260855532 shutdown_connections 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 --2- 172.21.15.164:0/2260855532 >> v2:172.21.15.172:3300/0 conn(0x7fabd00ce640 0x7fabd00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 --2- 172.21.15.164:0/2260855532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd0161230 0x7fabd00d0370 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 -- 172.21.15.164:0/2260855532 >> 172.21.15.164:0/2260855532 conn(0x7fabd009cea0 msgr2=0x7fabd00d3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 -- 172.21.15.164:0/2260855532 shutdown_connections 2024-05-08T04:40:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.881+0000 7fabd8200640 1 -- 172.21.15.164:0/2260855532 wait complete. 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 Processor -- start 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- start start 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 0x7fabd0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabd0161230 0x7fabd015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0161790 con 0x7fabd00d17a0 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0161610 con 0x7fabd00ce640 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd016aa90 con 0x7fabd0161230 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34818/0 (socket says 172.21.15.164:34818) 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 -- 172.21.15.164:0/819532835 learned_addr learned my addr 172.21.15.164:0/819532835 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd6200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabd0161230 0x7fabd015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd5800640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 0x7fabd0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 -- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabd0161230 msgr2=0x7fabd015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabd0161230 0x7fabd015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 -- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 msgr2=0x7fabd0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 0x7fabd0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 -- 172.21.15.164:0/819532835 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0157540 con 0x7fabd00d17a0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd5800640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 0x7fabd0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd4e00640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c780 tx=0x7fabbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc018070 con 0x7fabd00d17a0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0072f0 con 0x7fabd00d17a0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0047f0 con 0x7fabd00d17a0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd0155a70 con 0x7fabd00d17a0 2024-05-08T04:40:18.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd0156b70 con 0x7fabd00d17a0 2024-05-08T04:40:18.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.885+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0069d60 con 0x7fabd00d17a0 2024-05-08T04:40:18.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.889+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fabbc004990 con 0x7fabd00d17a0 2024-05-08T04:40:18.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.889+0000 7fabcd600640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 0x7fabb0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:18.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.889+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fabbc0848b0 con 0x7fabd00d17a0 2024-05-08T04:40:18.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.889+0000 7fabd5800640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 0x7fabb0069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:18.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.889+0000 7fabd5800640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 0x7fabb0069660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabc40ae000 tx=0x7fabc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:18.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:18.893+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabd0157540 con 0x7fabd00d17a0 2024-05-08T04:40:19.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.077+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd00b8870 con 0x7fabd00d17a0 2024-05-08T04:40:19.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.077+0000 7fabcd600640 1 -- 172.21.15.164:0/819532835 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fabd00b8440 con 0x7fabd00d17a0 2024-05-08T04:40:19.083 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:19.083 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:40:19.083 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 26 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 msgr2=0x7fabb0069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 0x7fabb0069660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabc40ae000 tx=0x7fabc4008040 comp rx=0 tx=0).stop 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 msgr2=0x7fabd015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c780 tx=0x7fabbc00cc40 comp rx=0 tx=0).stop 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 shutdown_connections 2024-05-08T04:40:19.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fabb00671a0 0x7fabb0069660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabd0161230 0x7fabd015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd00d17a0 0x7fabd015c190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 --2- 172.21.15.164:0/819532835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabd00ce640 0x7fabd0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 >> 172.21.15.164:0/819532835 conn(0x7fabd009cea0 msgr2=0x7fabd006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 shutdown_connections 2024-05-08T04:40:19.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.081+0000 7fabd8200640 1 -- 172.21.15.164:0/819532835 wait complete. 2024-05-08T04:40:19.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:19.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 Processor -- start 2024-05-08T04:40:19.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 -- start start 2024-05-08T04:40:19.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd4406b3a0 0x7fbd4406df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd44072320 con 0x7fbd440addd0 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4406d4e0 con 0x7fbd4406b3a0 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44072600 con 0x7fbd440ccdf0 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42304/0 (socket says 172.21.15.164:42304) 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 -- 172.21.15.164:0/4060992517 learned_addr learned my addr 172.21.15.164:0/4060992517 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:19.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd4a000640 1 --2- 172.21.15.164:0/4060992517 >> v2:172.21.15.172:3300/0 conn(0x7fbd4406b3a0 0x7fbd4406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 -- 172.21.15.164:0/4060992517 >> v2:172.21.15.172:3300/0 conn(0x7fbd4406b3a0 msgr2=0x7fbd4406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:19.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 --2- 172.21.15.164:0/4060992517 >> v2:172.21.15.172:3300/0 conn(0x7fbd4406b3a0 0x7fbd4406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 -- 172.21.15.164:0/4060992517 >> v1:172.21.15.172:6789/0 conn(0x7fbd440addd0 legacy=0x7fbd4406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd49600640 1 -- 172.21.15.164:0/4060992517 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4406d730 con 0x7fbd440ccdf0 2024-05-08T04:40:19.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.237+0000 7fbd43e00640 1 -- 172.21.15.164:0/4060992517 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562737256 0 0) 0x7fbd44072320 con 0x7fbd440addd0 2024-05-08T04:40:19.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd49600640 1 --2- 172.21.15.164:0/4060992517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009870 tx=0x7fbd3801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab10507ccc3871 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd43e00640 1 -- 172.21.15.164:0/4060992517 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b1070 con 0x7fbd440ccdf0 2024-05-08T04:40:19.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd43e00640 1 -- 172.21.15.164:0/4060992517 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd380a8050 con 0x7fbd440ccdf0 2024-05-08T04:40:19.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd43e00640 1 -- 172.21.15.164:0/4060992517 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380ac860 con 0x7fbd440ccdf0 2024-05-08T04:40:19.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 -- 172.21.15.164:0/4060992517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 msgr2=0x7fbd440ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 --2- 172.21.15.164:0/4060992517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009870 tx=0x7fbd3801c4d0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 -- 172.21.15.164:0/4060992517 shutdown_connections 2024-05-08T04:40:19.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 --2- 172.21.15.164:0/4060992517 >> v2:172.21.15.172:3300/0 conn(0x7fbd4406b3a0 0x7fbd4406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 --2- 172.21.15.164:0/4060992517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd440ad890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 -- 172.21.15.164:0/4060992517 >> 172.21.15.164:0/4060992517 conn(0x7fbd44066fa0 msgr2=0x7fbd4406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 -- 172.21.15.164:0/4060992517 shutdown_connections 2024-05-08T04:40:19.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.241+0000 7fbd4c000640 1 -- 172.21.15.164:0/4060992517 wait complete. 2024-05-08T04:40:19.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 Processor -- start 2024-05-08T04:40:19.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- start start 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd440addd0 0x7fbd440b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd441480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44157af0 con 0x7fbd4406b3a0 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44071ff0 con 0x7fbd440addd0 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44157c70 con 0x7fbd440ccdf0 2024-05-08T04:40:19.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44084/0 (socket says 172.21.15.164:44084) 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 -- 172.21.15.164:0/1230813156 learned_addr learned my addr 172.21.15.164:0/1230813156 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd48c00640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd440addd0 0x7fbd440b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4a000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd441480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 -- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 msgr2=0x7fbd441480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd441480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 -- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd440addd0 msgr2=0x7fbd440b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd440addd0 0x7fbd440b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 -- 172.21.15.164:0/1230813156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440b0780 con 0x7fbd4406b3a0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd49600640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbd3801a100 tx=0x7fbd3801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b1040 con 0x7fbd4406b3a0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd380bd070 con 0x7fbd4406b3a0 2024-05-08T04:40:19.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd440b0a60 con 0x7fbd4406b3a0 2024-05-08T04:40:19.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b7660 con 0x7fbd4406b3a0 2024-05-08T04:40:19.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.245+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd440b1020 con 0x7fbd4406b3a0 2024-05-08T04:40:19.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd44069d60 con 0x7fbd4406b3a0 2024-05-08T04:40:19.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fbd380b7800 con 0x7fbd4406b3a0 2024-05-08T04:40:19.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd41600640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 0x7fbd24069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbd3811f720 con 0x7fbd4406b3a0 2024-05-08T04:40:19.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd48c00640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 0x7fbd24069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.249+0000 7fbd48c00640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 0x7fbd24069660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbd4406dfe0 tx=0x7fbd30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.253+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd380c7020 con 0x7fbd4406b3a0 2024-05-08T04:40:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.409+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd440cdd10 con 0x7fbd4406b3a0 2024-05-08T04:40:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.409+0000 7fbd41600640 1 -- 172.21.15.164:0/1230813156 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fbd38016150 con 0x7fbd4406b3a0 2024-05-08T04:40:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:19.415 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":48,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:18.232734+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 msgr2=0x7fbd24069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 0x7fbd24069660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbd4406dfe0 tx=0x7fbd30009040 comp rx=0 tx=0).stop 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 msgr2=0x7fbd440d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbd3801a100 tx=0x7fbd3801ad10 comp rx=0 tx=0).stop 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 shutdown_connections 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd240671a0 0x7fbd24069660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd440ccdf0 0x7fbd441480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd440addd0 0x7fbd440b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 --2- 172.21.15.164:0/1230813156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4406b3a0 0x7fbd440d0700 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 >> 172.21.15.164:0/1230813156 conn(0x7fbd44066fa0 msgr2=0x7fbd440aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 shutdown_connections 2024-05-08T04:40:19.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.413+0000 7fbd4c000640 1 -- 172.21.15.164:0/1230813156 wait complete. 2024-05-08T04:40:19.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:39:50.888294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:39:52.156282+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:19.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:19.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.569+0000 7f5881400640 1 Processor -- start 2024-05-08T04:40:19.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.569+0000 7f5881400640 1 -- start start 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5881400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f587c06cce0 0x7f587c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5881400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 0x7f587c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5881400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c15e870 con 0x7f587c15ab10 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5881400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c06bb80 con 0x7f587c06cce0 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5881400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c0cf0f0 con 0x7f587c0cecc0 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f587c06cce0 0x7f587c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f587c06cce0 0x7f587c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44094/0 (socket says 172.21.15.164:44094) 2024-05-08T04:40:19.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 -- 172.21.15.164:0/3842793880 learned_addr learned my addr 172.21.15.164:0/3842793880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f587b200640 1 --2- 172.21.15.164:0/3842793880 >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 0x7f587c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 -- 172.21.15.164:0/3842793880 >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 msgr2=0x7f587c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 --2- 172.21.15.164:0/3842793880 >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 0x7f587c0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 -- 172.21.15.164:0/3842793880 >> v1:172.21.15.172:6789/0 conn(0x7f587c15ab10 legacy=0x7f587c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f5871e00640 1 -- 172.21.15.164:0/3842793880 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c15eb50 con 0x7f587c06cce0 2024-05-08T04:40:19.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.573+0000 7f587b200640 1 --2- 172.21.15.164:0/3842793880 >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 0x7f587c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5871e00640 1 --2- 172.21.15.164:0/3842793880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0ce780 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f587c0ca1f0 tx=0x7f586401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d1cb06cb8aa093 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5879e00640 1 -- 172.21.15.164:0/3842793880 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640ab660 con 0x7f587c06cce0 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5879e00640 1 -- 172.21.15.164:0/3842793880 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640b0070 con 0x7f587c06cce0 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5879e00640 1 -- 172.21.15.164:0/3842793880 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640aba20 con 0x7f587c06cce0 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- 172.21.15.164:0/3842793880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 msgr2=0x7f587c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- 172.21.15.164:0/3842793880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0ce780 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f587c0ca1f0 tx=0x7f586401c460 comp rx=0 tx=0).stop 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- 172.21.15.164:0/3842793880 shutdown_connections 2024-05-08T04:40:19.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- 172.21.15.164:0/3842793880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0ce780 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- 172.21.15.164:0/3842793880 >> v2:172.21.15.172:3300/0 conn(0x7f587c0cecc0 0x7f587c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- 172.21.15.164:0/3842793880 >> 172.21.15.164:0/3842793880 conn(0x7f587c0cf770 msgr2=0x7f587c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- 172.21.15.164:0/3842793880 shutdown_connections 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- 172.21.15.164:0/3842793880 wait complete. 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 Processor -- start 2024-05-08T04:40:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- start start 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587c0cecc0 0x7f587c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587c15ab10 0x7f587c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0aedf0 con 0x7f587c06cce0 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c15aef0 con 0x7f587c0cecc0 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f5881400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0aef70 con 0x7f587c15ab10 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.577+0000 7f587a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44108/0 (socket says 172.21.15.164:44108) 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 -- 172.21.15.164:0/3301250585 learned_addr learned my addr 172.21.15.164:0/3301250585 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5871e00640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587c0cecc0 0x7f587c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587b200640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587c15ab10 0x7f587c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 -- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587c15ab10 msgr2=0x7f587c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587c15ab10 0x7f587c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 -- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587c0cecc0 msgr2=0x7f587c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587c0cecc0 0x7f587c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 -- 172.21.15.164:0/3301250585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0b4000 con 0x7f587c06cce0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f587a800640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c790 tx=0x7f586c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c016020 con 0x7f587c06cce0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c006d60 con 0x7f587c06cce0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c003bc0 con 0x7f587c06cce0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c0b42e0 con 0x7f587c06cce0 2024-05-08T04:40:19.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c0c99e0 con 0x7f587c06cce0 2024-05-08T04:40:19.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c072d30 con 0x7f587c06cce0 2024-05-08T04:40:19.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f586c00a850 con 0x7f587c06cce0 2024-05-08T04:40:19.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 0x7f5858069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f586c085160 con 0x7f587c06cce0 2024-05-08T04:40:19.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.581+0000 7f5871e00640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 0x7f5858069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.585+0000 7f5871e00640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 0x7f5858069660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58640af000 tx=0x7f58640a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.585+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f586c020020 con 0x7f587c06cce0 2024-05-08T04:40:19.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.737+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f587c0b44b0 con 0x7f587c06cce0 2024-05-08T04:40:19.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.737+0000 7f5873400640 1 -- 172.21.15.164:0/3301250585 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f586c055e10 con 0x7f587c06cce0 2024-05-08T04:40:19.744 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 msgr2=0x7f5858069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 0x7f5858069660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58640af000 tx=0x7f58640a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 msgr2=0x7f587c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c790 tx=0x7f586c00cc50 comp rx=0 tx=0).stop 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 shutdown_connections 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f58580671a0 0x7f5858069660 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587c15ab10 0x7f587c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587c0cecc0 0x7f587c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 --2- 172.21.15.164:0/3301250585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06cce0 0x7f587c0b3960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 >> 172.21.15.164:0/3301250585 conn(0x7f587c0cf770 msgr2=0x7f587c159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 shutdown_connections 2024-05-08T04:40:19.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.741+0000 7f5881400640 1 -- 172.21.15.164:0/3301250585 wait complete. 2024-05-08T04:40:19.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:19.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 Processor -- start 2024-05-08T04:40:19.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 -- start start 2024-05-08T04:40:19.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f4c0cee90 0x7f3f4c165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c15e870 con 0x7f3f4c15ab10 2024-05-08T04:40:19.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c15eb50 con 0x7f3f4c06cce0 2024-05-08T04:40:19.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f51c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c06bb80 con 0x7f3f4c0cee90 2024-05-08T04:40:19.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3f4c15ab10 0x7f3f4c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38064/0 (socket says 172.21.15.164:38064) 2024-05-08T04:40:19.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4aa00640 1 -- 172.21.15.164:0/310208507 learned_addr learned my addr 172.21.15.164:0/310208507 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4b400640 1 --2- 172.21.15.164:0/310208507 >> v2:172.21.15.172:3300/0 conn(0x7f3f4c0cee90 0x7f3f4c165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951808301 0 0) 0x7f3f4c15e870 con 0x7f3f4c15ab10 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f30005740 con 0x7f3f4c15ab10 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458624198 0 0) 0x7f3f4c15eb50 con 0x7f3f4c06cce0 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c15e870 con 0x7f3f4c06cce0 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376991764 0 0) 0x7f3f30005740 con 0x7f3f4c15ab10 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4b400640 1 -- 172.21.15.164:0/310208507 >> v1:172.21.15.172:6789/0 conn(0x7f3f4c06cce0 legacy=0x7f3f4c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4b400640 1 -- 172.21.15.164:0/310208507 >> v1:172.21.15.164:6789/0 conn(0x7f3f4c15ab10 legacy=0x7f3f4c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.901+0000 7f3f4b400640 1 -- 172.21.15.164:0/310208507 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c15ee30 con 0x7f3f4c0cee90 2024-05-08T04:40:19.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f4b400640 1 --2- 172.21.15.164:0/310208507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c165ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009d90 tx=0x7f3f4401c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aaba9b14d78b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c001680 con 0x7f3f4c0cee90 2024-05-08T04:40:19.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440a6430 con 0x7f3f4c0cee90 2024-05-08T04:40:19.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f49600640 1 -- 172.21.15.164:0/310208507 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440a6800 con 0x7f3f4c0cee90 2024-05-08T04:40:19.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 -- 172.21.15.164:0/310208507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c0cee90 msgr2=0x7f3f4c165ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 --2- 172.21.15.164:0/310208507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c165ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009d90 tx=0x7f3f4401c7d0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 -- 172.21.15.164:0/310208507 shutdown_connections 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 --2- 172.21.15.164:0/310208507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c165ad0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 -- 172.21.15.164:0/310208507 >> 172.21.15.164:0/310208507 conn(0x7f3f4c0cf770 msgr2=0x7f3f4c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 -- 172.21.15.164:0/310208507 shutdown_connections 2024-05-08T04:40:19.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.905+0000 7f3f51c00640 1 -- 172.21.15.164:0/310208507 wait complete. 2024-05-08T04:40:19.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 Processor -- start 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- start start 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 0x7f3f4c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 0x7f3f4c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0aef70 con 0x7f3f4c0cee90 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0aedf0 con 0x7f3f4c15ab10 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0cde80 con 0x7f3f4c06cce0 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44116/0 (socket says 172.21.15.164:44116) 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 -- 172.21.15.164:0/3265245726 learned_addr learned my addr 172.21.15.164:0/3265245726 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4aa00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 0x7f3f4c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4b400640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 0x7f3f4c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 -- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 msgr2=0x7f3f4c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 0x7f3f4c151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 -- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 msgr2=0x7f3f4c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 0x7f3f4c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 -- 172.21.15.164:0/3265245726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0c7e20 con 0x7f3f4c0cee90 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4aa00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 0x7f3f4c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4b400640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 0x7f3f4c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:19.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f4a000640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c490 tx=0x7f3f3800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f380052d0 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38005470 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c0c80c0 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38012870 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c0c8680 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.909+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c072d30 con 0x7f3f4c0cee90 2024-05-08T04:40:19.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.913+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f3f38012a10 con 0x7f3f4c0cee90 2024-05-08T04:40:19.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.913+0000 7f3f42a00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 0x7f3f2c069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:19.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.913+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f3f38085170 con 0x7f3f4c0cee90 2024-05-08T04:40:19.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.913+0000 7f3f4aa00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 0x7f3f2c069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:19.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.913+0000 7f3f4aa00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 0x7f3f2c069690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0011f0 tx=0x7f3f3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:19.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:19.917+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f38055da0 con 0x7f3f4c0cee90 2024-05-08T04:40:20.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.069+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f4c0c4bf0 con 0x7f3f4c0cee90 2024-05-08T04:40:20.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f42a00640 1 -- 172.21.15.164:0/3265245726 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f380556a0 con 0x7f3f4c0cee90 2024-05-08T04:40:20.076 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:20.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 msgr2=0x7f3f2c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 0x7f3f2c069690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0011f0 tx=0x7f3f3c009040 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 msgr2=0x7f3f4c155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c490 tx=0x7f3f3800c950 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 shutdown_connections 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f2c0671d0 0x7f3f2c069690 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f4c15ab10 0x7f3f4c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f4c0cee90 0x7f3f4c155550 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 --2- 172.21.15.164:0/3265245726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f4c06cce0 0x7f3f4c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 >> 172.21.15.164:0/3265245726 conn(0x7f3f4c0cf770 msgr2=0x7f3f4c0ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 shutdown_connections 2024-05-08T04:40:20.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.073+0000 7f3f51c00640 1 -- 172.21.15.164:0/3265245726 wait complete. 2024-05-08T04:40:20.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:20.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.233+0000 7fbfe8200640 1 Processor -- start 2024-05-08T04:40:20.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.233+0000 7fbfe8200640 1 -- start start 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe8200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbfe006f100 0x7fbfe0166a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe8200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe015bfd0 con 0x7fbfe015bbf0 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe8200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe006f5d0 con 0x7fbfe00d0450 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe8200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00b0ef0 con 0x7fbfe006f100 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe5800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbfe015bbf0 0x7fbfe00cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38074/0 (socket says 172.21.15.164:38074) 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe5800640 1 -- 172.21.15.164:0/2932584917 learned_addr learned my addr 172.21.15.164:0/2932584917 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:20.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> v2:172.21.15.172:3300/0 conn(0x7fbfe006f100 0x7fbfe0166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643974369 0 0) 0x7fbfe015bfd0 con 0x7fbfe015bbf0 2024-05-08T04:40:20.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8003530 con 0x7fbfe015bbf0 2024-05-08T04:40:20.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272791578 0 0) 0x7fbfc8003530 con 0x7fbfe015bbf0 2024-05-08T04:40:20.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 >> v2:172.21.15.172:3300/0 conn(0x7fbfe006f100 msgr2=0x7fbfe0166a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:20.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 --2- 172.21.15.164:0/2932584917 >> v2:172.21.15.172:3300/0 conn(0x7fbfe006f100 0x7fbfe0166a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 >> v1:172.21.15.172:6789/0 conn(0x7fbfe00d0450 legacy=0x7fbfe006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe006f8b0 con 0x7fbfe015bbf0 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fbfd4001a00 con 0x7fbfe015bbf0 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 >> v1:172.21.15.164:6789/0 conn(0x7fbfe015bbf0 legacy=0x7fbfe00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfc8003a10 0x7fbfc8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 0x7fbfc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00b0ef0 con 0x7fbfc8005220 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc800c170 con 0x7fbfc8004810 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc800c370 con 0x7fbfc8003a10 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> v2:172.21.15.172:3300/0 conn(0x7fbfe006f100 0x7fbfe0166a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 0x7fbfc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe5800640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfc8003a10 0x7fbfc8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe4e00640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 -- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfc8003a10 msgr2=0x7fbfc8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:20.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfc8003a10 0x7fbfc8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 -- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8004810 msgr2=0x7fbfc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:20.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8004810 0x7fbfc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.237+0000 7fbfe6200640 1 -- 172.21.15.164:0/2932584917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8003530 con 0x7fbfc8005220 2024-05-08T04:40:20.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe6200640 1 -- 172.21.15.164:0/2932584917 reap_dead start 2024-05-08T04:40:20.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe6200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 0x7fbfc800b9d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00d2980 tx=0x7fbfd801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403a6c5376531178 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8009880 con 0x7fbfc8005220 2024-05-08T04:40:20.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80a58b0 con 0x7fbfc8005220 2024-05-08T04:40:20.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfdfe00640 1 -- 172.21.15.164:0/2932584917 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80a5c80 con 0x7fbfc8005220 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 msgr2=0x7fbfc800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 0x7fbfc800b9d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00d2980 tx=0x7fbfd801cbf0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- 172.21.15.164:0/2932584917 shutdown_connections 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 --2- 172.21.15.164:0/2932584917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8005220 0x7fbfc800b9d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- 172.21.15.164:0/2932584917 >> 172.21.15.164:0/2932584917 conn(0x7fbfe009aa20 msgr2=0x7fbfe009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- 172.21.15.164:0/2932584917 shutdown_connections 2024-05-08T04:40:20.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- 172.21.15.164:0/2932584917 wait complete. 2024-05-08T04:40:20.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 Processor -- start 2024-05-08T04:40:20.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.241+0000 7fbfe8200640 1 -- start start 2024-05-08T04:40:20.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8005220 0x7fbfe01f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfe00ca6c0 0x7fbfe00cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe006e9f0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe006fe70 con 0x7fbfc8005220 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe006eb70 con 0x7fbfe00ca6c0 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44136/0 (socket says 172.21.15.164:44136) 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 -- 172.21.15.164:0/1112453557 learned_addr learned my addr 172.21.15.164:0/1112453557 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe5800640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8005220 0x7fbfe01f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe6200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfe00ca6c0 0x7fbfe00cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 -- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfe00ca6c0 msgr2=0x7fbfe00cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfe00ca6c0 0x7fbfe00cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 -- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8005220 msgr2=0x7fbfe01f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8005220 0x7fbfe01f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 -- 172.21.15.164:0/1112453557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00c8eb0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe4e00640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00f820 tx=0x7fbfcc00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc001cf0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc018070 con 0x7fbfe00c8a50 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00d730 con 0x7fbfe00c8a50 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe00c9180 con 0x7fbfe00c8a50 2024-05-08T04:40:20.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe00c9740 con 0x7fbfe00c8a50 2024-05-08T04:40:20.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe009f9e0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.245+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fbfcc00d8d0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.249+0000 7fbfdd600640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 0x7fbfc0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.249+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbfcc0851b0 con 0x7fbfe00c8a50 2024-05-08T04:40:20.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.249+0000 7fbfe5800640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 0x7fbfc0069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.249+0000 7fbfe5800640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 0x7fbfc0069660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4000f90 tx=0x7fbfd4002140 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.253+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfcc055790 con 0x7fbfe00c8a50 2024-05-08T04:40:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.405+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfe015ef50 con 0x7fbfe00c8a50 2024-05-08T04:40:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfdd600640 1 -- 172.21.15.164:0/1112453557 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfcc059830 con 0x7fbfe00c8a50 2024-05-08T04:40:20.412 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 msgr2=0x7fbfc0069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 0x7fbfc0069660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4000f90 tx=0x7fbfd4002140 comp rx=0 tx=0).stop 2024-05-08T04:40:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 msgr2=0x7fbfe015df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00f820 tx=0x7fbfcc00fce0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 shutdown_connections 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbfc00671a0 0x7fbfc0069660 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbfe00ca6c0 0x7fbfe00cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8a50 0x7fbfe015df50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 --2- 172.21.15.164:0/1112453557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfc8005220 0x7fbfe01f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 >> 172.21.15.164:0/1112453557 conn(0x7fbfe009aa20 msgr2=0x7fbfe006ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 shutdown_connections 2024-05-08T04:40:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.409+0000 7fbfe8200640 1 -- 172.21.15.164:0/1112453557 wait complete. 2024-05-08T04:40:20.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 Processor -- start 2024-05-08T04:40:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 -- start start 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efc2015dd70 0x7efc201619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc200b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2006cc70 con 0x7efc2006c7b0 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2006cf50 con 0x7efc2015dd70 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc27000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc200b1570 con 0x7efc201625f0 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efc2015dd70 0x7efc201619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efc2015dd70 0x7efc201619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44152/0 (socket says 172.21.15.164:44152) 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 -- 172.21.15.164:0/4287751050 learned_addr learned my addr 172.21.15.164:0/4287751050 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc17400640 1 --1- 172.21.15.164:0/4287751050 >> v1:172.21.15.164:6789/0 conn(0x7efc2006c7b0 0x7efc20161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38076/0 (socket says 172.21.15.164:38076) 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc25000640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc200b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 -- 172.21.15.164:0/4287751050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 msgr2=0x7efc200b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc200b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 -- 172.21.15.164:0/4287751050 >> v1:172.21.15.164:6789/0 conn(0x7efc2006c7b0 legacy=0x7efc20161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.565+0000 7efc1fe00640 1 -- 172.21.15.164:0/4287751050 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200b1850 con 0x7efc2015dd70 2024-05-08T04:40:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc1fe00640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2015dd70 0x7efc201619a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc10009870 tx=0x7efc1001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b83dd144e67727 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc1f400640 1 -- 172.21.15.164:0/4287751050 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100b5070 con 0x7efc2015dd70 2024-05-08T04:40:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc1f400640 1 -- 172.21.15.164:0/4287751050 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc100a8a50 con 0x7efc2015dd70 2024-05-08T04:40:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc1f400640 1 -- 172.21.15.164:0/4287751050 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100a8e20 con 0x7efc2015dd70 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 -- 172.21.15.164:0/4287751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2015dd70 msgr2=0x7efc201619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2015dd70 0x7efc201619a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc10009870 tx=0x7efc1001ed50 comp rx=0 tx=0).stop 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 -- 172.21.15.164:0/4287751050 shutdown_connections 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc200b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 --2- 172.21.15.164:0/4287751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2015dd70 0x7efc201619a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 -- 172.21.15.164:0/4287751050 >> 172.21.15.164:0/4287751050 conn(0x7efc200b34f0 msgr2=0x7efc200b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 -- 172.21.15.164:0/4287751050 shutdown_connections 2024-05-08T04:40:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.569+0000 7efc27000640 1 -- 172.21.15.164:0/4287751050 wait complete. 2024-05-08T04:40:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 Processor -- start 2024-05-08T04:40:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- start start 2024-05-08T04:40:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 0x7efc20160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc201f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2015e2d0 con 0x7efc2006c7b0 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2015e150 con 0x7efc2015dd70 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc200b22b0 con 0x7efc201625f0 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44166/0 (socket says 172.21.15.164:44166) 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 -- 172.21.15.164:0/1299057389 learned_addr learned my addr 172.21.15.164:0/1299057389 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc17400640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 0x7efc20160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 -- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 msgr2=0x7efc201f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc25000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc201f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc201f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 -- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 msgr2=0x7efc20160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 0x7efc20160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 -- 172.21.15.164:0/1299057389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20150240 con 0x7efc2006c7b0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc17400640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 0x7efc20160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc25000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc201f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1fe00640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc100b3040 tx=0x7efc1001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100c0070 con 0x7efc2006c7b0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc100ae550 con 0x7efc2006c7b0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100b5070 con 0x7efc2006c7b0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2015e6b0 con 0x7efc2006c7b0 2024-05-08T04:40:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2015ec70 con 0x7efc2006c7b0 2024-05-08T04:40:20.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.573+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc20069d60 con 0x7efc2006c7b0 2024-05-08T04:40:20.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.577+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7efc100b9070 con 0x7efc2006c7b0 2024-05-08T04:40:20.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.577+0000 7efc1cc00640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 0x7efbfc069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.577+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7efc10122c60 con 0x7efc2006c7b0 2024-05-08T04:40:20.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.577+0000 7efc17400640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 0x7efbfc069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.577+0000 7efc17400640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 0x7efbfc069690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc08007fd0 tx=0x7efc08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.581+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc20150240 con 0x7efc2006c7b0 2024-05-08T04:40:20.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.765+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc2015d510 con 0x7efc2006c7b0 2024-05-08T04:40:20.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.765+0000 7efc1cc00640 1 -- 172.21.15.164:0/1299057389 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4944 (secure 0 0 0) 0x7efc100188b0 con 0x7efc2006c7b0 2024-05-08T04:40:20.772 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:20.772 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:40:20.772 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 26 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 msgr2=0x7efbfc069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 0x7efbfc069690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc08007fd0 tx=0x7efc08009040 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 msgr2=0x7efc2014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc100b3040 tx=0x7efc1001baf0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 shutdown_connections 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efbfc0671d0 0x7efbfc069690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc201625f0 0x7efc201f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc2015dd70 0x7efc20160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 --2- 172.21.15.164:0/1299057389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc2006c7b0 0x7efc2014fd00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 >> 172.21.15.164:0/1299057389 conn(0x7efc200b34f0 msgr2=0x7efc200b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 shutdown_connections 2024-05-08T04:40:20.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.769+0000 7efc27000640 1 -- 172.21.15.164:0/1299057389 wait complete. 2024-05-08T04:40:20.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:20.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 Processor -- start 2024-05-08T04:40:20.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 -- start start 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc15dd70 0x7f25fc1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25fc1625f0 0x7f25fc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc06cc70 con 0x7f25fc06c7b0 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc06cf50 con 0x7f25fc1625f0 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f2600a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b1570 con 0x7f25fc15dd70 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25fa800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25fc1625f0 0x7f25fc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25f9400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f25fc06c7b0 0x7f25fc161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38080/0 (socket says 172.21.15.164:38080) 2024-05-08T04:40:20.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25f9400640 1 -- 172.21.15.164:0/223980623 learned_addr learned my addr 172.21.15.164:0/223980623 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:20.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25f9e00640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc15dd70 0x7f25fc1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25f8a00640 1 -- 172.21.15.164:0/223980623 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322042326 0 0) 0x7f25fc06cc70 con 0x7f25fc06c7b0 2024-05-08T04:40:20.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25f8a00640 1 -- 172.21.15.164:0/223980623 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25e0003530 con 0x7f25fc06c7b0 2024-05-08T04:40:20.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25fa800640 1 -- 172.21.15.164:0/223980623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc15dd70 msgr2=0x7f25fc1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:20.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25fa800640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc15dd70 0x7f25fc1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25fa800640 1 -- 172.21.15.164:0/223980623 >> v1:172.21.15.164:6789/0 conn(0x7f25fc06c7b0 legacy=0x7f25fc161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.925+0000 7f25fa800640 1 -- 172.21.15.164:0/223980623 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0b1850 con 0x7f25fc1625f0 2024-05-08T04:40:20.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f25fa800640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc1625f0 0x7f25fc0b1d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009d90 tx=0x7f25f401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968ec0191192b834 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f25f8a00640 1 -- 172.21.15.164:0/223980623 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40b4070 con 0x7f25fc1625f0 2024-05-08T04:40:20.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f25f8a00640 1 -- 172.21.15.164:0/223980623 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40af600 con 0x7f25fc1625f0 2024-05-08T04:40:20.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f25f8a00640 1 -- 172.21.15.164:0/223980623 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40af9d0 con 0x7f25fc1625f0 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 -- 172.21.15.164:0/223980623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc1625f0 msgr2=0x7f25fc0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc1625f0 0x7f25fc0b1d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009d90 tx=0x7f25f401f150 comp rx=0 tx=0).stop 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 -- 172.21.15.164:0/223980623 shutdown_connections 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc1625f0 0x7f25fc0b1d70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 --2- 172.21.15.164:0/223980623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc15dd70 0x7f25fc1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 -- 172.21.15.164:0/223980623 >> 172.21.15.164:0/223980623 conn(0x7f25fc0b34f0 msgr2=0x7f25fc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 -- 172.21.15.164:0/223980623 shutdown_connections 2024-05-08T04:40:20.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.929+0000 7f2600a00640 1 -- 172.21.15.164:0/223980623 wait complete. 2024-05-08T04:40:20.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 Processor -- start 2024-05-08T04:40:20.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- start start 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25fc15dd70 0x7f25fc0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc1625f0 0x7f25fc158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc15e2d0 con 0x7f25fc06c7b0 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc15e150 con 0x7f25fc15dd70 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b22b0 con 0x7f25fc1625f0 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44184/0 (socket says 172.21.15.164:44184) 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 -- 172.21.15.164:0/111140770 learned_addr learned my addr 172.21.15.164:0/111140770 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25fa800640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc1625f0 0x7f25fc158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9400640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25fc15dd70 0x7f25fc0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 -- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc1625f0 msgr2=0x7f25fc158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc1625f0 0x7f25fc158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 -- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25fc15dd70 msgr2=0x7f25fc0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25fc15dd70 0x7f25fc0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 -- 172.21.15.164:0/111140770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0b5910 con 0x7f25fc06c7b0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f9e00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e960 tx=0x7f25ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec017070 con 0x7f25fc06c7b0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec00ca00 con 0x7f25fc06c7b0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc0d03a0 con 0x7f25fc06c7b0 2024-05-08T04:40:20.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00cdb0 con 0x7f25fc06c7b0 2024-05-08T04:40:20.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.933+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc0d0960 con 0x7f25fc06c7b0 2024-05-08T04:40:20.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc0677c0 con 0x7f25fc06c7b0 2024-05-08T04:40:20.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f25ec01a070 con 0x7f25fc06c7b0 2024-05-08T04:40:20.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f25f2000640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 0x7f25dc069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:20.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f25ec085470 con 0x7f25fc06c7b0 2024-05-08T04:40:20.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f25f9400640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 0x7f25dc069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:20.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.937+0000 7f25f9400640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 0x7f25dc069660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007fd0 tx=0x7f25e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:20.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:20.941+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25fc0b5910 con 0x7f25fc06c7b0 2024-05-08T04:40:21.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.093+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25fc069d60 con 0x7f25fc06c7b0 2024-05-08T04:40:21.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f25f2000640 1 -- 172.21.15.164:0/111140770 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f25ec055a50 con 0x7f25fc06c7b0 2024-05-08T04:40:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:21.101 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":48,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:18.232734+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 msgr2=0x7f25dc069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 0x7f25dc069660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007fd0 tx=0x7f25e8009040 comp rx=0 tx=0).stop 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 msgr2=0x7f25fc153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e960 tx=0x7f25ec00ee20 comp rx=0 tx=0).stop 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 shutdown_connections 2024-05-08T04:40:21.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f25dc0671a0 0x7f25dc069660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25fc1625f0 0x7f25fc158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25fc15dd70 0x7f25fc0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 --2- 172.21.15.164:0/111140770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25fc06c7b0 0x7f25fc153420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 >> 172.21.15.164:0/111140770 conn(0x7f25fc0b34f0 msgr2=0x7f25fc0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 shutdown_connections 2024-05-08T04:40:21.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.097+0000 7f2600a00640 1 -- 172.21.15.164:0/111140770 wait complete. 2024-05-08T04:40:21.123 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:40:21.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 Processor -- start 2024-05-08T04:40:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 -- start start 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc19415bc50 0x7fc1940b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1940d0780 0x7fc19415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940b42a0 con 0x7fc1940ccfe0 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940cd4e0 con 0x7fc19415bc50 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc198a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940cd7c0 con 0x7fc1940d0780 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc19415bc50 0x7fc1940b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc19415bc50 0x7fc1940b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44190/0 (socket says 172.21.15.164:44190) 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 -- 172.21.15.164:0/1365222587 learned_addr learned my addr 172.21.15.164:0/1365222587 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc192800640 1 --2- 172.21.15.164:0/1365222587 >> v2:172.21.15.172:3300/0 conn(0x7fc1940d0780 0x7fc19415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 -- 172.21.15.164:0/1365222587 >> v2:172.21.15.172:3300/0 conn(0x7fc1940d0780 msgr2=0x7fc19415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:21.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 --2- 172.21.15.164:0/1365222587 >> v2:172.21.15.172:3300/0 conn(0x7fc1940d0780 0x7fc19415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 -- 172.21.15.164:0/1365222587 >> v1:172.21.15.172:6789/0 conn(0x7fc1940ccfe0 legacy=0x7fc1940d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:21.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.257+0000 7fc191e00640 1 -- 172.21.15.164:0/1365222587 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940cf590 con 0x7fc19415bc50 2024-05-08T04:40:21.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc191e00640 1 --2- 172.21.15.164:0/1365222587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940b4810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc194071a00 tx=0x7fc18401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc61d8a6a62935c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc190a00640 1 -- 172.21.15.164:0/1365222587 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840ab600 con 0x7fc19415bc50 2024-05-08T04:40:21.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc190a00640 1 -- 172.21.15.164:0/1365222587 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1840b0070 con 0x7fc19415bc50 2024-05-08T04:40:21.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc190a00640 1 -- 172.21.15.164:0/1365222587 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840ab9c0 con 0x7fc19415bc50 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- 172.21.15.164:0/1365222587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 msgr2=0x7fc1940b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 --2- 172.21.15.164:0/1365222587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940b4810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc194071a00 tx=0x7fc18401c400 comp rx=0 tx=0).stop 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- 172.21.15.164:0/1365222587 shutdown_connections 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 --2- 172.21.15.164:0/1365222587 >> v2:172.21.15.172:3300/0 conn(0x7fc1940d0780 0x7fc19415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 --2- 172.21.15.164:0/1365222587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940b4810 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- 172.21.15.164:0/1365222587 >> 172.21.15.164:0/1365222587 conn(0x7fc1940b34f0 msgr2=0x7fc1940b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- 172.21.15.164:0/1365222587 shutdown_connections 2024-05-08T04:40:21.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- 172.21.15.164:0/1365222587 wait complete. 2024-05-08T04:40:21.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 Processor -- start 2024-05-08T04:40:21.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.261+0000 7fc198a00640 1 -- start start 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 0x7fc19415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1940d0780 0x7fc1940cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc194168c60 con 0x7fc19415bc50 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc194168ae0 con 0x7fc1940d0780 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc194168de0 con 0x7fc1940ccfe0 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44200/0 (socket says 172.21.15.164:44200) 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 -- 172.21.15.164:0/3286472767 learned_addr learned my addr 172.21.15.164:0/3286472767 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc191e00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 0x7fc19415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc191400640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1940d0780 0x7fc1940cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 -- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 msgr2=0x7fc19415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 0x7fc19415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 -- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1940d0780 msgr2=0x7fc1940cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1940d0780 0x7fc1940cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 -- 172.21.15.164:0/3286472767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19415e6a0 con 0x7fc19415bc50 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc191e00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 0x7fc19415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc192800640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007e10 tx=0x7fc18c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014620 con 0x7fc19415bc50 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c019070 con 0x7fc19415bc50 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1941552d0 con 0x7fc19415bc50 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014a20 con 0x7fc19415bc50 2024-05-08T04:40:21.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19415e940 con 0x7fc19415bc50 2024-05-08T04:40:21.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.265+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc194069d60 con 0x7fc19415bc50 2024-05-08T04:40:21.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.269+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fc18c014bc0 con 0x7fc19415bc50 2024-05-08T04:40:21.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.269+0000 7fc18a000640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 0x7fc174069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.269+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc18c085d40 con 0x7fc19415bc50 2024-05-08T04:40:21.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.269+0000 7fc191e00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 0x7fc174069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.269+0000 7fc191e00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 0x7fc174069660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc1840af000 tx=0x7fc1840a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.273+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc18c011260 con 0x7fc19415bc50 2024-05-08T04:40:21.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.457+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1940b7940 con 0x7fc19415bc50 2024-05-08T04:40:21.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc18a000640 1 -- 172.21.15.164:0/3286472767 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc18c0569b0 con 0x7fc19415bc50 2024-05-08T04:40:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:21.465 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:40:21.465 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 26 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 msgr2=0x7fc174069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 0x7fc174069660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc1840af000 tx=0x7fc1840a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 msgr2=0x7fc1940cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007e10 tx=0x7fc18c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 shutdown_connections 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1740671a0 0x7fc174069660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc19415bc50 0x7fc1940cdfe0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1940d0780 0x7fc1940cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 --2- 172.21.15.164:0/3286472767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1940ccfe0 0x7fc19415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 >> 172.21.15.164:0/3286472767 conn(0x7fc1940b34f0 msgr2=0x7fc19406fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 shutdown_connections 2024-05-08T04:40:21.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.461+0000 7fc198a00640 1 -- 172.21.15.164:0/3286472767 wait complete. 2024-05-08T04:40:21.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:21.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 Processor -- start 2024-05-08T04:40:21.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 -- start start 2024-05-08T04:40:21.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 0x7efcb00b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb00b42a0 con 0x7efcb00ccfe0 2024-05-08T04:40:21.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb00cd4e0 con 0x7efcb015bc50 2024-05-08T04:40:21.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcb6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00cd7c0 con 0x7efcb00d0780 2024-05-08T04:40:21.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcaf400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 0x7efcb00b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42398/0 (socket says 172.21.15.164:42398) 2024-05-08T04:40:21.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 -- 172.21.15.164:0/1112972190 learned_addr learned my addr 172.21.15.164:0/1112972190 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 -- 172.21.15.164:0/1112972190 >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 msgr2=0x7efcb00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:21.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 --2- 172.21.15.164:0/1112972190 >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 0x7efcb00b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 -- 172.21.15.164:0/1112972190 >> v1:172.21.15.172:6789/0 conn(0x7efcb00ccfe0 legacy=0x7efcb00d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcafe00640 1 -- 172.21.15.164:0/1112972190 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb00cf590 con 0x7efcb00d0780 2024-05-08T04:40:21.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.617+0000 7efcaf400640 1 --2- 172.21.15.164:0/1112972190 >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 0x7efcb00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:21.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcafe00640 1 --2- 172.21.15.164:0/1112972190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efca8009e50 tx=0x7efca801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a449cc6d40f39437 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcae000640 1 -- 172.21.15.164:0/1112972190 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80a5830 con 0x7efcb00d0780 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcae000640 1 -- 172.21.15.164:0/1112972190 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80a59d0 con 0x7efcb00d0780 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcae000640 1 -- 172.21.15.164:0/1112972190 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80a5da0 con 0x7efcb00d0780 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 -- 172.21.15.164:0/1112972190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 msgr2=0x7efcb015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 --2- 172.21.15.164:0/1112972190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efca8009e50 tx=0x7efca801c970 comp rx=0 tx=0).stop 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 -- 172.21.15.164:0/1112972190 shutdown_connections 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 --2- 172.21.15.164:0/1112972190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb015f7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 --2- 172.21.15.164:0/1112972190 >> v2:172.21.15.172:3300/0 conn(0x7efcb015bc50 0x7efcb00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 -- 172.21.15.164:0/1112972190 >> 172.21.15.164:0/1112972190 conn(0x7efcb00b34f0 msgr2=0x7efcb00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 -- 172.21.15.164:0/1112972190 shutdown_connections 2024-05-08T04:40:21.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.621+0000 7efcb6400640 1 -- 172.21.15.164:0/1112972190 wait complete. 2024-05-08T04:40:21.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 Processor -- start 2024-05-08T04:40:21.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- start start 2024-05-08T04:40:21.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb00b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 0x7efcb00b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0168c60 con 0x7efcb00ccfe0 2024-05-08T04:40:21.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0168ae0 con 0x7efcb015bc50 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0168de0 con 0x7efcb00d0780 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcaf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcaf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44214/0 (socket says 172.21.15.164:44214) 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcaf400640 1 -- 172.21.15.164:0/3523134250 learned_addr learned my addr 172.21.15.164:0/3523134250 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 0x7efcb00b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcaea00640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb00b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 -- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 msgr2=0x7efcb00b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb00b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 -- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 msgr2=0x7efcb015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 -- 172.21.15.164:0/3523134250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb015a7e0 con 0x7efcb015bc50 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcaf400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcafe00640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 0x7efcb00b56a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efca80ab000 tx=0x7efca8019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80bc030 con 0x7efcb015bc50 2024-05-08T04:40:21.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80b8470 con 0x7efcb015bc50 2024-05-08T04:40:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb00c8b30 con 0x7efcb015bc50 2024-05-08T04:40:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80b8860 con 0x7efcb015bc50 2024-05-08T04:40:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.625+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb00c90f0 con 0x7efcb015bc50 2024-05-08T04:40:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb0069d60 con 0x7efcb015bc50 2024-05-08T04:40:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7efca80bf070 con 0x7efcb015bc50 2024-05-08T04:40:21.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efca7400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 0x7efc90069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7efca81209f0 con 0x7efcb015bc50 2024-05-08T04:40:21.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efcaf400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 0x7efc90069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.629+0000 7efcaf400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 0x7efc90069660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efca0009580 tx=0x7efca0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.633+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efca8016620 con 0x7efcb015bc50 2024-05-08T04:40:21.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcb00b7940 con 0x7efcb015bc50 2024-05-08T04:40:21.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efca7400640 1 -- 172.21.15.164:0/3523134250 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4944 (secure 0 0 0) 0x7efca80f10e0 con 0x7efcb015bc50 2024-05-08T04:40:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:21.826 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:39:53.147237+0000","modified":"2024-05-08T04:39:56.208727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5523":{"gid":5523,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/4206585796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4206585796},{"type":"v1","addr":"172.21.15.164:6835","nonce":4206585796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T04:40:21.826 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 26 2024-05-08T04:40:21.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 msgr2=0x7efc90069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 0x7efc90069660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efca0009580 tx=0x7efca0009190 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 msgr2=0x7efcb00b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 0x7efcb00b56a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efca80ab000 tx=0x7efca8019710 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 shutdown_connections 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efc900671a0 0x7efc90069660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcb00d0780 0x7efcb00b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015bc50 0x7efcb00b56a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 --2- 172.21.15.164:0/3523134250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcb00ccfe0 0x7efcb015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 >> 172.21.15.164:0/3523134250 conn(0x7efcb00b34f0 msgr2=0x7efcb01652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 shutdown_connections 2024-05-08T04:40:21.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.821+0000 7efcb6400640 1 -- 172.21.15.164:0/3523134250 wait complete. 2024-05-08T04:40:21.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:21.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.977+0000 7fd1d0200640 1 Processor -- start 2024-05-08T04:40:21.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.977+0000 7fd1d0200640 1 -- start start 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1d0200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1c815ba70 0x7fd1c8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1d0200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1c815be50 0x7fd1c80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1d0200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c80cd9f0 con 0x7fd1c80aeda0 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1d0200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c815fe70 con 0x7fd1c815ba70 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1d0200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80cdcd0 con 0x7fd1c815be50 2024-05-08T04:40:21.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1c815ba70 0x7fd1c8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1c815ba70 0x7fd1c8160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44224/0 (socket says 172.21.15.164:44224) 2024-05-08T04:40:21.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 -- 172.21.15.164:0/3331383353 learned_addr learned my addr 172.21.15.164:0/3331383353 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cce00640 1 --2- 172.21.15.164:0/3331383353 >> v2:172.21.15.172:3300/0 conn(0x7fd1c815be50 0x7fd1c80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1ce200640 1 --1- 172.21.15.164:0/3331383353 >> v1:172.21.15.164:6789/0 conn(0x7fd1c80aeda0 0x7fd1c80b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38086/0 (socket says 172.21.15.164:38086) 2024-05-08T04:40:21.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 -- 172.21.15.164:0/3331383353 >> v2:172.21.15.172:3300/0 conn(0x7fd1c815be50 msgr2=0x7fd1c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:21.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 --2- 172.21.15.164:0/3331383353 >> v2:172.21.15.172:3300/0 conn(0x7fd1c815be50 0x7fd1c80cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 -- 172.21.15.164:0/3331383353 >> v1:172.21.15.164:6789/0 conn(0x7fd1c80aeda0 legacy=0x7fd1c80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 -- 172.21.15.164:0/3331383353 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c815f820 con 0x7fd1c815ba70 2024-05-08T04:40:21.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1c7e00640 1 -- 172.21.15.164:0/3331383353 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196516149 0 0) 0x7fd1c80cd9f0 con 0x7fd1c80aeda0 2024-05-08T04:40:21.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.981+0000 7fd1cd800640 1 --2- 172.21.15.164:0/3331383353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c8160300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd1c815f780 tx=0x7fd1bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48376e103d59514c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1c7e00640 1 -- 172.21.15.164:0/3331383353 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0ab600 con 0x7fd1c815ba70 2024-05-08T04:40:21.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1c7e00640 1 -- 172.21.15.164:0/3331383353 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc0b0070 con 0x7fd1c815ba70 2024-05-08T04:40:21.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1c7e00640 1 -- 172.21.15.164:0/3331383353 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0ab9c0 con 0x7fd1c815ba70 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- 172.21.15.164:0/3331383353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 msgr2=0x7fd1c8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3331383353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c8160300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd1c815f780 tx=0x7fd1bc01c4f0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- 172.21.15.164:0/3331383353 shutdown_connections 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3331383353 >> v2:172.21.15.172:3300/0 conn(0x7fd1c815be50 0x7fd1c80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3331383353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c8160300 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- 172.21.15.164:0/3331383353 >> 172.21.15.164:0/3331383353 conn(0x7fd1c806ac90 msgr2=0x7fd1c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- 172.21.15.164:0/3331383353 shutdown_connections 2024-05-08T04:40:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- 172.21.15.164:0/3331383353 wait complete. 2024-05-08T04:40:21.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 Processor -- start 2024-05-08T04:40:21.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- start start 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c80aeda0 0x7fd1c814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c815be50 0x7fd1c80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8168d40 con 0x7fd1c815ba70 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8168bc0 con 0x7fd1c815be50 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1d0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8168ec0 con 0x7fd1c80aeda0 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1cce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1cce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44228/0 (socket says 172.21.15.164:44228) 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1cce00640 1 -- 172.21.15.164:0/3170891561 learned_addr learned my addr 172.21.15.164:0/3170891561 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1ce200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c815be50 0x7fd1c80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.985+0000 7fd1cd800640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c80aeda0 0x7fd1c814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 -- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c80aeda0 msgr2=0x7fd1c814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c80aeda0 0x7fd1c814b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 -- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c815be50 msgr2=0x7fd1c80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:21.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c815be50 0x7fd1c80714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:21.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 -- 172.21.15.164:0/3170891561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8071e90 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cce00640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e8a0 tx=0x7fd1b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400c940 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400ccf0 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c815c4f0 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c815d510 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c8067820 con 0x7fd1c815ba70 2024-05-08T04:40:21.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fd1b401a070 con 0x7fd1c815ba70 2024-05-08T04:40:21.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 0x7fd1a8069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:21.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd1b4085710 con 0x7fd1c815ba70 2024-05-08T04:40:21.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.989+0000 7fd1cd800640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 0x7fd1a8069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:21.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.993+0000 7fd1cd800640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 0x7fd1a8069660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0af000 tx=0x7fd1bc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:21.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:21.993+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1b4055c60 con 0x7fd1c815ba70 2024-05-08T04:40:22.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.149+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c8069d60 con 0x7fd1c815ba70 2024-05-08T04:40:22.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1c5600640 1 -- 172.21.15.164:0/3170891561 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fd1b4059d00 con 0x7fd1c815ba70 2024-05-08T04:40:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:22.157 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":48,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:18.232734+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:39:50.888294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-08T04:39:52.156282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 msgr2=0x7fd1a8069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 0x7fd1a8069660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0af000 tx=0x7fd1bc0a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 msgr2=0x7fd1c815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e8a0 tx=0x7fd1b400eca0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 shutdown_connections 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a80671a0 0x7fd1a8069660 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c815be50 0x7fd1c80714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c815ba70 0x7fd1c815cfd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 --2- 172.21.15.164:0/3170891561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c80aeda0 0x7fd1c814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 >> 172.21.15.164:0/3170891561 conn(0x7fd1c806ac90 msgr2=0x7fd1c81593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:22.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 shutdown_connections 2024-05-08T04:40:22.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.153+0000 7fd1d0200640 1 -- 172.21.15.164:0/3170891561 wait complete. 2024-05-08T04:40:22.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:40:22.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 Processor -- start 2024-05-08T04:40:22.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 -- start start 2024-05-08T04:40:22.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a240706b0 0x7f5a240cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 0x7f5a240cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a240ce3a0 con 0x7f5a2415bae0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a240ce680 con 0x7f5a240706b0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a2a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240ce960 con 0x7f5a240cc4a0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a240706b0 0x7f5a240cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5a240706b0 0x7f5a240cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44232/0 (socket says 172.21.15.164:44232) 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 -- 172.21.15.164:0/2555445908 learned_addr learned my addr 172.21.15.164:0/2555445908 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29a00640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 0x7f5a240cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 -- 172.21.15.164:0/2555445908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 msgr2=0x7f5a240cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 0x7f5a240cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 -- 172.21.15.164:0/2555445908 >> v1:172.21.15.172:6789/0 conn(0x7f5a2415bae0 legacy=0x7f5a24070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29000640 1 -- 172.21.15.164:0/2555445908 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240cec40 con 0x7f5a240706b0 2024-05-08T04:40:22.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.313+0000 7f5a29a00640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 0x7f5a240cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:22.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a29000640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240706b0 0x7f5a240cdd50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009870 tx=0x7f5a1801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95280ec09d577a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a23400640 1 -- 172.21.15.164:0/2555445908 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a180aa430 con 0x7f5a240706b0 2024-05-08T04:40:22.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a23400640 1 -- 172.21.15.164:0/2555445908 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a180af070 con 0x7f5a240706b0 2024-05-08T04:40:22.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a23400640 1 -- 172.21.15.164:0/2555445908 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a180aa7f0 con 0x7f5a240706b0 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 -- 172.21.15.164:0/2555445908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240706b0 msgr2=0x7f5a240cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240706b0 0x7f5a240cdd50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009870 tx=0x7f5a1801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 -- 172.21.15.164:0/2555445908 shutdown_connections 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a240cc4a0 0x7f5a240cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 --2- 172.21.15.164:0/2555445908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240706b0 0x7f5a240cdd50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 -- 172.21.15.164:0/2555445908 >> 172.21.15.164:0/2555445908 conn(0x7f5a240cfaf0 msgr2=0x7f5a240cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 -- 172.21.15.164:0/2555445908 shutdown_connections 2024-05-08T04:40:22.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.317+0000 7f5a2a400640 1 -- 172.21.15.164:0/2555445908 wait complete. 2024-05-08T04:40:22.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 Processor -- start 2024-05-08T04:40:22.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- start start 2024-05-08T04:40:22.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a240706b0 0x7f5a2414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2415bae0 0x7f5a241521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2406d2b0 con 0x7f5a240cc4a0 2024-05-08T04:40:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2406d130 con 0x7f5a240706b0 2024-05-08T04:40:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2415e260 con 0x7f5a2415bae0 2024-05-08T04:40:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44248/0 (socket says 172.21.15.164:44248) 2024-05-08T04:40:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 -- 172.21.15.164:0/857926757 learned_addr learned my addr 172.21.15.164:0/857926757 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a29000640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a240706b0 0x7f5a2414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a29a00640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2415bae0 0x7f5a241521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 -- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2415bae0 msgr2=0x7f5a241521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2415bae0 0x7f5a241521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 -- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a240706b0 msgr2=0x7f5a2414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a240706b0 0x7f5a2414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 -- 172.21.15.164:0/857926757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2415dca0 con 0x7f5a240cc4a0 2024-05-08T04:40:22.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a23e00640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000b410 tx=0x7f5a1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10015030 con 0x7f5a240cc4a0 2024-05-08T04:40:22.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10004020 con 0x7f5a240cc4a0 2024-05-08T04:40:22.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004450 con 0x7f5a240cc4a0 2024-05-08T04:40:22.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2415df80 con 0x7f5a240cc4a0 2024-05-08T04:40:22.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a24157d70 con 0x7f5a240cc4a0 2024-05-08T04:40:22.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.321+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2404ef30 con 0x7f5a240cc4a0 2024-05-08T04:40:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.325+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f5a10018070 con 0x7f5a240cc4a0 2024-05-08T04:40:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.325+0000 7f5a20c00640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 0x7f5a04069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.325+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f5a10084fd0 con 0x7f5a240cc4a0 2024-05-08T04:40:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.325+0000 7f5a29000640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 0x7f5a04069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.325+0000 7f5a29000640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 0x7f5a04069690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ae000 tx=0x7f5a180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.329+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a10055520 con 0x7f5a240cc4a0 2024-05-08T04:40:22.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.517+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5a24072420 con 0x7f5a240cc4a0 2024-05-08T04:40:22.523 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:40:22.517+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:40:22.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.521+0000 7f5a20c00640 1 -- 172.21.15.164:0/857926757 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5a100595c0 con 0x7f5a240cc4a0 2024-05-08T04:40:22.527 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:40:22.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 msgr2=0x7f5a04069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 0x7f5a04069690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ae000 tx=0x7f5a180a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 msgr2=0x7f5a24158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000b410 tx=0x7f5a1000b8d0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 shutdown_connections 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5a040671d0 0x7f5a04069690 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2415bae0 0x7f5a241521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a240cc4a0 0x7f5a24158570 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 --2- 172.21.15.164:0/857926757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a240706b0 0x7f5a2414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 >> 172.21.15.164:0/857926757 conn(0x7f5a240cfaf0 msgr2=0x7f5a240cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 shutdown_connections 2024-05-08T04:40:22.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.525+0000 7f5a2a400640 1 -- 172.21.15.164:0/857926757 wait complete. 2024-05-08T04:40:22.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:40:22.595 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:40:22.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.685+0000 7f5876a00640 1 Processor -- start 2024-05-08T04:40:22.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.685+0000 7f5876a00640 1 -- start start 2024-05-08T04:40:22.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5876a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f587015ab10 0x7f587006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5876a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5876a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587015e870 con 0x7f58700ae6e0 2024-05-08T04:40:22.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5876a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587006bb80 con 0x7f587015ab10 2024-05-08T04:40:22.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5876a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587015eb50 con 0x7f58700cee90 2024-05-08T04:40:22.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42446/0 (socket says 172.21.15.164:42446) 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 -- 172.21.15.164:0/3295446530 learned_addr learned my addr 172.21.15.164:0/3295446530 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f586fe00640 1 --2- 172.21.15.164:0/3295446530 >> v2:172.21.15.172:3300/0 conn(0x7f587015ab10 0x7f587006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 -- 172.21.15.164:0/3295446530 >> v2:172.21.15.172:3300/0 conn(0x7f587015ab10 msgr2=0x7f587006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 --2- 172.21.15.164:0/3295446530 >> v2:172.21.15.172:3300/0 conn(0x7f587015ab10 0x7f587006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 -- 172.21.15.164:0/3295446530 >> v1:172.21.15.172:6789/0 conn(0x7f58700ae6e0 legacy=0x7f58700ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.689+0000 7f5874a00640 1 -- 172.21.15.164:0/3295446530 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587015ee30 con 0x7f58700cee90 2024-05-08T04:40:22.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5874a00640 1 --2- 172.21.15.164:0/3295446530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5868009e50 tx=0x7f586801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f61976d22e0a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f586f400640 1 -- 172.21.15.164:0/3295446530 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680a57f0 con 0x7f58700cee90 2024-05-08T04:40:22.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f586f400640 1 -- 172.21.15.164:0/3295446530 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58680a5990 con 0x7f58700cee90 2024-05-08T04:40:22.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f586f400640 1 -- 172.21.15.164:0/3295446530 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680a5d60 con 0x7f58700cee90 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- 172.21.15.164:0/3295446530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 msgr2=0x7f58700aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 --2- 172.21.15.164:0/3295446530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5868009e50 tx=0x7f586801c970 comp rx=0 tx=0).stop 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- 172.21.15.164:0/3295446530 shutdown_connections 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 --2- 172.21.15.164:0/3295446530 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58700cee90 0x7f58700aeb70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 --2- 172.21.15.164:0/3295446530 >> v2:172.21.15.172:3300/0 conn(0x7f587015ab10 0x7f587006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- 172.21.15.164:0/3295446530 >> 172.21.15.164:0/3295446530 conn(0x7f58700cf770 msgr2=0x7f58700cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- 172.21.15.164:0/3295446530 shutdown_connections 2024-05-08T04:40:22.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- 172.21.15.164:0/3295446530 wait complete. 2024-05-08T04:40:22.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 Processor -- start 2024-05-08T04:40:22.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.693+0000 7f5876a00640 1 -- start start 2024-05-08T04:40:22.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 0x7f5870146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587015ab10 0x7f5870071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700cb070 con 0x7f58700cee90 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700caef0 con 0x7f58700ae6e0 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700cb1f0 con 0x7f587015ab10 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5867400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5867400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44280/0 (socket says 172.21.15.164:44280) 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5867400640 1 -- 172.21.15.164:0/1692038825 learned_addr learned my addr 172.21.15.164:0/1692038825 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 0x7f5870146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5874a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587015ab10 0x7f5870071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 -- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587015ab10 msgr2=0x7f5870071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587015ab10 0x7f5870071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 -- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 msgr2=0x7f58701506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 -- 172.21.15.164:0/1692038825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870072140 con 0x7f58700ae6e0 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5867400640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:22.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586fe00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 0x7f5870146c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f586000e870 tx=0x7f586000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860017070 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000ee60 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000cb60 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701599e0 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5870159fa0 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870072d30 con 0x7f58700ae6e0 2024-05-08T04:40:22.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.697+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f586001a070 con 0x7f58700ae6e0 2024-05-08T04:40:22.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.701+0000 7f586cc00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 0x7f584c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:22.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.701+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f58600852c0 con 0x7f58700ae6e0 2024-05-08T04:40:22.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.701+0000 7f5867400640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 0x7f584c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:22.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.701+0000 7f5867400640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 0x7f584c069660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58700cdd10 tx=0x7f5858009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:22.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.705+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5870072140 con 0x7f58700ae6e0 2024-05-08T04:40:22.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:22.889+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f587004f010 con 0x7f58700ae6e0 2024-05-08T04:40:23.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.529+0000 7f586cc00640 1 -- 172.21.15.164:0/1692038825 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5860055860 con 0x7f58700ae6e0 2024-05-08T04:40:23.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 msgr2=0x7f584c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 0x7f584c069660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58700cdd10 tx=0x7f5858009040 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 msgr2=0x7f5870146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 0x7f5870146c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f586000e870 tx=0x7f586000ed30 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 shutdown_connections 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f584c0671a0 0x7f584c069660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f587015ab10 0x7f5870071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58700cee90 0x7f58701506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 --2- 172.21.15.164:0/1692038825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58700ae6e0 0x7f5870146c90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 >> 172.21.15.164:0/1692038825 conn(0x7f58700cf770 msgr2=0x7f58700ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 shutdown_connections 2024-05-08T04:40:23.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.533+0000 7f5876a00640 1 -- 172.21.15.164:0/1692038825 wait complete. 2024-05-08T04:40:23.559 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:40:23.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.693+0000 7f55ffe00640 1 Processor -- start 2024-05-08T04:40:23.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ffe00640 1 -- start start 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f5600071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ffe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015c900 con 0x7f56000b1d40 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ffe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015cbe0 con 0x7f56000cec40 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560015cec0 con 0x7f56000b1960 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ff400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f56000b1d40 0x7f56000ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38106/0 (socket says 172.21.15.164:38106) 2024-05-08T04:40:23.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ff400640 1 -- 172.21.15.164:0/2289577863 learned_addr learned my addr 172.21.15.164:0/2289577863 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:23.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f5600071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951133265 0 0) 0x7f560015c900 con 0x7f56000b1d40 2024-05-08T04:40:23.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e4003530 con 0x7f56000b1d40 2024-05-08T04:40:23.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1244563625 0 0) 0x7f55e4003530 con 0x7f56000b1d40 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 msgr2=0x7f5600071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f5600071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 >> v1:172.21.15.172:6789/0 conn(0x7f56000cec40 legacy=0x7f560015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000712a0 con 0x7f56000b1d40 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f55f8001a00 con 0x7f56000b1d40 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 >> v1:172.21.15.164:6789/0 conn(0x7f56000b1d40 legacy=0x7f56000ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55e40039f0 0x7f55e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e4004810 0x7f55e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f55e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560015cec0 con 0x7f55e40039f0 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e400bb00 con 0x7f55e4004810 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e400bd00 con 0x7f56000b1960 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55e40039f0 0x7f55e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fe000640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e4004810 0x7f55e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55ff400640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f55e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 -- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 msgr2=0x7f55e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:23.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000b1960 0x7f55e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 -- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e4004810 msgr2=0x7f55e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:23.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e4004810 0x7f55e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.697+0000 7f55fea00640 1 -- 172.21.15.164:0/2289577863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4003530 con 0x7f55e40039f0 2024-05-08T04:40:23.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55fea00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55e40039f0 0x7f55e4003dd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ae0 tx=0x7f55f001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2728d94d398a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:23.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00b2070 con 0x7f55e40039f0 2024-05-08T04:40:23.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f00a5530 con 0x7f55e40039f0 2024-05-08T04:40:23.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55fd600640 1 -- 172.21.15.164:0/2289577863 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00096b0 con 0x7f55e40039f0 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 -- 172.21.15.164:0/2289577863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55e40039f0 msgr2=0x7f55e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 --2- 172.21.15.164:0/2289577863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55e40039f0 0x7f55e4003dd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ae0 tx=0x7f55f001ca70 comp rx=0 tx=0).stop 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ff400640 1 -- 172.21.15.164:0/2289577863 reap_dead start 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 -- 172.21.15.164:0/2289577863 shutdown_connections 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 -- 172.21.15.164:0/2289577863 >> 172.21.15.164:0/2289577863 conn(0x7f560006ac90 msgr2=0x7f560006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 -- 172.21.15.164:0/2289577863 shutdown_connections 2024-05-08T04:40:23.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.701+0000 7f55ffe00640 1 -- 172.21.15.164:0/2289577863 wait complete. 2024-05-08T04:40:23.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 Processor -- start 2024-05-08T04:40:23.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- start start 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e40039f0 0x7f56001fa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 0x7f56001f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000b4780 con 0x7f560015b400 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000b4600 con 0x7f55e40039f0 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000b4900 con 0x7f56001610b0 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44306/0 (socket says 172.21.15.164:44306) 2024-05-08T04:40:23.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 -- 172.21.15.164:0/389118443 learned_addr learned my addr 172.21.15.164:0/389118443 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ff400640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 0x7f56001f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 -- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 msgr2=0x7f56001f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 0x7f56001f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 -- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e40039f0 msgr2=0x7f56001fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e40039f0 0x7f56001fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 -- 172.21.15.164:0/389118443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56001f7560 con 0x7f560015b400 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ff400640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 0x7f56001f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55fe000640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00f7c0 tx=0x7f55ec00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00fe70 con 0x7f560015b400 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec018070 con 0x7f560015b400 2024-05-08T04:40:23.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00d730 con 0x7f560015b400 2024-05-08T04:40:23.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56001f7280 con 0x7f560015b400 2024-05-08T04:40:23.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.705+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56001fa010 con 0x7f560015b400 2024-05-08T04:40:23.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560004ef30 con 0x7f560015b400 2024-05-08T04:40:23.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f55ec00d8d0 con 0x7f560015b400 2024-05-08T04:40:23.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55f7400640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 0x7f55e0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:23.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f55ec085450 con 0x7f560015b400 2024-05-08T04:40:23.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55fea00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 0x7f55e0069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:23.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.709+0000 7f55fea00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 0x7f55e0069660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f55f00af000 tx=0x7f55f0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:23.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.713+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55ec056040 con 0x7f560015b400 2024-05-08T04:40:23.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:23.869+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5600068b10 con 0x7f560015b400 2024-05-08T04:40:24.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55f7400640 1 -- 172.21.15.164:0/389118443 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7f55ec055940 con 0x7f560015b400 2024-05-08T04:40:24.532 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 msgr2=0x7f55e0069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 0x7f55e0069660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f55f00af000 tx=0x7f55f0004740 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 msgr2=0x7f5600160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00f7c0 tx=0x7f55ec00fc80 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 shutdown_connections 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f55e00671a0 0x7f55e0069660 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56001610b0 0x7f56001f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f560015b400 0x7f5600160b70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 --2- 172.21.15.164:0/389118443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55e40039f0 0x7f56001fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 >> 172.21.15.164:0/389118443 conn(0x7f560006ac90 msgr2=0x7f5600073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:24.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 shutdown_connections 2024-05-08T04:40:24.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.529+0000 7f55ffe00640 1 -- 172.21.15.164:0/389118443 wait complete. 2024-05-08T04:40:24.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:40:24.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 Processor -- start 2024-05-08T04:40:24.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 -- start start 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a5c06cce0 0x7f2a5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c15e870 con 0x7f2a5c15ab10 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c15eb50 con 0x7f2a5c0cecc0 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a61a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c06bb80 con 0x7f2a5c06cce0 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2a5c15ab10 0x7f2a5c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38118/0 (socket says 172.21.15.164:38118) 2024-05-08T04:40:24.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5aa00640 1 -- 172.21.15.164:0/1707482226 learned_addr learned my addr 172.21.15.164:0/1707482226 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058943128 0 0) 0x7f2a5c15e870 con 0x7f2a5c15ab10 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a40003530 con 0x7f2a5c15ab10 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c06cce0 0x7f2a5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221212607 0 0) 0x7f2a40003530 con 0x7f2a5c15ab10 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c06cce0 msgr2=0x7f2a5c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:24.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 --2- 172.21.15.164:0/1707482226 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c06cce0 0x7f2a5c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 >> v1:172.21.15.172:6789/0 conn(0x7f2a5c0cecc0 legacy=0x7f2a5c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15ee30 con 0x7f2a5c15ab10 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2a4c001680 con 0x7f2a5c15ab10 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c06cce0 0x7f2a5c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 >> v1:172.21.15.164:6789/0 conn(0x7f2a5c15ab10 legacy=0x7f2a5c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a400039c0 0x7f2a40003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 0x7f2a40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a400053d0 0x7f2a4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c06bb80 con 0x7f2a40004810 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4000c270 con 0x7f2a400053d0 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4000c470 con 0x7f2a400039c0 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 0x7f2a40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5b400640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a400053d0 0x7f2a4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5aa00640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a400039c0 0x7f2a40003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 -- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a400039c0 msgr2=0x7f2a40003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a400039c0 0x7f2a40003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 -- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a400053d0 msgr2=0x7f2a4000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a400053d0 0x7f2a4000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5a000640 1 -- 172.21.15.164:0/1707482226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40003530 con 0x7f2a40004810 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.701+0000 7f2a5b400640 1 -- 172.21.15.164:0/1707482226 reap_dead start 2024-05-08T04:40:24.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a5a000640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 0x7f2a40004c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800aaa0 tx=0x7f2a4801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ed250f488668ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:24.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480b2070 con 0x7f2a40004810 2024-05-08T04:40:24.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480a9070 con 0x7f2a40004810 2024-05-08T04:40:24.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a59600640 1 -- 172.21.15.164:0/1707482226 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800a670 con 0x7f2a40004810 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 -- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 msgr2=0x7f2a40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 0x7f2a40004c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800aaa0 tx=0x7f2a4801c630 comp rx=0 tx=0).stop 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 -- 172.21.15.164:0/1707482226 shutdown_connections 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 --2- 172.21.15.164:0/1707482226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a40004810 0x7f2a40004c90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 -- 172.21.15.164:0/1707482226 >> 172.21.15.164:0/1707482226 conn(0x7f2a5c0cf770 msgr2=0x7f2a5c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.705+0000 7f2a61a00640 1 -- 172.21.15.164:0/1707482226 shutdown_connections 2024-05-08T04:40:24.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- 172.21.15.164:0/1707482226 wait complete. 2024-05-08T04:40:24.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 Processor -- start 2024-05-08T04:40:24.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- start start 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a40004810 0x7f2a5c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0b4570 0x7f2a5c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c54e0 con 0x7f2a5c15c280 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c5360 con 0x7f2a5c0b4570 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0c5660 con 0x7f2a40004810 2024-05-08T04:40:24.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44320/0 (socket says 172.21.15.164:44320) 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 -- 172.21.15.164:0/4033187902 learned_addr learned my addr 172.21.15.164:0/4033187902 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5aa00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a40004810 0x7f2a5c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5a000640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0b4570 0x7f2a5c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 -- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a40004810 msgr2=0x7f2a5c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a40004810 0x7f2a5c154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 -- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0b4570 msgr2=0x7f2a5c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0b4570 0x7f2a5c0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 -- 172.21.15.164:0/4033187902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15cc40 con 0x7f2a5c15c280 2024-05-08T04:40:24.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a5b400640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06be30 tx=0x7f2a5400e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54019070 con 0x7f2a5c15c280 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54006d70 con 0x7f2a5c15c280 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540146c0 con 0x7f2a5c15c280 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.709+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c154aa0 con 0x7f2a5c15c280 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c15d1d0 con 0x7f2a5c15c280 2024-05-08T04:40:24.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c072d30 con 0x7f2a5c15c280 2024-05-08T04:40:24.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f2a5401c070 con 0x7f2a5c15c280 2024-05-08T04:40:24.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a52a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 0x7f2a3c069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:24.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f2a54084870 con 0x7f2a5c15c280 2024-05-08T04:40:24.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a5aa00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 0x7f2a3c069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:24.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.713+0000 7f2a5aa00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 0x7f2a3c069690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001d00 tx=0x7f2a4c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:24.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.717+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a54054fc0 con 0x7f2a5c15c280 2024-05-08T04:40:24.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:24.881+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a5c0ceaa0 con 0x7f2a5c15c280 2024-05-08T04:40:25.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.529+0000 7f2a52a00640 1 -- 172.21.15.164:0/4033187902 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2a54059060 con 0x7f2a5c15c280 2024-05-08T04:40:25.535 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:40:25.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 msgr2=0x7f2a3c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 0x7f2a3c069690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001d00 tx=0x7f2a4c007040 comp rx=0 tx=0).stop 2024-05-08T04:40:25.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 msgr2=0x7f2a5c15c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06be30 tx=0x7f2a5400e8f0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 shutdown_connections 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a3c0671d0 0x7f2a3c069690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a5c15c280 0x7f2a5c15c700 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0b4570 0x7f2a5c0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 --2- 172.21.15.164:0/4033187902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a40004810 0x7f2a5c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 >> 172.21.15.164:0/4033187902 conn(0x7f2a5c0cf770 msgr2=0x7f2a5c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:25.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 shutdown_connections 2024-05-08T04:40:25.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.533+0000 7f2a61a00640 1 -- 172.21.15.164:0/4033187902 wait complete. 2024-05-08T04:40:25.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:40:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 Processor -- start 2024-05-08T04:40:25.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 -- start start 2024-05-08T04:40:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5dcc0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0c6600 con 0x7f5dcc15ba20 2024-05-08T04:40:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0c6040 con 0x7f5dcc15ce10 2024-05-08T04:40:25.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dd2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc15fbe0 con 0x7f5dcc0ae6e0 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcb400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5dcc15ba20 0x7f5dcc160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38122/0 (socket says 172.21.15.164:38122) 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcb400640 1 -- 172.21.15.164:0/1979761731 learned_addr learned my addr 172.21.15.164:0/1979761731 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcbe00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5dcc0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417380308 0 0) 0x7f5dcc0c6600 con 0x7f5dcc15ba20 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db0003530 con 0x7f5dcc15ba20 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3454820413 0 0) 0x7f5db0003530 con 0x7f5dcc15ba20 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 msgr2=0x7f5dcc0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5dcc0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 >> v1:172.21.15.172:6789/0 conn(0x7f5dcc15ce10 legacy=0x7f5dcc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0c6320 con 0x7f5dcc15ba20 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5dbc001680 con 0x7f5dcc15ba20 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 >> v1:172.21.15.164:6789/0 conn(0x7f5dcc15ba20 legacy=0x7f5dcc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db00039f0 0x7f5db0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db0004810 0x7f5db0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5db00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc15fbe0 con 0x7f5db0004810 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db000bab0 con 0x7f5db00039f0 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db000bcb0 con 0x7f5dcc0ae6e0 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db0004810 0x7f5db0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcb400640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db00039f0 0x7f5db0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcbe00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5db00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 -- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 msgr2=0x7f5db00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0ae6e0 0x7f5db00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 -- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db00039f0 msgr2=0x7f5db0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db00039f0 0x7f5db0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.701+0000 7f5dcaa00640 1 -- 172.21.15.164:0/1979761731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0003530 con 0x7f5db0004810 2024-05-08T04:40:25.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dcaa00640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db0004810 0x7f5db0004c90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5db8001750 tx=0x7f5db801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3eaa1fe52ef045 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:25.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80013a0 con 0x7f5db0004810 2024-05-08T04:40:25.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db80052f0 con 0x7f5db0004810 2024-05-08T04:40:25.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dca000640 1 -- 172.21.15.164:0/1979761731 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8003840 con 0x7f5db0004810 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 -- 172.21.15.164:0/1979761731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db0004810 msgr2=0x7f5db0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 --2- 172.21.15.164:0/1979761731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db0004810 0x7f5db0004c90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5db8001750 tx=0x7f5db801c3c0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dcbe00640 1 -- 172.21.15.164:0/1979761731 reap_dead start 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 -- 172.21.15.164:0/1979761731 shutdown_connections 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 -- 172.21.15.164:0/1979761731 >> 172.21.15.164:0/1979761731 conn(0x7f5dcc0cf770 msgr2=0x7f5dcc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:25.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 -- 172.21.15.164:0/1979761731 shutdown_connections 2024-05-08T04:40:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.705+0000 7f5dd2000640 1 -- 172.21.15.164:0/1979761731 wait complete. 2024-05-08T04:40:25.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 Processor -- start 2024-05-08T04:40:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- start start 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db0004810 0x7f5dcc1f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0b50c0 0x7f5dcc0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc06e560 con 0x7f5dcc06ac90 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0cf410 con 0x7f5db0004810 2024-05-08T04:40:25.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc06e6e0 con 0x7f5dcc0b50c0 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44340/0 (socket says 172.21.15.164:44340) 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 -- 172.21.15.164:0/2909487599 learned_addr learned my addr 172.21.15.164:0/2909487599 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcb400640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db0004810 0x7f5dcc1f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcaa00640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0b50c0 0x7f5dcc0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 -- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0b50c0 msgr2=0x7f5dcc0b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0b50c0 0x7f5dcc0b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 -- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db0004810 msgr2=0x7f5dcc1f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db0004810 0x7f5dcc1f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 -- 172.21.15.164:0/2909487599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0b29f0 con 0x7f5dcc06ac90 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dcbe00640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009e90 tx=0x7f5dc40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4011e40 con 0x7f5dcc06ac90 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc401b070 con 0x7f5dcc06ac90 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40039a0 con 0x7f5dcc06ac90 2024-05-08T04:40:25.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc0c8d70 con 0x7f5dcc06ac90 2024-05-08T04:40:25.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.709+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc0c9330 con 0x7f5dcc06ac90 2024-05-08T04:40:25.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc072d30 con 0x7f5dcc06ac90 2024-05-08T04:40:25.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f5dc4003b40 con 0x7f5dcc06ac90 2024-05-08T04:40:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dc3400640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 0x7f5dac069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5dc40859b0 con 0x7f5dcc06ac90 2024-05-08T04:40:25.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dcb400640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 0x7f5dac069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:25.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.713+0000 7f5dcb400640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 0x7f5dac069660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc001060 tx=0x7f5dbc007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:25.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.717+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dc4056790 con 0x7f5dcc06ac90 2024-05-08T04:40:25.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.877+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5dcc04f010 con 0x7f5dcc06ac90 2024-05-08T04:40:25.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.881+0000 7f5dc3400640 1 -- 172.21.15.164:0/2909487599 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5dc4056090 con 0x7f5dcc06ac90 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 msgr2=0x7f5dac069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 0x7f5dac069660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc001060 tx=0x7f5dbc007900 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 msgr2=0x7f5dcc06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009e90 tx=0x7f5dc40115a0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 shutdown_connections 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5dac0671a0 0x7f5dac069660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dcc0b50c0 0x7f5dcc0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dcc06ac90 0x7f5dcc06b110 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 --2- 172.21.15.164:0/2909487599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db0004810 0x7f5dcc1f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 >> 172.21.15.164:0/2909487599 conn(0x7f5dcc0cf770 msgr2=0x7f5dcc160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:25.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 shutdown_connections 2024-05-08T04:40:25.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:25.885+0000 7f5dd2000640 1 -- 172.21.15.164:0/2909487599 wait complete. 2024-05-08T04:40:25.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2024-05-08T04:40:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 Processor -- start 2024-05-08T04:40:26.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 -- start start 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ae4160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40cd9f0 con 0x7f3ae415ba70 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40cdcd0 con 0x7f3ae40cfda0 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3aebc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae415fe70 con 0x7f3ae415be50 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae9200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3ae415ba70 0x7f3ae40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38124/0 (socket says 172.21.15.164:38124) 2024-05-08T04:40:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae9200640 1 -- 172.21.15.164:0/2824855015 learned_addr learned my addr 172.21.15.164:0/2824855015 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:26.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae9c00640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ae4160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191442221 0 0) 0x7f3ae40cd9f0 con 0x7f3ae415ba70 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac8003530 con 0x7f3ae415ba70 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2439257662 0 0) 0x7f3ac8003530 con 0x7f3ae415ba70 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 msgr2=0x7f3ae4160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ae4160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 >> v1:172.21.15.172:6789/0 conn(0x7f3ae40cfda0 legacy=0x7f3ae40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae415f820 con 0x7f3ae415ba70 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f3ad4001a00 con 0x7f3ae415ba70 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 >> v1:172.21.15.164:6789/0 conn(0x7f3ae415ba70 legacy=0x7f3ae40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ac8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ac80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae415fe70 con 0x7f3ac80039f0 2024-05-08T04:40:26.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800ba80 con 0x7f3ac8004810 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800bc80 con 0x7f3ae415be50 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae9200640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ac8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae3e00640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.041+0000 7f3ae9c00640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ac80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 -- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 msgr2=0x7f3ac80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ac80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 -- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac8004810 msgr2=0x7f3ac8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 -- 172.21.15.164:0/2824855015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8003530 con 0x7f3ac80039f0 2024-05-08T04:40:26.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9c00640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae415be50 0x7f3ac80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:26.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9200640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ac8003e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400a810 tx=0x7f3ad401d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43ac63bcc711978 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400a460 con 0x7f3ac80039f0 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad40ad650 con 0x7f3ac80039f0 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae3400640 1 -- 172.21.15.164:0/2824855015 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40ada40 con 0x7f3ac80039f0 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 -- 172.21.15.164:0/2824855015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 msgr2=0x7f3ac8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 --2- 172.21.15.164:0/2824855015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ac8003e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400a810 tx=0x7f3ad401d2f0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3ae9c00640 1 -- 172.21.15.164:0/2824855015 reap_dead start 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 -- 172.21.15.164:0/2824855015 shutdown_connections 2024-05-08T04:40:26.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 -- 172.21.15.164:0/2824855015 >> 172.21.15.164:0/2824855015 conn(0x7f3ae406ac90 msgr2=0x7f3ae406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:26.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 -- 172.21.15.164:0/2824855015 shutdown_connections 2024-05-08T04:40:26.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.045+0000 7f3aebc00640 1 -- 172.21.15.164:0/2824855015 wait complete. 2024-05-08T04:40:26.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 Processor -- start 2024-05-08T04:40:26.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- start start 2024-05-08T04:40:26.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 0x7f3ae40b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae4158590 0x7f3ae4141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4160670 con 0x7f3ac80039f0 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41604f0 con 0x7f3ae4158110 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4167bc0 con 0x7f3ae4158590 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44366/0 (socket says 172.21.15.164:44366) 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 -- 172.21.15.164:0/1018107324 learned_addr learned my addr 172.21.15.164:0/1018107324 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae3e00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 0x7f3ae40b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 -- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae4158590 msgr2=0x7f3ae4141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae4158590 0x7f3ae4141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 -- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 msgr2=0x7f3ae40b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 0x7f3ae40b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 -- 172.21.15.164:0/1018107324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4071520 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae3e00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 0x7f3ae40b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae9200640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400a810 tx=0x7f3ac8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40b2040 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad40bc070 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae40b1e30 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40019c0 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.049+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41516f0 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4069d60 con 0x7f3ac80039f0 2024-05-08T04:40:26.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f3ad40ad650 con 0x7f3ac80039f0 2024-05-08T04:40:26.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3ae0c00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 0x7f3ac4069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3ad4120060 con 0x7f3ac80039f0 2024-05-08T04:40:26.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3ae3e00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 0x7f3ac4069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.053+0000 7f3ae3e00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 0x7f3ac4069690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c10 tx=0x7f3ad0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.057+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ad4017030 con 0x7f3ac80039f0 2024-05-08T04:40:26.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.265+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f3ae406e210 con 0x7f3ac80039f0 2024-05-08T04:40:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3ae0c00640 1 -- 172.21.15.164:0/1018107324 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v135) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3ad40f10c0 con 0x7f3ac80039f0 2024-05-08T04:40:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 msgr2=0x7f3ac4069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 0x7f3ac4069690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c10 tx=0x7f3ad0009040 comp rx=0 tx=0).stop 2024-05-08T04:40:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 msgr2=0x7f3ae4071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400a810 tx=0x7f3ac8005840 comp rx=0 tx=0).stop 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 shutdown_connections 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ac40671d0 0x7f3ac4069690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ae4158590 0x7f3ae4141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ae4158110 0x7f3ae40b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 --2- 172.21.15.164:0/1018107324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ac80039f0 0x7f3ae4071fc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 >> 172.21.15.164:0/1018107324 conn(0x7f3ae406ac90 msgr2=0x7f3ae40b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 shutdown_connections 2024-05-08T04:40:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.533+0000 7f3aebc00640 1 -- 172.21.15.164:0/1018107324 wait complete. 2024-05-08T04:40:26.558 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2024-05-08T04:40:26.559 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:40:26.559 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:26.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 Processor -- start 2024-05-08T04:40:26.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 -- start start 2024-05-08T04:40:26.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fe4163540 0x7f6fe40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fe40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40ceac0 con 0x7f6fe40aedb0 2024-05-08T04:40:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40ce860 con 0x7f6fe4163540 2024-05-08T04:40:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40cc1e0 con 0x7f6fe40cf6c0 2024-05-08T04:40:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6fe40aedb0 0x7f6fe40ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38126/0 (socket says 172.21.15.164:38126) 2024-05-08T04:40:26.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2a00640 1 -- 172.21.15.164:0/2490514056 learned_addr learned my addr 172.21.15.164:0/2490514056 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2490514056 >> v2:172.21.15.172:3300/0 conn(0x7f6fe4163540 0x7f6fe40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe3400640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fe40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033839483 0 0) 0x7f6fe40ceac0 con 0x7f6fe40aedb0 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc8003530 con 0x7f6fe40aedb0 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770332251 0 0) 0x7f6fc8003530 con 0x7f6fe40aedb0 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 msgr2=0x7f6fe40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fe40cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 >> v2:172.21.15.172:3300/0 conn(0x7f6fe4163540 msgr2=0x7f6fe40cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 --2- 172.21.15.164:0/2490514056 >> v2:172.21.15.172:3300/0 conn(0x7f6fe4163540 0x7f6fe40cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40cc4c0 con 0x7f6fe40aedb0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f6fd4001680 con 0x7f6fe40aedb0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 >> v1:172.21.15.164:6789/0 conn(0x7f6fe40aedb0 legacy=0x7f6fe40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fc80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fc8003f40 0x7f6fc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8004d50 0x7f6fc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40ce860 con 0x7f6fc8003f40 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40cc1e0 con 0x7f6fc8004d50 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc800ba20 con 0x7f6fe40cf6c0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fc8003f40 0x7f6fc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2a00640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fc80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe3400640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8004d50 0x7f6fc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 -- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 msgr2=0x7f6fc80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe40cf6c0 0x7f6fc80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 -- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8004d50 msgr2=0x7f6fc80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8004d50 0x7f6fc80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.693+0000 7f6fe2000640 1 -- 172.21.15.164:0/2490514056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8003530 con 0x7f6fc8003f40 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fc8003f40 0x7f6fc8004810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009b10 tx=0x7f6fd001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18f432aa2caed1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00b2070 con 0x7f6fc8003f40 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd00a5670 con 0x7f6fc8003f40 2024-05-08T04:40:26.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe1600640 1 -- 172.21.15.164:0/2490514056 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00096b0 con 0x7f6fc8003f40 2024-05-08T04:40:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 -- 172.21.15.164:0/2490514056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fc8003f40 msgr2=0x7f6fc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2490514056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fc8003f40 0x7f6fc8004810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009b10 tx=0x7f6fd001ca80 comp rx=0 tx=0).stop 2024-05-08T04:40:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe3400640 1 -- 172.21.15.164:0/2490514056 reap_dead start 2024-05-08T04:40:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 -- 172.21.15.164:0/2490514056 shutdown_connections 2024-05-08T04:40:26.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 -- 172.21.15.164:0/2490514056 >> 172.21.15.164:0/2490514056 conn(0x7f6fe406ac90 msgr2=0x7f6fe406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:26.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 -- 172.21.15.164:0/2490514056 shutdown_connections 2024-05-08T04:40:26.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.697+0000 7f6fe9800640 1 -- 172.21.15.164:0/2490514056 wait complete. 2024-05-08T04:40:26.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 Processor -- start 2024-05-08T04:40:26.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- start start 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8003f40 0x7f6fe40b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 0x7f6fe41560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe406d970 con 0x7f6fe41571a0 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe406d7f0 con 0x7f6fc8003f40 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe406daf0 con 0x7f6fe41ff2d0 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44390/0 (socket says 172.21.15.164:44390) 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 -- 172.21.15.164:0/2965802035 learned_addr learned my addr 172.21.15.164:0/2965802035 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2a00640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8003f40 0x7f6fe40b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe3400640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 0x7f6fe41560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 -- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 msgr2=0x7f6fe41560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 0x7f6fe41560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 -- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8003f40 msgr2=0x7f6fe40b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8003f40 0x7f6fe40b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 -- 172.21.15.164:0/2965802035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4159cb0 con 0x7f6fe41571a0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe3400640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 0x7f6fe41560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe2000640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fd001c8b0 tx=0x7f6fc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00bc030 con 0x7f6fe41571a0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd00a5e40 con 0x7f6fe41571a0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00b2040 con 0x7f6fe41571a0 2024-05-08T04:40:26.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe4159f50 con 0x7f6fe41571a0 2024-05-08T04:40:26.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.701+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41feaf0 con 0x7f6fe41571a0 2024-05-08T04:40:26.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe4069d60 con 0x7f6fe41571a0 2024-05-08T04:40:26.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f6fd00bf070 con 0x7f6fe41571a0 2024-05-08T04:40:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fdaa00640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 0x7f6fc4069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6fd011ee20 con 0x7f6fe41571a0 2024-05-08T04:40:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fe2a00640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 0x7f6fc4069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.705+0000 7f6fe2a00640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 0x7f6fc4069690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c10 tx=0x7f6fd4007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:26.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.709+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fd00c7020 con 0x7f6fe41571a0 2024-05-08T04:40:26.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fe40cbad0 con 0x7f6fe41571a0 2024-05-08T04:40:26.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fdaa00640 1 -- 172.21.15.164:0/2965802035 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13125 (secure 0 0 0) 0x7f6fd0016800 con 0x7f6fe41571a0 2024-05-08T04:40:26.873 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:26.873 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":51,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:25.532256+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6835/4206585796":"2024-05-09T04:40:22.523295+0000","172.21.15.164:6834/4206585796":"2024-05-09T04:40:22.523295+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 msgr2=0x7f6fc4069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 0x7f6fc4069690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c10 tx=0x7f6fd4007900 comp rx=0 tx=0).stop 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 msgr2=0x7f6fe415a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fd001c8b0 tx=0x7f6fc8005840 comp rx=0 tx=0).stop 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 shutdown_connections 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc40671d0 0x7f6fc4069690 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe41ff2d0 0x7f6fe41560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe41571a0 0x7f6fe415a7c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 --2- 172.21.15.164:0/2965802035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fc8003f40 0x7f6fe40b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:26.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 >> 172.21.15.164:0/2965802035 conn(0x7f6fe406ac90 msgr2=0x7f6fe415fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:26.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 shutdown_connections 2024-05-08T04:40:26.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:26.869+0000 7f6fe9800640 1 -- 172.21.15.164:0/2965802035 wait complete. 2024-05-08T04:40:26.894 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:26.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:27.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 Processor -- start 2024-05-08T04:40:27.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 -- start start 2024-05-08T04:40:27.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88700b2c60 0x7f88700cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 0x7f8870161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88700b2a20 con 0x7f88700b1f70 2024-05-08T04:40:27.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88700b3240 con 0x7f88700b2c60 2024-05-08T04:40:27.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8878200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887006be40 con 0x7f88700d0870 2024-05-08T04:40:27.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88700b2c60 0x7f88700cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88700b2c60 0x7f88700cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44400/0 (socket says 172.21.15.164:44400) 2024-05-08T04:40:27.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 -- 172.21.15.164:0/2147422994 learned_addr learned my addr 172.21.15.164:0/2147422994 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:27.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8876200640 1 --2- 172.21.15.164:0/2147422994 >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 0x7f8870161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 -- 172.21.15.164:0/2147422994 >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 msgr2=0x7f8870161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:27.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 --2- 172.21.15.164:0/2147422994 >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 0x7f8870161be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 -- 172.21.15.164:0/2147422994 >> v1:172.21.15.172:6789/0 conn(0x7f88700b1f70 legacy=0x7f88700d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:27.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8874e00640 1 -- 172.21.15.164:0/2147422994 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887006c120 con 0x7f88700b2c60 2024-05-08T04:40:27.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.029+0000 7f8876200640 1 --2- 172.21.15.164:0/2147422994 >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 0x7f8870161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:27.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8874e00640 1 --2- 172.21.15.164:0/2147422994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f88700cbfd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f88700d1320 tx=0x7f885c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdefdea926778ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f886fe00640 1 -- 172.21.15.164:0/2147422994 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0ab5f0 con 0x7f88700b2c60 2024-05-08T04:40:27.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f886fe00640 1 -- 172.21.15.164:0/2147422994 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c0b0070 con 0x7f88700b2c60 2024-05-08T04:40:27.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f886fe00640 1 -- 172.21.15.164:0/2147422994 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0ab9b0 con 0x7f88700b2c60 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 -- 172.21.15.164:0/2147422994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 msgr2=0x7f88700cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 --2- 172.21.15.164:0/2147422994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f88700cbfd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f88700d1320 tx=0x7f885c01c460 comp rx=0 tx=0).stop 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 -- 172.21.15.164:0/2147422994 shutdown_connections 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 --2- 172.21.15.164:0/2147422994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f88700cbfd0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 --2- 172.21.15.164:0/2147422994 >> v2:172.21.15.172:3300/0 conn(0x7f88700d0870 0x7f8870161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 -- 172.21.15.164:0/2147422994 >> 172.21.15.164:0/2147422994 conn(0x7f88700d1980 msgr2=0x7f88700d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 -- 172.21.15.164:0/2147422994 shutdown_connections 2024-05-08T04:40:27.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.033+0000 7f8878200640 1 -- 172.21.15.164:0/2147422994 wait complete. 2024-05-08T04:40:27.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 Processor -- start 2024-05-08T04:40:27.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- start start 2024-05-08T04:40:27.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88700b1f70 0x7f8870152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88700d0870 0x7f887015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700d0c80 con 0x7f88700b2c60 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700b2560 con 0x7f88700d0870 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700d0e00 con 0x7f88700b1f70 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44410/0 (socket says 172.21.15.164:44410) 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 -- 172.21.15.164:0/1534032797 learned_addr learned my addr 172.21.15.164:0/1534032797 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8875800640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88700b1f70 0x7f8870152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8876200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88700d0870 0x7f887015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 -- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88700b1f70 msgr2=0x7f8870152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88700b1f70 0x7f8870152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 -- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88700d0870 msgr2=0x7f887015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88700d0870 0x7f887015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 -- 172.21.15.164:0/1534032797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700c8f50 con 0x7f88700b2c60 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8874e00640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f885c01ab10 tx=0x7f885c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0be5f0 con 0x7f88700b2c60 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c0be7b0 con 0x7f88700b2c60 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88700c9230 con 0x7f88700b2c60 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0beba0 con 0x7f88700b2c60 2024-05-08T04:40:27.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.037+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88700c97f0 con 0x7f88700b2c60 2024-05-08T04:40:27.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887009fa50 con 0x7f88700b2c60 2024-05-08T04:40:27.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f885c0b43f0 con 0x7f88700b2c60 2024-05-08T04:40:27.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f886d600640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 0x7f8850069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f885c11f740 con 0x7f88700b2c60 2024-05-08T04:40:27.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f8875800640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 0x7f8850069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.041+0000 7f8875800640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 0x7f8850069690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88700b1a90 tx=0x7f8864009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.045+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f885c0c4060 con 0x7f88700b2c60 2024-05-08T04:40:27.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.197+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8870072890 con 0x7f88700b2c60 2024-05-08T04:40:27.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f886d600640 1 -- 172.21.15.164:0/1534032797 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f885c016110 con 0x7f88700b2c60 2024-05-08T04:40:27.204 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 msgr2=0x7f8850069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 0x7f8850069690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88700b1a90 tx=0x7f8864009040 comp rx=0 tx=0).stop 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 msgr2=0x7f8870157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f885c01ab10 tx=0x7f885c01aca0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 shutdown_connections 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88500671d0 0x7f8850069690 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88700d0870 0x7f887015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88700b2c60 0x7f8870157fb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 --2- 172.21.15.164:0/1534032797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88700b1f70 0x7f8870152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 >> 172.21.15.164:0/1534032797 conn(0x7f88700d1980 msgr2=0x7f887015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:27.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 shutdown_connections 2024-05-08T04:40:27.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.201+0000 7f8878200640 1 -- 172.21.15.164:0/1534032797 wait complete. 2024-05-08T04:40:27.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2024-05-08T04:40:27.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 Processor -- start 2024-05-08T04:40:27.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 -- start start 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff300ad890 0x7eff30071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300add10 0x7eff300719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff30072120 con 0x7eff3006ac90 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff30071f00 con 0x7eff300ad890 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff35200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff30072400 con 0x7eff300add10 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff2de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7eff3006ac90 0x7eff3006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:38132/0 (socket says 172.21.15.164:38132) 2024-05-08T04:40:27.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff2de00640 1 -- 172.21.15.164:0/3235004727 learned_addr learned my addr 172.21.15.164:0/3235004727 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:27.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff25400640 1 --2- 172.21.15.164:0/3235004727 >> v2:172.21.15.172:3300/0 conn(0x7eff300ad890 0x7eff30071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff2e800640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300add10 0x7eff300719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff2d400640 1 -- 172.21.15.164:0/3235004727 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304984220 0 0) 0x7eff30072120 con 0x7eff3006ac90 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff2d400640 1 -- 172.21.15.164:0/3235004727 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14003530 con 0x7eff3006ac90 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff25400640 1 -- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300add10 msgr2=0x7eff300719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff25400640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300add10 0x7eff300719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff25400640 1 -- 172.21.15.164:0/3235004727 >> v1:172.21.15.164:6789/0 conn(0x7eff3006ac90 legacy=0x7eff3006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.357+0000 7eff25400640 1 -- 172.21.15.164:0/3235004727 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff300726e0 con 0x7eff300ad890 2024-05-08T04:40:27.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff25400640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300ad890 0x7eff30071480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7eff180097c0 tx=0x7eff1801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fb509142e0741a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff2d400640 1 -- 172.21.15.164:0/3235004727 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180b2070 con 0x7eff300ad890 2024-05-08T04:40:27.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff2d400640 1 -- 172.21.15.164:0/3235004727 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180ad640 con 0x7eff300ad890 2024-05-08T04:40:27.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff2d400640 1 -- 172.21.15.164:0/3235004727 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180ada10 con 0x7eff300ad890 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 -- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300ad890 msgr2=0x7eff30071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300ad890 0x7eff30071480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7eff180097c0 tx=0x7eff1801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 -- 172.21.15.164:0/3235004727 shutdown_connections 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300add10 0x7eff300719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 --2- 172.21.15.164:0/3235004727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300ad890 0x7eff30071480 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 -- 172.21.15.164:0/3235004727 >> 172.21.15.164:0/3235004727 conn(0x7eff30066fa0 msgr2=0x7eff3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:27.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 -- 172.21.15.164:0/3235004727 shutdown_connections 2024-05-08T04:40:27.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.361+0000 7eff35200640 1 -- 172.21.15.164:0/3235004727 wait complete. 2024-05-08T04:40:27.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 Processor -- start 2024-05-08T04:40:27.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- start start 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300ad890 0x7eff3015df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300add10 0x7eff30144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3006e710 con 0x7eff3006ac90 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3006e590 con 0x7eff300add10 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3016ad60 con 0x7eff300ad890 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44426/0 (socket says 172.21.15.164:44426) 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 -- 172.21.15.164:0/3347194868 learned_addr learned my addr 172.21.15.164:0/3347194868 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2e800640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300add10 0x7eff30144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff25400640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300ad890 0x7eff3015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 -- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300ad890 msgr2=0x7eff3015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300ad890 0x7eff3015df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 -- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300add10 msgr2=0x7eff30144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300add10 0x7eff30144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 -- 172.21.15.164:0/3347194868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff30157b30 con 0x7eff3006ac90 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff2de00640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c450 tx=0x7eff2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20017070 con 0x7eff3006ac90 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20005cf0 con 0x7eff3006ac90 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff300b2b40 con 0x7eff3006ac90 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20007290 con 0x7eff3006ac90 2024-05-08T04:40:27.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.365+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff300b3100 con 0x7eff3006ac90 2024-05-08T04:40:27.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff30069d60 con 0x7eff3006ac90 2024-05-08T04:40:27.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7eff2001a070 con 0x7eff3006ac90 2024-05-08T04:40:27.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff26a00640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 0x7eff0c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:27.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7eff200844e0 con 0x7eff3006ac90 2024-05-08T04:40:27.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff25400640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 0x7eff0c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:27.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.369+0000 7eff25400640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 0x7eff0c069660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff180af000 tx=0x7eff18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:27.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.373+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff20054d80 con 0x7eff3006ac90 2024-05-08T04:40:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:27.585+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7eff300b18e0 con 0x7eff3006ac90 2024-05-08T04:40:28.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:40:28.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:40:28.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.537+0000 7eff26a00640 1 -- 172.21.15.164:0/3347194868 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v137) v1 ==== 169+0+0 (secure 0 0 0) 0x7eff2005b0d0 con 0x7eff3006ac90 2024-05-08T04:40:28.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 msgr2=0x7eff0c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 0x7eff0c069660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff180af000 tx=0x7eff18008040 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 msgr2=0x7eff300b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c450 tx=0x7eff2000c910 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 shutdown_connections 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff0c0671a0 0x7eff0c069660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300ad890 0x7eff3015df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300add10 0x7eff30144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 --2- 172.21.15.164:0/3347194868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff3006ac90 0x7eff300b24b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 >> 172.21.15.164:0/3347194868 conn(0x7eff30066fa0 msgr2=0x7eff3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 shutdown_connections 2024-05-08T04:40:28.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.541+0000 7eff35200640 1 -- 172.21.15.164:0/3347194868 wait complete. 2024-05-08T04:40:28.568 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:40:28.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.705+0000 7f78bc800640 1 Processor -- start 2024-05-08T04:40:28.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 -- start start 2024-05-08T04:40:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78b415f2b0 0x7f78b40d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f78b40b3fe0 0x7f78b40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b4070280 con 0x7f78b40cd390 2024-05-08T04:40:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b415f690 con 0x7f78b40b3fe0 2024-05-08T04:40:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78bc800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40cff80 con 0x7f78b415f2b0 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f78b40b3fe0 0x7f78b40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78b415f2b0 0x7f78b40d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f78b40b3fe0 0x7f78b40cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44440/0 (socket says 172.21.15.164:44440) 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 -- 172.21.15.164:0/2895464153 learned_addr learned my addr 172.21.15.164:0/2895464153 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78ba800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f78b40cd390 0x7f78b406e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:35270/0 (socket says 172.21.15.164:35270) 2024-05-08T04:40:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 -- 172.21.15.164:0/2895464153 >> v2:172.21.15.172:3300/0 conn(0x7f78b415f2b0 msgr2=0x7f78b40d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 --2- 172.21.15.164:0/2895464153 >> v2:172.21.15.172:3300/0 conn(0x7f78b415f2b0 0x7f78b40d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 -- 172.21.15.164:0/2895464153 >> v1:172.21.15.172:6789/0 conn(0x7f78b40cd390 legacy=0x7f78b406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b9400640 1 -- 172.21.15.164:0/2895464153 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b4070560 con 0x7f78b40b3fe0 2024-05-08T04:40:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.709+0000 7f78b8a00640 1 -- 172.21.15.164:0/2895464153 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447707755 0 0) 0x7f78b4070280 con 0x7f78b40cd390 2024-05-08T04:40:28.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78b9400640 1 --2- 172.21.15.164:0/2895464153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40b3fe0 0x7f78b40cce50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f78b4070870 tx=0x7f78a401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5fcef9269ef11c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:28.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78b8a00640 1 -- 172.21.15.164:0/2895464153 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40aa650 con 0x7f78b40b3fe0 2024-05-08T04:40:28.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78b8a00640 1 -- 172.21.15.164:0/2895464153 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40af070 con 0x7f78b40b3fe0 2024-05-08T04:40:28.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78b8a00640 1 -- 172.21.15.164:0/2895464153 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40aaa10 con 0x7f78b40b3fe0 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- 172.21.15.164:0/2895464153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40b3fe0 msgr2=0x7f78b40cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 --2- 172.21.15.164:0/2895464153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40b3fe0 0x7f78b40cce50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f78b4070870 tx=0x7f78a401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- 172.21.15.164:0/2895464153 shutdown_connections 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 --2- 172.21.15.164:0/2895464153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40b3fe0 0x7f78b40cce50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 --2- 172.21.15.164:0/2895464153 >> v2:172.21.15.172:3300/0 conn(0x7f78b415f2b0 0x7f78b40d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- 172.21.15.164:0/2895464153 >> 172.21.15.164:0/2895464153 conn(0x7f78b40b34f0 msgr2=0x7f78b40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- 172.21.15.164:0/2895464153 shutdown_connections 2024-05-08T04:40:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- 172.21.15.164:0/2895464153 wait complete. 2024-05-08T04:40:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 Processor -- start 2024-05-08T04:40:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.713+0000 7f78bc800640 1 -- start start 2024-05-08T04:40:28.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78b40b3fe0 0x7f78b415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78b415f2b0 0x7f78b4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4168bf0 con 0x7f78b40cd390 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406fda0 con 0x7f78b40b3fe0 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4168d70 con 0x7f78b415f2b0 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44446/0 (socket says 172.21.15.164:44446) 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 -- 172.21.15.164:0/1338253923 learned_addr learned my addr 172.21.15.164:0/1338253923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9e00640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78b40b3fe0 0x7f78b415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78ba800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78b415f2b0 0x7f78b4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 -- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78b415f2b0 msgr2=0x7f78b4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78b415f2b0 0x7f78b4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 -- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78b40b3fe0 msgr2=0x7f78b415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78b40b3fe0 0x7f78b415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 -- 172.21.15.164:0/1338253923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40c7400 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78b9400640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f78a4019100 tx=0x7f78a401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40bd5f0 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40a6040 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40bd990 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40c76e0 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40c7ca0 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b409fae0 con 0x7f78b40cd390 2024-05-08T04:40:28.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.717+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f78a40b33f0 con 0x7f78b40cd390 2024-05-08T04:40:28.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.721+0000 7f78ae000640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 0x7f7898069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:28.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.721+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f78a411f500 con 0x7f78b40cd390 2024-05-08T04:40:28.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.721+0000 7f78b9e00640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 0x7f7898069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:28.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.721+0000 7f78b9e00640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 0x7f7898069660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009ac0 tx=0x7f78a8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:28.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.725+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78a40c3060 con 0x7f78b40cd390 2024-05-08T04:40:28.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.877+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f78b40b7920 con 0x7f78b40cd390 2024-05-08T04:40:28.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.881+0000 7f78ae000640 1 -- 172.21.15.164:0/1338253923 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f78a4016110 con 0x7f78b40cd390 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 msgr2=0x7f7898069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 0x7f7898069660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009ac0 tx=0x7f78a8009190 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 msgr2=0x7f78b4150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f78a4019100 tx=0x7f78a401ad30 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 shutdown_connections 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78980671a0 0x7f7898069660 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78b415f2b0 0x7f78b4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78b40cd390 0x7f78b4150a90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 --2- 172.21.15.164:0/1338253923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78b40b3fe0 0x7f78b415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 >> 172.21.15.164:0/1338253923 conn(0x7f78b40b34f0 msgr2=0x7f78b40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 shutdown_connections 2024-05-08T04:40:28.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:28.885+0000 7f78bc800640 1 -- 172.21.15.164:0/1338253923 wait complete. 2024-05-08T04:40:28.908 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:40:28.908 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:28.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:29.069 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:40:29.069 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:29.069 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:29.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:40:29.078 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:29.078 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:29.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:40:29.087 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:29.088 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:29.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:40:29.127 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:29.127 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:29.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:40:29.142 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:40:29.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:29.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 Processor -- start 2024-05-08T04:40:29.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 -- start start 2024-05-08T04:40:29.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f80500cfda0 0x7f8050160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f805015be50 0x7f80500b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80500cd9f0 con 0x7f805015ba70 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805015fe70 con 0x7f80500cfda0 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8056a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500cdcd0 con 0x7f805015be50 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8047400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f80500cfda0 0x7f8050160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8047400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f80500cfda0 0x7f8050160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41176/0 (socket says 172.21.15.164:41176) 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.301+0000 7f8047400640 1 -- 172.21.15.164:0/1341482426 learned_addr learned my addr 172.21.15.164:0/1341482426 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:29.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8054a00640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f805015be50 0x7f80500b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8047400640 1 -- 172.21.15.164:0/1341482426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f805015be50 msgr2=0x7f80500b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:29.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8047400640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f805015be50 0x7f80500b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8047400640 1 -- 172.21.15.164:0/1341482426 >> v1:172.21.15.172:6789/0 conn(0x7f805015ba70 legacy=0x7f80500cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8047400640 1 -- 172.21.15.164:0/1341482426 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805015f820 con 0x7f80500cfda0 2024-05-08T04:40:29.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f8047400640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80500cfda0 0x7f8050160300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8038009870 tx=0x7f803801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392e9ee2e2c8d492 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f804f400640 1 -- 172.21.15.164:0/1341482426 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380ab640 con 0x7f80500cfda0 2024-05-08T04:40:29.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f804f400640 1 -- 172.21.15.164:0/1341482426 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380b0070 con 0x7f80500cfda0 2024-05-08T04:40:29.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.305+0000 7f804f400640 1 -- 172.21.15.164:0/1341482426 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380aba00 con 0x7f80500cfda0 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- 172.21.15.164:0/1341482426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80500cfda0 msgr2=0x7f8050160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80500cfda0 0x7f8050160300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8038009870 tx=0x7f803801c530 comp rx=0 tx=0).stop 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- 172.21.15.164:0/1341482426 shutdown_connections 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f805015be50 0x7f80500b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- 172.21.15.164:0/1341482426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80500cfda0 0x7f8050160300 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- 172.21.15.164:0/1341482426 >> 172.21.15.164:0/1341482426 conn(0x7f805006ac90 msgr2=0x7f805006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:29.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- 172.21.15.164:0/1341482426 shutdown_connections 2024-05-08T04:40:29.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- 172.21.15.164:0/1341482426 wait complete. 2024-05-08T04:40:29.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 Processor -- start 2024-05-08T04:40:29.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- start start 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80500cfda0 0x7f8050148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f805015be50 0x7f805015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050168d40 con 0x7f805015ba70 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050168bc0 con 0x7f805015be50 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8056a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8050168ec0 con 0x7f80500cfda0 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41190/0 (socket says 172.21.15.164:41190) 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 -- 172.21.15.164:0/869830262 learned_addr learned my addr 172.21.15.164:0/869830262 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f804fe00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80500cfda0 0x7f8050148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8054a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f805015be50 0x7f805015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 -- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80500cfda0 msgr2=0x7f8050148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80500cfda0 0x7f8050148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 -- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f805015be50 msgr2=0x7f805015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f805015be50 0x7f805015ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.309+0000 7f8047400640 1 -- 172.21.15.164:0/869830262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805014cdc0 con 0x7f805015ba70 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f8047400640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f80380ad040 tx=0x7f803801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380be3f0 con 0x7f805015ba70 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380be590 con 0x7f805015ba70 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380be980 con 0x7f805015ba70 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805014d0a0 con 0x7f805015ba70 2024-05-08T04:40:29.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805014d660 con 0x7f805015ba70 2024-05-08T04:40:29.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8050069d60 con 0x7f805015ba70 2024-05-08T04:40:29.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f80380b43f0 con 0x7f805015ba70 2024-05-08T04:40:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 0x7f802c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f803811f490 con 0x7f805015ba70 2024-05-08T04:40:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.313+0000 7f804fe00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 0x7f802c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.317+0000 7f804fe00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 0x7f802c069660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8040001810 tx=0x7f8040009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.317+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8038016150 con 0x7f805015ba70 2024-05-08T04:40:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.505+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f805006e210 con 0x7f805015ba70 2024-05-08T04:40:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.505+0000 7f804cc00640 1 -- 172.21.15.164:0/869830262 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f80380efcb0 con 0x7f805015ba70 2024-05-08T04:40:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:29.512 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/934638396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":934638396},{"type":"v1","addr":"172.21.15.164:6837","nonce":934638396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5105,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/163044006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":163044006},{"type":"v1","addr":"172.21.15.172:6834","nonce":163044006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3535735957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3535735957},{"type":"v1","addr":"172.21.15.172:6835","nonce":3535735957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2750360994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2750360994},{"type":"v1","addr":"172.21.15.164:6835","nonce":2750360994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2024-05-08T04:40:29.512 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-05-08T04:40:29.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 msgr2=0x7f802c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 0x7f802c069660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8040001810 tx=0x7f8040009040 comp rx=0 tx=0).stop 2024-05-08T04:40:29.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 msgr2=0x7f805014c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f80380ad040 tx=0x7f803801ad70 comp rx=0 tx=0).stop 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 shutdown_connections 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f802c0671a0 0x7f802c069660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f805015be50 0x7f805015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f805015ba70 0x7f805014c4c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 --2- 172.21.15.164:0/869830262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80500cfda0 0x7f8050148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 >> 172.21.15.164:0/869830262 conn(0x7f805006ac90 msgr2=0x7f80501509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 shutdown_connections 2024-05-08T04:40:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.509+0000 7f8056a00640 1 -- 172.21.15.164:0/869830262 wait complete. 2024-05-08T04:40:29.532 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:40:29.532 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:40:29.532 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:40:29.533 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:40:29.533 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:40:29.533 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:40:29.533 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:40:29.533 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:40:29.533 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:40:29.533 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:40:29.534 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:40:29.534 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:40:29.534 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:40:29.534 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:40:29.534 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:40:29.534 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:40:29.546 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:40:29.547 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:40:29.548 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:40:29.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:40:29.551 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:40:29.551 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:40:29.552 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:40:29.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:40:29.594 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:40:29.594 DEBUG:teuthology.parallel:result is None 2024-05-08T04:40:29.595 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:40:29.596 DEBUG:teuthology.parallel:result is None 2024-05-08T04:40:29.597 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:40:29.597 DEBUG:teuthology.parallel:result is None 2024-05-08T04:40:29.598 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:40:29.599 DEBUG:teuthology.parallel:result is None 2024-05-08T04:40:29.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:40:29.680 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:40:29.681 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:40:29.686 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:40:29.687 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:40:29.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 Processor -- start 2024-05-08T04:40:29.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 -- start start 2024-05-08T04:40:29.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c5415bc50 0x7f3c540b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540b42a0 con 0x7f3c540ccfe0 2024-05-08T04:40:29.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540cd4e0 con 0x7f3c540d0780 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.737+0000 7f3c5b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540cd7c0 con 0x7f3c5415bc50 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3c540ccfe0 0x7f3c540d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45866/0 (socket says 172.21.15.164:45866) 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53e00640 1 -- 172.21.15.164:0/3954228164 learned_addr learned my addr 172.21.15.164:0/3954228164 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c58a00640 1 --2- 172.21.15.164:0/3954228164 >> v2:172.21.15.172:3300/0 conn(0x7f3c5415bc50 0x7f3c540b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53400640 1 -- 172.21.15.164:0/3954228164 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355355704 0 0) 0x7f3c540b42a0 con 0x7f3c540ccfe0 2024-05-08T04:40:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c58a00640 1 -- 172.21.15.164:0/3954228164 >> v1:172.21.15.172:6789/0 conn(0x7f3c540d0780 legacy=0x7f3c5415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c58a00640 1 -- 172.21.15.164:0/3954228164 >> v1:172.21.15.164:6789/0 conn(0x7f3c540ccfe0 legacy=0x7f3c540d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c58a00640 1 -- 172.21.15.164:0/3954228164 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540cf590 con 0x7f3c5415bc50 2024-05-08T04:40:29.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c58a00640 1 --2- 172.21.15.164:0/3954228164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c5415bc50 0x7f3c540b4810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009740 tx=0x7f3c4801c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a539519a18920b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53400640 1 -- 172.21.15.164:0/3954228164 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480b0070 con 0x7f3c5415bc50 2024-05-08T04:40:29.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53400640 1 -- 172.21.15.164:0/3954228164 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c480ab600 con 0x7f3c5415bc50 2024-05-08T04:40:29.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.741+0000 7f3c53400640 1 -- 172.21.15.164:0/3954228164 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480ab9d0 con 0x7f3c5415bc50 2024-05-08T04:40:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- 172.21.15.164:0/3954228164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c5415bc50 msgr2=0x7f3c540b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3954228164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c5415bc50 0x7f3c540b4810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009740 tx=0x7f3c4801c210 comp rx=0 tx=0).stop 2024-05-08T04:40:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- 172.21.15.164:0/3954228164 shutdown_connections 2024-05-08T04:40:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3954228164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c5415bc50 0x7f3c540b4810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- 172.21.15.164:0/3954228164 >> 172.21.15.164:0/3954228164 conn(0x7f3c540b34f0 msgr2=0x7f3c540b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- 172.21.15.164:0/3954228164 shutdown_connections 2024-05-08T04:40:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- 172.21.15.164:0/3954228164 wait complete. 2024-05-08T04:40:29.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 Processor -- start 2024-05-08T04:40:29.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- start start 2024-05-08T04:40:29.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c540ccfe0 0x7f3c5414a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 0x7f3c540ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54169c70 con 0x7f3c5415bc50 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54169af0 con 0x7f3c540d0780 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c5b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54169df0 con 0x7f3c540ccfe0 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41236/0 (socket says 172.21.15.164:41236) 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 -- 172.21.15.164:0/3489829989 learned_addr learned my addr 172.21.15.164:0/3489829989 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c53e00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 0x7f3c540ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c58a00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c540ccfe0 0x7f3c5414a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 -- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c540ccfe0 msgr2=0x7f3c5414a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c540ccfe0 0x7f3c5414a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 -- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 msgr2=0x7f3c540ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:29.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 0x7f3c540ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:29.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.745+0000 7f3c59400640 1 -- 172.21.15.164:0/3489829989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5414afe0 con 0x7f3c5415bc50 2024-05-08T04:40:29.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c53e00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 0x7f3c540ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:29.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c59400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cb20 tx=0x7f3c4c002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c017070 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c003db0 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5414b2c0 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00a940 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5414b880 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54069d60 con 0x7f3c5415bc50 2024-05-08T04:40:29.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-05-08T04:40:29.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-05-08T04:40:29.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-05-08T04:40:29.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f3c4c01a070 con 0x7f3c5415bc50 2024-05-08T04:40:29.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 0x7f3c34069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:29.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3c4c084920 con 0x7f3c5415bc50 2024-05-08T04:40:29.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.749+0000 7f3c58a00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 0x7f3c34069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:29.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.753+0000 7f3c58a00640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 0x7f3c34069690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c480ad000 tx=0x7f3c480a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:29.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.753+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c4c0551c0 con 0x7f3c5415bc50 2024-05-08T04:40:29.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:29.921+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c5406e3c0 con 0x7f3c5415bc50 2024-05-08T04:40:30.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.897+0000 7f3c50c00640 1 -- 172.21.15.164:0/3489829989 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c4c059260 con 0x7f3c5415bc50 2024-05-08T04:40:30.903 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:40:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 msgr2=0x7f3c34069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 0x7f3c34069690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c480ad000 tx=0x7f3c480a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 msgr2=0x7f3c540ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cb20 tx=0x7f3c4c002080 comp rx=0 tx=0).stop 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 shutdown_connections 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c340671d0 0x7f3c34069690 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c5415bc50 0x7f3c540ca8c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c540d0780 0x7f3c540ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 --2- 172.21.15.164:0/3489829989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c540ccfe0 0x7f3c5414a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 >> 172.21.15.164:0/3489829989 conn(0x7f3c540b34f0 msgr2=0x7f3c5406fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 shutdown_connections 2024-05-08T04:40:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:30.901+0000 7f3c5b400640 1 -- 172.21.15.164:0/3489829989 wait complete. 2024-05-08T04:40:30.930 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:40:31.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 Processor -- start 2024-05-08T04:40:31.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 -- start start 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c15dd70 0x7f199c1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f199c1625f0 0x7f199c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c06cc70 con 0x7f199c06c7b0 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c06cf50 con 0x7f199c1625f0 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f19a1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0b1570 con 0x7f199c15dd70 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f199c1625f0 0x7f199c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f199c1625f0 0x7f199c0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41252/0 (socket says 172.21.15.164:41252) 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 -- 172.21.15.164:0/27232198 learned_addr learned my addr 172.21.15.164:0/27232198 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f1992000640 1 --1- 172.21.15.164:0/27232198 >> v1:172.21.15.172:6789/0 conn(0x7f199c06c7b0 0x7f199c161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:47438/0 (socket says 172.21.15.164:47438) 2024-05-08T04:40:31.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199aa00640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c15dd70 0x7f199c1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 -- 172.21.15.164:0/27232198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c15dd70 msgr2=0x7f199c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:31.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c15dd70 0x7f199c1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 -- 172.21.15.164:0/27232198 >> v1:172.21.15.172:6789/0 conn(0x7f199c06c7b0 legacy=0x7f199c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.065+0000 7f199b400640 1 -- 172.21.15.164:0/27232198 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0b1850 con 0x7f199c1625f0 2024-05-08T04:40:31.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f199b400640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c1625f0 0x7f199c0b1d70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1994009e50 tx=0x7f199401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a093b8d002488e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f199a000640 1 -- 172.21.15.164:0/27232198 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940b4070 con 0x7f199c1625f0 2024-05-08T04:40:31.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f199a000640 1 -- 172.21.15.164:0/27232198 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19940af400 con 0x7f199c1625f0 2024-05-08T04:40:31.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f199a000640 1 -- 172.21.15.164:0/27232198 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940af7d0 con 0x7f199c1625f0 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 -- 172.21.15.164:0/27232198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c1625f0 msgr2=0x7f199c0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c1625f0 0x7f199c0b1d70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1994009e50 tx=0x7f199401f210 comp rx=0 tx=0).stop 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 -- 172.21.15.164:0/27232198 shutdown_connections 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c1625f0 0x7f199c0b1d70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 --2- 172.21.15.164:0/27232198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c15dd70 0x7f199c1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 -- 172.21.15.164:0/27232198 >> 172.21.15.164:0/27232198 conn(0x7f199c0b34f0 msgr2=0x7f199c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 -- 172.21.15.164:0/27232198 shutdown_connections 2024-05-08T04:40:31.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.069+0000 7f19a1e00640 1 -- 172.21.15.164:0/27232198 wait complete. 2024-05-08T04:40:31.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 Processor -- start 2024-05-08T04:40:31.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- start start 2024-05-08T04:40:31.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c15dd70 0x7f199c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 0x7f199c14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c15e2d0 con 0x7f199c06c7b0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c15e150 con 0x7f199c15dd70 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0b22b0 con 0x7f199c1625f0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41266/0 (socket says 172.21.15.164:41266) 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 -- 172.21.15.164:0/2502665488 learned_addr learned my addr 172.21.15.164:0/2502665488 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f1992000640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c15dd70 0x7f199c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199b400640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 0x7f199c14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 -- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 msgr2=0x7f199c14f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 0x7f199c14f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 -- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c15dd70 msgr2=0x7f199c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c15dd70 0x7f199c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 -- 172.21.15.164:0/2502665488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c1f7f00 con 0x7f199c06c7b0 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199b400640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 0x7f199c14f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:31.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f199aa00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e930 tx=0x7f198c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c017070 con 0x7f199c06c7b0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00ca00 con 0x7f199c06c7b0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00cdb0 con 0x7f199c06c7b0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c1f81c0 con 0x7f199c06c7b0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.073+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c1603b0 con 0x7f199c06c7b0 2024-05-08T04:40:31.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c1f8390 con 0x7f199c06c7b0 2024-05-08T04:40:31.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f198c01a070 con 0x7f199c06c7b0 2024-05-08T04:40:31.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f1993e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 0x7f1978069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f198c085150 con 0x7f199c06c7b0 2024-05-08T04:40:31.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f1992000640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 0x7f1978069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.077+0000 7f1992000640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 0x7f1978069660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f199c0afe90 tx=0x7f1984007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.081+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f198c055b00 con 0x7f199c06c7b0 2024-05-08T04:40:31.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.289+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f199c069d60 con 0x7f199c06c7b0 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.289+0000 7f1993e00640 1 -- 172.21.15.164:0/2502665488 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f198c059ba0 con 0x7f199c06c7b0 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:40:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:40:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:40:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:40:31.307 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:40:31.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 msgr2=0x7f1978069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 0x7f1978069660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f199c0afe90 tx=0x7f1984007e90 comp rx=0 tx=0).stop 2024-05-08T04:40:31.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 msgr2=0x7f199c0b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e930 tx=0x7f198c00edf0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 shutdown_connections 2024-05-08T04:40:31.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f19780671a0 0x7f1978069660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c1625f0 0x7f199c14f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c15dd70 0x7f199c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 --2- 172.21.15.164:0/2502665488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f199c06c7b0 0x7f199c0b0240 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 >> 172.21.15.164:0/2502665488 conn(0x7f199c0b34f0 msgr2=0x7f199c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 shutdown_connections 2024-05-08T04:40:31.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.293+0000 7f19a1e00640 1 -- 172.21.15.164:0/2502665488 wait complete. 2024-05-08T04:40:31.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:31.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 Processor -- start 2024-05-08T04:40:31.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 -- start start 2024-05-08T04:40:31.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e4815ab60 0x7f6e48072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480b26c0 0x7f6e480cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e48072640 con 0x7f6e480cfbd0 2024-05-08T04:40:31.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480d0060 con 0x7f6e4815ab60 2024-05-08T04:40:31.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e4d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4806c770 con 0x7f6e480b26c0 2024-05-08T04:40:31.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.445+0000 7f6e45e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e4815ab60 0x7f6e48072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e4815ab60 0x7f6e48072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41276/0 (socket says 172.21.15.164:41276) 2024-05-08T04:40:31.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 -- 172.21.15.164:0/2316518872 learned_addr learned my addr 172.21.15.164:0/2316518872 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e46800640 1 --1- 172.21.15.164:0/2316518872 >> v1:172.21.15.164:6789/0 conn(0x7f6e480cfbd0 0x7f6e480cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45876/0 (socket says 172.21.15.164:45876) 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45400640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480b26c0 0x7f6e480cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 -- 172.21.15.164:0/2316518872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480b26c0 msgr2=0x7f6e480cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480b26c0 0x7f6e480cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 -- 172.21.15.164:0/2316518872 >> v1:172.21.15.164:6789/0 conn(0x7f6e480cfbd0 legacy=0x7f6e480cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 -- 172.21.15.164:0/2316518872 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4806ca50 con 0x7f6e4815ab60 2024-05-08T04:40:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e45e00640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48072100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958529b25cd6ca03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e44a00640 1 -- 172.21.15.164:0/2316518872 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380b2070 con 0x7f6e4815ab60 2024-05-08T04:40:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e44a00640 1 -- 172.21.15.164:0/2316518872 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e380a9070 con 0x7f6e4815ab60 2024-05-08T04:40:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.449+0000 7f6e44a00640 1 -- 172.21.15.164:0/2316518872 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380ad880 con 0x7f6e4815ab60 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- 172.21.15.164:0/2316518872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 msgr2=0x7f6e48072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48072100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3801c560 comp rx=0 tx=0).stop 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- 172.21.15.164:0/2316518872 shutdown_connections 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480b26c0 0x7f6e480cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- 172.21.15.164:0/2316518872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48072100 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- 172.21.15.164:0/2316518872 >> 172.21.15.164:0/2316518872 conn(0x7f6e4806ac90 msgr2=0x7f6e4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- 172.21.15.164:0/2316518872 shutdown_connections 2024-05-08T04:40:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- 172.21.15.164:0/2316518872 wait complete. 2024-05-08T04:40:31.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 Processor -- start 2024-05-08T04:40:31.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- start start 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 0x7f6e481546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480cfbd0 0x7f6e48155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4806c550 con 0x7f6e4815ab60 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4806c3d0 con 0x7f6e480b26c0 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e4d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480aed30 con 0x7f6e480cfbd0 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41280/0 (socket says 172.21.15.164:41280) 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 -- 172.21.15.164:0/246537583 learned_addr learned my addr 172.21.15.164:0/246537583 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:31.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e45400640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480cfbd0 0x7f6e48155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 -- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480cfbd0 msgr2=0x7f6e48155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e45e00640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 0x7f6e481546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480cfbd0 0x7f6e48155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 -- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 msgr2=0x7f6e481546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 0x7f6e481546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e46800640 1 -- 172.21.15.164:0/246537583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4814c600 con 0x7f6e4815ab60 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.453+0000 7f6e45e00640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 0x7f6e481546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e46800640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000caa0 tx=0x7f6e4000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40017070 con 0x7f6e4815ab60 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000a730 con 0x7f6e4815ab60 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000ab00 con 0x7f6e4815ab60 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e48154cc0 con 0x7f6e4815ab60 2024-05-08T04:40:31.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e48155280 con 0x7f6e4815ab60 2024-05-08T04:40:31.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48069d60 con 0x7f6e4815ab60 2024-05-08T04:40:31.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f6e4001a070 con 0x7f6e4815ab60 2024-05-08T04:40:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 0x7f6e28069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f6e400849b0 con 0x7f6e4815ab60 2024-05-08T04:40:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.457+0000 7f6e45e00640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 0x7f6e28069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.461+0000 7f6e45e00640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 0x7f6e28069690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e380af000 tx=0x7f6e38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.461+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e4814c600 con 0x7f6e4815ab60 2024-05-08T04:40:31.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e480cec90 con 0x7f6e4815ab60 2024-05-08T04:40:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e3e000640 1 -- 172.21.15.164:0/246537583 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12994 (secure 0 0 0) 0x7f6e40055980 con 0x7f6e4815ab60 2024-05-08T04:40:31.625 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:31.625 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":56,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:30.899230+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 msgr2=0x7f6e28069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 0x7f6e28069690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e380af000 tx=0x7f6e38008040 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 msgr2=0x7f6e48153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000caa0 tx=0x7f6e4000cf60 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 shutdown_connections 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e280671d0 0x7f6e28069690 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e480cfbd0 0x7f6e48155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4815ab60 0x7f6e48153b50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 --2- 172.21.15.164:0/246537583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e480b26c0 0x7f6e481546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 >> 172.21.15.164:0/246537583 conn(0x7f6e4806ac90 msgr2=0x7f6e480b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 shutdown_connections 2024-05-08T04:40:31.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.621+0000 7f6e4d000640 1 -- 172.21.15.164:0/246537583 wait complete. 2024-05-08T04:40:31.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:31.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:31.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 Processor -- start 2024-05-08T04:40:31.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 -- start start 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 0x7f0f04160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0f0406de50 0x7f0f0406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f040c6600 con 0x7f0f040ae6e0 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0415fbe0 con 0x7f0f0406de50 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040c6040 con 0x7f0f0415ba20 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f08e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0f0406de50 0x7f0f0406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0a200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0f040ae6e0 0x7f0f040cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45882/0 (socket says 172.21.15.164:45882) 2024-05-08T04:40:31.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f0a200640 1 -- 172.21.15.164:0/3047878890 learned_addr learned my addr 172.21.15.164:0/3047878890 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:31.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f09800640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 0x7f0f04160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f08e00640 1 -- 172.21.15.164:0/3047878890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 msgr2=0x7f0f04160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:31.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f08e00640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 0x7f0f04160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.781+0000 7f0f08e00640 1 -- 172.21.15.164:0/3047878890 >> v1:172.21.15.164:6789/0 conn(0x7f0f040ae6e0 legacy=0x7f0f040cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f08e00640 1 -- 172.21.15.164:0/3047878890 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f040c6320 con 0x7f0f0406de50 2024-05-08T04:40:31.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f09800640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 0x7f0f04160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:31.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f08e00640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0406de50 0x7f0f0406e290 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0f0415fd60 tx=0x7f0ef801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c924c1edf49230 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f03e00640 1 -- 172.21.15.164:0/3047878890 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80ab610 con 0x7f0f0406de50 2024-05-08T04:40:31.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f03e00640 1 -- 172.21.15.164:0/3047878890 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef80b0070 con 0x7f0f0406de50 2024-05-08T04:40:31.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f03e00640 1 -- 172.21.15.164:0/3047878890 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80ab9d0 con 0x7f0f0406de50 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 -- 172.21.15.164:0/3047878890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0406de50 msgr2=0x7f0f0406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0406de50 0x7f0f0406e290 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0f0415fd60 tx=0x7f0ef801c470 comp rx=0 tx=0).stop 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 -- 172.21.15.164:0/3047878890 shutdown_connections 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0406de50 0x7f0f0406e290 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 --2- 172.21.15.164:0/3047878890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0415ba20 0x7f0f04160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 -- 172.21.15.164:0/3047878890 >> 172.21.15.164:0/3047878890 conn(0x7f0f040cf770 msgr2=0x7f0f040cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 -- 172.21.15.164:0/3047878890 shutdown_connections 2024-05-08T04:40:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.785+0000 7f0f0c200640 1 -- 172.21.15.164:0/3047878890 wait complete. 2024-05-08T04:40:31.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 Processor -- start 2024-05-08T04:40:31.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- start start 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0406de50 0x7f0f041556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f040ae6e0 0x7f0f04071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040aec40 con 0x7f0f0415ba20 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040aeac0 con 0x7f0f040ae6e0 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040aedc0 con 0x7f0f0406de50 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41306/0 (socket says 172.21.15.164:41306) 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 -- 172.21.15.164:0/239263454 learned_addr learned my addr 172.21.15.164:0/239263454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f09800640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0406de50 0x7f0f041556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f08e00640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f040ae6e0 0x7f0f04071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 -- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0406de50 msgr2=0x7f0f041556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0406de50 0x7f0f041556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 -- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f040ae6e0 msgr2=0x7f0f04071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f040ae6e0 0x7f0f04071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 -- 172.21.15.164:0/239263454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f041510e0 con 0x7f0f0415ba20 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0a200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00cbc0 tx=0x7f0efc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc017070 con 0x7f0f0415ba20 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc00ad40 con 0x7f0f0415ba20 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f04070ab0 con 0x7f0f0415ba20 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc012600 con 0x7f0f0415ba20 2024-05-08T04:40:31.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.789+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04071070 con 0x7f0f0415ba20 2024-05-08T04:40:31.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f04072d30 con 0x7f0f0415ba20 2024-05-08T04:40:31.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f0efc01a070 con 0x7f0f0415ba20 2024-05-08T04:40:31.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f01600640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 0x7f0ee8069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:31.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f0efc084e30 con 0x7f0f0415ba20 2024-05-08T04:40:31.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f09800640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 0x7f0ee8069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:31.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.793+0000 7f0f09800640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 0x7f0ee8069690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400a9a0 tx=0x7f0ef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:31.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.797+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0efc055720 con 0x7f0f0415ba20 2024-05-08T04:40:31.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f0404f010 con 0x7f0f0415ba20 2024-05-08T04:40:31.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f01600640 1 -- 172.21.15.164:0/239263454 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0efc0597c0 con 0x7f0f0415ba20 2024-05-08T04:40:31.957 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 msgr2=0x7f0ee8069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 0x7f0ee8069690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400a9a0 tx=0x7f0ef4008040 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 msgr2=0x7f0f040b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00cbc0 tx=0x7f0efc00a430 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 shutdown_connections 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ee80671d0 0x7f0ee8069690 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f0415ba20 0x7f0f040b0970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f040ae6e0 0x7f0f04071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 --2- 172.21.15.164:0/239263454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0406de50 0x7f0f041556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.953+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 >> 172.21.15.164:0/239263454 conn(0x7f0f040cf770 msgr2=0x7f0f0415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.957+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 shutdown_connections 2024-05-08T04:40:31.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:31.957+0000 7f0f0c200640 1 -- 172.21.15.164:0/239263454 wait complete. 2024-05-08T04:40:31.980 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:40:32.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 Processor -- start 2024-05-08T04:40:32.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 -- start start 2024-05-08T04:40:32.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 0x7f8d4c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c15cf90 con 0x7f8d4c06efd0 2024-05-08T04:40:32.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4c15c150 con 0x7f8d4c15f3b0 2024-05-08T04:40:32.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d51000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c15c410 con 0x7f8d4c15b830 2024-05-08T04:40:32.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 0x7f8d4c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:46292/0 (socket says 172.21.15.164:46292) 2024-05-08T04:40:32.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 -- 172.21.15.164:0/4061469621 learned_addr learned my addr 172.21.15.164:0/4061469621 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4be00640 1 --1- 172.21.15.164:0/4061469621 >> v1:172.21.15.172:6789/0 conn(0x7f8d4c06efd0 0x7f8d4c06cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:47448/0 (socket says 172.21.15.164:47448) 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 -- 172.21.15.164:0/4061469621 >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 msgr2=0x7f8d4c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 --2- 172.21.15.164:0/4061469621 >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 0x7f8d4c06ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 -- 172.21.15.164:0/4061469621 >> v1:172.21.15.172:6789/0 conn(0x7f8d4c06efd0 legacy=0x7f8d4c06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4b400640 1 -- 172.21.15.164:0/4061469621 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c15c6f0 con 0x7f8d4c15b830 2024-05-08T04:40:32.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.113+0000 7f8d4aa00640 1 --2- 172.21.15.164:0/4061469621 >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 0x7f8d4c06ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:32.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d4b400640 1 --2- 172.21.15.164:0/4061469621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009870 tx=0x7f8d3c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9f5701a39903e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d4a000640 1 -- 172.21.15.164:0/4061469621 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0b2070 con 0x7f8d4c15b830 2024-05-08T04:40:32.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d4a000640 1 -- 172.21.15.164:0/4061469621 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c0a9070 con 0x7f8d4c15b830 2024-05-08T04:40:32.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d4a000640 1 -- 172.21.15.164:0/4061469621 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0ad880 con 0x7f8d4c15b830 2024-05-08T04:40:32.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 -- 172.21.15.164:0/4061469621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 msgr2=0x7f8d4c15bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 --2- 172.21.15.164:0/4061469621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009870 tx=0x7f8d3c01c560 comp rx=0 tx=0).stop 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 -- 172.21.15.164:0/4061469621 shutdown_connections 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 --2- 172.21.15.164:0/4061469621 >> v2:172.21.15.172:3300/0 conn(0x7f8d4c15f3b0 0x7f8d4c06ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 --2- 172.21.15.164:0/4061469621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15b830 0x7f8d4c15bc10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 -- 172.21.15.164:0/4061469621 >> 172.21.15.164:0/4061469621 conn(0x7f8d4c0d0410 msgr2=0x7f8d4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 -- 172.21.15.164:0/4061469621 shutdown_connections 2024-05-08T04:40:32.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.117+0000 7f8d51000640 1 -- 172.21.15.164:0/4061469621 wait complete. 2024-05-08T04:40:32.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 Processor -- start 2024-05-08T04:40:32.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- start start 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d4c06efd0 0x7f8d4c071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15f3b0 0x7f8d4c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1686c0 con 0x7f8d4c15b830 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c168540 con 0x7f8d4c06efd0 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c168840 con 0x7f8d4c15f3b0 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41314/0 (socket says 172.21.15.164:41314) 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 -- 172.21.15.164:0/3159720112 learned_addr learned my addr 172.21.15.164:0/3159720112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4be00640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15f3b0 0x7f8d4c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4b400640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d4c06efd0 0x7f8d4c071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 -- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15f3b0 msgr2=0x7f8d4c149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15f3b0 0x7f8d4c149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 -- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d4c06efd0 msgr2=0x7f8d4c071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d4c06efd0 0x7f8d4c071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 -- 172.21.15.164:0/3159720112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c067d40 con 0x7f8d4c15b830 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d4aa00640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ebe0 tx=0x7f8d3800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38017070 con 0x7f8d4c15b830 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800cc30 con 0x7f8d4c15b830 2024-05-08T04:40:32.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38013470 con 0x7f8d4c15b830 2024-05-08T04:40:32.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c15ed00 con 0x7f8d4c15b830 2024-05-08T04:40:32.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c071080 con 0x7f8d4c15b830 2024-05-08T04:40:32.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.121+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c04ef30 con 0x7f8d4c15b830 2024-05-08T04:40:32.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.125+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f8d3801a070 con 0x7f8d4c15b830 2024-05-08T04:40:32.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.125+0000 7f8d43400640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 0x7f8d2c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.125+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8d38085080 con 0x7f8d4c15b830 2024-05-08T04:40:32.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.125+0000 7f8d4b400640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 0x7f8d2c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.125+0000 7f8d4b400640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 0x7f8d2c069660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0af000 tx=0x7f8d3c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.129+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d38055970 con 0x7f8d4c15b830 2024-05-08T04:40:32.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.285+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8d4c156e10 con 0x7f8d4c15b830 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.285+0000 7f8d43400640 1 -- 172.21.15.164:0/3159720112 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8d38059a10 con 0x7f8d4c15b830 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:32.291 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 msgr2=0x7f8d2c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 0x7f8d2c069660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0af000 tx=0x7f8d3c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 msgr2=0x7f8d4c15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ebe0 tx=0x7f8d3800c520 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 shutdown_connections 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8d2c0671a0 0x7f8d2c069660 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d4c15f3b0 0x7f8d4c149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d4c15b830 0x7f8d4c15e4b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 --2- 172.21.15.164:0/3159720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d4c06efd0 0x7f8d4c071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 >> 172.21.15.164:0/3159720112 conn(0x7f8d4c0d0410 msgr2=0x7f8d4c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:32.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 shutdown_connections 2024-05-08T04:40:32.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.289+0000 7f8d51000640 1 -- 172.21.15.164:0/3159720112 wait complete. 2024-05-08T04:40:32.312 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:40:32.312 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:40:32.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 Processor -- start 2024-05-08T04:40:32.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 -- start start 2024-05-08T04:40:32.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e3815ab10 0x7f5e3806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e380aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e380ce780 0x7f5e380cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380cf130 con 0x7f5e3815ab10 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3806bb80 con 0x7f5e380ce780 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3815e870 con 0x7f5e380ae6e0 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e3815ab10 0x7f5e3806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e3815ab10 0x7f5e3806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41326/0 (socket says 172.21.15.164:41326) 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 -- 172.21.15.164:0/4147400643 learned_addr learned my addr 172.21.15.164:0/4147400643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e3cc00640 1 --2- 172.21.15.164:0/4147400643 >> v2:172.21.15.172:3300/0 conn(0x7f5e380ce780 0x7f5e380cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37400640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e380aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 -- 172.21.15.164:0/4147400643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 msgr2=0x7f5e380aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:32.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e380aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 -- 172.21.15.164:0/4147400643 >> v2:172.21.15.172:3300/0 conn(0x7f5e380ce780 msgr2=0x7f5e380cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:32.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 --2- 172.21.15.164:0/4147400643 >> v2:172.21.15.172:3300/0 conn(0x7f5e380ce780 0x7f5e380cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37e00640 1 -- 172.21.15.164:0/4147400643 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3815eb50 con 0x7f5e3815ab10 2024-05-08T04:40:32.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.449+0000 7f5e37400640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e380aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:32.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e37e00640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e3815ab10 0x7f5e3806cce0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e3806be30 tx=0x7f5e2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7d14a069b6d730 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e36a00640 1 -- 172.21.15.164:0/4147400643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280ab620 con 0x7f5e3815ab10 2024-05-08T04:40:32.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e36a00640 1 -- 172.21.15.164:0/4147400643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280b0070 con 0x7f5e3815ab10 2024-05-08T04:40:32.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e36a00640 1 -- 172.21.15.164:0/4147400643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280ab9e0 con 0x7f5e3815ab10 2024-05-08T04:40:32.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 -- 172.21.15.164:0/4147400643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e3815ab10 msgr2=0x7f5e3806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e3815ab10 0x7f5e3806cce0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e3806be30 tx=0x7f5e2801c460 comp rx=0 tx=0).stop 2024-05-08T04:40:32.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 -- 172.21.15.164:0/4147400643 shutdown_connections 2024-05-08T04:40:32.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/4147400643 >> v2:172.21.15.172:3300/0 conn(0x7f5e380ce780 0x7f5e380cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e380aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/4147400643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e3815ab10 0x7f5e3806cce0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 -- 172.21.15.164:0/4147400643 >> 172.21.15.164:0/4147400643 conn(0x7f5e380cf770 msgr2=0x7f5e380cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:32.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 -- 172.21.15.164:0/4147400643 shutdown_connections 2024-05-08T04:40:32.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.453+0000 7f5e3ec00640 1 -- 172.21.15.164:0/4147400643 wait complete. 2024-05-08T04:40:32.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 Processor -- start 2024-05-08T04:40:32.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- start start 2024-05-08T04:40:32.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e38152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 0x7f5e3815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c5b70 con 0x7f5e380ce780 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3815aef0 con 0x7f5e3815ab10 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c5cf0 con 0x7f5e380ae6e0 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41338/0 (socket says 172.21.15.164:41338) 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 -- 172.21.15.164:0/2273410191 learned_addr learned my addr 172.21.15.164:0/2273410191 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3cc00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 0x7f5e3815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37e00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e38152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 -- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 msgr2=0x7f5e38152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e38152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 -- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 msgr2=0x7f5e3815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:32.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 0x7f5e3815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 -- 172.21.15.164:0/2273410191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e381526b0 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3cc00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 0x7f5e3815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e37400640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c7c0 tx=0x7f5e2400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24007a30 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24007bd0 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e380c89f0 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240113f0 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.457+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e380c8fb0 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38072d30 con 0x7f5e380ce780 2024-05-08T04:40:32.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f5e24011590 con 0x7f5e380ce780 2024-05-08T04:40:32.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e2fe00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 0x7f5e18069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:32.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5e240853a0 con 0x7f5e380ce780 2024-05-08T04:40:32.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e37e00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 0x7f5e18069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:32.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.461+0000 7f5e37e00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 0x7f5e18069660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e280af000 tx=0x7f5e280a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:32.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.465+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e24015020 con 0x7f5e380ce780 2024-05-08T04:40:32.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:32.621+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e380c4bf0 con 0x7f5e380ce780 2024-05-08T04:40:33.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e2fe00640 1 -- 172.21.15.164:0/2273410191 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e24055c10 con 0x7f5e380ce780 2024-05-08T04:40:33.564 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 msgr2=0x7f5e18069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 0x7f5e18069660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e280af000 tx=0x7f5e280a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 msgr2=0x7f5e381500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c7c0 tx=0x7f5e2400cc80 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 shutdown_connections 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5e180671a0 0x7f5e18069660 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3815ab10 0x7f5e3815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e380ce780 0x7f5e381500f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 --2- 172.21.15.164:0/2273410191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e380ae6e0 0x7f5e38152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 >> 172.21.15.164:0/2273410191 conn(0x7f5e380cf770 msgr2=0x7f5e3806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 shutdown_connections 2024-05-08T04:40:33.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.561+0000 7f5e3ec00640 1 -- 172.21.15.164:0/2273410191 wait complete. 2024-05-08T04:40:33.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:40:33.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 Processor -- start 2024-05-08T04:40:33.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 -- start start 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f049415ba20 0x7f0494160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04940aedf0 0x7f049406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940c6600 con 0x7f049406de50 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049415fbe0 con 0x7f049415ba20 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f049a400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940c6040 con 0x7f04940aedf0 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f049415ba20 0x7f0494160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f049415ba20 0x7f0494160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41350/0 (socket says 172.21.15.164:41350) 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 -- 172.21.15.164:0/741676858 learned_addr learned my addr 172.21.15.164:0/741676858 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:33.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493e00640 1 --2- 172.21.15.164:0/741676858 >> v2:172.21.15.172:3300/0 conn(0x7f04940aedf0 0x7f049406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 -- 172.21.15.164:0/741676858 >> v2:172.21.15.172:3300/0 conn(0x7f04940aedf0 msgr2=0x7f049406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:33.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 --2- 172.21.15.164:0/741676858 >> v2:172.21.15.172:3300/0 conn(0x7f04940aedf0 0x7f049406e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 -- 172.21.15.164:0/741676858 >> v1:172.21.15.172:6789/0 conn(0x7f049406de50 legacy=0x7f04940ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:33.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.737+0000 7f0493400640 1 -- 172.21.15.164:0/741676858 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940c6320 con 0x7f049415ba20 2024-05-08T04:40:33.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f0493400640 1 --2- 172.21.15.164:0/741676858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049415ba20 0x7f0494160070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f049406bd80 tx=0x7f048401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1943b622f5083d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:33.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f0492000640 1 -- 172.21.15.164:0/741676858 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840ab650 con 0x7f049415ba20 2024-05-08T04:40:33.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f0492000640 1 -- 172.21.15.164:0/741676858 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840b0070 con 0x7f049415ba20 2024-05-08T04:40:33.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f0492000640 1 -- 172.21.15.164:0/741676858 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840aba10 con 0x7f049415ba20 2024-05-08T04:40:33.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- 172.21.15.164:0/741676858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049415ba20 msgr2=0x7f0494160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:33.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 --2- 172.21.15.164:0/741676858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049415ba20 0x7f0494160070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f049406bd80 tx=0x7f048401c4b0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- 172.21.15.164:0/741676858 shutdown_connections 2024-05-08T04:40:33.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 --2- 172.21.15.164:0/741676858 >> v2:172.21.15.172:3300/0 conn(0x7f04940aedf0 0x7f049406e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 --2- 172.21.15.164:0/741676858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049415ba20 0x7f0494160070 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- 172.21.15.164:0/741676858 >> 172.21.15.164:0/741676858 conn(0x7f04940cf770 msgr2=0x7f04940cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- 172.21.15.164:0/741676858 shutdown_connections 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- 172.21.15.164:0/741676858 wait complete. 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 Processor -- start 2024-05-08T04:40:33.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.741+0000 7f049a400640 1 -- start start 2024-05-08T04:40:33.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04940aedf0 0x7f04940c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f049415ba20 0x7f049414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940ce000 con 0x7f049406de50 2024-05-08T04:40:33.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cde80 con 0x7f049415ba20 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494168b70 con 0x7f04940aedf0 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41360/0 (socket says 172.21.15.164:41360) 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 -- 172.21.15.164:0/1498163379 learned_addr learned my addr 172.21.15.164:0/1498163379 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0492a00640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04940aedf0 0x7f04940c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493e00640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f049415ba20 0x7f049414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 -- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04940aedf0 msgr2=0x7f04940c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04940aedf0 0x7f04940c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 -- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f049415ba20 msgr2=0x7f049414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f049415ba20 0x7f049414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 -- 172.21.15.164:0/1498163379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940c9010 con 0x7f049406de50 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f0493400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad040 tx=0x7f048401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840be5f0 con 0x7f049406de50 2024-05-08T04:40:33.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840be7b0 con 0x7f049406de50 2024-05-08T04:40:33.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940c92f0 con 0x7f049406de50 2024-05-08T04:40:33.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840beba0 con 0x7f049406de50 2024-05-08T04:40:33.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940c98b0 con 0x7f049406de50 2024-05-08T04:40:33.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04940c9af0 con 0x7f049406de50 2024-05-08T04:40:33.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.745+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f04840b43f0 con 0x7f049406de50 2024-05-08T04:40:33.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.749+0000 7f048b400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 0x7f0474069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:33.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.749+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f04841201f0 con 0x7f049406de50 2024-05-08T04:40:33.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.749+0000 7f0492a00640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 0x7f0474069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:33.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.749+0000 7f0492a00640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 0x7f0474069690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04940b4f00 tx=0x7f0480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:33.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.753+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04940c9af0 con 0x7f049406de50 2024-05-08T04:40:33.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:33.909+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0494072d30 con 0x7f049406de50 2024-05-08T04:40:34.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.561+0000 7f048b400640 1 -- 172.21.15.164:0/1498163379 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f04840c4060 con 0x7f049406de50 2024-05-08T04:40:34.568 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:40:34.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 msgr2=0x7f0474069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:34.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 0x7f0474069690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04940b4f00 tx=0x7f0480009040 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 msgr2=0x7f04940c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad040 tx=0x7f048401acf0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 shutdown_connections 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f04740671d0 0x7f0474069690 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04940aedf0 0x7f04940c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f049415ba20 0x7f049414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 --2- 172.21.15.164:0/1498163379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f049406de50 0x7f04940c6840 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 >> 172.21.15.164:0/1498163379 conn(0x7f04940cf770 msgr2=0x7f049415e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:34.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 shutdown_connections 2024-05-08T04:40:34.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.565+0000 7f049a400640 1 -- 172.21.15.164:0/1498163379 wait complete. 2024-05-08T04:40:34.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:40:34.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.749+0000 7f7566a00640 1 Processor -- start 2024-05-08T04:40:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.749+0000 7f7566a00640 1 -- start start 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f756015ab10 0x7f756006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75600ae6e0 0x7f75600aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ce780 0x7f75600cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600cf130 con 0x7f756015ab10 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756006bb80 con 0x7f75600ae6e0 2024-05-08T04:40:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7566a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756015e870 con 0x7f75600ce780 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f756015ab10 0x7f756006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f756015ab10 0x7f756006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41364/0 (socket says 172.21.15.164:41364) 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 -- 172.21.15.164:0/351061324 learned_addr learned my addr 172.21.15.164:0/351061324 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7564a00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ce780 0x7f75600cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f7557400640 1 --2- 172.21.15.164:0/351061324 >> v2:172.21.15.172:3300/0 conn(0x7f75600ae6e0 0x7f75600aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 -- 172.21.15.164:0/351061324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ce780 msgr2=0x7f75600cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ce780 0x7f75600cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 -- 172.21.15.164:0/351061324 >> v2:172.21.15.172:3300/0 conn(0x7f75600ae6e0 msgr2=0x7f75600aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:34.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 --2- 172.21.15.164:0/351061324 >> v2:172.21.15.172:3300/0 conn(0x7f75600ae6e0 0x7f75600aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.753+0000 7f755fe00640 1 -- 172.21.15.164:0/351061324 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756015eb50 con 0x7f756015ab10 2024-05-08T04:40:34.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f755fe00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756015ab10 0x7f756006cce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f75600cf3c0 tx=0x7f755001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf76d03c44ea8c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:34.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f755f400640 1 -- 172.21.15.164:0/351061324 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500ab620 con 0x7f756015ab10 2024-05-08T04:40:34.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f755f400640 1 -- 172.21.15.164:0/351061324 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75500b0070 con 0x7f756015ab10 2024-05-08T04:40:34.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f755f400640 1 -- 172.21.15.164:0/351061324 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500ab9e0 con 0x7f756015ab10 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- 172.21.15.164:0/351061324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756015ab10 msgr2=0x7f756006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756015ab10 0x7f756006cce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f75600cf3c0 tx=0x7f755001c460 comp rx=0 tx=0).stop 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- 172.21.15.164:0/351061324 shutdown_connections 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ce780 0x7f75600cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 --2- 172.21.15.164:0/351061324 >> v2:172.21.15.172:3300/0 conn(0x7f75600ae6e0 0x7f75600aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 --2- 172.21.15.164:0/351061324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756015ab10 0x7f756006cce0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- 172.21.15.164:0/351061324 >> 172.21.15.164:0/351061324 conn(0x7f75600cf770 msgr2=0x7f75600cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- 172.21.15.164:0/351061324 shutdown_connections 2024-05-08T04:40:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- 172.21.15.164:0/351061324 wait complete. 2024-05-08T04:40:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 Processor -- start 2024-05-08T04:40:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.757+0000 7f7566a00640 1 -- start start 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ae6e0 0x7f756015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 0x7f75600c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600c5b70 con 0x7f75600ce780 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756015aef0 con 0x7f756015ab10 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600c5cf0 con 0x7f75600ae6e0 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41370/0 (socket says 172.21.15.164:41370) 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 -- 172.21.15.164:0/2213598342 learned_addr learned my addr 172.21.15.164:0/2213598342 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7564a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 0x7f75600c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f755fe00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ae6e0 0x7f756015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 -- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ae6e0 msgr2=0x7f756015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ae6e0 0x7f756015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 -- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 msgr2=0x7f75600c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 0x7f75600c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 -- 172.21.15.164:0/2213598342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756015c600 con 0x7f75600ce780 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7564a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 0x7f75600c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7557400640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f754800e930 tx=0x7f754800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548017070 con 0x7f75600ce780 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800ca00 con 0x7f75600ce780 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756015c880 con 0x7f75600ce780 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800cdb0 con 0x7f75600ce780 2024-05-08T04:40:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756015d120 con 0x7f75600ce780 2024-05-08T04:40:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.761+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7560072d30 con 0x7f75600ce780 2024-05-08T04:40:34.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.765+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f754801a070 con 0x7f75600ce780 2024-05-08T04:40:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.765+0000 7f755cc00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 0x7f753c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.765+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f7548085bd0 con 0x7f75600ce780 2024-05-08T04:40:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.765+0000 7f755fe00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 0x7f753c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:34.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.765+0000 7f755fe00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 0x7f753c069660 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75500af000 tx=0x7f75500a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:34.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.769+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f756015c600 con 0x7f75600ce780 2024-05-08T04:40:34.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:34.953+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f756004f010 con 0x7f75600ce780 2024-05-08T04:40:35.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.585+0000 7f755cc00640 1 -- 172.21.15.164:0/2213598342 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7548056310 con 0x7f75600ce780 2024-05-08T04:40:35.596 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 8 and data pool 9 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 msgr2=0x7f753c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 0x7f753c069660 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75500af000 tx=0x7f75500a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 msgr2=0x7f75600c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f754800e930 tx=0x7f754800edf0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 shutdown_connections 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f753c0671a0 0x7f753c069660 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f756015ab10 0x7f75600c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75600ce780 0x7f75600c6840 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 --2- 172.21.15.164:0/2213598342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75600ae6e0 0x7f756015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 >> 172.21.15.164:0/2213598342 conn(0x7f75600cf770 msgr2=0x7f756006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:35.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 shutdown_connections 2024-05-08T04:40:35.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.597+0000 7f7566a00640 1 -- 172.21.15.164:0/2213598342 wait complete. 2024-05-08T04:40:35.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:35.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 Processor -- start 2024-05-08T04:40:35.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 -- start start 2024-05-08T04:40:35.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa75c0cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0ccac0 con 0x7fa75c0b2b80 2024-05-08T04:40:35.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0ccda0 con 0x7fa75c0d0120 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa75b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0cdb60 con 0x7fa75c0cc470 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.769+0000 7fa759e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa75c0b2b80 0x7fa75c0cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45896/0 (socket says 172.21.15.164:45896) 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 -- 172.21.15.164:0/4091761967 learned_addr learned my addr 172.21.15.164:0/4091761967 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa751400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa75c0cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94608772 0 0) 0x7fa75c0ccac0 con 0x7fa75c0b2b80 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa740003530 con 0x7fa75c0b2b80 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751481820 0 0) 0x7fa75c0ccda0 con 0x7fa75c0d0120 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c0ccac0 con 0x7fa75c0d0120 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445182646 0 0) 0x7fa740003530 con 0x7fa75c0b2b80 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 msgr2=0x7fa75c0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa75c0cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 >> v1:172.21.15.172:6789/0 conn(0x7fa75c0d0120 legacy=0x7fa75c0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0cd080 con 0x7fa75c0b2b80 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa74c001a00 con 0x7fa75c0b2b80 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 >> v1:172.21.15.164:6789/0 conn(0x7fa75c0b2b80 legacy=0x7fa75c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740003b80 0x7fa740003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa740004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740004d50 0x7fa7400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0cdb60 con 0x7fa740003b80 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74000ba80 con 0x7fa740004d50 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74000bc30 con 0x7fa75c0cc470 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740003b80 0x7fa740003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa751400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa740004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa75a800640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740004d50 0x7fa7400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 -- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 msgr2=0x7fa740004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa740004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 -- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740004d50 msgr2=0x7fa7400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740004d50 0x7fa7400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa759e00640 1 -- 172.21.15.164:0/4091761967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740003530 con 0x7fa740003b80 2024-05-08T04:40:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.773+0000 7fa751400640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0cc470 0x7fa740004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa759e00640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740003b80 0x7fa740003f80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00a800 tx=0x7fa74c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0deb7ff188f067 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00a430 con 0x7fa740003b80 2024-05-08T04:40:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0ad650 con 0x7fa740003b80 2024-05-08T04:40:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa759400640 1 -- 172.21.15.164:0/4091761967 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0ada40 con 0x7fa740003b80 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- 172.21.15.164:0/4091761967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740003b80 msgr2=0x7fa740003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 --2- 172.21.15.164:0/4091761967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740003b80 0x7fa740003f80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00a800 tx=0x7fa74c01d320 comp rx=0 tx=0).stop 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75a800640 1 -- 172.21.15.164:0/4091761967 reap_dead start 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- 172.21.15.164:0/4091761967 shutdown_connections 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- 172.21.15.164:0/4091761967 >> 172.21.15.164:0/4091761967 conn(0x7fa75c09aa20 msgr2=0x7fa75c0d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- 172.21.15.164:0/4091761967 shutdown_connections 2024-05-08T04:40:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- 172.21.15.164:0/4091761967 wait complete. 2024-05-08T04:40:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 Processor -- start 2024-05-08T04:40:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.777+0000 7fa75b200640 1 -- start start 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740003b80 0x7fa75c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0ae4e0 0x7fa75c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0d0530 con 0x7fa75c0aeea0 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0cc850 con 0x7fa740003b80 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0b1f50 con 0x7fa75c0ae4e0 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41390/0 (socket says 172.21.15.164:41390) 2024-05-08T04:40:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 -- 172.21.15.164:0/281708332 learned_addr learned my addr 172.21.15.164:0/281708332 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa751400640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0ae4e0 0x7fa75c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa759e00640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740003b80 0x7fa75c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 -- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0ae4e0 msgr2=0x7fa75c0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0ae4e0 0x7fa75c0ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 -- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740003b80 msgr2=0x7fa75c155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740003b80 0x7fa75c155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 -- 172.21.15.164:0/281708332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0af320 con 0x7fa75c0aeea0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75a800640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa75c072040 tx=0x7fa75400fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754019070 con 0x7fa75c0aeea0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75400fe80 con 0x7fa75c0aeea0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c0c5530 con 0x7fa75c0aeea0 2024-05-08T04:40:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7440096b0 con 0x7fa75c0aeea0 2024-05-08T04:40:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c0c5af0 con 0x7fa75c0aeea0 2024-05-08T04:40:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c04ef30 con 0x7fa75c0aeea0 2024-05-08T04:40:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.781+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fa75401c070 con 0x7fa75c0aeea0 2024-05-08T04:40:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.785+0000 7fa752a00640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 0x7fa738069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.785+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa7540840f0 con 0x7fa75c0aeea0 2024-05-08T04:40:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.785+0000 7fa759e00640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 0x7fa738069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.785+0000 7fa759e00640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 0x7fa738069690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c001dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:35.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.789+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa754054720 con 0x7fa75c0aeea0 2024-05-08T04:40:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.941+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa75c162990 con 0x7fa75c0aeea0 2024-05-08T04:40:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.941+0000 7fa752a00640 1 -- 172.21.15.164:0/281708332 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16781 (secure 0 0 0) 0x7fa7540587c0 con 0x7fa75c0aeea0 2024-05-08T04:40:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:35.948 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":59,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:34.959911+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 msgr2=0x7fa738069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 0x7fa738069690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c001dc0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 msgr2=0x7fa75c0c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa75c072040 tx=0x7fa75400fb40 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 shutdown_connections 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7380671d0 0x7fa738069690 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa75c0ae4e0 0x7fa75c0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa75c0aeea0 0x7fa75c0c62f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 --2- 172.21.15.164:0/281708332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa740003b80 0x7fa75c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 >> 172.21.15.164:0/281708332 conn(0x7fa75c09aa20 msgr2=0x7fa75c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 shutdown_connections 2024-05-08T04:40:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:35.945+0000 7fa75b200640 1 -- 172.21.15.164:0/281708332 wait complete. 2024-05-08T04:40:35.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:36.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 Processor -- start 2024-05-08T04:40:36.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 -- start start 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa0280cfda0 0x7fa0280cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02806c770 con 0x7fa02815ab60 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02806ca50 con 0x7fa0280b26c0 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02806cd30 con 0x7fa0280cfda0 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa0280cfda0 0x7fa0280cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa0280cfda0 0x7fa0280cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41404/0 (socket says 172.21.15.164:41404) 2024-05-08T04:40:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 -- 172.21.15.164:0/3108814496 learned_addr learned my addr 172.21.15.164:0/3108814496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02d400640 1 --1- 172.21.15.164:0/3108814496 >> v1:172.21.15.164:6789/0 conn(0x7fa02815ab60 0x7fa028072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45910/0 (socket says 172.21.15.164:45910) 2024-05-08T04:40:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 -- 172.21.15.164:0/3108814496 >> v1:172.21.15.172:6789/0 conn(0x7fa0280b26c0 legacy=0x7fa0280cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 -- 172.21.15.164:0/3108814496 >> v1:172.21.15.164:6789/0 conn(0x7fa02815ab60 legacy=0x7fa028072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.101+0000 7fa02de00640 1 -- 172.21.15.164:0/3108814496 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02806d010 con 0x7fa0280cfda0 2024-05-08T04:40:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02de00640 1 --2- 172.21.15.164:0/3108814496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280cfda0 0x7fa0280cd910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa020009e50 tx=0x7fa02001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587e3141a7b06473 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa027e00640 1 -- 172.21.15.164:0/3108814496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200a5700 con 0x7fa0280cfda0 2024-05-08T04:40:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa027e00640 1 -- 172.21.15.164:0/3108814496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0200a58a0 con 0x7fa0280cfda0 2024-05-08T04:40:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa027e00640 1 -- 172.21.15.164:0/3108814496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200a5c70 con 0x7fa0280cfda0 2024-05-08T04:40:36.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- 172.21.15.164:0/3108814496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280cfda0 msgr2=0x7fa0280cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 --2- 172.21.15.164:0/3108814496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280cfda0 0x7fa0280cd910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa020009e50 tx=0x7fa02001ca30 comp rx=0 tx=0).stop 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- 172.21.15.164:0/3108814496 shutdown_connections 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 --2- 172.21.15.164:0/3108814496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280cfda0 0x7fa0280cd910 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- 172.21.15.164:0/3108814496 >> 172.21.15.164:0/3108814496 conn(0x7fa02806ac90 msgr2=0x7fa02806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- 172.21.15.164:0/3108814496 shutdown_connections 2024-05-08T04:40:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- 172.21.15.164:0/3108814496 wait complete. 2024-05-08T04:40:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 Processor -- start 2024-05-08T04:40:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.105+0000 7fa02fe00640 1 -- start start 2024-05-08T04:40:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0280cfda0 0x7fa02813b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa02815ab60 0x7fa0280b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028168b00 con 0x7fa0280b26c0 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028168980 con 0x7fa0280cfda0 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa028168c80 con 0x7fa02815ab60 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41412/0 (socket says 172.21.15.164:41412) 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 -- 172.21.15.164:0/4020623149 learned_addr learned my addr 172.21.15.164:0/4020623149 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02de00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa02815ab60 0x7fa0280b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02ca00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0280cfda0 0x7fa02813b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 -- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa02815ab60 msgr2=0x7fa0280b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa02815ab60 0x7fa0280b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 -- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0280cfda0 msgr2=0x7fa02813b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0280cfda0 0x7fa02813b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 -- 172.21.15.164:0/4020623149 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280b3110 con 0x7fa0280b26c0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02d400640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00c430 tx=0x7fa01c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0074e0 con 0x7fa0280b26c0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c007680 con 0x7fa0280b26c0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c004a10 con 0x7fa0280b26c0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280b33f0 con 0x7fa0280b26c0 2024-05-08T04:40:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280b39b0 con 0x7fa0280b26c0 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.109+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa028069d60 con 0x7fa0280b26c0 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.113+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fa01c01a020 con 0x7fa0280b26c0 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.113+0000 7fa025600640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 0x7fa008069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.113+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa01c084bb0 con 0x7fa0280b26c0 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.113+0000 7fa02ca00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 0x7fa008069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.113+0000 7fa02ca00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 0x7fa008069660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa0281547d0 tx=0x7fa014009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.117+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa01c055150 con 0x7fa0280b26c0 2024-05-08T04:40:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.273+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa028068d20 con 0x7fa0280b26c0 2024-05-08T04:40:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.273+0000 7fa025600640 1 -- 172.21.15.164:0/4020623149 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16781 (secure 0 0 0) 0x7fa01c0591f0 con 0x7fa0280b26c0 2024-05-08T04:40:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:36.279 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":59,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:34.959911+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 msgr2=0x7fa008069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 0x7fa008069660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa0281547d0 tx=0x7fa014009040 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 msgr2=0x7fa0280b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00c430 tx=0x7fa01c00c8f0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 shutdown_connections 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0080671a0 0x7fa008069660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa02815ab60 0x7fa0280b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0280cfda0 0x7fa02813b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 --2- 172.21.15.164:0/4020623149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0280b26c0 0x7fa0280b41b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 >> 172.21.15.164:0/4020623149 conn(0x7fa02806ac90 msgr2=0x7fa0281577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 shutdown_connections 2024-05-08T04:40:36.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.277+0000 7fa02fe00640 1 -- 172.21.15.164:0/4020623149 wait complete. 2024-05-08T04:40:36.301 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:40:36.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 Processor -- start 2024-05-08T04:40:36.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 -- start start 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 0x7fb9a40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9a40b2fb0 0x7fb9a40cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a40b2a80 con 0x7fb9a40b2510 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a40cce70 con 0x7fb9a40b2fb0 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9aa000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a40cd0c0 con 0x7fb9a415f200 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb9a40b2510 0x7fb9a415fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45924/0 (socket says 172.21.15.164:45924) 2024-05-08T04:40:36.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3e00640 1 -- 172.21.15.164:0/3739871802 learned_addr learned my addr 172.21.15.164:0/3739871802 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3739871802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 0x7fb9a40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a2a00640 1 --2- 172.21.15.164:0/3739871802 >> v2:172.21.15.172:3300/0 conn(0x7fb9a40b2fb0 0x7fb9a40cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984490478 0 0) 0x7fb9a40b2a80 con 0x7fb9a40b2510 2024-05-08T04:40:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb988003530 con 0x7fb9a40b2510 2024-05-08T04:40:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3400640 1 -- 172.21.15.164:0/3739871802 >> v2:172.21.15.172:3300/0 conn(0x7fb9a40b2fb0 msgr2=0x7fb9a40cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3739871802 >> v2:172.21.15.172:3300/0 conn(0x7fb9a40b2fb0 0x7fb9a40cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198858865 0 0) 0x7fb988003530 con 0x7fb9a40b2510 2024-05-08T04:40:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3400640 1 -- 172.21.15.164:0/3739871802 >> v1:172.21.15.164:6789/0 conn(0x7fb9a40b2510 legacy=0x7fb9a415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.437+0000 7fb9a3400640 1 -- 172.21.15.164:0/3739871802 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a40cd3a0 con 0x7fb9a415f200 2024-05-08T04:40:36.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3739871802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 0x7fb9a40cf9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb994009d90 tx=0x7fb99401c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5601d78fcdd85e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940b2070 con 0x7fb9a415f200 2024-05-08T04:40:36.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9940a55f0 con 0x7fb9a415f200 2024-05-08T04:40:36.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9a2000640 1 -- 172.21.15.164:0/3739871802 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c001680 con 0x7fb9a415f200 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 -- 172.21.15.164:0/3739871802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 msgr2=0x7fb9a40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3739871802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 0x7fb9a40cf9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb994009d90 tx=0x7fb99401c7d0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 -- 172.21.15.164:0/3739871802 shutdown_connections 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3739871802 >> v2:172.21.15.172:3300/0 conn(0x7fb9a40b2fb0 0x7fb9a40cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3739871802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a415f200 0x7fb9a40cf9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 -- 172.21.15.164:0/3739871802 >> 172.21.15.164:0/3739871802 conn(0x7fb9a40d1980 msgr2=0x7fb9a40d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 -- 172.21.15.164:0/3739871802 shutdown_connections 2024-05-08T04:40:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.441+0000 7fb9aa000640 1 -- 172.21.15.164:0/3739871802 wait complete. 2024-05-08T04:40:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 Processor -- start 2024-05-08T04:40:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- start start 2024-05-08T04:40:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a40b2fb0 0x7fb9a4151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9a415f200 0x7fb9a4153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4168d40 con 0x7fb9a40b2510 2024-05-08T04:40:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a415f7f0 con 0x7fb9a415f200 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4168ec0 con 0x7fb9a40b2fb0 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41420/0 (socket says 172.21.15.164:41420) 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 -- 172.21.15.164:0/3546598345 learned_addr learned my addr 172.21.15.164:0/3546598345 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a2a00640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a40b2fb0 0x7fb9a4151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3e00640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9a415f200 0x7fb9a4153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 -- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a40b2fb0 msgr2=0x7fb9a4151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a40b2fb0 0x7fb9a4151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 -- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9a415f200 msgr2=0x7fb9a4153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9a415f200 0x7fb9a4153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 -- 172.21.15.164:0/3546598345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a41f71b0 con 0x7fb9a40b2510 2024-05-08T04:40:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9a3400640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb99401ae80 tx=0x7fb994019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940bd020 con 0x7fb9a40b2510 2024-05-08T04:40:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9940b8670 con 0x7fb9a40b2510 2024-05-08T04:40:36.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940b2040 con 0x7fb9a40b2510 2024-05-08T04:40:36.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a41f7490 con 0x7fb9a40b2510 2024-05-08T04:40:36.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.445+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a414fc20 con 0x7fb9a40b2510 2024-05-08T04:40:36.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a409fa50 con 0x7fb9a40b2510 2024-05-08T04:40:36.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fb9940c0020 con 0x7fb9a40b2510 2024-05-08T04:40:36.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb99b400640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 0x7fb984069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:36.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb994120230 con 0x7fb9a40b2510 2024-05-08T04:40:36.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb9a2a00640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 0x7fb984069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:36.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.449+0000 7fb9a2a00640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 0x7fb984069660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9a406d990 tx=0x7fb990009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:36.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.453+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9940c7030 con 0x7fb9a40b2510 2024-05-08T04:40:36.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-05-08T04:40:36.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-05-08T04:40:36.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-05-08T04:40:36.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:36.645+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9a4154840 con 0x7fb9a40b2510 2024-05-08T04:40:37.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.597+0000 7fb99b400640 1 -- 172.21.15.164:0/3546598345 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb994016480 con 0x7fb9a40b2510 2024-05-08T04:40:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 msgr2=0x7fb984069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 0x7fb984069660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9a406d990 tx=0x7fb990009190 comp rx=0 tx=0).stop 2024-05-08T04:40:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 msgr2=0x7fb9a41f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb99401ae80 tx=0x7fb994019570 comp rx=0 tx=0).stop 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 shutdown_connections 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb9840671a0 0x7fb984069660 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9a40b2fb0 0x7fb9a4151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9a415f200 0x7fb9a4153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 --2- 172.21.15.164:0/3546598345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40b2510 0x7fb9a41f6b80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 >> 172.21.15.164:0/3546598345 conn(0x7fb9a40d1980 msgr2=0x7fb9a40b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 shutdown_connections 2024-05-08T04:40:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.601+0000 7fb9aa000640 1 -- 172.21.15.164:0/3546598345 wait complete. 2024-05-08T04:40:37.625 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:40:37.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:40:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 Processor -- start 2024-05-08T04:40:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 -- start start 2024-05-08T04:40:37.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f21cc15ce10 0x7f21cc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 0x7f21cc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0c6600 con 0x7f21cc15ba20 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc15fbe0 con 0x7f21cc15ce10 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.761+0000 7f21d3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0c6040 con 0x7f21cc06e390 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f21cc15ba20 0x7f21cc160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:47488/0 (socket says 172.21.15.164:47488) 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1000640 1 -- 172.21.15.164:0/3097433453 learned_addr learned my addr 172.21.15.164:0/3097433453 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21cbe00640 1 --2- 172.21.15.164:0/3097433453 >> v2:172.21.15.172:3300/0 conn(0x7f21cc15ce10 0x7f21cc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1a00640 1 --2- 172.21.15.164:0/3097433453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 0x7f21cc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1a00640 1 -- 172.21.15.164:0/3097433453 >> v2:172.21.15.172:3300/0 conn(0x7f21cc15ce10 msgr2=0x7f21cc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1a00640 1 --2- 172.21.15.164:0/3097433453 >> v2:172.21.15.172:3300/0 conn(0x7f21cc15ce10 0x7f21cc06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1a00640 1 -- 172.21.15.164:0/3097433453 >> v1:172.21.15.172:6789/0 conn(0x7f21cc15ba20 legacy=0x7f21cc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.765+0000 7f21d1a00640 1 -- 172.21.15.164:0/3097433453 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc0c6320 con 0x7f21cc06e390 2024-05-08T04:40:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d1a00640 1 --2- 172.21.15.164:0/3097433453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 0x7f21cc0ae6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21c40097c0 tx=0x7f21c401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446e7c0f516cd620 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21cb400640 1 -- 172.21.15.164:0/3097433453 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40b0070 con 0x7f21cc06e390 2024-05-08T04:40:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21cb400640 1 -- 172.21.15.164:0/3097433453 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40ab600 con 0x7f21cc06e390 2024-05-08T04:40:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21cb400640 1 -- 172.21.15.164:0/3097433453 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40ab9d0 con 0x7f21cc06e390 2024-05-08T04:40:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- 172.21.15.164:0/3097433453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 msgr2=0x7f21cc0ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 --2- 172.21.15.164:0/3097433453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 0x7f21cc0ae6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21c40097c0 tx=0x7f21c401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- 172.21.15.164:0/3097433453 shutdown_connections 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 --2- 172.21.15.164:0/3097433453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc06e390 0x7f21cc0ae6e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 --2- 172.21.15.164:0/3097433453 >> v2:172.21.15.172:3300/0 conn(0x7f21cc15ce10 0x7f21cc06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- 172.21.15.164:0/3097433453 >> 172.21.15.164:0/3097433453 conn(0x7f21cc0cf770 msgr2=0x7f21cc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- 172.21.15.164:0/3097433453 shutdown_connections 2024-05-08T04:40:37.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- 172.21.15.164:0/3097433453 wait complete. 2024-05-08T04:40:37.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 Processor -- start 2024-05-08T04:40:37.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.769+0000 7f21d3a00640 1 -- start start 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc06e390 0x7f21cc147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc15ba20 0x7f21cc0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc168b70 con 0x7f21cc15ce10 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0ce090 con 0x7f21cc06e390 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc168cf0 con 0x7f21cc15ba20 2024-05-08T04:40:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41428/0 (socket says 172.21.15.164:41428) 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 -- 172.21.15.164:0/1975858401 learned_addr learned my addr 172.21.15.164:0/1975858401 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1000640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc06e390 0x7f21cc147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21cbe00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc15ba20 0x7f21cc0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 -- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc15ba20 msgr2=0x7f21cc0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc15ba20 0x7f21cc0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 -- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc06e390 msgr2=0x7f21cc147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc06e390 0x7f21cc147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 -- 172.21.15.164:0/1975858401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc1509d0 con 0x7f21cc15ce10 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d1a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f21c401a0d0 tx=0x7f21c4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40abe60 con 0x7f21cc15ce10 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40b9070 con 0x7f21cc15ce10 2024-05-08T04:40:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc1524d0 con 0x7f21cc15ce10 2024-05-08T04:40:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40b0040 con 0x7f21cc15ce10 2024-05-08T04:40:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc14fc30 con 0x7f21cc15ce10 2024-05-08T04:40:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc072d30 con 0x7f21cc15ce10 2024-05-08T04:40:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.773+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f21c40b4650 con 0x7f21cc15ce10 2024-05-08T04:40:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.777+0000 7f21c8c00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 0x7f21ac069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.777+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f21c411efe0 con 0x7f21cc15ce10 2024-05-08T04:40:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.777+0000 7f21d1000640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 0x7f21ac069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.777+0000 7f21d1000640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 0x7f21ac069660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f21cc138d60 tx=0x7f21bc001a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:37.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.781+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21c4016090 con 0x7f21cc15ce10 2024-05-08T04:40:37.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:37.965+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f21cc04f010 con 0x7f21cc15ce10 2024-05-08T04:40:38.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.605+0000 7f21c8c00640 1 -- 172.21.15.164:0/1975858401 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21c40ef560 con 0x7f21cc15ce10 2024-05-08T04:40:38.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 msgr2=0x7f21ac069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 0x7f21ac069660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f21cc138d60 tx=0x7f21bc001a40 comp rx=0 tx=0).stop 2024-05-08T04:40:38.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 msgr2=0x7f21cc0c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f21c401a0d0 tx=0x7f21c4019240 comp rx=0 tx=0).stop 2024-05-08T04:40:38.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 shutdown_connections 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21ac0671a0 0x7f21ac069660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21cc15ba20 0x7f21cc0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21cc15ce10 0x7f21cc0c6d80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 --2- 172.21.15.164:0/1975858401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc06e390 0x7f21cc147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 >> 172.21.15.164:0/1975858401 conn(0x7f21cc0cf770 msgr2=0x7f21cc0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 shutdown_connections 2024-05-08T04:40:38.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.609+0000 7f21d3a00640 1 -- 172.21.15.164:0/1975858401 wait complete. 2024-05-08T04:40:38.634 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:40:38.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:38.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 Processor -- start 2024-05-08T04:40:38.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 -- start start 2024-05-08T04:40:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 0x7f4b500cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4b50163540 0x7f4b500ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4b500cf6c0 0x7f4b500cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500ceac0 con 0x7f4b50163540 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500cc1e0 con 0x7f4b500cf6c0 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.769+0000 7f4b55200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500cc4c0 con 0x7f4b500aedb0 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4b50163540 0x7f4b500ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4b50163540 0x7f4b500ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41442/0 (socket says 172.21.15.164:41442) 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 -- 172.21.15.164:0/826256217 learned_addr learned my addr 172.21.15.164:0/826256217 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4de00640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 0x7f4b500cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4e800640 1 --2- 172.21.15.164:0/826256217 >> v2:172.21.15.172:3300/0 conn(0x7f4b500cf6c0 0x7f4b500cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 -- 172.21.15.164:0/826256217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 msgr2=0x7f4b500cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 0x7f4b500cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 -- 172.21.15.164:0/826256217 >> v2:172.21.15.172:3300/0 conn(0x7f4b500cf6c0 msgr2=0x7f4b500cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 --2- 172.21.15.164:0/826256217 >> v2:172.21.15.172:3300/0 conn(0x7f4b500cf6c0 0x7f4b500cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 -- 172.21.15.164:0/826256217 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500cc7a0 con 0x7f4b50163540 2024-05-08T04:40:38.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4de00640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 0x7f4b500cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b45400640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b500ce150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006e860 tx=0x7f4b3801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11d9ce119d9e307 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4d400640 1 -- 172.21.15.164:0/826256217 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380b2070 con 0x7f4b50163540 2024-05-08T04:40:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4d400640 1 -- 172.21.15.164:0/826256217 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b380a9070 con 0x7f4b50163540 2024-05-08T04:40:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.773+0000 7f4b4d400640 1 -- 172.21.15.164:0/826256217 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380ad880 con 0x7f4b50163540 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- 172.21.15.164:0/826256217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 msgr2=0x7f4b500ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b500ce150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006e860 tx=0x7f4b3801c490 comp rx=0 tx=0).stop 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- 172.21.15.164:0/826256217 shutdown_connections 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- 172.21.15.164:0/826256217 >> v2:172.21.15.172:3300/0 conn(0x7f4b500cf6c0 0x7f4b500cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b500ce150 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- 172.21.15.164:0/826256217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500aedb0 0x7f4b500cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- 172.21.15.164:0/826256217 >> 172.21.15.164:0/826256217 conn(0x7f4b5006ac90 msgr2=0x7f4b5006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- 172.21.15.164:0/826256217 shutdown_connections 2024-05-08T04:40:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- 172.21.15.164:0/826256217 wait complete. 2024-05-08T04:40:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 Processor -- start 2024-05-08T04:40:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- start start 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 0x7f4b5015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500cf6c0 0x7f4b5015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5006db80 con 0x7f4b50163540 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5006da00 con 0x7f4b500aedb0 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b55200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5006dd00 con 0x7f4b500cf6c0 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41458/0 (socket says 172.21.15.164:41458) 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 -- 172.21.15.164:0/129247033 learned_addr learned my addr 172.21.15.164:0/129247033 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b45400640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500cf6c0 0x7f4b5015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4de00640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 0x7f4b5015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 -- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500cf6c0 msgr2=0x7f4b5015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500cf6c0 0x7f4b5015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 -- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 msgr2=0x7f4b5015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 0x7f4b5015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 -- 172.21.15.164:0/129247033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50163dc0 con 0x7f4b50163540 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4de00640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 0x7f4b5015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.777+0000 7f4b4e800640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d560 tx=0x7f4b4800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480125f0 con 0x7f4b50163540 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48019070 con 0x7f4b50163540 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480129f0 con 0x7f4b50163540 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b50164060 con 0x7f4b50163540 2024-05-08T04:40:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50164620 con 0x7f4b50163540 2024-05-08T04:40:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b50069d60 con 0x7f4b50163540 2024-05-08T04:40:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f4b48012b90 con 0x7f4b50163540 2024-05-08T04:40:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 0x7f4b2c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4b48085890 con 0x7f4b50163540 2024-05-08T04:40:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.781+0000 7f4b4de00640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 0x7f4b2c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:38.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.785+0000 7f4b4de00640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 0x7f4b2c069660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e490 tx=0x7f4b400075f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:38.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.785+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b48056480 con 0x7f4b50163540 2024-05-08T04:40:38.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b5006c870 con 0x7f4b50163540 2024-05-08T04:40:38.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b46a00640 1 -- 172.21.15.164:0/129247033 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f4b48055d80 con 0x7f4b50163540 2024-05-08T04:40:38.977 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:38.977 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:38.977 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 34 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 msgr2=0x7f4b2c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 0x7f4b2c069660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e490 tx=0x7f4b400075f0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 msgr2=0x7f4b5015c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d560 tx=0x7f4b4800da20 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 shutdown_connections 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4b2c0671a0 0x7f4b2c069660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b500cf6c0 0x7f4b5015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b50163540 0x7f4b5015c690 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 --2- 172.21.15.164:0/129247033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b500aedb0 0x7f4b5015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 >> 172.21.15.164:0/129247033 conn(0x7f4b5006ac90 msgr2=0x7f4b501531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:38.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 shutdown_connections 2024-05-08T04:40:38.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:38.973+0000 7f4b55200640 1 -- 172.21.15.164:0/129247033 wait complete. 2024-05-08T04:40:38.998 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:39.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 Processor -- start 2024-05-08T04:40:39.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 -- start start 2024-05-08T04:40:39.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f236c15bc50 0x7f236c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 0x7f236c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0b42a0 con 0x7f236c0b4d50 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cd4e0 con 0x7f236c15bc50 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2371200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0cd7c0 con 0x7f236c0ccfe0 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f236a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f236c0b4d50 0x7f236c0ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45934/0 (socket says 172.21.15.164:45934) 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f236a800640 1 -- 172.21.15.164:0/4283700303 learned_addr learned my addr 172.21.15.164:0/4283700303 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2361400640 1 --2- 172.21.15.164:0/4283700303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 0x7f236c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2369e00640 1 --2- 172.21.15.164:0/4283700303 >> v2:172.21.15.172:3300/0 conn(0x7f236c15bc50 0x7f236c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2369400640 1 -- 172.21.15.164:0/4283700303 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929089183 0 0) 0x7f236c0b42a0 con 0x7f236c0b4d50 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2369400640 1 -- 172.21.15.164:0/4283700303 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003530 con 0x7f236c0b4d50 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2361400640 1 -- 172.21.15.164:0/4283700303 >> v2:172.21.15.172:3300/0 conn(0x7f236c15bc50 msgr2=0x7f236c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2361400640 1 --2- 172.21.15.164:0/4283700303 >> v2:172.21.15.172:3300/0 conn(0x7f236c15bc50 0x7f236c0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2361400640 1 -- 172.21.15.164:0/4283700303 >> v1:172.21.15.164:6789/0 conn(0x7f236c0b4d50 legacy=0x7f236c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.129+0000 7f2361400640 1 -- 172.21.15.164:0/4283700303 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0cf590 con 0x7f236c0ccfe0 2024-05-08T04:40:39.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2361400640 1 --2- 172.21.15.164:0/4283700303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 0x7f236c0d0070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23540097c0 tx=0x7f235401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e72a5d298005fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2369400640 1 -- 172.21.15.164:0/4283700303 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540b0070 con 0x7f236c0ccfe0 2024-05-08T04:40:39.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2369400640 1 -- 172.21.15.164:0/4283700303 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23540ab600 con 0x7f236c0ccfe0 2024-05-08T04:40:39.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2369400640 1 -- 172.21.15.164:0/4283700303 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540ab9d0 con 0x7f236c0ccfe0 2024-05-08T04:40:39.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 -- 172.21.15.164:0/4283700303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 msgr2=0x7f236c0d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 --2- 172.21.15.164:0/4283700303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 0x7f236c0d0070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23540097c0 tx=0x7f235401c3e0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 -- 172.21.15.164:0/4283700303 shutdown_connections 2024-05-08T04:40:39.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 --2- 172.21.15.164:0/4283700303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c0ccfe0 0x7f236c0d0070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 --2- 172.21.15.164:0/4283700303 >> v2:172.21.15.172:3300/0 conn(0x7f236c15bc50 0x7f236c0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 -- 172.21.15.164:0/4283700303 >> 172.21.15.164:0/4283700303 conn(0x7f236c0b34f0 msgr2=0x7f236c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:39.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 -- 172.21.15.164:0/4283700303 shutdown_connections 2024-05-08T04:40:39.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.133+0000 7f2371200640 1 -- 172.21.15.164:0/4283700303 wait complete. 2024-05-08T04:40:39.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 Processor -- start 2024-05-08T04:40:39.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- start start 2024-05-08T04:40:39.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f236c0ccfe0 0x7f236c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 0x7f236c166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c168c60 con 0x7f236c0b4d50 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c168ae0 con 0x7f236c0ccfe0 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c168de0 con 0x7f236c15bc50 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41466/0 (socket says 172.21.15.164:41466) 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 -- 172.21.15.164:0/1171648419 learned_addr learned my addr 172.21.15.164:0/1171648419 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2361400640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f236c0ccfe0 0x7f236c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f236a800640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 0x7f236c166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 -- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 msgr2=0x7f236c166120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 0x7f236c166120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 -- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f236c0ccfe0 msgr2=0x7f236c0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f236c0ccfe0 0x7f236c0c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 -- 172.21.15.164:0/1171648419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c164da0 con 0x7f236c0b4d50 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f236a800640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 0x7f236c166120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2369e00640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e960 tx=0x7f235c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c017070 con 0x7f236c0b4d50 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00ca00 con 0x7f236c0b4d50 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c165080 con 0x7f236c0b4d50 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00cdb0 con 0x7f236c0b4d50 2024-05-08T04:40:39.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.137+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c165640 con 0x7f236c0b4d50 2024-05-08T04:40:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c069d60 con 0x7f236c0b4d50 2024-05-08T04:40:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f235c01a070 con 0x7f236c0b4d50 2024-05-08T04:40:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2362a00640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 0x7f2348069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f235c085940 con 0x7f236c0b4d50 2024-05-08T04:40:39.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2361400640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 0x7f2348069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.141+0000 7f2361400640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 0x7f2348069660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f23540ad000 tx=0x7f23540a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.145+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f235c056530 con 0x7f236c0b4d50 2024-05-08T04:40:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.301+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236c15b120 con 0x7f236c0b4d50 2024-05-08T04:40:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.301+0000 7f2362a00640 1 -- 172.21.15.164:0/1171648419 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f235c055e30 con 0x7f236c0b4d50 2024-05-08T04:40:39.307 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:39.307 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":60,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:36.590286+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 msgr2=0x7f2348069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 0x7f2348069660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f23540ad000 tx=0x7f23540a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 msgr2=0x7f236c14b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e960 tx=0x7f235c00ee20 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 shutdown_connections 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f23480671a0 0x7f2348069660 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f236c15bc50 0x7f236c166120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f236c0ccfe0 0x7f236c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 --2- 172.21.15.164:0/1171648419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f236c0b4d50 0x7f236c14b960 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 >> 172.21.15.164:0/1171648419 conn(0x7f236c0b34f0 msgr2=0x7f236c0b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 shutdown_connections 2024-05-08T04:40:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.305+0000 7f2371200640 1 -- 172.21.15.164:0/1171648419 wait complete. 2024-05-08T04:40:39.328 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:40:39.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:40:39.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 Processor -- start 2024-05-08T04:40:39.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 -- start start 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f437815ba70 0x7f4378160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f43780cfda0 0x7f43780b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780cd9f0 con 0x7f437815be50 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437815fe70 con 0x7f437815ba70 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43780cdcd0 con 0x7f43780cfda0 2024-05-08T04:40:39.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f437815ba70 0x7f4378160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437ca00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f437815be50 0x7f43780cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37006/0 (socket says 172.21.15.164:37006) 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437ca00640 1 -- 172.21.15.164:0/3939392952 learned_addr learned my addr 172.21.15.164:0/3939392952 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437de00640 1 --2- 172.21.15.164:0/3939392952 >> v2:172.21.15.172:3300/0 conn(0x7f43780cfda0 0x7f43780b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f4377e00640 1 -- 172.21.15.164:0/3939392952 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672119967 0 0) 0x7f43780cd9f0 con 0x7f437815be50 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f4377e00640 1 -- 172.21.15.164:0/3939392952 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4360003530 con 0x7f437815be50 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437d400640 1 -- 172.21.15.164:0/3939392952 >> v2:172.21.15.172:3300/0 conn(0x7f43780cfda0 msgr2=0x7f43780b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437d400640 1 --2- 172.21.15.164:0/3939392952 >> v2:172.21.15.172:3300/0 conn(0x7f43780cfda0 0x7f43780b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437d400640 1 -- 172.21.15.164:0/3939392952 >> v1:172.21.15.164:6789/0 conn(0x7f437815be50 legacy=0x7f43780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.461+0000 7f437d400640 1 -- 172.21.15.164:0/3939392952 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437815f820 con 0x7f437815ba70 2024-05-08T04:40:39.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437d400640 1 --2- 172.21.15.164:0/3939392952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437815ba70 0x7f4378160300 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f437815fde0 tx=0x7f436c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ee985ed6f3745f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f4377e00640 1 -- 172.21.15.164:0/3939392952 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b2070 con 0x7f437815ba70 2024-05-08T04:40:39.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f4377e00640 1 -- 172.21.15.164:0/3939392952 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c0a9070 con 0x7f437815ba70 2024-05-08T04:40:39.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f4377e00640 1 -- 172.21.15.164:0/3939392952 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0ad880 con 0x7f437815ba70 2024-05-08T04:40:39.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 -- 172.21.15.164:0/3939392952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437815ba70 msgr2=0x7f4378160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 --2- 172.21.15.164:0/3939392952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437815ba70 0x7f4378160300 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f437815fde0 tx=0x7f436c01c450 comp rx=0 tx=0).stop 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 -- 172.21.15.164:0/3939392952 shutdown_connections 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 --2- 172.21.15.164:0/3939392952 >> v2:172.21.15.172:3300/0 conn(0x7f43780cfda0 0x7f43780b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 --2- 172.21.15.164:0/3939392952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f437815ba70 0x7f4378160300 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 -- 172.21.15.164:0/3939392952 >> 172.21.15.164:0/3939392952 conn(0x7f437806ac90 msgr2=0x7f437806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 -- 172.21.15.164:0/3939392952 shutdown_connections 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.465+0000 7f437fe00640 1 -- 172.21.15.164:0/3939392952 wait complete. 2024-05-08T04:40:39.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 Processor -- start 2024-05-08T04:40:39.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- start start 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f437815ba70 0x7f437814df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 0x7f437814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378168d40 con 0x7f43780cfda0 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378168bc0 con 0x7f437815be50 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4378168ec0 con 0x7f437815ba70 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46212/0 (socket says 172.21.15.164:46212) 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 -- 172.21.15.164:0/1559892999 learned_addr learned my addr 172.21.15.164:0/1559892999 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437ca00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f437815ba70 0x7f437814df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437de00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 0x7f437814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 -- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f437815ba70 msgr2=0x7f437814df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f437815ba70 0x7f437814df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 -- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 msgr2=0x7f437814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 0x7f437814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 -- 172.21.15.164:0/1559892999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43781490e0 con 0x7f43780cfda0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437de00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 0x7f437814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437d400640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f436c0b0040 tx=0x7f436c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b2050 con 0x7f43780cfda0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c0be070 con 0x7f43780cfda0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43781493c0 con 0x7f43780cfda0 2024-05-08T04:40:39.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0ade60 con 0x7f43780cfda0 2024-05-08T04:40:39.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.469+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4378149980 con 0x7f43780cfda0 2024-05-08T04:40:39.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378069d60 con 0x7f43780cfda0 2024-05-08T04:40:39.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f436c0ab070 con 0x7f43780cfda0 2024-05-08T04:40:39.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f4375600640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 0x7f4358069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f436c11fae0 con 0x7f43780cfda0 2024-05-08T04:40:39.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f437ca00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 0x7f4358069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.473+0000 7f437ca00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 0x7f4358069660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f437814cdd0 tx=0x7f4364009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.477+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f436c016070 con 0x7f43780cfda0 2024-05-08T04:40:39.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.633+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4378155600 con 0x7f43580671a0 2024-05-08T04:40:39.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.637+0000 7f4375600640 1 -- 172.21.15.164:0/1559892999 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7f4378155600 con 0x7f43580671a0 2024-05-08T04:40:39.646 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:39.647 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 msgr2=0x7f4358069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 0x7f4358069660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f437814cdd0 tx=0x7f4364009040 comp rx=0 tx=0).stop 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 msgr2=0x7f437815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f436c0b0040 tx=0x7f436c01ac90 comp rx=0 tx=0).stop 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 shutdown_connections 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43580671a0 0x7f4358069660 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f437815ba70 0x7f437814df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437815be50 0x7f437814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 --2- 172.21.15.164:0/1559892999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780cfda0 0x7f437815ccb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 >> 172.21.15.164:0/1559892999 conn(0x7f437806ac90 msgr2=0x7f4378151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 shutdown_connections 2024-05-08T04:40:39.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.641+0000 7f437fe00640 1 -- 172.21.15.164:0/1559892999 wait complete. 2024-05-08T04:40:39.666 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-05-08T04:40:39.456711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227176,"kb_used_data":6832,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747544,"statfs":{"total":773094113280,"available":772861485056,"internally_reserved":0,"allocated":6995968,"data_stored":3593364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.891369"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898982+0000","last_change":"2024-05-08T04:40:35.611583+0000","last_active":"2024-05-08T04:40:36.898982+0000","last_peered":"2024-05-08T04:40:36.898982+0000","last_clean":"2024-05-08T04:40:36.898982+0000","last_became_active":"2024-05-08T04:40:35.611306+0000","last_became_peered":"2024-05-08T04:40:35.611306+0000","last_unstale":"2024-05-08T04:40:36.898982+0000","last_undegraded":"2024-05-08T04:40:36.898982+0000","last_fullsized":"2024-05-08T04:40:36.898982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.907283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599872+0000","last_change":"2024-05-08T04:40:35.623953+0000","last_active":"2024-05-08T04:40:36.599872+0000","last_peered":"2024-05-08T04:40:36.599872+0000","last_clean":"2024-05-08T04:40:36.599872+0000","last_became_active":"2024-05-08T04:40:35.623589+0000","last_became_peered":"2024-05-08T04:40:35.623589+0000","last_unstale":"2024-05-08T04:40:36.599872+0000","last_undegraded":"2024-05-08T04:40:36.599872+0000","last_fullsized":"2024-05-08T04:40:36.599872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:10:34.811364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899281+0000","last_change":"2024-05-08T04:40:35.604418+0000","last_active":"2024-05-08T04:40:36.899281+0000","last_peered":"2024-05-08T04:40:36.899281+0000","last_clean":"2024-05-08T04:40:36.899281+0000","last_became_active":"2024-05-08T04:40:35.602188+0000","last_became_peered":"2024-05-08T04:40:35.602188+0000","last_unstale":"2024-05-08T04:40:36.899281+0000","last_undegraded":"2024-05-08T04:40:36.899281+0000","last_fullsized":"2024-05-08T04:40:36.899281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:03.198334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601195+0000","last_change":"2024-05-08T04:40:35.599410+0000","last_active":"2024-05-08T04:40:36.601195+0000","last_peered":"2024-05-08T04:40:36.601195+0000","last_clean":"2024-05-08T04:40:36.601195+0000","last_became_active":"2024-05-08T04:40:35.599115+0000","last_became_peered":"2024-05-08T04:40:35.599115+0000","last_unstale":"2024-05-08T04:40:36.601195+0000","last_undegraded":"2024-05-08T04:40:36.601195+0000","last_fullsized":"2024-05-08T04:40:36.601195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:42.695403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898622+0000","last_change":"2024-05-08T04:40:35.609307+0000","last_active":"2024-05-08T04:40:36.898622+0000","last_peered":"2024-05-08T04:40:36.898622+0000","last_clean":"2024-05-08T04:40:36.898622+0000","last_became_active":"2024-05-08T04:40:35.609091+0000","last_became_peered":"2024-05-08T04:40:35.609091+0000","last_unstale":"2024-05-08T04:40:36.898622+0000","last_undegraded":"2024-05-08T04:40:36.898622+0000","last_fullsized":"2024-05-08T04:40:36.898622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:20.641727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602285+0000","last_change":"2024-05-08T04:40:35.596918+0000","last_active":"2024-05-08T04:40:36.602285+0000","last_peered":"2024-05-08T04:40:36.602285+0000","last_clean":"2024-05-08T04:40:36.602285+0000","last_became_active":"2024-05-08T04:40:35.596634+0000","last_became_peered":"2024-05-08T04:40:35.596634+0000","last_unstale":"2024-05-08T04:40:36.602285+0000","last_undegraded":"2024-05-08T04:40:36.602285+0000","last_fullsized":"2024-05-08T04:40:36.602285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:22.085977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599872+0000","last_change":"2024-05-08T04:40:35.598093+0000","last_active":"2024-05-08T04:40:36.599872+0000","last_peered":"2024-05-08T04:40:36.599872+0000","last_clean":"2024-05-08T04:40:36.599872+0000","last_became_active":"2024-05-08T04:40:35.597827+0000","last_became_peered":"2024-05-08T04:40:35.597827+0000","last_unstale":"2024-05-08T04:40:36.599872+0000","last_undegraded":"2024-05-08T04:40:36.599872+0000","last_fullsized":"2024-05-08T04:40:36.599872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:29.340734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599344+0000","last_change":"2024-05-08T04:40:35.612889+0000","last_active":"2024-05-08T04:40:36.599344+0000","last_peered":"2024-05-08T04:40:36.599344+0000","last_clean":"2024-05-08T04:40:36.599344+0000","last_became_active":"2024-05-08T04:40:35.612567+0000","last_became_peered":"2024-05-08T04:40:35.612567+0000","last_unstale":"2024-05-08T04:40:36.599344+0000","last_undegraded":"2024-05-08T04:40:36.599344+0000","last_fullsized":"2024-05-08T04:40:36.599344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:24.028947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600765+0000","last_change":"2024-05-08T04:40:35.603549+0000","last_active":"2024-05-08T04:40:36.600765+0000","last_peered":"2024-05-08T04:40:36.600765+0000","last_clean":"2024-05-08T04:40:36.600765+0000","last_became_active":"2024-05-08T04:40:35.598125+0000","last_became_peered":"2024-05-08T04:40:35.598125+0000","last_unstale":"2024-05-08T04:40:36.600765+0000","last_undegraded":"2024-05-08T04:40:36.600765+0000","last_fullsized":"2024-05-08T04:40:36.600765+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:37.494892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898915+0000","last_change":"2024-05-08T04:40:35.611098+0000","last_active":"2024-05-08T04:40:36.898915+0000","last_peered":"2024-05-08T04:40:36.898915+0000","last_clean":"2024-05-08T04:40:36.898915+0000","last_became_active":"2024-05-08T04:40:35.610380+0000","last_became_peered":"2024-05-08T04:40:35.610380+0000","last_unstale":"2024-05-08T04:40:36.898915+0000","last_undegraded":"2024-05-08T04:40:36.898915+0000","last_fullsized":"2024-05-08T04:40:36.898915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:33.168942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898995+0000","last_change":"2024-05-08T04:40:35.618617+0000","last_active":"2024-05-08T04:40:36.898995+0000","last_peered":"2024-05-08T04:40:36.898995+0000","last_clean":"2024-05-08T04:40:36.898995+0000","last_became_active":"2024-05-08T04:40:35.618438+0000","last_became_peered":"2024-05-08T04:40:35.618438+0000","last_unstale":"2024-05-08T04:40:36.898995+0000","last_undegraded":"2024-05-08T04:40:36.898995+0000","last_fullsized":"2024-05-08T04:40:36.898995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:01.802314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601454+0000","last_change":"2024-05-08T04:40:35.606582+0000","last_active":"2024-05-08T04:40:36.601454+0000","last_peered":"2024-05-08T04:40:36.601454+0000","last_clean":"2024-05-08T04:40:36.601454+0000","last_became_active":"2024-05-08T04:40:35.606211+0000","last_became_peered":"2024-05-08T04:40:35.606211+0000","last_unstale":"2024-05-08T04:40:36.601454+0000","last_undegraded":"2024-05-08T04:40:36.601454+0000","last_fullsized":"2024-05-08T04:40:36.601454+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:52.918881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605605+0000","last_change":"2024-05-08T04:40:35.602441+0000","last_active":"2024-05-08T04:40:36.605605+0000","last_peered":"2024-05-08T04:40:36.605605+0000","last_clean":"2024-05-08T04:40:36.605605+0000","last_became_active":"2024-05-08T04:40:35.601934+0000","last_became_peered":"2024-05-08T04:40:35.601934+0000","last_unstale":"2024-05-08T04:40:36.605605+0000","last_undegraded":"2024-05-08T04:40:36.605605+0000","last_fullsized":"2024-05-08T04:40:36.605605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:26.028767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600210+0000","last_change":"2024-05-08T04:40:35.598805+0000","last_active":"2024-05-08T04:40:36.600210+0000","last_peered":"2024-05-08T04:40:36.600210+0000","last_clean":"2024-05-08T04:40:36.600210+0000","last_became_active":"2024-05-08T04:40:35.598497+0000","last_became_peered":"2024-05-08T04:40:35.598497+0000","last_unstale":"2024-05-08T04:40:36.600210+0000","last_undegraded":"2024-05-08T04:40:36.600210+0000","last_fullsized":"2024-05-08T04:40:36.600210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:35.157666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600169+0000","last_change":"2024-05-08T04:40:35.603457+0000","last_active":"2024-05-08T04:40:36.600169+0000","last_peered":"2024-05-08T04:40:36.600169+0000","last_clean":"2024-05-08T04:40:36.600169+0000","last_became_active":"2024-05-08T04:40:35.603124+0000","last_became_peered":"2024-05-08T04:40:35.603124+0000","last_unstale":"2024-05-08T04:40:36.600169+0000","last_undegraded":"2024-05-08T04:40:36.600169+0000","last_fullsized":"2024-05-08T04:40:36.600169+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:42.983311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602302+0000","last_change":"2024-05-08T04:40:35.601731+0000","last_active":"2024-05-08T04:40:36.602302+0000","last_peered":"2024-05-08T04:40:36.602302+0000","last_clean":"2024-05-08T04:40:36.602302+0000","last_became_active":"2024-05-08T04:40:35.601366+0000","last_became_peered":"2024-05-08T04:40:35.601366+0000","last_unstale":"2024-05-08T04:40:36.602302+0000","last_undegraded":"2024-05-08T04:40:36.602302+0000","last_fullsized":"2024-05-08T04:40:36.602302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:01.803268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600241+0000","last_change":"2024-05-08T04:40:35.612272+0000","last_active":"2024-05-08T04:40:36.600241+0000","last_peered":"2024-05-08T04:40:36.600241+0000","last_clean":"2024-05-08T04:40:36.600241+0000","last_became_active":"2024-05-08T04:40:35.608761+0000","last_became_peered":"2024-05-08T04:40:35.608761+0000","last_unstale":"2024-05-08T04:40:36.600241+0000","last_undegraded":"2024-05-08T04:40:36.600241+0000","last_fullsized":"2024-05-08T04:40:36.600241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:22.363547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899457+0000","last_change":"2024-05-08T04:40:35.601871+0000","last_active":"2024-05-08T04:40:36.899457+0000","last_peered":"2024-05-08T04:40:36.899457+0000","last_clean":"2024-05-08T04:40:36.899457+0000","last_became_active":"2024-05-08T04:40:35.601571+0000","last_became_peered":"2024-05-08T04:40:35.601571+0000","last_unstale":"2024-05-08T04:40:36.899457+0000","last_undegraded":"2024-05-08T04:40:36.899457+0000","last_fullsized":"2024-05-08T04:40:36.899457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:22.085977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898625+0000","last_change":"2024-05-08T04:40:35.617954+0000","last_active":"2024-05-08T04:40:36.898625+0000","last_peered":"2024-05-08T04:40:36.898625+0000","last_clean":"2024-05-08T04:40:36.898625+0000","last_became_active":"2024-05-08T04:40:35.617691+0000","last_became_peered":"2024-05-08T04:40:35.617691+0000","last_unstale":"2024-05-08T04:40:36.898625+0000","last_undegraded":"2024-05-08T04:40:36.898625+0000","last_fullsized":"2024-05-08T04:40:36.898625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:29:11.001448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599821+0000","last_change":"2024-05-08T04:40:35.613721+0000","last_active":"2024-05-08T04:40:36.599821+0000","last_peered":"2024-05-08T04:40:36.599821+0000","last_clean":"2024-05-08T04:40:36.599821+0000","last_became_active":"2024-05-08T04:40:35.613310+0000","last_became_peered":"2024-05-08T04:40:35.613310+0000","last_unstale":"2024-05-08T04:40:36.599821+0000","last_undegraded":"2024-05-08T04:40:36.599821+0000","last_fullsized":"2024-05-08T04:40:36.599821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:24.735268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899034+0000","last_change":"2024-05-08T04:40:35.608182+0000","last_active":"2024-05-08T04:40:36.899034+0000","last_peered":"2024-05-08T04:40:36.899034+0000","last_clean":"2024-05-08T04:40:36.899034+0000","last_became_active":"2024-05-08T04:40:35.607909+0000","last_became_peered":"2024-05-08T04:40:35.607909+0000","last_unstale":"2024-05-08T04:40:36.899034+0000","last_undegraded":"2024-05-08T04:40:36.899034+0000","last_fullsized":"2024-05-08T04:40:36.899034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:46:43.291290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599613+0000","last_change":"2024-05-08T04:40:35.610112+0000","last_active":"2024-05-08T04:40:36.599613+0000","last_peered":"2024-05-08T04:40:36.599613+0000","last_clean":"2024-05-08T04:40:36.599613+0000","last_became_active":"2024-05-08T04:40:35.609344+0000","last_became_peered":"2024-05-08T04:40:35.609344+0000","last_unstale":"2024-05-08T04:40:36.599613+0000","last_undegraded":"2024-05-08T04:40:36.599613+0000","last_fullsized":"2024-05-08T04:40:36.599613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:16.603914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899157+0000","last_change":"2024-05-08T04:40:35.618464+0000","last_active":"2024-05-08T04:40:36.899157+0000","last_peered":"2024-05-08T04:40:36.899157+0000","last_clean":"2024-05-08T04:40:36.899157+0000","last_became_active":"2024-05-08T04:40:35.617795+0000","last_became_peered":"2024-05-08T04:40:35.617795+0000","last_unstale":"2024-05-08T04:40:36.899157+0000","last_undegraded":"2024-05-08T04:40:36.899157+0000","last_fullsized":"2024-05-08T04:40:36.899157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:50.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605539+0000","last_change":"2024-05-08T04:40:35.601191+0000","last_active":"2024-05-08T04:40:36.605539+0000","last_peered":"2024-05-08T04:40:36.605539+0000","last_clean":"2024-05-08T04:40:36.605539+0000","last_became_active":"2024-05-08T04:40:35.600825+0000","last_became_peered":"2024-05-08T04:40:35.600825+0000","last_unstale":"2024-05-08T04:40:36.605539+0000","last_undegraded":"2024-05-08T04:40:36.605539+0000","last_fullsized":"2024-05-08T04:40:36.605539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:01.802314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899163+0000","last_change":"2024-05-08T04:40:35.615715+0000","last_active":"2024-05-08T04:40:36.899163+0000","last_peered":"2024-05-08T04:40:36.899163+0000","last_clean":"2024-05-08T04:40:36.899163+0000","last_became_active":"2024-05-08T04:40:35.615471+0000","last_became_peered":"2024-05-08T04:40:35.615471+0000","last_unstale":"2024-05-08T04:40:36.899163+0000","last_undegraded":"2024-05-08T04:40:36.899163+0000","last_fullsized":"2024-05-08T04:40:36.899163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:00.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601728+0000","last_change":"2024-05-08T04:40:35.601742+0000","last_active":"2024-05-08T04:40:36.601728+0000","last_peered":"2024-05-08T04:40:36.601728+0000","last_clean":"2024-05-08T04:40:36.601728+0000","last_became_active":"2024-05-08T04:40:35.601378+0000","last_became_peered":"2024-05-08T04:40:35.601378+0000","last_unstale":"2024-05-08T04:40:36.601728+0000","last_undegraded":"2024-05-08T04:40:36.601728+0000","last_fullsized":"2024-05-08T04:40:36.601728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:05.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602651+0000","last_change":"2024-05-08T04:40:35.603124+0000","last_active":"2024-05-08T04:40:36.602651+0000","last_peered":"2024-05-08T04:40:36.602651+0000","last_clean":"2024-05-08T04:40:36.602651+0000","last_became_active":"2024-05-08T04:40:35.602781+0000","last_became_peered":"2024-05-08T04:40:35.602781+0000","last_unstale":"2024-05-08T04:40:36.602651+0000","last_undegraded":"2024-05-08T04:40:36.602651+0000","last_fullsized":"2024-05-08T04:40:36.602651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:53.268007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898600+0000","last_change":"2024-05-08T04:40:35.605408+0000","last_active":"2024-05-08T04:40:36.898600+0000","last_peered":"2024-05-08T04:40:36.898600+0000","last_clean":"2024-05-08T04:40:36.898600+0000","last_became_active":"2024-05-08T04:40:35.605048+0000","last_became_peered":"2024-05-08T04:40:35.605048+0000","last_unstale":"2024-05-08T04:40:36.898600+0000","last_undegraded":"2024-05-08T04:40:36.898600+0000","last_fullsized":"2024-05-08T04:40:36.898600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:02.084663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600216+0000","last_change":"2024-05-08T04:40:35.609883+0000","last_active":"2024-05-08T04:40:36.600216+0000","last_peered":"2024-05-08T04:40:36.600216+0000","last_clean":"2024-05-08T04:40:36.600216+0000","last_became_active":"2024-05-08T04:40:35.609239+0000","last_became_peered":"2024-05-08T04:40:35.609239+0000","last_unstale":"2024-05-08T04:40:36.600216+0000","last_undegraded":"2024-05-08T04:40:36.600216+0000","last_fullsized":"2024-05-08T04:40:36.600216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:25:56.900214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899030+0000","last_change":"2024-05-08T04:40:35.603619+0000","last_active":"2024-05-08T04:40:36.899030+0000","last_peered":"2024-05-08T04:40:36.899030+0000","last_clean":"2024-05-08T04:40:36.899030+0000","last_became_active":"2024-05-08T04:40:35.603284+0000","last_became_peered":"2024-05-08T04:40:35.603284+0000","last_unstale":"2024-05-08T04:40:36.899030+0000","last_undegraded":"2024-05-08T04:40:36.899030+0000","last_fullsized":"2024-05-08T04:40:36.899030+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:27.931796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898248+0000","last_change":"2024-05-08T04:40:35.606994+0000","last_active":"2024-05-08T04:40:36.898248+0000","last_peered":"2024-05-08T04:40:36.898248+0000","last_clean":"2024-05-08T04:40:36.898248+0000","last_became_active":"2024-05-08T04:40:35.604475+0000","last_became_peered":"2024-05-08T04:40:35.604475+0000","last_unstale":"2024-05-08T04:40:36.898248+0000","last_undegraded":"2024-05-08T04:40:36.898248+0000","last_fullsized":"2024-05-08T04:40:36.898248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:22.065594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602267+0000","last_change":"2024-05-08T04:40:35.617966+0000","last_active":"2024-05-08T04:40:36.602267+0000","last_peered":"2024-05-08T04:40:36.602267+0000","last_clean":"2024-05-08T04:40:36.602267+0000","last_became_active":"2024-05-08T04:40:35.617707+0000","last_became_peered":"2024-05-08T04:40:35.617707+0000","last_unstale":"2024-05-08T04:40:36.602267+0000","last_undegraded":"2024-05-08T04:40:36.602267+0000","last_fullsized":"2024-05-08T04:40:36.602267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:57.335849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.603026+0000","last_change":"2024-05-08T04:40:35.598013+0000","last_active":"2024-05-08T04:40:36.603026+0000","last_peered":"2024-05-08T04:40:36.603026+0000","last_clean":"2024-05-08T04:40:36.603026+0000","last_became_active":"2024-05-08T04:40:35.597707+0000","last_became_peered":"2024-05-08T04:40:35.597707+0000","last_unstale":"2024-05-08T04:40:36.603026+0000","last_undegraded":"2024-05-08T04:40:36.603026+0000","last_fullsized":"2024-05-08T04:40:36.603026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.404913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602787+0000","last_change":"2024-05-08T04:40:34.592658+0000","last_active":"2024-05-08T04:40:36.602787+0000","last_peered":"2024-05-08T04:40:36.602787+0000","last_clean":"2024-05-08T04:40:36.602787+0000","last_became_active":"2024-05-08T04:40:34.592398+0000","last_became_peered":"2024-05-08T04:40:34.592398+0000","last_unstale":"2024-05-08T04:40:36.602787+0000","last_undegraded":"2024-05-08T04:40:36.602787+0000","last_fullsized":"2024-05-08T04:40:36.602787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:36.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899055+0000","last_change":"2024-05-08T04:40:35.610732+0000","last_active":"2024-05-08T04:40:36.899055+0000","last_peered":"2024-05-08T04:40:36.899055+0000","last_clean":"2024-05-08T04:40:36.899055+0000","last_became_active":"2024-05-08T04:40:35.610448+0000","last_became_peered":"2024-05-08T04:40:35.610448+0000","last_unstale":"2024-05-08T04:40:36.899055+0000","last_undegraded":"2024-05-08T04:40:36.899055+0000","last_fullsized":"2024-05-08T04:40:36.899055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:26.028767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600012+0000","last_change":"2024-05-08T04:40:34.598869+0000","last_active":"2024-05-08T04:40:36.600012+0000","last_peered":"2024-05-08T04:40:36.600012+0000","last_clean":"2024-05-08T04:40:36.600012+0000","last_became_active":"2024-05-08T04:40:34.598616+0000","last_became_peered":"2024-05-08T04:40:34.598616+0000","last_unstale":"2024-05-08T04:40:36.600012+0000","last_undegraded":"2024-05-08T04:40:36.600012+0000","last_fullsized":"2024-05-08T04:40:36.600012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:02.190842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602707+0000","last_change":"2024-05-08T04:40:35.623259+0000","last_active":"2024-05-08T04:40:36.602707+0000","last_peered":"2024-05-08T04:40:36.602707+0000","last_clean":"2024-05-08T04:40:36.602707+0000","last_became_active":"2024-05-08T04:40:35.622905+0000","last_became_peered":"2024-05-08T04:40:35.622905+0000","last_unstale":"2024-05-08T04:40:36.602707+0000","last_undegraded":"2024-05-08T04:40:36.602707+0000","last_fullsized":"2024-05-08T04:40:36.602707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:13.065627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602802+0000","last_change":"2024-05-08T04:40:34.595613+0000","last_active":"2024-05-08T04:40:36.602802+0000","last_peered":"2024-05-08T04:40:36.602802+0000","last_clean":"2024-05-08T04:40:36.602802+0000","last_became_active":"2024-05-08T04:40:34.595245+0000","last_became_peered":"2024-05-08T04:40:34.595245+0000","last_unstale":"2024-05-08T04:40:36.602802+0000","last_undegraded":"2024-05-08T04:40:36.602802+0000","last_fullsized":"2024-05-08T04:40:36.602802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:32.793521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602700+0000","last_change":"2024-05-08T04:40:35.602040+0000","last_active":"2024-05-08T04:40:36.602700+0000","last_peered":"2024-05-08T04:40:36.602700+0000","last_clean":"2024-05-08T04:40:36.602700+0000","last_became_active":"2024-05-08T04:40:35.601716+0000","last_became_peered":"2024-05-08T04:40:35.601716+0000","last_unstale":"2024-05-08T04:40:36.602700+0000","last_undegraded":"2024-05-08T04:40:36.602700+0000","last_fullsized":"2024-05-08T04:40:36.602700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:31:14.018668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601391+0000","last_change":"2024-05-08T04:40:34.589406+0000","last_active":"2024-05-08T04:40:36.601391+0000","last_peered":"2024-05-08T04:40:36.601391+0000","last_clean":"2024-05-08T04:40:36.601391+0000","last_became_active":"2024-05-08T04:40:34.588554+0000","last_became_peered":"2024-05-08T04:40:34.588554+0000","last_unstale":"2024-05-08T04:40:36.601391+0000","last_undegraded":"2024-05-08T04:40:36.601391+0000","last_fullsized":"2024-05-08T04:40:36.601391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:25.771576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599657+0000","last_change":"2024-05-08T04:40:35.609852+0000","last_active":"2024-05-08T04:40:36.599657+0000","last_peered":"2024-05-08T04:40:36.599657+0000","last_clean":"2024-05-08T04:40:36.599657+0000","last_became_active":"2024-05-08T04:40:35.608870+0000","last_became_peered":"2024-05-08T04:40:35.608870+0000","last_unstale":"2024-05-08T04:40:36.599657+0000","last_undegraded":"2024-05-08T04:40:36.599657+0000","last_fullsized":"2024-05-08T04:40:36.599657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:08.103190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600310+0000","last_change":"2024-05-08T04:40:34.586666+0000","last_active":"2024-05-08T04:40:36.600310+0000","last_peered":"2024-05-08T04:40:36.600310+0000","last_clean":"2024-05-08T04:40:36.600310+0000","last_became_active":"2024-05-08T04:40:34.586295+0000","last_became_peered":"2024-05-08T04:40:34.586295+0000","last_unstale":"2024-05-08T04:40:36.600310+0000","last_undegraded":"2024-05-08T04:40:36.600310+0000","last_fullsized":"2024-05-08T04:40:36.600310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:00.794821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599926+0000","last_change":"2024-05-08T04:40:35.613673+0000","last_active":"2024-05-08T04:40:36.599926+0000","last_peered":"2024-05-08T04:40:36.599926+0000","last_clean":"2024-05-08T04:40:36.599926+0000","last_became_active":"2024-05-08T04:40:35.609519+0000","last_became_peered":"2024-05-08T04:40:35.609519+0000","last_unstale":"2024-05-08T04:40:36.599926+0000","last_undegraded":"2024-05-08T04:40:36.599926+0000","last_fullsized":"2024-05-08T04:40:36.599926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:16:43.538626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602384+0000","last_change":"2024-05-08T04:40:34.596409+0000","last_active":"2024-05-08T04:40:36.602384+0000","last_peered":"2024-05-08T04:40:36.602384+0000","last_clean":"2024-05-08T04:40:36.602384+0000","last_became_active":"2024-05-08T04:40:34.596131+0000","last_became_peered":"2024-05-08T04:40:34.596131+0000","last_unstale":"2024-05-08T04:40:36.602384+0000","last_undegraded":"2024-05-08T04:40:36.602384+0000","last_fullsized":"2024-05-08T04:40:36.602384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:34.150174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605523+0000","last_change":"2024-05-08T04:40:35.605488+0000","last_active":"2024-05-08T04:40:36.605523+0000","last_peered":"2024-05-08T04:40:36.605523+0000","last_clean":"2024-05-08T04:40:36.605523+0000","last_became_active":"2024-05-08T04:40:35.605224+0000","last_became_peered":"2024-05-08T04:40:35.605224+0000","last_unstale":"2024-05-08T04:40:36.605523+0000","last_undegraded":"2024-05-08T04:40:36.605523+0000","last_fullsized":"2024-05-08T04:40:36.605523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:19.456105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601111+0000","last_change":"2024-05-08T04:40:34.589217+0000","last_active":"2024-05-08T04:40:36.601111+0000","last_peered":"2024-05-08T04:40:36.601111+0000","last_clean":"2024-05-08T04:40:36.601111+0000","last_became_active":"2024-05-08T04:40:34.588859+0000","last_became_peered":"2024-05-08T04:40:34.588859+0000","last_unstale":"2024-05-08T04:40:36.601111+0000","last_undegraded":"2024-05-08T04:40:36.601111+0000","last_fullsized":"2024-05-08T04:40:36.601111+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:35:58.250941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599636+0000","last_change":"2024-05-08T04:40:35.613121+0000","last_active":"2024-05-08T04:40:36.599636+0000","last_peered":"2024-05-08T04:40:36.599636+0000","last_clean":"2024-05-08T04:40:36.599636+0000","last_became_active":"2024-05-08T04:40:35.612308+0000","last_became_peered":"2024-05-08T04:40:35.612308+0000","last_unstale":"2024-05-08T04:40:36.599636+0000","last_undegraded":"2024-05-08T04:40:36.599636+0000","last_fullsized":"2024-05-08T04:40:36.599636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:10.353996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600145+0000","last_change":"2024-05-08T04:40:34.592271+0000","last_active":"2024-05-08T04:40:36.600145+0000","last_peered":"2024-05-08T04:40:36.600145+0000","last_clean":"2024-05-08T04:40:36.600145+0000","last_became_active":"2024-05-08T04:40:34.591958+0000","last_became_peered":"2024-05-08T04:40:34.591958+0000","last_unstale":"2024-05-08T04:40:36.600145+0000","last_undegraded":"2024-05-08T04:40:36.600145+0000","last_fullsized":"2024-05-08T04:40:36.600145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:18.448613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600726+0000","last_change":"2024-05-08T04:40:35.620986+0000","last_active":"2024-05-08T04:40:36.600726+0000","last_peered":"2024-05-08T04:40:36.600726+0000","last_clean":"2024-05-08T04:40:36.600726+0000","last_became_active":"2024-05-08T04:40:35.620624+0000","last_became_peered":"2024-05-08T04:40:35.620624+0000","last_unstale":"2024-05-08T04:40:36.600726+0000","last_undegraded":"2024-05-08T04:40:36.600726+0000","last_fullsized":"2024-05-08T04:40:36.600726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:33.801014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899489+0000","last_change":"2024-05-08T04:40:34.591906+0000","last_active":"2024-05-08T04:40:36.899489+0000","last_peered":"2024-05-08T04:40:36.899489+0000","last_clean":"2024-05-08T04:40:36.899489+0000","last_became_active":"2024-05-08T04:40:34.591483+0000","last_became_peered":"2024-05-08T04:40:34.591483+0000","last_unstale":"2024-05-08T04:40:36.899489+0000","last_undegraded":"2024-05-08T04:40:36.899489+0000","last_fullsized":"2024-05-08T04:40:36.899489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:25.771576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899554+0000","last_change":"2024-05-08T04:40:35.620487+0000","last_active":"2024-05-08T04:40:36.899554+0000","last_peered":"2024-05-08T04:40:36.899554+0000","last_clean":"2024-05-08T04:40:36.899554+0000","last_became_active":"2024-05-08T04:40:35.620136+0000","last_became_peered":"2024-05-08T04:40:35.620136+0000","last_unstale":"2024-05-08T04:40:36.899554+0000","last_undegraded":"2024-05-08T04:40:36.899554+0000","last_fullsized":"2024-05-08T04:40:36.899554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.324262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600134+0000","last_change":"2024-05-08T04:40:34.598271+0000","last_active":"2024-05-08T04:40:36.600134+0000","last_peered":"2024-05-08T04:40:36.600134+0000","last_clean":"2024-05-08T04:40:36.600134+0000","last_became_active":"2024-05-08T04:40:34.597945+0000","last_became_peered":"2024-05-08T04:40:34.597945+0000","last_unstale":"2024-05-08T04:40:36.600134+0000","last_undegraded":"2024-05-08T04:40:36.600134+0000","last_fullsized":"2024-05-08T04:40:36.600134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:21.078485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899110+0000","last_change":"2024-05-08T04:40:35.619932+0000","last_active":"2024-05-08T04:40:36.899110+0000","last_peered":"2024-05-08T04:40:36.899110+0000","last_clean":"2024-05-08T04:40:36.899110+0000","last_became_active":"2024-05-08T04:40:35.619664+0000","last_became_peered":"2024-05-08T04:40:35.619664+0000","last_unstale":"2024-05-08T04:40:36.899110+0000","last_undegraded":"2024-05-08T04:40:36.899110+0000","last_fullsized":"2024-05-08T04:40:36.899110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:05.401749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899615+0000","last_change":"2024-05-08T04:40:34.590862+0000","last_active":"2024-05-08T04:40:36.899615+0000","last_peered":"2024-05-08T04:40:36.899615+0000","last_clean":"2024-05-08T04:40:36.899615+0000","last_became_active":"2024-05-08T04:40:34.590593+0000","last_became_peered":"2024-05-08T04:40:34.590593+0000","last_unstale":"2024-05-08T04:40:36.899615+0000","last_undegraded":"2024-05-08T04:40:36.899615+0000","last_fullsized":"2024-05-08T04:40:36.899615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:15.840633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899524+0000","last_change":"2024-05-08T04:40:35.613436+0000","last_active":"2024-05-08T04:40:36.899524+0000","last_peered":"2024-05-08T04:40:36.899524+0000","last_clean":"2024-05-08T04:40:36.899524+0000","last_became_active":"2024-05-08T04:40:35.612002+0000","last_became_peered":"2024-05-08T04:40:35.612002+0000","last_unstale":"2024-05-08T04:40:36.899524+0000","last_undegraded":"2024-05-08T04:40:36.899524+0000","last_fullsized":"2024-05-08T04:40:36.899524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:42.227926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600463+0000","last_change":"2024-05-08T04:40:34.596197+0000","last_active":"2024-05-08T04:40:36.600463+0000","last_peered":"2024-05-08T04:40:36.600463+0000","last_clean":"2024-05-08T04:40:36.600463+0000","last_became_active":"2024-05-08T04:40:34.595952+0000","last_became_peered":"2024-05-08T04:40:34.595952+0000","last_unstale":"2024-05-08T04:40:36.600463+0000","last_undegraded":"2024-05-08T04:40:36.600463+0000","last_fullsized":"2024-05-08T04:40:36.600463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.161450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601187+0000","last_change":"2024-05-08T04:40:35.621759+0000","last_active":"2024-05-08T04:40:36.601187+0000","last_peered":"2024-05-08T04:40:36.601187+0000","last_clean":"2024-05-08T04:40:36.601187+0000","last_became_active":"2024-05-08T04:40:35.621467+0000","last_became_peered":"2024-05-08T04:40:35.621467+0000","last_unstale":"2024-05-08T04:40:36.601187+0000","last_undegraded":"2024-05-08T04:40:36.601187+0000","last_fullsized":"2024-05-08T04:40:36.601187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:54.919040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600012+0000","last_change":"2024-05-08T04:40:34.598168+0000","last_active":"2024-05-08T04:40:36.600012+0000","last_peered":"2024-05-08T04:40:36.600012+0000","last_clean":"2024-05-08T04:40:36.600012+0000","last_became_active":"2024-05-08T04:40:34.597926+0000","last_became_peered":"2024-05-08T04:40:34.597926+0000","last_unstale":"2024-05-08T04:40:36.600012+0000","last_undegraded":"2024-05-08T04:40:36.600012+0000","last_fullsized":"2024-05-08T04:40:36.600012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:49.590106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899099+0000","last_change":"2024-05-08T04:40:35.605470+0000","last_active":"2024-05-08T04:40:36.899099+0000","last_peered":"2024-05-08T04:40:36.899099+0000","last_clean":"2024-05-08T04:40:36.899099+0000","last_became_active":"2024-05-08T04:40:35.605105+0000","last_became_peered":"2024-05-08T04:40:35.605105+0000","last_unstale":"2024-05-08T04:40:36.899099+0000","last_undegraded":"2024-05-08T04:40:36.899099+0000","last_fullsized":"2024-05-08T04:40:36.899099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:05.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605347+0000","last_change":"2024-05-08T04:40:34.585624+0000","last_active":"2024-05-08T04:40:36.605347+0000","last_peered":"2024-05-08T04:40:36.605347+0000","last_clean":"2024-05-08T04:40:36.605347+0000","last_became_active":"2024-05-08T04:40:34.585330+0000","last_became_peered":"2024-05-08T04:40:34.585330+0000","last_unstale":"2024-05-08T04:40:36.605347+0000","last_undegraded":"2024-05-08T04:40:36.605347+0000","last_fullsized":"2024-05-08T04:40:36.605347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:40.862844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599951+0000","last_change":"2024-05-08T04:40:35.602568+0000","last_active":"2024-05-08T04:40:36.599951+0000","last_peered":"2024-05-08T04:40:36.599951+0000","last_clean":"2024-05-08T04:40:36.599951+0000","last_became_active":"2024-05-08T04:40:35.602251+0000","last_became_peered":"2024-05-08T04:40:35.602251+0000","last_unstale":"2024-05-08T04:40:36.599951+0000","last_undegraded":"2024-05-08T04:40:36.599951+0000","last_fullsized":"2024-05-08T04:40:36.599951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:53.122303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601275+0000","last_change":"2024-05-08T04:40:34.584809+0000","last_active":"2024-05-08T04:40:36.601275+0000","last_peered":"2024-05-08T04:40:36.601275+0000","last_clean":"2024-05-08T04:40:36.601275+0000","last_became_active":"2024-05-08T04:40:34.584533+0000","last_became_peered":"2024-05-08T04:40:34.584533+0000","last_unstale":"2024-05-08T04:40:36.601275+0000","last_undegraded":"2024-05-08T04:40:36.601275+0000","last_fullsized":"2024-05-08T04:40:36.601275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:29.762914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899146+0000","last_change":"2024-05-08T04:40:35.618679+0000","last_active":"2024-05-08T04:40:36.899146+0000","last_peered":"2024-05-08T04:40:36.899146+0000","last_clean":"2024-05-08T04:40:36.899146+0000","last_became_active":"2024-05-08T04:40:35.618108+0000","last_became_peered":"2024-05-08T04:40:35.618108+0000","last_unstale":"2024-05-08T04:40:36.899146+0000","last_undegraded":"2024-05-08T04:40:36.899146+0000","last_fullsized":"2024-05-08T04:40:36.899146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:38.824580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898992+0000","last_change":"2024-05-08T04:40:34.585095+0000","last_active":"2024-05-08T04:40:36.898992+0000","last_peered":"2024-05-08T04:40:36.898992+0000","last_clean":"2024-05-08T04:40:36.898992+0000","last_became_active":"2024-05-08T04:40:34.584831+0000","last_became_peered":"2024-05-08T04:40:34.584831+0000","last_unstale":"2024-05-08T04:40:36.898992+0000","last_undegraded":"2024-05-08T04:40:36.898992+0000","last_fullsized":"2024-05-08T04:40:36.898992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:51.895822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899609+0000","last_change":"2024-05-08T04:40:35.602621+0000","last_active":"2024-05-08T04:40:36.899609+0000","last_peered":"2024-05-08T04:40:36.899609+0000","last_clean":"2024-05-08T04:40:36.899609+0000","last_became_active":"2024-05-08T04:40:35.602288+0000","last_became_peered":"2024-05-08T04:40:35.602288+0000","last_unstale":"2024-05-08T04:40:36.899609+0000","last_undegraded":"2024-05-08T04:40:36.899609+0000","last_fullsized":"2024-05-08T04:40:36.899609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:53.268007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600400+0000","last_change":"2024-05-08T04:40:34.597788+0000","last_active":"2024-05-08T04:40:36.600400+0000","last_peered":"2024-05-08T04:40:36.600400+0000","last_clean":"2024-05-08T04:40:36.600400+0000","last_became_active":"2024-05-08T04:40:34.597523+0000","last_became_peered":"2024-05-08T04:40:34.597523+0000","last_unstale":"2024-05-08T04:40:36.600400+0000","last_undegraded":"2024-05-08T04:40:36.600400+0000","last_fullsized":"2024-05-08T04:40:36.600400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:00.795776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601726+0000","last_change":"2024-05-08T04:40:35.622729+0000","last_active":"2024-05-08T04:40:36.601726+0000","last_peered":"2024-05-08T04:40:36.601726+0000","last_clean":"2024-05-08T04:40:36.601726+0000","last_became_active":"2024-05-08T04:40:35.622458+0000","last_became_peered":"2024-05-08T04:40:35.622458+0000","last_unstale":"2024-05-08T04:40:36.601726+0000","last_undegraded":"2024-05-08T04:40:36.601726+0000","last_fullsized":"2024-05-08T04:40:36.601726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:38.824580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602741+0000","last_change":"2024-05-08T04:40:34.590615+0000","last_active":"2024-05-08T04:40:36.602741+0000","last_peered":"2024-05-08T04:40:36.602741+0000","last_clean":"2024-05-08T04:40:36.602741+0000","last_became_active":"2024-05-08T04:40:34.590357+0000","last_became_peered":"2024-05-08T04:40:34.590357+0000","last_unstale":"2024-05-08T04:40:36.602741+0000","last_undegraded":"2024-05-08T04:40:36.602741+0000","last_fullsized":"2024-05-08T04:40:36.602741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:26.131502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599900+0000","last_change":"2024-05-08T04:40:35.612791+0000","last_active":"2024-05-08T04:40:36.599900+0000","last_peered":"2024-05-08T04:40:36.599900+0000","last_clean":"2024-05-08T04:40:36.599900+0000","last_became_active":"2024-05-08T04:40:35.612430+0000","last_became_peered":"2024-05-08T04:40:35.612430+0000","last_unstale":"2024-05-08T04:40:36.599900+0000","last_undegraded":"2024-05-08T04:40:36.599900+0000","last_fullsized":"2024-05-08T04:40:36.599900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:26.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601606+0000","last_change":"2024-05-08T04:40:34.589019+0000","last_active":"2024-05-08T04:40:36.601606+0000","last_peered":"2024-05-08T04:40:36.601606+0000","last_clean":"2024-05-08T04:40:36.601606+0000","last_became_active":"2024-05-08T04:40:34.588690+0000","last_became_peered":"2024-05-08T04:40:34.588690+0000","last_unstale":"2024-05-08T04:40:36.601606+0000","last_undegraded":"2024-05-08T04:40:36.601606+0000","last_fullsized":"2024-05-08T04:40:36.601606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:28.333241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899471+0000","last_change":"2024-05-08T04:40:35.613665+0000","last_active":"2024-05-08T04:40:36.899471+0000","last_peered":"2024-05-08T04:40:36.899471+0000","last_clean":"2024-05-08T04:40:36.899471+0000","last_became_active":"2024-05-08T04:40:35.612485+0000","last_became_peered":"2024-05-08T04:40:35.612485+0000","last_unstale":"2024-05-08T04:40:36.899471+0000","last_undegraded":"2024-05-08T04:40:36.899471+0000","last_fullsized":"2024-05-08T04:40:36.899471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:01.803268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602894+0000","last_change":"2024-05-08T04:40:34.600472+0000","last_active":"2024-05-08T04:40:36.602894+0000","last_peered":"2024-05-08T04:40:36.602894+0000","last_clean":"2024-05-08T04:40:36.602894+0000","last_became_active":"2024-05-08T04:40:34.600197+0000","last_became_peered":"2024-05-08T04:40:34.600197+0000","last_unstale":"2024-05-08T04:40:36.602894+0000","last_undegraded":"2024-05-08T04:40:36.602894+0000","last_fullsized":"2024-05-08T04:40:36.602894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:42:54.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601359+0000","last_change":"2024-05-08T04:40:35.621461+0000","last_active":"2024-05-08T04:40:36.601359+0000","last_peered":"2024-05-08T04:40:36.601359+0000","last_clean":"2024-05-08T04:40:36.601359+0000","last_became_active":"2024-05-08T04:40:35.621125+0000","last_became_peered":"2024-05-08T04:40:35.621125+0000","last_unstale":"2024-05-08T04:40:36.601359+0000","last_undegraded":"2024-05-08T04:40:36.601359+0000","last_fullsized":"2024-05-08T04:40:36.601359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.021416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600244+0000","last_change":"2024-05-08T04:40:34.593711+0000","last_active":"2024-05-08T04:40:36.600244+0000","last_peered":"2024-05-08T04:40:36.600244+0000","last_clean":"2024-05-08T04:40:36.600244+0000","last_became_active":"2024-05-08T04:40:34.593430+0000","last_became_peered":"2024-05-08T04:40:34.593430+0000","last_unstale":"2024-05-08T04:40:36.600244+0000","last_undegraded":"2024-05-08T04:40:36.600244+0000","last_fullsized":"2024-05-08T04:40:36.600244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:56.328356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602548+0000","last_change":"2024-05-08T04:40:35.605161+0000","last_active":"2024-05-08T04:40:36.602548+0000","last_peered":"2024-05-08T04:40:36.602548+0000","last_clean":"2024-05-08T04:40:36.602548+0000","last_became_active":"2024-05-08T04:40:35.603972+0000","last_became_peered":"2024-05-08T04:40:35.603972+0000","last_unstale":"2024-05-08T04:40:36.602548+0000","last_undegraded":"2024-05-08T04:40:36.602548+0000","last_fullsized":"2024-05-08T04:40:36.602548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.324262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898610+0000","last_change":"2024-05-08T04:40:34.587688+0000","last_active":"2024-05-08T04:40:36.898610+0000","last_peered":"2024-05-08T04:40:36.898610+0000","last_clean":"2024-05-08T04:40:36.898610+0000","last_became_active":"2024-05-08T04:40:34.586205+0000","last_became_peered":"2024-05-08T04:40:34.586205+0000","last_unstale":"2024-05-08T04:40:36.898610+0000","last_undegraded":"2024-05-08T04:40:36.898610+0000","last_fullsized":"2024-05-08T04:40:36.898610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.112943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602302+0000","last_change":"2024-05-08T04:40:35.594941+0000","last_active":"2024-05-08T04:40:36.602302+0000","last_peered":"2024-05-08T04:40:36.602302+0000","last_clean":"2024-05-08T04:40:36.602302+0000","last_became_active":"2024-05-08T04:40:35.594677+0000","last_became_peered":"2024-05-08T04:40:35.594677+0000","last_unstale":"2024-05-08T04:40:36.602302+0000","last_undegraded":"2024-05-08T04:40:36.602302+0000","last_fullsized":"2024-05-08T04:40:36.602302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:42.227926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600911+0000","last_change":"2024-05-08T04:40:34.593499+0000","last_active":"2024-05-08T04:40:36.600911+0000","last_peered":"2024-05-08T04:40:36.600911+0000","last_clean":"2024-05-08T04:40:36.600911+0000","last_became_active":"2024-05-08T04:40:34.593156+0000","last_became_peered":"2024-05-08T04:40:34.593156+0000","last_unstale":"2024-05-08T04:40:36.600911+0000","last_undegraded":"2024-05-08T04:40:36.600911+0000","last_fullsized":"2024-05-08T04:40:36.600911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:51.895822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601711+0000","last_change":"2024-05-08T04:40:35.600197+0000","last_active":"2024-05-08T04:40:36.601711+0000","last_peered":"2024-05-08T04:40:36.601711+0000","last_clean":"2024-05-08T04:40:36.601711+0000","last_became_active":"2024-05-08T04:40:35.599884+0000","last_became_peered":"2024-05-08T04:40:35.599884+0000","last_unstale":"2024-05-08T04:40:36.601711+0000","last_undegraded":"2024-05-08T04:40:36.601711+0000","last_fullsized":"2024-05-08T04:40:36.601711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:27.931796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899068+0000","last_change":"2024-05-08T04:40:34.599233+0000","last_active":"2024-05-08T04:40:36.899068+0000","last_peered":"2024-05-08T04:40:36.899068+0000","last_clean":"2024-05-08T04:40:36.899068+0000","last_became_active":"2024-05-08T04:40:34.598972+0000","last_became_peered":"2024-05-08T04:40:34.598972+0000","last_unstale":"2024-05-08T04:40:36.899068+0000","last_undegraded":"2024-05-08T04:40:36.899068+0000","last_fullsized":"2024-05-08T04:40:36.899068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:18.792909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601719+0000","last_change":"2024-05-08T04:40:35.606936+0000","last_active":"2024-05-08T04:40:36.601719+0000","last_peered":"2024-05-08T04:40:36.601719+0000","last_clean":"2024-05-08T04:40:36.601719+0000","last_became_active":"2024-05-08T04:40:35.606640+0000","last_became_peered":"2024-05-08T04:40:35.606640+0000","last_unstale":"2024-05-08T04:40:36.601719+0000","last_undegraded":"2024-05-08T04:40:36.601719+0000","last_fullsized":"2024-05-08T04:40:36.601719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:46:43.291290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.604508+0000","last_change":"2024-05-08T04:40:34.592667+0000","last_active":"2024-05-08T04:40:36.604508+0000","last_peered":"2024-05-08T04:40:36.604508+0000","last_clean":"2024-05-08T04:40:36.604508+0000","last_became_active":"2024-05-08T04:40:34.591674+0000","last_became_peered":"2024-05-08T04:40:34.591674+0000","last_unstale":"2024-05-08T04:40:36.604508+0000","last_undegraded":"2024-05-08T04:40:36.604508+0000","last_fullsized":"2024-05-08T04:40:36.604508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:00.801964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898968+0000","last_change":"2024-05-08T04:40:35.621359+0000","last_active":"2024-05-08T04:40:36.898968+0000","last_peered":"2024-05-08T04:40:36.898968+0000","last_clean":"2024-05-08T04:40:36.898968+0000","last_became_active":"2024-05-08T04:40:35.621154+0000","last_became_peered":"2024-05-08T04:40:35.621154+0000","last_unstale":"2024-05-08T04:40:36.898968+0000","last_undegraded":"2024-05-08T04:40:36.898968+0000","last_fullsized":"2024-05-08T04:40:36.898968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:19.456105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605587+0000","last_change":"2024-05-08T04:40:34.588114+0000","last_active":"2024-05-08T04:40:36.605587+0000","last_peered":"2024-05-08T04:40:36.605587+0000","last_clean":"2024-05-08T04:40:36.605587+0000","last_became_active":"2024-05-08T04:40:34.587831+0000","last_became_peered":"2024-05-08T04:40:34.587831+0000","last_unstale":"2024-05-08T04:40:36.605587+0000","last_undegraded":"2024-05-08T04:40:36.605587+0000","last_fullsized":"2024-05-08T04:40:36.605587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:44.449340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602789+0000","last_change":"2024-05-08T04:40:35.599281+0000","last_active":"2024-05-08T04:40:36.602789+0000","last_peered":"2024-05-08T04:40:36.602789+0000","last_clean":"2024-05-08T04:40:36.602789+0000","last_became_active":"2024-05-08T04:40:35.598619+0000","last_became_peered":"2024-05-08T04:40:35.598619+0000","last_unstale":"2024-05-08T04:40:36.602789+0000","last_undegraded":"2024-05-08T04:40:36.602789+0000","last_fullsized":"2024-05-08T04:40:36.602789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:49.474365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601540+0000","last_change":"2024-05-08T04:40:34.587738+0000","last_active":"2024-05-08T04:40:36.601540+0000","last_peered":"2024-05-08T04:40:36.601540+0000","last_clean":"2024-05-08T04:40:36.601540+0000","last_became_active":"2024-05-08T04:40:34.587464+0000","last_became_peered":"2024-05-08T04:40:34.587464+0000","last_unstale":"2024-05-08T04:40:36.601540+0000","last_undegraded":"2024-05-08T04:40:36.601540+0000","last_fullsized":"2024-05-08T04:40:36.601540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:15.840633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599907+0000","last_change":"2024-05-08T04:40:35.609632+0000","last_active":"2024-05-08T04:40:36.599907+0000","last_peered":"2024-05-08T04:40:36.599907+0000","last_clean":"2024-05-08T04:40:36.599907+0000","last_became_active":"2024-05-08T04:40:35.608561+0000","last_became_peered":"2024-05-08T04:40:35.608561+0000","last_unstale":"2024-05-08T04:40:36.599907+0000","last_undegraded":"2024-05-08T04:40:36.599907+0000","last_fullsized":"2024-05-08T04:40:36.599907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:18.103016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898629+0000","last_change":"2024-05-08T04:40:34.587522+0000","last_active":"2024-05-08T04:40:36.898629+0000","last_peered":"2024-05-08T04:40:36.898629+0000","last_clean":"2024-05-08T04:40:36.898629+0000","last_became_active":"2024-05-08T04:40:34.587224+0000","last_became_peered":"2024-05-08T04:40:34.587224+0000","last_unstale":"2024-05-08T04:40:36.898629+0000","last_undegraded":"2024-05-08T04:40:36.898629+0000","last_fullsized":"2024-05-08T04:40:36.898629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:31.556397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899349+0000","last_change":"2024-05-08T04:40:35.617761+0000","last_active":"2024-05-08T04:40:36.899349+0000","last_peered":"2024-05-08T04:40:36.899349+0000","last_clean":"2024-05-08T04:40:36.899349+0000","last_became_active":"2024-05-08T04:40:35.617469+0000","last_became_peered":"2024-05-08T04:40:35.617469+0000","last_unstale":"2024-05-08T04:40:36.899349+0000","last_undegraded":"2024-05-08T04:40:36.899349+0000","last_fullsized":"2024-05-08T04:40:36.899349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:57.335849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600190+0000","last_change":"2024-05-08T04:40:34.596968+0000","last_active":"2024-05-08T04:40:36.600190+0000","last_peered":"2024-05-08T04:40:36.600190+0000","last_clean":"2024-05-08T04:40:36.600190+0000","last_became_active":"2024-05-08T04:40:34.596581+0000","last_became_peered":"2024-05-08T04:40:34.596581+0000","last_unstale":"2024-05-08T04:40:36.600190+0000","last_undegraded":"2024-05-08T04:40:36.600190+0000","last_fullsized":"2024-05-08T04:40:36.600190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:25.021274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898328+0000","last_change":"2024-05-08T04:40:35.602339+0000","last_active":"2024-05-08T04:40:36.898328+0000","last_peered":"2024-05-08T04:40:36.898328+0000","last_clean":"2024-05-08T04:40:36.898328+0000","last_became_active":"2024-05-08T04:40:35.602081+0000","last_became_peered":"2024-05-08T04:40:35.602081+0000","last_unstale":"2024-05-08T04:40:36.898328+0000","last_undegraded":"2024-05-08T04:40:36.898328+0000","last_fullsized":"2024-05-08T04:40:36.898328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:06.052427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898436+0000","last_change":"2024-05-08T04:40:34.595685+0000","last_active":"2024-05-08T04:40:36.898436+0000","last_peered":"2024-05-08T04:40:36.898436+0000","last_clean":"2024-05-08T04:40:36.898436+0000","last_became_active":"2024-05-08T04:40:34.595382+0000","last_became_peered":"2024-05-08T04:40:34.595382+0000","last_unstale":"2024-05-08T04:40:36.898436+0000","last_undegraded":"2024-05-08T04:40:36.898436+0000","last_fullsized":"2024-05-08T04:40:36.898436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:30.768476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599610+0000","last_change":"2024-05-08T04:40:35.613111+0000","last_active":"2024-05-08T04:40:36.599610+0000","last_peered":"2024-05-08T04:40:36.599610+0000","last_clean":"2024-05-08T04:40:36.599610+0000","last_became_active":"2024-05-08T04:40:35.610355+0000","last_became_peered":"2024-05-08T04:40:35.610355+0000","last_unstale":"2024-05-08T04:40:36.599610+0000","last_undegraded":"2024-05-08T04:40:36.599610+0000","last_fullsized":"2024-05-08T04:40:36.599610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:47.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599453+0000","last_change":"2024-05-08T04:40:34.592173+0000","last_active":"2024-05-08T04:40:36.599453+0000","last_peered":"2024-05-08T04:40:36.599453+0000","last_clean":"2024-05-08T04:40:36.599453+0000","last_became_active":"2024-05-08T04:40:34.588946+0000","last_became_peered":"2024-05-08T04:40:34.588946+0000","last_unstale":"2024-05-08T04:40:36.599453+0000","last_undegraded":"2024-05-08T04:40:36.599453+0000","last_fullsized":"2024-05-08T04:40:36.599453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:37.817087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599344+0000","last_change":"2024-05-08T04:40:35.607436+0000","last_active":"2024-05-08T04:40:36.599344+0000","last_peered":"2024-05-08T04:40:36.599344+0000","last_clean":"2024-05-08T04:40:36.599344+0000","last_became_active":"2024-05-08T04:40:35.607141+0000","last_became_peered":"2024-05-08T04:40:35.607141+0000","last_unstale":"2024-05-08T04:40:36.599344+0000","last_undegraded":"2024-05-08T04:40:36.599344+0000","last_fullsized":"2024-05-08T04:40:36.599344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:43.108589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605445+0000","last_change":"2024-05-08T04:40:34.592421+0000","last_active":"2024-05-08T04:40:36.605445+0000","last_peered":"2024-05-08T04:40:36.605445+0000","last_clean":"2024-05-08T04:40:36.605445+0000","last_became_active":"2024-05-08T04:40:34.590067+0000","last_became_peered":"2024-05-08T04:40:34.590067+0000","last_unstale":"2024-05-08T04:40:36.605445+0000","last_undegraded":"2024-05-08T04:40:36.605445+0000","last_fullsized":"2024-05-08T04:40:36.605445+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:45.643834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898878+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:40:36.898878+0000","last_peered":"2024-05-08T04:40:36.898878+0000","last_clean":"2024-05-08T04:40:36.898878+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:40:36.898878+0000","last_undegraded":"2024-05-08T04:40:36.898878+0000","last_fullsized":"2024-05-08T04:40:36.898878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1323008,"data_stored":893086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":296230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":729088,"data_stored":303210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":304170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":302614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":302614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":296320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":895120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:40:39.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:40:39.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.841+0000 7fd5ddc00640 1 Processor -- start 2024-05-08T04:40:39.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.841+0000 7fd5ddc00640 1 -- start start 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ddc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5d80cfda0 0x7fd5d8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ddc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d80cd9f0 con 0x7fd5d815be50 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ddc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d80cdcd0 con 0x7fd5d815ba70 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ddc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d815fe70 con 0x7fd5d80cfda0 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d7400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd5d815be50 0x7fd5d80b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37010/0 (socket says 172.21.15.164:37010) 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d7400640 1 -- 172.21.15.164:0/218768934 learned_addr learned my addr 172.21.15.164:0/218768934 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:39.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ce000640 1 --2- 172.21.15.164:0/218768934 >> v2:172.21.15.172:3300/0 conn(0x7fd5d80cfda0 0x7fd5d8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321222760 0 0) 0x7fd5d80cd9f0 con 0x7fd5d815be50 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc003530 con 0x7fd5d815be50 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551849574 0 0) 0x7fd5d80cdcd0 con 0x7fd5d815ba70 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d80cd9f0 con 0x7fd5d815ba70 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ce000640 1 -- 172.21.15.164:0/218768934 >> v1:172.21.15.172:6789/0 conn(0x7fd5d815ba70 legacy=0x7fd5d80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ce000640 1 -- 172.21.15.164:0/218768934 >> v1:172.21.15.164:6789/0 conn(0x7fd5d815be50 legacy=0x7fd5d80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001656867 0 0) 0x7fd5bc003530 con 0x7fd5d815be50 2024-05-08T04:40:39.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.845+0000 7fd5ce000640 1 -- 172.21.15.164:0/218768934 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d815f820 con 0x7fd5d80cfda0 2024-05-08T04:40:39.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ce000640 1 --2- 172.21.15.164:0/218768934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d8160300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097c0 tx=0x7fd5c001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6393415f2331877 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00b0070 con 0x7fd5d80cfda0 2024-05-08T04:40:39.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c00ab400 con 0x7fd5d80cfda0 2024-05-08T04:40:39.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5d6000640 1 -- 172.21.15.164:0/218768934 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0001680 con 0x7fd5d80cfda0 2024-05-08T04:40:39.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- 172.21.15.164:0/218768934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 msgr2=0x7fd5d8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/218768934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d8160300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097c0 tx=0x7fd5c001c390 comp rx=0 tx=0).stop 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- 172.21.15.164:0/218768934 shutdown_connections 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/218768934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d8160300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- 172.21.15.164:0/218768934 >> 172.21.15.164:0/218768934 conn(0x7fd5d806ac90 msgr2=0x7fd5d806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- 172.21.15.164:0/218768934 shutdown_connections 2024-05-08T04:40:39.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- 172.21.15.164:0/218768934 wait complete. 2024-05-08T04:40:39.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 Processor -- start 2024-05-08T04:40:39.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.849+0000 7fd5ddc00640 1 -- start start 2024-05-08T04:40:39.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5d815ba70 0x7fd5d814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8169f60 con 0x7fd5d815be50 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8169de0 con 0x7fd5d80cfda0 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d816a0e0 con 0x7fd5d815ba70 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46214/0 (socket says 172.21.15.164:46214) 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 -- 172.21.15.164:0/178901174 learned_addr learned my addr 172.21.15.164:0/178901174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d6a00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ce000640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5d815ba70 0x7fd5d814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 -- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5d815ba70 msgr2=0x7fd5d814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5d815ba70 0x7fd5d814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 -- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 msgr2=0x7fd5d813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 -- 172.21.15.164:0/178901174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d814cea0 con 0x7fd5d815be50 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5d7400640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c520 tx=0x7fd5d000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0017070 con 0x7fd5d815be50 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0005cf0 con 0x7fd5d815be50 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d814d140 con 0x7fd5d815be50 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0007290 con 0x7fd5d815be50 2024-05-08T04:40:39.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d814d700 con 0x7fd5d815be50 2024-05-08T04:40:39.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.853+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d8067820 con 0x7fd5d815be50 2024-05-08T04:40:39.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.857+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fd5d001a070 con 0x7fd5d815be50 2024-05-08T04:40:39.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.857+0000 7fd5cf400640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 0x7fd5b4069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:39.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.857+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd5d0084970 con 0x7fd5d815be50 2024-05-08T04:40:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.857+0000 7fd5d6a00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 0x7fd5b4069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.857+0000 7fd5d6a00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 0x7fd5b4069660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001570 tx=0x7fd5c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:39.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:39.861+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5d0054e90 con 0x7fd5d815be50 2024-05-08T04:40:40.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.013+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5d8069d60 con 0x7fd5b40671a0 2024-05-08T04:40:40.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.021+0000 7fd5cf400640 1 -- 172.21.15.164:0/178901174 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7fd5d8069d60 con 0x7fd5b40671a0 2024-05-08T04:40:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:40.028 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 msgr2=0x7fd5b4069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 0x7fd5b4069660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001570 tx=0x7fd5c8009040 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 msgr2=0x7fd5d814e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c520 tx=0x7fd5d000c9e0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 shutdown_connections 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd5b40671a0 0x7fd5b4069660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5d815ba70 0x7fd5d814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d815be50 0x7fd5d814e6b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 --2- 172.21.15.164:0/178901174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d80cfda0 0x7fd5d813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 >> 172.21.15.164:0/178901174 conn(0x7fd5d806ac90 msgr2=0x7fd5d806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:40.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 shutdown_connections 2024-05-08T04:40:40.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.025+0000 7fd5ddc00640 1 -- 172.21.15.164:0/178901174 wait complete. 2024-05-08T04:40:40.047 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-05-08T04:40:39.456711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227176,"kb_used_data":6832,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747544,"statfs":{"total":773094113280,"available":772861485056,"internally_reserved":0,"allocated":6995968,"data_stored":3593364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.891369"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898982+0000","last_change":"2024-05-08T04:40:35.611583+0000","last_active":"2024-05-08T04:40:36.898982+0000","last_peered":"2024-05-08T04:40:36.898982+0000","last_clean":"2024-05-08T04:40:36.898982+0000","last_became_active":"2024-05-08T04:40:35.611306+0000","last_became_peered":"2024-05-08T04:40:35.611306+0000","last_unstale":"2024-05-08T04:40:36.898982+0000","last_undegraded":"2024-05-08T04:40:36.898982+0000","last_fullsized":"2024-05-08T04:40:36.898982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.907283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599872+0000","last_change":"2024-05-08T04:40:35.623953+0000","last_active":"2024-05-08T04:40:36.599872+0000","last_peered":"2024-05-08T04:40:36.599872+0000","last_clean":"2024-05-08T04:40:36.599872+0000","last_became_active":"2024-05-08T04:40:35.623589+0000","last_became_peered":"2024-05-08T04:40:35.623589+0000","last_unstale":"2024-05-08T04:40:36.599872+0000","last_undegraded":"2024-05-08T04:40:36.599872+0000","last_fullsized":"2024-05-08T04:40:36.599872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:10:34.811364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899281+0000","last_change":"2024-05-08T04:40:35.604418+0000","last_active":"2024-05-08T04:40:36.899281+0000","last_peered":"2024-05-08T04:40:36.899281+0000","last_clean":"2024-05-08T04:40:36.899281+0000","last_became_active":"2024-05-08T04:40:35.602188+0000","last_became_peered":"2024-05-08T04:40:35.602188+0000","last_unstale":"2024-05-08T04:40:36.899281+0000","last_undegraded":"2024-05-08T04:40:36.899281+0000","last_fullsized":"2024-05-08T04:40:36.899281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:03.198334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601195+0000","last_change":"2024-05-08T04:40:35.599410+0000","last_active":"2024-05-08T04:40:36.601195+0000","last_peered":"2024-05-08T04:40:36.601195+0000","last_clean":"2024-05-08T04:40:36.601195+0000","last_became_active":"2024-05-08T04:40:35.599115+0000","last_became_peered":"2024-05-08T04:40:35.599115+0000","last_unstale":"2024-05-08T04:40:36.601195+0000","last_undegraded":"2024-05-08T04:40:36.601195+0000","last_fullsized":"2024-05-08T04:40:36.601195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:42.695403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898622+0000","last_change":"2024-05-08T04:40:35.609307+0000","last_active":"2024-05-08T04:40:36.898622+0000","last_peered":"2024-05-08T04:40:36.898622+0000","last_clean":"2024-05-08T04:40:36.898622+0000","last_became_active":"2024-05-08T04:40:35.609091+0000","last_became_peered":"2024-05-08T04:40:35.609091+0000","last_unstale":"2024-05-08T04:40:36.898622+0000","last_undegraded":"2024-05-08T04:40:36.898622+0000","last_fullsized":"2024-05-08T04:40:36.898622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:20.641727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602285+0000","last_change":"2024-05-08T04:40:35.596918+0000","last_active":"2024-05-08T04:40:36.602285+0000","last_peered":"2024-05-08T04:40:36.602285+0000","last_clean":"2024-05-08T04:40:36.602285+0000","last_became_active":"2024-05-08T04:40:35.596634+0000","last_became_peered":"2024-05-08T04:40:35.596634+0000","last_unstale":"2024-05-08T04:40:36.602285+0000","last_undegraded":"2024-05-08T04:40:36.602285+0000","last_fullsized":"2024-05-08T04:40:36.602285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:22.085977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599872+0000","last_change":"2024-05-08T04:40:35.598093+0000","last_active":"2024-05-08T04:40:36.599872+0000","last_peered":"2024-05-08T04:40:36.599872+0000","last_clean":"2024-05-08T04:40:36.599872+0000","last_became_active":"2024-05-08T04:40:35.597827+0000","last_became_peered":"2024-05-08T04:40:35.597827+0000","last_unstale":"2024-05-08T04:40:36.599872+0000","last_undegraded":"2024-05-08T04:40:36.599872+0000","last_fullsized":"2024-05-08T04:40:36.599872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:29.340734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599344+0000","last_change":"2024-05-08T04:40:35.612889+0000","last_active":"2024-05-08T04:40:36.599344+0000","last_peered":"2024-05-08T04:40:36.599344+0000","last_clean":"2024-05-08T04:40:36.599344+0000","last_became_active":"2024-05-08T04:40:35.612567+0000","last_became_peered":"2024-05-08T04:40:35.612567+0000","last_unstale":"2024-05-08T04:40:36.599344+0000","last_undegraded":"2024-05-08T04:40:36.599344+0000","last_fullsized":"2024-05-08T04:40:36.599344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:24.028947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600765+0000","last_change":"2024-05-08T04:40:35.603549+0000","last_active":"2024-05-08T04:40:36.600765+0000","last_peered":"2024-05-08T04:40:36.600765+0000","last_clean":"2024-05-08T04:40:36.600765+0000","last_became_active":"2024-05-08T04:40:35.598125+0000","last_became_peered":"2024-05-08T04:40:35.598125+0000","last_unstale":"2024-05-08T04:40:36.600765+0000","last_undegraded":"2024-05-08T04:40:36.600765+0000","last_fullsized":"2024-05-08T04:40:36.600765+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:37.494892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898915+0000","last_change":"2024-05-08T04:40:35.611098+0000","last_active":"2024-05-08T04:40:36.898915+0000","last_peered":"2024-05-08T04:40:36.898915+0000","last_clean":"2024-05-08T04:40:36.898915+0000","last_became_active":"2024-05-08T04:40:35.610380+0000","last_became_peered":"2024-05-08T04:40:35.610380+0000","last_unstale":"2024-05-08T04:40:36.898915+0000","last_undegraded":"2024-05-08T04:40:36.898915+0000","last_fullsized":"2024-05-08T04:40:36.898915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:33.168942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898995+0000","last_change":"2024-05-08T04:40:35.618617+0000","last_active":"2024-05-08T04:40:36.898995+0000","last_peered":"2024-05-08T04:40:36.898995+0000","last_clean":"2024-05-08T04:40:36.898995+0000","last_became_active":"2024-05-08T04:40:35.618438+0000","last_became_peered":"2024-05-08T04:40:35.618438+0000","last_unstale":"2024-05-08T04:40:36.898995+0000","last_undegraded":"2024-05-08T04:40:36.898995+0000","last_fullsized":"2024-05-08T04:40:36.898995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:01.802314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601454+0000","last_change":"2024-05-08T04:40:35.606582+0000","last_active":"2024-05-08T04:40:36.601454+0000","last_peered":"2024-05-08T04:40:36.601454+0000","last_clean":"2024-05-08T04:40:36.601454+0000","last_became_active":"2024-05-08T04:40:35.606211+0000","last_became_peered":"2024-05-08T04:40:35.606211+0000","last_unstale":"2024-05-08T04:40:36.601454+0000","last_undegraded":"2024-05-08T04:40:36.601454+0000","last_fullsized":"2024-05-08T04:40:36.601454+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:52.918881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605605+0000","last_change":"2024-05-08T04:40:35.602441+0000","last_active":"2024-05-08T04:40:36.605605+0000","last_peered":"2024-05-08T04:40:36.605605+0000","last_clean":"2024-05-08T04:40:36.605605+0000","last_became_active":"2024-05-08T04:40:35.601934+0000","last_became_peered":"2024-05-08T04:40:35.601934+0000","last_unstale":"2024-05-08T04:40:36.605605+0000","last_undegraded":"2024-05-08T04:40:36.605605+0000","last_fullsized":"2024-05-08T04:40:36.605605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:26.028767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600210+0000","last_change":"2024-05-08T04:40:35.598805+0000","last_active":"2024-05-08T04:40:36.600210+0000","last_peered":"2024-05-08T04:40:36.600210+0000","last_clean":"2024-05-08T04:40:36.600210+0000","last_became_active":"2024-05-08T04:40:35.598497+0000","last_became_peered":"2024-05-08T04:40:35.598497+0000","last_unstale":"2024-05-08T04:40:36.600210+0000","last_undegraded":"2024-05-08T04:40:36.600210+0000","last_fullsized":"2024-05-08T04:40:36.600210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:35.157666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600169+0000","last_change":"2024-05-08T04:40:35.603457+0000","last_active":"2024-05-08T04:40:36.600169+0000","last_peered":"2024-05-08T04:40:36.600169+0000","last_clean":"2024-05-08T04:40:36.600169+0000","last_became_active":"2024-05-08T04:40:35.603124+0000","last_became_peered":"2024-05-08T04:40:35.603124+0000","last_unstale":"2024-05-08T04:40:36.600169+0000","last_undegraded":"2024-05-08T04:40:36.600169+0000","last_fullsized":"2024-05-08T04:40:36.600169+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:42.983311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602302+0000","last_change":"2024-05-08T04:40:35.601731+0000","last_active":"2024-05-08T04:40:36.602302+0000","last_peered":"2024-05-08T04:40:36.602302+0000","last_clean":"2024-05-08T04:40:36.602302+0000","last_became_active":"2024-05-08T04:40:35.601366+0000","last_became_peered":"2024-05-08T04:40:35.601366+0000","last_unstale":"2024-05-08T04:40:36.602302+0000","last_undegraded":"2024-05-08T04:40:36.602302+0000","last_fullsized":"2024-05-08T04:40:36.602302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:01.803268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600241+0000","last_change":"2024-05-08T04:40:35.612272+0000","last_active":"2024-05-08T04:40:36.600241+0000","last_peered":"2024-05-08T04:40:36.600241+0000","last_clean":"2024-05-08T04:40:36.600241+0000","last_became_active":"2024-05-08T04:40:35.608761+0000","last_became_peered":"2024-05-08T04:40:35.608761+0000","last_unstale":"2024-05-08T04:40:36.600241+0000","last_undegraded":"2024-05-08T04:40:36.600241+0000","last_fullsized":"2024-05-08T04:40:36.600241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:22.363547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899457+0000","last_change":"2024-05-08T04:40:35.601871+0000","last_active":"2024-05-08T04:40:36.899457+0000","last_peered":"2024-05-08T04:40:36.899457+0000","last_clean":"2024-05-08T04:40:36.899457+0000","last_became_active":"2024-05-08T04:40:35.601571+0000","last_became_peered":"2024-05-08T04:40:35.601571+0000","last_unstale":"2024-05-08T04:40:36.899457+0000","last_undegraded":"2024-05-08T04:40:36.899457+0000","last_fullsized":"2024-05-08T04:40:36.899457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:22.085977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898625+0000","last_change":"2024-05-08T04:40:35.617954+0000","last_active":"2024-05-08T04:40:36.898625+0000","last_peered":"2024-05-08T04:40:36.898625+0000","last_clean":"2024-05-08T04:40:36.898625+0000","last_became_active":"2024-05-08T04:40:35.617691+0000","last_became_peered":"2024-05-08T04:40:35.617691+0000","last_unstale":"2024-05-08T04:40:36.898625+0000","last_undegraded":"2024-05-08T04:40:36.898625+0000","last_fullsized":"2024-05-08T04:40:36.898625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:29:11.001448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599821+0000","last_change":"2024-05-08T04:40:35.613721+0000","last_active":"2024-05-08T04:40:36.599821+0000","last_peered":"2024-05-08T04:40:36.599821+0000","last_clean":"2024-05-08T04:40:36.599821+0000","last_became_active":"2024-05-08T04:40:35.613310+0000","last_became_peered":"2024-05-08T04:40:35.613310+0000","last_unstale":"2024-05-08T04:40:36.599821+0000","last_undegraded":"2024-05-08T04:40:36.599821+0000","last_fullsized":"2024-05-08T04:40:36.599821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:24.735268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899034+0000","last_change":"2024-05-08T04:40:35.608182+0000","last_active":"2024-05-08T04:40:36.899034+0000","last_peered":"2024-05-08T04:40:36.899034+0000","last_clean":"2024-05-08T04:40:36.899034+0000","last_became_active":"2024-05-08T04:40:35.607909+0000","last_became_peered":"2024-05-08T04:40:35.607909+0000","last_unstale":"2024-05-08T04:40:36.899034+0000","last_undegraded":"2024-05-08T04:40:36.899034+0000","last_fullsized":"2024-05-08T04:40:36.899034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:46:43.291290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599613+0000","last_change":"2024-05-08T04:40:35.610112+0000","last_active":"2024-05-08T04:40:36.599613+0000","last_peered":"2024-05-08T04:40:36.599613+0000","last_clean":"2024-05-08T04:40:36.599613+0000","last_became_active":"2024-05-08T04:40:35.609344+0000","last_became_peered":"2024-05-08T04:40:35.609344+0000","last_unstale":"2024-05-08T04:40:36.599613+0000","last_undegraded":"2024-05-08T04:40:36.599613+0000","last_fullsized":"2024-05-08T04:40:36.599613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:16.603914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899157+0000","last_change":"2024-05-08T04:40:35.618464+0000","last_active":"2024-05-08T04:40:36.899157+0000","last_peered":"2024-05-08T04:40:36.899157+0000","last_clean":"2024-05-08T04:40:36.899157+0000","last_became_active":"2024-05-08T04:40:35.617795+0000","last_became_peered":"2024-05-08T04:40:35.617795+0000","last_unstale":"2024-05-08T04:40:36.899157+0000","last_undegraded":"2024-05-08T04:40:36.899157+0000","last_fullsized":"2024-05-08T04:40:36.899157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:50.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605539+0000","last_change":"2024-05-08T04:40:35.601191+0000","last_active":"2024-05-08T04:40:36.605539+0000","last_peered":"2024-05-08T04:40:36.605539+0000","last_clean":"2024-05-08T04:40:36.605539+0000","last_became_active":"2024-05-08T04:40:35.600825+0000","last_became_peered":"2024-05-08T04:40:35.600825+0000","last_unstale":"2024-05-08T04:40:36.605539+0000","last_undegraded":"2024-05-08T04:40:36.605539+0000","last_fullsized":"2024-05-08T04:40:36.605539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:01.802314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899163+0000","last_change":"2024-05-08T04:40:35.615715+0000","last_active":"2024-05-08T04:40:36.899163+0000","last_peered":"2024-05-08T04:40:36.899163+0000","last_clean":"2024-05-08T04:40:36.899163+0000","last_became_active":"2024-05-08T04:40:35.615471+0000","last_became_peered":"2024-05-08T04:40:35.615471+0000","last_unstale":"2024-05-08T04:40:36.899163+0000","last_undegraded":"2024-05-08T04:40:36.899163+0000","last_fullsized":"2024-05-08T04:40:36.899163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:00.699214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601728+0000","last_change":"2024-05-08T04:40:35.601742+0000","last_active":"2024-05-08T04:40:36.601728+0000","last_peered":"2024-05-08T04:40:36.601728+0000","last_clean":"2024-05-08T04:40:36.601728+0000","last_became_active":"2024-05-08T04:40:35.601378+0000","last_became_peered":"2024-05-08T04:40:35.601378+0000","last_unstale":"2024-05-08T04:40:36.601728+0000","last_undegraded":"2024-05-08T04:40:36.601728+0000","last_fullsized":"2024-05-08T04:40:36.601728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:05.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602651+0000","last_change":"2024-05-08T04:40:35.603124+0000","last_active":"2024-05-08T04:40:36.602651+0000","last_peered":"2024-05-08T04:40:36.602651+0000","last_clean":"2024-05-08T04:40:36.602651+0000","last_became_active":"2024-05-08T04:40:35.602781+0000","last_became_peered":"2024-05-08T04:40:35.602781+0000","last_unstale":"2024-05-08T04:40:36.602651+0000","last_undegraded":"2024-05-08T04:40:36.602651+0000","last_fullsized":"2024-05-08T04:40:36.602651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:53.268007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898600+0000","last_change":"2024-05-08T04:40:35.605408+0000","last_active":"2024-05-08T04:40:36.898600+0000","last_peered":"2024-05-08T04:40:36.898600+0000","last_clean":"2024-05-08T04:40:36.898600+0000","last_became_active":"2024-05-08T04:40:35.605048+0000","last_became_peered":"2024-05-08T04:40:35.605048+0000","last_unstale":"2024-05-08T04:40:36.898600+0000","last_undegraded":"2024-05-08T04:40:36.898600+0000","last_fullsized":"2024-05-08T04:40:36.898600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:02.084663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600216+0000","last_change":"2024-05-08T04:40:35.609883+0000","last_active":"2024-05-08T04:40:36.600216+0000","last_peered":"2024-05-08T04:40:36.600216+0000","last_clean":"2024-05-08T04:40:36.600216+0000","last_became_active":"2024-05-08T04:40:35.609239+0000","last_became_peered":"2024-05-08T04:40:35.609239+0000","last_unstale":"2024-05-08T04:40:36.600216+0000","last_undegraded":"2024-05-08T04:40:36.600216+0000","last_fullsized":"2024-05-08T04:40:36.600216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:25:56.900214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899030+0000","last_change":"2024-05-08T04:40:35.603619+0000","last_active":"2024-05-08T04:40:36.899030+0000","last_peered":"2024-05-08T04:40:36.899030+0000","last_clean":"2024-05-08T04:40:36.899030+0000","last_became_active":"2024-05-08T04:40:35.603284+0000","last_became_peered":"2024-05-08T04:40:35.603284+0000","last_unstale":"2024-05-08T04:40:36.899030+0000","last_undegraded":"2024-05-08T04:40:36.899030+0000","last_fullsized":"2024-05-08T04:40:36.899030+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:27.931796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898248+0000","last_change":"2024-05-08T04:40:35.606994+0000","last_active":"2024-05-08T04:40:36.898248+0000","last_peered":"2024-05-08T04:40:36.898248+0000","last_clean":"2024-05-08T04:40:36.898248+0000","last_became_active":"2024-05-08T04:40:35.604475+0000","last_became_peered":"2024-05-08T04:40:35.604475+0000","last_unstale":"2024-05-08T04:40:36.898248+0000","last_undegraded":"2024-05-08T04:40:36.898248+0000","last_fullsized":"2024-05-08T04:40:36.898248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:22.065594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602267+0000","last_change":"2024-05-08T04:40:35.617966+0000","last_active":"2024-05-08T04:40:36.602267+0000","last_peered":"2024-05-08T04:40:36.602267+0000","last_clean":"2024-05-08T04:40:36.602267+0000","last_became_active":"2024-05-08T04:40:35.617707+0000","last_became_peered":"2024-05-08T04:40:35.617707+0000","last_unstale":"2024-05-08T04:40:36.602267+0000","last_undegraded":"2024-05-08T04:40:36.602267+0000","last_fullsized":"2024-05-08T04:40:36.602267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:57.335849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.603026+0000","last_change":"2024-05-08T04:40:35.598013+0000","last_active":"2024-05-08T04:40:36.603026+0000","last_peered":"2024-05-08T04:40:36.603026+0000","last_clean":"2024-05-08T04:40:36.603026+0000","last_became_active":"2024-05-08T04:40:35.597707+0000","last_became_peered":"2024-05-08T04:40:35.597707+0000","last_unstale":"2024-05-08T04:40:36.603026+0000","last_undegraded":"2024-05-08T04:40:36.603026+0000","last_fullsized":"2024-05-08T04:40:36.603026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.404913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602787+0000","last_change":"2024-05-08T04:40:34.592658+0000","last_active":"2024-05-08T04:40:36.602787+0000","last_peered":"2024-05-08T04:40:36.602787+0000","last_clean":"2024-05-08T04:40:36.602787+0000","last_became_active":"2024-05-08T04:40:34.592398+0000","last_became_peered":"2024-05-08T04:40:34.592398+0000","last_unstale":"2024-05-08T04:40:36.602787+0000","last_undegraded":"2024-05-08T04:40:36.602787+0000","last_fullsized":"2024-05-08T04:40:36.602787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:36.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899055+0000","last_change":"2024-05-08T04:40:35.610732+0000","last_active":"2024-05-08T04:40:36.899055+0000","last_peered":"2024-05-08T04:40:36.899055+0000","last_clean":"2024-05-08T04:40:36.899055+0000","last_became_active":"2024-05-08T04:40:35.610448+0000","last_became_peered":"2024-05-08T04:40:35.610448+0000","last_unstale":"2024-05-08T04:40:36.899055+0000","last_undegraded":"2024-05-08T04:40:36.899055+0000","last_fullsized":"2024-05-08T04:40:36.899055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:26.028767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600012+0000","last_change":"2024-05-08T04:40:34.598869+0000","last_active":"2024-05-08T04:40:36.600012+0000","last_peered":"2024-05-08T04:40:36.600012+0000","last_clean":"2024-05-08T04:40:36.600012+0000","last_became_active":"2024-05-08T04:40:34.598616+0000","last_became_peered":"2024-05-08T04:40:34.598616+0000","last_unstale":"2024-05-08T04:40:36.600012+0000","last_undegraded":"2024-05-08T04:40:36.600012+0000","last_fullsized":"2024-05-08T04:40:36.600012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:02.190842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602707+0000","last_change":"2024-05-08T04:40:35.623259+0000","last_active":"2024-05-08T04:40:36.602707+0000","last_peered":"2024-05-08T04:40:36.602707+0000","last_clean":"2024-05-08T04:40:36.602707+0000","last_became_active":"2024-05-08T04:40:35.622905+0000","last_became_peered":"2024-05-08T04:40:35.622905+0000","last_unstale":"2024-05-08T04:40:36.602707+0000","last_undegraded":"2024-05-08T04:40:36.602707+0000","last_fullsized":"2024-05-08T04:40:36.602707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:13.065627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602802+0000","last_change":"2024-05-08T04:40:34.595613+0000","last_active":"2024-05-08T04:40:36.602802+0000","last_peered":"2024-05-08T04:40:36.602802+0000","last_clean":"2024-05-08T04:40:36.602802+0000","last_became_active":"2024-05-08T04:40:34.595245+0000","last_became_peered":"2024-05-08T04:40:34.595245+0000","last_unstale":"2024-05-08T04:40:36.602802+0000","last_undegraded":"2024-05-08T04:40:36.602802+0000","last_fullsized":"2024-05-08T04:40:36.602802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:32.793521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602700+0000","last_change":"2024-05-08T04:40:35.602040+0000","last_active":"2024-05-08T04:40:36.602700+0000","last_peered":"2024-05-08T04:40:36.602700+0000","last_clean":"2024-05-08T04:40:36.602700+0000","last_became_active":"2024-05-08T04:40:35.601716+0000","last_became_peered":"2024-05-08T04:40:35.601716+0000","last_unstale":"2024-05-08T04:40:36.602700+0000","last_undegraded":"2024-05-08T04:40:36.602700+0000","last_fullsized":"2024-05-08T04:40:36.602700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:31:14.018668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601391+0000","last_change":"2024-05-08T04:40:34.589406+0000","last_active":"2024-05-08T04:40:36.601391+0000","last_peered":"2024-05-08T04:40:36.601391+0000","last_clean":"2024-05-08T04:40:36.601391+0000","last_became_active":"2024-05-08T04:40:34.588554+0000","last_became_peered":"2024-05-08T04:40:34.588554+0000","last_unstale":"2024-05-08T04:40:36.601391+0000","last_undegraded":"2024-05-08T04:40:36.601391+0000","last_fullsized":"2024-05-08T04:40:36.601391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:25.771576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599657+0000","last_change":"2024-05-08T04:40:35.609852+0000","last_active":"2024-05-08T04:40:36.599657+0000","last_peered":"2024-05-08T04:40:36.599657+0000","last_clean":"2024-05-08T04:40:36.599657+0000","last_became_active":"2024-05-08T04:40:35.608870+0000","last_became_peered":"2024-05-08T04:40:35.608870+0000","last_unstale":"2024-05-08T04:40:36.599657+0000","last_undegraded":"2024-05-08T04:40:36.599657+0000","last_fullsized":"2024-05-08T04:40:36.599657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:08.103190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600310+0000","last_change":"2024-05-08T04:40:34.586666+0000","last_active":"2024-05-08T04:40:36.600310+0000","last_peered":"2024-05-08T04:40:36.600310+0000","last_clean":"2024-05-08T04:40:36.600310+0000","last_became_active":"2024-05-08T04:40:34.586295+0000","last_became_peered":"2024-05-08T04:40:34.586295+0000","last_unstale":"2024-05-08T04:40:36.600310+0000","last_undegraded":"2024-05-08T04:40:36.600310+0000","last_fullsized":"2024-05-08T04:40:36.600310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:00.794821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599926+0000","last_change":"2024-05-08T04:40:35.613673+0000","last_active":"2024-05-08T04:40:36.599926+0000","last_peered":"2024-05-08T04:40:36.599926+0000","last_clean":"2024-05-08T04:40:36.599926+0000","last_became_active":"2024-05-08T04:40:35.609519+0000","last_became_peered":"2024-05-08T04:40:35.609519+0000","last_unstale":"2024-05-08T04:40:36.599926+0000","last_undegraded":"2024-05-08T04:40:36.599926+0000","last_fullsized":"2024-05-08T04:40:36.599926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:16:43.538626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602384+0000","last_change":"2024-05-08T04:40:34.596409+0000","last_active":"2024-05-08T04:40:36.602384+0000","last_peered":"2024-05-08T04:40:36.602384+0000","last_clean":"2024-05-08T04:40:36.602384+0000","last_became_active":"2024-05-08T04:40:34.596131+0000","last_became_peered":"2024-05-08T04:40:34.596131+0000","last_unstale":"2024-05-08T04:40:36.602384+0000","last_undegraded":"2024-05-08T04:40:36.602384+0000","last_fullsized":"2024-05-08T04:40:36.602384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:34.150174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605523+0000","last_change":"2024-05-08T04:40:35.605488+0000","last_active":"2024-05-08T04:40:36.605523+0000","last_peered":"2024-05-08T04:40:36.605523+0000","last_clean":"2024-05-08T04:40:36.605523+0000","last_became_active":"2024-05-08T04:40:35.605224+0000","last_became_peered":"2024-05-08T04:40:35.605224+0000","last_unstale":"2024-05-08T04:40:36.605523+0000","last_undegraded":"2024-05-08T04:40:36.605523+0000","last_fullsized":"2024-05-08T04:40:36.605523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:19.456105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601111+0000","last_change":"2024-05-08T04:40:34.589217+0000","last_active":"2024-05-08T04:40:36.601111+0000","last_peered":"2024-05-08T04:40:36.601111+0000","last_clean":"2024-05-08T04:40:36.601111+0000","last_became_active":"2024-05-08T04:40:34.588859+0000","last_became_peered":"2024-05-08T04:40:34.588859+0000","last_unstale":"2024-05-08T04:40:36.601111+0000","last_undegraded":"2024-05-08T04:40:36.601111+0000","last_fullsized":"2024-05-08T04:40:36.601111+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:35:58.250941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599636+0000","last_change":"2024-05-08T04:40:35.613121+0000","last_active":"2024-05-08T04:40:36.599636+0000","last_peered":"2024-05-08T04:40:36.599636+0000","last_clean":"2024-05-08T04:40:36.599636+0000","last_became_active":"2024-05-08T04:40:35.612308+0000","last_became_peered":"2024-05-08T04:40:35.612308+0000","last_unstale":"2024-05-08T04:40:36.599636+0000","last_undegraded":"2024-05-08T04:40:36.599636+0000","last_fullsized":"2024-05-08T04:40:36.599636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:10.353996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600145+0000","last_change":"2024-05-08T04:40:34.592271+0000","last_active":"2024-05-08T04:40:36.600145+0000","last_peered":"2024-05-08T04:40:36.600145+0000","last_clean":"2024-05-08T04:40:36.600145+0000","last_became_active":"2024-05-08T04:40:34.591958+0000","last_became_peered":"2024-05-08T04:40:34.591958+0000","last_unstale":"2024-05-08T04:40:36.600145+0000","last_undegraded":"2024-05-08T04:40:36.600145+0000","last_fullsized":"2024-05-08T04:40:36.600145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:18.448613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600726+0000","last_change":"2024-05-08T04:40:35.620986+0000","last_active":"2024-05-08T04:40:36.600726+0000","last_peered":"2024-05-08T04:40:36.600726+0000","last_clean":"2024-05-08T04:40:36.600726+0000","last_became_active":"2024-05-08T04:40:35.620624+0000","last_became_peered":"2024-05-08T04:40:35.620624+0000","last_unstale":"2024-05-08T04:40:36.600726+0000","last_undegraded":"2024-05-08T04:40:36.600726+0000","last_fullsized":"2024-05-08T04:40:36.600726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:33.801014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899489+0000","last_change":"2024-05-08T04:40:34.591906+0000","last_active":"2024-05-08T04:40:36.899489+0000","last_peered":"2024-05-08T04:40:36.899489+0000","last_clean":"2024-05-08T04:40:36.899489+0000","last_became_active":"2024-05-08T04:40:34.591483+0000","last_became_peered":"2024-05-08T04:40:34.591483+0000","last_unstale":"2024-05-08T04:40:36.899489+0000","last_undegraded":"2024-05-08T04:40:36.899489+0000","last_fullsized":"2024-05-08T04:40:36.899489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:25.771576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899554+0000","last_change":"2024-05-08T04:40:35.620487+0000","last_active":"2024-05-08T04:40:36.899554+0000","last_peered":"2024-05-08T04:40:36.899554+0000","last_clean":"2024-05-08T04:40:36.899554+0000","last_became_active":"2024-05-08T04:40:35.620136+0000","last_became_peered":"2024-05-08T04:40:35.620136+0000","last_unstale":"2024-05-08T04:40:36.899554+0000","last_undegraded":"2024-05-08T04:40:36.899554+0000","last_fullsized":"2024-05-08T04:40:36.899554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.324262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600134+0000","last_change":"2024-05-08T04:40:34.598271+0000","last_active":"2024-05-08T04:40:36.600134+0000","last_peered":"2024-05-08T04:40:36.600134+0000","last_clean":"2024-05-08T04:40:36.600134+0000","last_became_active":"2024-05-08T04:40:34.597945+0000","last_became_peered":"2024-05-08T04:40:34.597945+0000","last_unstale":"2024-05-08T04:40:36.600134+0000","last_undegraded":"2024-05-08T04:40:36.600134+0000","last_fullsized":"2024-05-08T04:40:36.600134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:21.078485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899110+0000","last_change":"2024-05-08T04:40:35.619932+0000","last_active":"2024-05-08T04:40:36.899110+0000","last_peered":"2024-05-08T04:40:36.899110+0000","last_clean":"2024-05-08T04:40:36.899110+0000","last_became_active":"2024-05-08T04:40:35.619664+0000","last_became_peered":"2024-05-08T04:40:35.619664+0000","last_unstale":"2024-05-08T04:40:36.899110+0000","last_undegraded":"2024-05-08T04:40:36.899110+0000","last_fullsized":"2024-05-08T04:40:36.899110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:05.401749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899615+0000","last_change":"2024-05-08T04:40:34.590862+0000","last_active":"2024-05-08T04:40:36.899615+0000","last_peered":"2024-05-08T04:40:36.899615+0000","last_clean":"2024-05-08T04:40:36.899615+0000","last_became_active":"2024-05-08T04:40:34.590593+0000","last_became_peered":"2024-05-08T04:40:34.590593+0000","last_unstale":"2024-05-08T04:40:36.899615+0000","last_undegraded":"2024-05-08T04:40:36.899615+0000","last_fullsized":"2024-05-08T04:40:36.899615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:15.840633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899524+0000","last_change":"2024-05-08T04:40:35.613436+0000","last_active":"2024-05-08T04:40:36.899524+0000","last_peered":"2024-05-08T04:40:36.899524+0000","last_clean":"2024-05-08T04:40:36.899524+0000","last_became_active":"2024-05-08T04:40:35.612002+0000","last_became_peered":"2024-05-08T04:40:35.612002+0000","last_unstale":"2024-05-08T04:40:36.899524+0000","last_undegraded":"2024-05-08T04:40:36.899524+0000","last_fullsized":"2024-05-08T04:40:36.899524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:42.227926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600463+0000","last_change":"2024-05-08T04:40:34.596197+0000","last_active":"2024-05-08T04:40:36.600463+0000","last_peered":"2024-05-08T04:40:36.600463+0000","last_clean":"2024-05-08T04:40:36.600463+0000","last_became_active":"2024-05-08T04:40:34.595952+0000","last_became_peered":"2024-05-08T04:40:34.595952+0000","last_unstale":"2024-05-08T04:40:36.600463+0000","last_undegraded":"2024-05-08T04:40:36.600463+0000","last_fullsized":"2024-05-08T04:40:36.600463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.161450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601187+0000","last_change":"2024-05-08T04:40:35.621759+0000","last_active":"2024-05-08T04:40:36.601187+0000","last_peered":"2024-05-08T04:40:36.601187+0000","last_clean":"2024-05-08T04:40:36.601187+0000","last_became_active":"2024-05-08T04:40:35.621467+0000","last_became_peered":"2024-05-08T04:40:35.621467+0000","last_unstale":"2024-05-08T04:40:36.601187+0000","last_undegraded":"2024-05-08T04:40:36.601187+0000","last_fullsized":"2024-05-08T04:40:36.601187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:54.919040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600012+0000","last_change":"2024-05-08T04:40:34.598168+0000","last_active":"2024-05-08T04:40:36.600012+0000","last_peered":"2024-05-08T04:40:36.600012+0000","last_clean":"2024-05-08T04:40:36.600012+0000","last_became_active":"2024-05-08T04:40:34.597926+0000","last_became_peered":"2024-05-08T04:40:34.597926+0000","last_unstale":"2024-05-08T04:40:36.600012+0000","last_undegraded":"2024-05-08T04:40:36.600012+0000","last_fullsized":"2024-05-08T04:40:36.600012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:49.590106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899099+0000","last_change":"2024-05-08T04:40:35.605470+0000","last_active":"2024-05-08T04:40:36.899099+0000","last_peered":"2024-05-08T04:40:36.899099+0000","last_clean":"2024-05-08T04:40:36.899099+0000","last_became_active":"2024-05-08T04:40:35.605105+0000","last_became_peered":"2024-05-08T04:40:35.605105+0000","last_unstale":"2024-05-08T04:40:36.899099+0000","last_undegraded":"2024-05-08T04:40:36.899099+0000","last_fullsized":"2024-05-08T04:40:36.899099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:05.006199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605347+0000","last_change":"2024-05-08T04:40:34.585624+0000","last_active":"2024-05-08T04:40:36.605347+0000","last_peered":"2024-05-08T04:40:36.605347+0000","last_clean":"2024-05-08T04:40:36.605347+0000","last_became_active":"2024-05-08T04:40:34.585330+0000","last_became_peered":"2024-05-08T04:40:34.585330+0000","last_unstale":"2024-05-08T04:40:36.605347+0000","last_undegraded":"2024-05-08T04:40:36.605347+0000","last_fullsized":"2024-05-08T04:40:36.605347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:40.862844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599951+0000","last_change":"2024-05-08T04:40:35.602568+0000","last_active":"2024-05-08T04:40:36.599951+0000","last_peered":"2024-05-08T04:40:36.599951+0000","last_clean":"2024-05-08T04:40:36.599951+0000","last_became_active":"2024-05-08T04:40:35.602251+0000","last_became_peered":"2024-05-08T04:40:35.602251+0000","last_unstale":"2024-05-08T04:40:36.599951+0000","last_undegraded":"2024-05-08T04:40:36.599951+0000","last_fullsized":"2024-05-08T04:40:36.599951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:53.122303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601275+0000","last_change":"2024-05-08T04:40:34.584809+0000","last_active":"2024-05-08T04:40:36.601275+0000","last_peered":"2024-05-08T04:40:36.601275+0000","last_clean":"2024-05-08T04:40:36.601275+0000","last_became_active":"2024-05-08T04:40:34.584533+0000","last_became_peered":"2024-05-08T04:40:34.584533+0000","last_unstale":"2024-05-08T04:40:36.601275+0000","last_undegraded":"2024-05-08T04:40:36.601275+0000","last_fullsized":"2024-05-08T04:40:36.601275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:09:29.762914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899146+0000","last_change":"2024-05-08T04:40:35.618679+0000","last_active":"2024-05-08T04:40:36.899146+0000","last_peered":"2024-05-08T04:40:36.899146+0000","last_clean":"2024-05-08T04:40:36.899146+0000","last_became_active":"2024-05-08T04:40:35.618108+0000","last_became_peered":"2024-05-08T04:40:35.618108+0000","last_unstale":"2024-05-08T04:40:36.899146+0000","last_undegraded":"2024-05-08T04:40:36.899146+0000","last_fullsized":"2024-05-08T04:40:36.899146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:38.824580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898992+0000","last_change":"2024-05-08T04:40:34.585095+0000","last_active":"2024-05-08T04:40:36.898992+0000","last_peered":"2024-05-08T04:40:36.898992+0000","last_clean":"2024-05-08T04:40:36.898992+0000","last_became_active":"2024-05-08T04:40:34.584831+0000","last_became_peered":"2024-05-08T04:40:34.584831+0000","last_unstale":"2024-05-08T04:40:36.898992+0000","last_undegraded":"2024-05-08T04:40:36.898992+0000","last_fullsized":"2024-05-08T04:40:36.898992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:51.895822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899609+0000","last_change":"2024-05-08T04:40:35.602621+0000","last_active":"2024-05-08T04:40:36.899609+0000","last_peered":"2024-05-08T04:40:36.899609+0000","last_clean":"2024-05-08T04:40:36.899609+0000","last_became_active":"2024-05-08T04:40:35.602288+0000","last_became_peered":"2024-05-08T04:40:35.602288+0000","last_unstale":"2024-05-08T04:40:36.899609+0000","last_undegraded":"2024-05-08T04:40:36.899609+0000","last_fullsized":"2024-05-08T04:40:36.899609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:53.268007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600400+0000","last_change":"2024-05-08T04:40:34.597788+0000","last_active":"2024-05-08T04:40:36.600400+0000","last_peered":"2024-05-08T04:40:36.600400+0000","last_clean":"2024-05-08T04:40:36.600400+0000","last_became_active":"2024-05-08T04:40:34.597523+0000","last_became_peered":"2024-05-08T04:40:34.597523+0000","last_unstale":"2024-05-08T04:40:36.600400+0000","last_undegraded":"2024-05-08T04:40:36.600400+0000","last_fullsized":"2024-05-08T04:40:36.600400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:00.795776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601726+0000","last_change":"2024-05-08T04:40:35.622729+0000","last_active":"2024-05-08T04:40:36.601726+0000","last_peered":"2024-05-08T04:40:36.601726+0000","last_clean":"2024-05-08T04:40:36.601726+0000","last_became_active":"2024-05-08T04:40:35.622458+0000","last_became_peered":"2024-05-08T04:40:35.622458+0000","last_unstale":"2024-05-08T04:40:36.601726+0000","last_undegraded":"2024-05-08T04:40:36.601726+0000","last_fullsized":"2024-05-08T04:40:36.601726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:38.824580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602741+0000","last_change":"2024-05-08T04:40:34.590615+0000","last_active":"2024-05-08T04:40:36.602741+0000","last_peered":"2024-05-08T04:40:36.602741+0000","last_clean":"2024-05-08T04:40:36.602741+0000","last_became_active":"2024-05-08T04:40:34.590357+0000","last_became_peered":"2024-05-08T04:40:34.590357+0000","last_unstale":"2024-05-08T04:40:36.602741+0000","last_undegraded":"2024-05-08T04:40:36.602741+0000","last_fullsized":"2024-05-08T04:40:36.602741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:26.131502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599900+0000","last_change":"2024-05-08T04:40:35.612791+0000","last_active":"2024-05-08T04:40:36.599900+0000","last_peered":"2024-05-08T04:40:36.599900+0000","last_clean":"2024-05-08T04:40:36.599900+0000","last_became_active":"2024-05-08T04:40:35.612430+0000","last_became_peered":"2024-05-08T04:40:35.612430+0000","last_unstale":"2024-05-08T04:40:36.599900+0000","last_undegraded":"2024-05-08T04:40:36.599900+0000","last_fullsized":"2024-05-08T04:40:36.599900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:26.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601606+0000","last_change":"2024-05-08T04:40:34.589019+0000","last_active":"2024-05-08T04:40:36.601606+0000","last_peered":"2024-05-08T04:40:36.601606+0000","last_clean":"2024-05-08T04:40:36.601606+0000","last_became_active":"2024-05-08T04:40:34.588690+0000","last_became_peered":"2024-05-08T04:40:34.588690+0000","last_unstale":"2024-05-08T04:40:36.601606+0000","last_undegraded":"2024-05-08T04:40:36.601606+0000","last_fullsized":"2024-05-08T04:40:36.601606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:28.333241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899471+0000","last_change":"2024-05-08T04:40:35.613665+0000","last_active":"2024-05-08T04:40:36.899471+0000","last_peered":"2024-05-08T04:40:36.899471+0000","last_clean":"2024-05-08T04:40:36.899471+0000","last_became_active":"2024-05-08T04:40:35.612485+0000","last_became_peered":"2024-05-08T04:40:35.612485+0000","last_unstale":"2024-05-08T04:40:36.899471+0000","last_undegraded":"2024-05-08T04:40:36.899471+0000","last_fullsized":"2024-05-08T04:40:36.899471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:01.803268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602894+0000","last_change":"2024-05-08T04:40:34.600472+0000","last_active":"2024-05-08T04:40:36.602894+0000","last_peered":"2024-05-08T04:40:36.602894+0000","last_clean":"2024-05-08T04:40:36.602894+0000","last_became_active":"2024-05-08T04:40:34.600197+0000","last_became_peered":"2024-05-08T04:40:34.600197+0000","last_unstale":"2024-05-08T04:40:36.602894+0000","last_undegraded":"2024-05-08T04:40:36.602894+0000","last_fullsized":"2024-05-08T04:40:36.602894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:42:54.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601359+0000","last_change":"2024-05-08T04:40:35.621461+0000","last_active":"2024-05-08T04:40:36.601359+0000","last_peered":"2024-05-08T04:40:36.601359+0000","last_clean":"2024-05-08T04:40:36.601359+0000","last_became_active":"2024-05-08T04:40:35.621125+0000","last_became_peered":"2024-05-08T04:40:35.621125+0000","last_unstale":"2024-05-08T04:40:36.601359+0000","last_undegraded":"2024-05-08T04:40:36.601359+0000","last_fullsized":"2024-05-08T04:40:36.601359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.021416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600244+0000","last_change":"2024-05-08T04:40:34.593711+0000","last_active":"2024-05-08T04:40:36.600244+0000","last_peered":"2024-05-08T04:40:36.600244+0000","last_clean":"2024-05-08T04:40:36.600244+0000","last_became_active":"2024-05-08T04:40:34.593430+0000","last_became_peered":"2024-05-08T04:40:34.593430+0000","last_unstale":"2024-05-08T04:40:36.600244+0000","last_undegraded":"2024-05-08T04:40:36.600244+0000","last_fullsized":"2024-05-08T04:40:36.600244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:56.328356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602548+0000","last_change":"2024-05-08T04:40:35.605161+0000","last_active":"2024-05-08T04:40:36.602548+0000","last_peered":"2024-05-08T04:40:36.602548+0000","last_clean":"2024-05-08T04:40:36.602548+0000","last_became_active":"2024-05-08T04:40:35.603972+0000","last_became_peered":"2024-05-08T04:40:35.603972+0000","last_unstale":"2024-05-08T04:40:36.602548+0000","last_undegraded":"2024-05-08T04:40:36.602548+0000","last_fullsized":"2024-05-08T04:40:36.602548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.324262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898610+0000","last_change":"2024-05-08T04:40:34.587688+0000","last_active":"2024-05-08T04:40:36.898610+0000","last_peered":"2024-05-08T04:40:36.898610+0000","last_clean":"2024-05-08T04:40:36.898610+0000","last_became_active":"2024-05-08T04:40:34.586205+0000","last_became_peered":"2024-05-08T04:40:34.586205+0000","last_unstale":"2024-05-08T04:40:36.898610+0000","last_undegraded":"2024-05-08T04:40:36.898610+0000","last_fullsized":"2024-05-08T04:40:36.898610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:49.112943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602302+0000","last_change":"2024-05-08T04:40:35.594941+0000","last_active":"2024-05-08T04:40:36.602302+0000","last_peered":"2024-05-08T04:40:36.602302+0000","last_clean":"2024-05-08T04:40:36.602302+0000","last_became_active":"2024-05-08T04:40:35.594677+0000","last_became_peered":"2024-05-08T04:40:35.594677+0000","last_unstale":"2024-05-08T04:40:36.602302+0000","last_undegraded":"2024-05-08T04:40:36.602302+0000","last_fullsized":"2024-05-08T04:40:36.602302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:42.227926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600911+0000","last_change":"2024-05-08T04:40:34.593499+0000","last_active":"2024-05-08T04:40:36.600911+0000","last_peered":"2024-05-08T04:40:36.600911+0000","last_clean":"2024-05-08T04:40:36.600911+0000","last_became_active":"2024-05-08T04:40:34.593156+0000","last_became_peered":"2024-05-08T04:40:34.593156+0000","last_unstale":"2024-05-08T04:40:36.600911+0000","last_undegraded":"2024-05-08T04:40:36.600911+0000","last_fullsized":"2024-05-08T04:40:36.600911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:51.895822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601711+0000","last_change":"2024-05-08T04:40:35.600197+0000","last_active":"2024-05-08T04:40:36.601711+0000","last_peered":"2024-05-08T04:40:36.601711+0000","last_clean":"2024-05-08T04:40:36.601711+0000","last_became_active":"2024-05-08T04:40:35.599884+0000","last_became_peered":"2024-05-08T04:40:35.599884+0000","last_unstale":"2024-05-08T04:40:36.601711+0000","last_undegraded":"2024-05-08T04:40:36.601711+0000","last_fullsized":"2024-05-08T04:40:36.601711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:27.931796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899068+0000","last_change":"2024-05-08T04:40:34.599233+0000","last_active":"2024-05-08T04:40:36.899068+0000","last_peered":"2024-05-08T04:40:36.899068+0000","last_clean":"2024-05-08T04:40:36.899068+0000","last_became_active":"2024-05-08T04:40:34.598972+0000","last_became_peered":"2024-05-08T04:40:34.598972+0000","last_unstale":"2024-05-08T04:40:36.899068+0000","last_undegraded":"2024-05-08T04:40:36.899068+0000","last_fullsized":"2024-05-08T04:40:36.899068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:18.792909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601719+0000","last_change":"2024-05-08T04:40:35.606936+0000","last_active":"2024-05-08T04:40:36.601719+0000","last_peered":"2024-05-08T04:40:36.601719+0000","last_clean":"2024-05-08T04:40:36.601719+0000","last_became_active":"2024-05-08T04:40:35.606640+0000","last_became_peered":"2024-05-08T04:40:35.606640+0000","last_unstale":"2024-05-08T04:40:36.601719+0000","last_undegraded":"2024-05-08T04:40:36.601719+0000","last_fullsized":"2024-05-08T04:40:36.601719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:46:43.291290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.604508+0000","last_change":"2024-05-08T04:40:34.592667+0000","last_active":"2024-05-08T04:40:36.604508+0000","last_peered":"2024-05-08T04:40:36.604508+0000","last_clean":"2024-05-08T04:40:36.604508+0000","last_became_active":"2024-05-08T04:40:34.591674+0000","last_became_peered":"2024-05-08T04:40:34.591674+0000","last_unstale":"2024-05-08T04:40:36.604508+0000","last_undegraded":"2024-05-08T04:40:36.604508+0000","last_fullsized":"2024-05-08T04:40:36.604508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:00.801964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898968+0000","last_change":"2024-05-08T04:40:35.621359+0000","last_active":"2024-05-08T04:40:36.898968+0000","last_peered":"2024-05-08T04:40:36.898968+0000","last_clean":"2024-05-08T04:40:36.898968+0000","last_became_active":"2024-05-08T04:40:35.621154+0000","last_became_peered":"2024-05-08T04:40:35.621154+0000","last_unstale":"2024-05-08T04:40:36.898968+0000","last_undegraded":"2024-05-08T04:40:36.898968+0000","last_fullsized":"2024-05-08T04:40:36.898968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:19.456105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605587+0000","last_change":"2024-05-08T04:40:34.588114+0000","last_active":"2024-05-08T04:40:36.605587+0000","last_peered":"2024-05-08T04:40:36.605587+0000","last_clean":"2024-05-08T04:40:36.605587+0000","last_became_active":"2024-05-08T04:40:34.587831+0000","last_became_peered":"2024-05-08T04:40:34.587831+0000","last_unstale":"2024-05-08T04:40:36.605587+0000","last_undegraded":"2024-05-08T04:40:36.605587+0000","last_fullsized":"2024-05-08T04:40:36.605587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:44.449340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.602789+0000","last_change":"2024-05-08T04:40:35.599281+0000","last_active":"2024-05-08T04:40:36.602789+0000","last_peered":"2024-05-08T04:40:36.602789+0000","last_clean":"2024-05-08T04:40:36.602789+0000","last_became_active":"2024-05-08T04:40:35.598619+0000","last_became_peered":"2024-05-08T04:40:35.598619+0000","last_unstale":"2024-05-08T04:40:36.602789+0000","last_undegraded":"2024-05-08T04:40:36.602789+0000","last_fullsized":"2024-05-08T04:40:36.602789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:49.474365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.601540+0000","last_change":"2024-05-08T04:40:34.587738+0000","last_active":"2024-05-08T04:40:36.601540+0000","last_peered":"2024-05-08T04:40:36.601540+0000","last_clean":"2024-05-08T04:40:36.601540+0000","last_became_active":"2024-05-08T04:40:34.587464+0000","last_became_peered":"2024-05-08T04:40:34.587464+0000","last_unstale":"2024-05-08T04:40:36.601540+0000","last_undegraded":"2024-05-08T04:40:36.601540+0000","last_fullsized":"2024-05-08T04:40:36.601540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:15.840633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599907+0000","last_change":"2024-05-08T04:40:35.609632+0000","last_active":"2024-05-08T04:40:36.599907+0000","last_peered":"2024-05-08T04:40:36.599907+0000","last_clean":"2024-05-08T04:40:36.599907+0000","last_became_active":"2024-05-08T04:40:35.608561+0000","last_became_peered":"2024-05-08T04:40:35.608561+0000","last_unstale":"2024-05-08T04:40:36.599907+0000","last_undegraded":"2024-05-08T04:40:36.599907+0000","last_fullsized":"2024-05-08T04:40:36.599907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:18.103016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898629+0000","last_change":"2024-05-08T04:40:34.587522+0000","last_active":"2024-05-08T04:40:36.898629+0000","last_peered":"2024-05-08T04:40:36.898629+0000","last_clean":"2024-05-08T04:40:36.898629+0000","last_became_active":"2024-05-08T04:40:34.587224+0000","last_became_peered":"2024-05-08T04:40:34.587224+0000","last_unstale":"2024-05-08T04:40:36.898629+0000","last_undegraded":"2024-05-08T04:40:36.898629+0000","last_fullsized":"2024-05-08T04:40:36.898629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:31.556397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.899349+0000","last_change":"2024-05-08T04:40:35.617761+0000","last_active":"2024-05-08T04:40:36.899349+0000","last_peered":"2024-05-08T04:40:36.899349+0000","last_clean":"2024-05-08T04:40:36.899349+0000","last_became_active":"2024-05-08T04:40:35.617469+0000","last_became_peered":"2024-05-08T04:40:35.617469+0000","last_unstale":"2024-05-08T04:40:36.899349+0000","last_undegraded":"2024-05-08T04:40:36.899349+0000","last_fullsized":"2024-05-08T04:40:36.899349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:57.335849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.600190+0000","last_change":"2024-05-08T04:40:34.596968+0000","last_active":"2024-05-08T04:40:36.600190+0000","last_peered":"2024-05-08T04:40:36.600190+0000","last_clean":"2024-05-08T04:40:36.600190+0000","last_became_active":"2024-05-08T04:40:34.596581+0000","last_became_peered":"2024-05-08T04:40:34.596581+0000","last_unstale":"2024-05-08T04:40:36.600190+0000","last_undegraded":"2024-05-08T04:40:36.600190+0000","last_fullsized":"2024-05-08T04:40:36.600190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:25.021274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898328+0000","last_change":"2024-05-08T04:40:35.602339+0000","last_active":"2024-05-08T04:40:36.898328+0000","last_peered":"2024-05-08T04:40:36.898328+0000","last_clean":"2024-05-08T04:40:36.898328+0000","last_became_active":"2024-05-08T04:40:35.602081+0000","last_became_peered":"2024-05-08T04:40:35.602081+0000","last_unstale":"2024-05-08T04:40:36.898328+0000","last_undegraded":"2024-05-08T04:40:36.898328+0000","last_fullsized":"2024-05-08T04:40:36.898328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:06.052427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898436+0000","last_change":"2024-05-08T04:40:34.595685+0000","last_active":"2024-05-08T04:40:36.898436+0000","last_peered":"2024-05-08T04:40:36.898436+0000","last_clean":"2024-05-08T04:40:36.898436+0000","last_became_active":"2024-05-08T04:40:34.595382+0000","last_became_peered":"2024-05-08T04:40:34.595382+0000","last_unstale":"2024-05-08T04:40:36.898436+0000","last_undegraded":"2024-05-08T04:40:36.898436+0000","last_fullsized":"2024-05-08T04:40:36.898436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:30.768476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599610+0000","last_change":"2024-05-08T04:40:35.613111+0000","last_active":"2024-05-08T04:40:36.599610+0000","last_peered":"2024-05-08T04:40:36.599610+0000","last_clean":"2024-05-08T04:40:36.599610+0000","last_became_active":"2024-05-08T04:40:35.610355+0000","last_became_peered":"2024-05-08T04:40:35.610355+0000","last_unstale":"2024-05-08T04:40:36.599610+0000","last_undegraded":"2024-05-08T04:40:36.599610+0000","last_fullsized":"2024-05-08T04:40:36.599610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:47.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599453+0000","last_change":"2024-05-08T04:40:34.592173+0000","last_active":"2024-05-08T04:40:36.599453+0000","last_peered":"2024-05-08T04:40:36.599453+0000","last_clean":"2024-05-08T04:40:36.599453+0000","last_became_active":"2024-05-08T04:40:34.588946+0000","last_became_peered":"2024-05-08T04:40:34.588946+0000","last_unstale":"2024-05-08T04:40:36.599453+0000","last_undegraded":"2024-05-08T04:40:36.599453+0000","last_fullsized":"2024-05-08T04:40:36.599453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:37.817087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.599344+0000","last_change":"2024-05-08T04:40:35.607436+0000","last_active":"2024-05-08T04:40:36.599344+0000","last_peered":"2024-05-08T04:40:36.599344+0000","last_clean":"2024-05-08T04:40:36.599344+0000","last_became_active":"2024-05-08T04:40:35.607141+0000","last_became_peered":"2024-05-08T04:40:35.607141+0000","last_unstale":"2024-05-08T04:40:36.599344+0000","last_undegraded":"2024-05-08T04:40:36.599344+0000","last_fullsized":"2024-05-08T04:40:36.599344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:34.564028+0000","last_clean_scrub_stamp":"2024-05-08T04:40:34.564028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:43.108589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.605445+0000","last_change":"2024-05-08T04:40:34.592421+0000","last_active":"2024-05-08T04:40:36.605445+0000","last_peered":"2024-05-08T04:40:36.605445+0000","last_clean":"2024-05-08T04:40:36.605445+0000","last_became_active":"2024-05-08T04:40:34.590067+0000","last_became_peered":"2024-05-08T04:40:34.590067+0000","last_unstale":"2024-05-08T04:40:36.605445+0000","last_undegraded":"2024-05-08T04:40:36.605445+0000","last_fullsized":"2024-05-08T04:40:36.605445+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:40:33.556536+0000","last_clean_scrub_stamp":"2024-05-08T04:40:33.556536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:45.643834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T04:40:36.898878+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:40:36.898878+0000","last_peered":"2024-05-08T04:40:36.898878+0000","last_clean":"2024-05-08T04:40:36.898878+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:40:36.898878+0000","last_undegraded":"2024-05-08T04:40:36.898878+0000","last_fullsized":"2024-05-08T04:40:36.898878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1323008,"data_stored":893086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":296230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":729088,"data_stored":303210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":304170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":302614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":302614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":296320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":895120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:40:40.050 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:40:40.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:40:40.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 Processor -- start 2024-05-08T04:40:40.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 -- start start 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac80aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac80c6600 con 0x7ffac815ba20 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac80c6040 con 0x7ffac815ce10 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffacde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac815fbe0 con 0x7ffac80ae6e0 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac6a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ffac815ba20 0x7ffac8160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37012/0 (socket says 172.21.15.164:37012) 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac6a00640 1 -- 172.21.15.164:0/1254265977 learned_addr learned my addr 172.21.15.164:0/1254265977 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac7400640 1 --2- 172.21.15.164:0/1254265977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac80aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707786552 0 0) 0x7ffac80c6600 con 0x7ffac815ba20 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaac003530 con 0x7ffac815ba20 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202088781 0 0) 0x7ffac80c6040 con 0x7ffac815ce10 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac80c6600 con 0x7ffac815ce10 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac7400640 1 -- 172.21.15.164:0/1254265977 >> v1:172.21.15.172:6789/0 conn(0x7ffac815ce10 legacy=0x7ffac806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac7400640 1 -- 172.21.15.164:0/1254265977 >> v1:172.21.15.164:6789/0 conn(0x7ffac815ba20 legacy=0x7ffac8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.229+0000 7ffac7400640 1 -- 172.21.15.164:0/1254265977 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80c6320 con 0x7ffac80ae6e0 2024-05-08T04:40:40.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffac7400640 1 --2- 172.21.15.164:0/1254265977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac80aeb20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009dc0 tx=0x7ffac001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f94772939e520f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b2070 con 0x7ffac80ae6e0 2024-05-08T04:40:40.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00a5850 con 0x7ffac80ae6e0 2024-05-08T04:40:40.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffac5600640 1 -- 172.21.15.164:0/1254265977 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00a5c20 con 0x7ffac80ae6e0 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 -- 172.21.15.164:0/1254265977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 msgr2=0x7ffac80aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 --2- 172.21.15.164:0/1254265977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac80aeb20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009dc0 tx=0x7ffac001c990 comp rx=0 tx=0).stop 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 -- 172.21.15.164:0/1254265977 shutdown_connections 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 --2- 172.21.15.164:0/1254265977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac80aeb20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 -- 172.21.15.164:0/1254265977 >> 172.21.15.164:0/1254265977 conn(0x7ffac80cf770 msgr2=0x7ffac80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 -- 172.21.15.164:0/1254265977 shutdown_connections 2024-05-08T04:40:40.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.233+0000 7ffacde00640 1 -- 172.21.15.164:0/1254265977 wait complete. 2024-05-08T04:40:40.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 Processor -- start 2024-05-08T04:40:40.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- start start 2024-05-08T04:40:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac815ce10 0x7ffac80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80ce000 con 0x7ffac815ba20 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80cde80 con 0x7ffac815ce10 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8169b80 con 0x7ffac80ae6e0 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46218/0 (socket says 172.21.15.164:46218) 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 -- 172.21.15.164:0/291459064 learned_addr learned my addr 172.21.15.164:0/291459064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac7400640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac815ce10 0x7ffac80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6a00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 -- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 msgr2=0x7ffac815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 -- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac815ce10 msgr2=0x7ffac80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac815ce10 0x7ffac80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 -- 172.21.15.164:0/291459064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80c7570 con 0x7ffac815ba20 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6a00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffac6000640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c460 tx=0x7ffab400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4017070 con 0x7ffac815ba20 2024-05-08T04:40:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab40052f0 con 0x7ffac815ba20 2024-05-08T04:40:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac80b28f0 con 0x7ffac815ba20 2024-05-08T04:40:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4003840 con 0x7ffac815ba20 2024-05-08T04:40:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.237+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80c6fb0 con 0x7ffac815ba20 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8072d30 con 0x7ffac815ba20 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7ffab401a070 con 0x7ffac815ba20 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffabf400640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 0x7ffaa8069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ffab40846c0 con 0x7ffac815ba20 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffac6a00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 0x7ffaa8069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.241+0000 7ffac6a00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 0x7ffaa8069660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffab80011f0 tx=0x7ffab8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.245+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffab4054dd0 con 0x7ffac815ba20 2024-05-08T04:40:40.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.453+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffac8167950 con 0x7ffac815ba20 2024-05-08T04:40:40.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.457+0000 7ffabf400640 1 -- 172.21.15.164:0/291459064 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffab4058e70 con 0x7ffac815ba20 2024-05-08T04:40:40.462 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:40:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:40:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:40:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:40:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:40:40.463 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 msgr2=0x7ffaa8069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 0x7ffaa8069660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffab80011f0 tx=0x7ffab8009040 comp rx=0 tx=0).stop 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 msgr2=0x7ffac8152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c460 tx=0x7ffab400c920 comp rx=0 tx=0).stop 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 shutdown_connections 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaa80671a0 0x7ffaa8069660 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac815ce10 0x7ffac80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffac815ba20 0x7ffac8152b60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 --2- 172.21.15.164:0/291459064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac80ae6e0 0x7ffac815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 >> 172.21.15.164:0/291459064 conn(0x7ffac80cf770 msgr2=0x7ffac806de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:40.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 shutdown_connections 2024-05-08T04:40:40.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.461+0000 7ffacde00640 1 -- 172.21.15.164:0/291459064 wait complete. 2024-05-08T04:40:40.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:40.559 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-05-08T04:40:40.553+0000 7f64dea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:40:40.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 Processor -- start 2024-05-08T04:40:40.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 -- start start 2024-05-08T04:40:40.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08e815ab60 0x7f08e8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 0x7f08e80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e8072640 con 0x7f08e80b26c0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e806c770 con 0x7f08e815ab60 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08ed400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e806ca50 con 0x7f08e80cfda0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e5e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f08e80b26c0 0x7f08e80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37022/0 (socket says 172.21.15.164:37022) 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e5e00640 1 -- 172.21.15.164:0/450569214 learned_addr learned my addr 172.21.15.164:0/450569214 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e6800640 1 --2- 172.21.15.164:0/450569214 >> v2:172.21.15.172:3300/0 conn(0x7f08e815ab60 0x7f08e8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e7200640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 0x7f08e80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531663284 0 0) 0x7f08e8072640 con 0x7f08e80b26c0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08cc003530 con 0x7f08e80b26c0 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e6800640 1 -- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 msgr2=0x7f08e80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e6800640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 0x7f08e80cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e6800640 1 -- 172.21.15.164:0/450569214 >> v1:172.21.15.164:6789/0 conn(0x7f08e80b26c0 legacy=0x7f08e80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121211220 0 0) 0x7f08cc003530 con 0x7f08e80b26c0 2024-05-08T04:40:40.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e6800640 1 -- 172.21.15.164:0/450569214 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e806cd30 con 0x7f08e815ab60 2024-05-08T04:40:40.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.617+0000 7f08e7200640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 0x7f08e80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:40.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08e6800640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e815ab60 0x7f08e8072100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08d80097c0 tx=0x7f08d801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eafc3e1080b02571 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80b0070 con 0x7f08e815ab60 2024-05-08T04:40:40.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d80ab600 con 0x7f08e815ab60 2024-05-08T04:40:40.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08e5400640 1 -- 172.21.15.164:0/450569214 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80ab9d0 con 0x7f08e815ab60 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 -- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e815ab60 msgr2=0x7f08e8072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e815ab60 0x7f08e8072100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08d80097c0 tx=0x7f08d801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 -- 172.21.15.164:0/450569214 shutdown_connections 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80cfda0 0x7f08e80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 --2- 172.21.15.164:0/450569214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e815ab60 0x7f08e8072100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 -- 172.21.15.164:0/450569214 >> 172.21.15.164:0/450569214 conn(0x7f08e806ac90 msgr2=0x7f08e806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 -- 172.21.15.164:0/450569214 shutdown_connections 2024-05-08T04:40:40.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.621+0000 7f08ed400640 1 -- 172.21.15.164:0/450569214 wait complete. 2024-05-08T04:40:40.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 Processor -- start 2024-05-08T04:40:40.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- start start 2024-05-08T04:40:40.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 0x7f08e81f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e80cfda0 0x7f08e8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80aeeb0 con 0x7f08e815ab60 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80aed30 con 0x7f08e80cfda0 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8167970 con 0x7f08e80b26c0 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46232/0 (socket says 172.21.15.164:46232) 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 -- 172.21.15.164:0/1249837514 learned_addr learned my addr 172.21.15.164:0/1249837514 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e5e00640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e80cfda0 0x7f08e8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e6800640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 0x7f08e81f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 -- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 msgr2=0x7f08e81f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 0x7f08e81f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 -- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e80cfda0 msgr2=0x7f08e8159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e80cfda0 0x7f08e8159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 -- 172.21.15.164:0/1249837514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e81f6bc0 con 0x7f08e815ab60 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e6800640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 0x7f08e81f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08e7200640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f08e000efc0 tx=0x7f08e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0018030 con 0x7f08e815ab60 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e000cde0 con 0x7f08e815ab60 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003bc0 con 0x7f08e815ab60 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e81f6ea0 con 0x7f08e815ab60 2024-05-08T04:40:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.625+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e81f7460 con 0x7f08e815ab60 2024-05-08T04:40:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e8069d60 con 0x7f08e815ab60 2024-05-08T04:40:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f08e001b070 con 0x7f08e815ab60 2024-05-08T04:40:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08dea00640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 0x7f08c8069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f08e00860c0 con 0x7f08e815ab60 2024-05-08T04:40:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08e6800640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 0x7f08c8069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:40.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.629+0000 7f08e6800640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 0x7f08c8069660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ad000 tx=0x7f08d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:40.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.633+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08e0010040 con 0x7f08e815ab60 2024-05-08T04:40:40.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.817+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e806d9a0 con 0x7f08e815ab60 2024-05-08T04:40:40.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.817+0000 7f08dea00640 1 -- 172.21.15.164:0/1249837514 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f08e0056ba0 con 0x7f08e815ab60 2024-05-08T04:40:40.823 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:40.823 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:40.823 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 34 2024-05-08T04:40:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 msgr2=0x7f08c8069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 0x7f08c8069660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ad000 tx=0x7f08d8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 msgr2=0x7f08e8158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f08e000efc0 tx=0x7f08e0009f90 comp rx=0 tx=0).stop 2024-05-08T04:40:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 shutdown_connections 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08c80671a0 0x7f08c8069660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e815ab60 0x7f08e8158e70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e80cfda0 0x7f08e8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 --2- 172.21.15.164:0/1249837514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08e80b26c0 0x7f08e81f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 >> 172.21.15.164:0/1249837514 conn(0x7f08e806ac90 msgr2=0x7f08e80b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 shutdown_connections 2024-05-08T04:40:40.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:40.821+0000 7f08ed400640 1 -- 172.21.15.164:0/1249837514 wait complete. 2024-05-08T04:40:41.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:40:34.960304+0000', 'modified': '2024-05-08T04:40:37.971888+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6018}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6018': {'gid': 6018, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/1114280106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 1114280106}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1114280106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:40:41.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:40:41.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T04:40:41.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 Processor -- start 2024-05-08T04:40:41.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 -- start start 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e60166c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6006f2e0 con 0x7f4e600b3a80 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e600702f0 con 0x7f4e6015f0f0 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e64c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600ae5e0 con 0x7f4e60164890 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e56a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4e600b3a80 0x7f4e6006ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37028/0 (socket says 172.21.15.164:37028) 2024-05-08T04:40:41.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e56a00640 1 -- 172.21.15.164:0/3711015704 learned_addr learned my addr 172.21.15.164:0/3711015704 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267023044 0 0) 0x7f4e6006f2e0 con 0x7f4e600b3a80 2024-05-08T04:40:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e60166c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.381+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e44003530 con 0x7f4e600b3a80 2024-05-08T04:40:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3891577665 0 0) 0x7f4e44003530 con 0x7f4e600b3a80 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 msgr2=0x7f4e60166c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e60166c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 >> v1:172.21.15.172:6789/0 conn(0x7f4e6015f0f0 legacy=0x7f4e6006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600cce50 con 0x7f4e600b3a80 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f4e48001680 con 0x7f4e600b3a80 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 >> v1:172.21.15.164:6789/0 conn(0x7f4e600b3a80 legacy=0x7f4e6006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e44004d50 0x7f4e440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400bb00 con 0x7f4e440039f0 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400bcb0 con 0x7f4e44004d50 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400beb0 con 0x7f4e60164890 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e56a00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e44004d50 0x7f4e440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 -- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 msgr2=0x7f4e44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 -- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e44004d50 msgr2=0x7f4e440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e44004d50 0x7f4e440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 -- 172.21.15.164:0/3711015704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44003530 con 0x7f4e440039f0 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e44004d50 0x7f4e440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:41.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e56a00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e60164890 0x7f4e44004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e44003e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001780 tx=0x7f4e5001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c503a11be4106 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.385+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500013a0 con 0x7f4e440039f0 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500050d0 con 0x7f4e440039f0 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5ea00640 1 -- 172.21.15.164:0/3711015704 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500054a0 con 0x7f4e440039f0 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- 172.21.15.164:0/3711015704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 msgr2=0x7f4e44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 --2- 172.21.15.164:0/3711015704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e44003e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001780 tx=0x7f4e5001c360 comp rx=0 tx=0).stop 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5fe00640 1 -- 172.21.15.164:0/3711015704 reap_dead start 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- 172.21.15.164:0/3711015704 shutdown_connections 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- 172.21.15.164:0/3711015704 >> 172.21.15.164:0/3711015704 conn(0x7f4e6009aa20 msgr2=0x7f4e600b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- 172.21.15.164:0/3711015704 shutdown_connections 2024-05-08T04:40:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- 172.21.15.164:0/3711015704 wait complete. 2024-05-08T04:40:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 Processor -- start 2024-05-08T04:40:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- start start 2024-05-08T04:40:41.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e60163be0 0x7f4e60164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 0x7f4e600b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015f4d0 con 0x7f4e440039f0 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6006f660 con 0x7f4e60163be0 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e64c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015f650 con 0x7f4e600b1190 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46238/0 (socket says 172.21.15.164:46238) 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5f400640 1 -- 172.21.15.164:0/912534721 learned_addr learned my addr 172.21.15.164:0/912534721 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e56a00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e60163be0 0x7f4e60164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.389+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 0x7f4e600b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 -- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 msgr2=0x7f4e600b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 0x7f4e600b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 -- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e60163be0 msgr2=0x7f4e60164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e60163be0 0x7f4e60164060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 -- 172.21.15.164:0/912534721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600b1da0 con 0x7f4e440039f0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 0x7f4e600b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e5f400640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005040 tx=0x7f4e44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 -- 172.21.15.164:0/912534721 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50007040 con 0x7f4e440039f0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 -- 172.21.15.164:0/912534721 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500bb070 con 0x7f4e440039f0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 -- 172.21.15.164:0/912534721 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500073e0 con 0x7f4e440039f0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e600b2080 con 0x7f4e440039f0 2024-05-08T04:40:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e600b2640 con 0x7f4e440039f0 2024-05-08T04:40:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 -- 172.21.15.164:0/912534721 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f4e500b4040 con 0x7f4e440039f0 2024-05-08T04:40:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 0x7f4e3c069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e57e00640 1 -- 172.21.15.164:0/912534721 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e5011f2e0 con 0x7f4e440039f0 2024-05-08T04:40:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.393+0000 7f4e56a00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 0x7f4e3c069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 Processor -- start 2024-05-08T04:40:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- start start 2024-05-08T04:40:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e56a00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 0x7f4e3c069690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e6015c580 tx=0x7f4e48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c82dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c6cb5f0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c632da0 con 0x55cc3c82d920 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c5bb150 con 0x55cc3c82e240 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46254/0 (socket says 172.21.15.164:46254) 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 -- 172.21.15.164:0/2109859058 learned_addr learned my addr 172.21.15.164:0/2109859058 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5f400640 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c82dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e56a00640 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 -- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 msgr2=0x55cc3c7e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 -- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 msgr2=0x55cc3c82dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c82dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 -- 172.21.15.164:0/2109859058 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc3c5d4ca0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4e600723d0 tx=0x7f4e58009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30df49426308ea27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e4fe00640 1 -- 172.21.15.164:0/2109859058 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5801a070 con 0x55cc3c7e3f30 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e4fe00640 1 -- 172.21.15.164:0/2109859058 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58016830 con 0x55cc3c7e3f30 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e4fe00640 1 -- 172.21.15.164:0/2109859058 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58016be0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- 172.21.15.164:0/2109859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 msgr2=0x55cc3c7e4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4e600723d0 tx=0x7f4e58009750 comp rx=0 tx=0).stop 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- 172.21.15.164:0/2109859058 shutdown_connections 2024-05-08T04:40:41.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c7e4310 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 --2- 172.21.15.164:0/2109859058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c82dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- 172.21.15.164:0/2109859058 >> 172.21.15.164:0/2109859058 conn(0x55cc3c7ea2a0 msgr2=0x55cc3c7e22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- 172.21.15.164:0/2109859058 shutdown_connections 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- 172.21.15.164:0/2109859058 wait complete. 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.397+0000 7f4e69651000 1 -- start start 2024-05-08T04:40:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 Processor -- start 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c8353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c5ed990 con 0x55cc3c7e3f30 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c62ee40 con 0x55cc3c82d920 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c5be900 con 0x55cc3c82e240 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 10 client.0 _ll_register_callbacks cb 0x55cc3c6e61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46270/0 (socket says 172.21.15.164:46270) 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 -- 172.21.15.164:0/3710260737 learned_addr learned my addr 172.21.15.164:0/3710260737 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e56a00640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c8353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 -- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 msgr2=0x55cc3c7e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 -- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 msgr2=0x55cc3c8353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c8353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 -- 172.21.15.164:0/3710260737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc3c5eb630 con 0x55cc3c7e3f30 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e5f400640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4e50097c40 tx=0x7f4e5011f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ef08d3009ee73a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500f7eb0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e501070d0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e58016830 con 0x55cc3c7e3f30 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e501072d0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 10 client.6150 fetch_fsmap learned FSMap version 34 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55cc3c5bb3a0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58016830 con 0x55cc3c7e3f30 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e500efa00 con 0x55cc3c7e3f30 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 10 client.6150 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 10 client.6150 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 20 client.6150 populate_metadata read hostname 'smithi164' 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c7e9160 0x55cc3c83eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 4 client.6150 mds_command: new command op to 6018 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 --> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc3c5cea50 con 0x55cc3c7e9160 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e56a00640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c7e9160 0x55cc3c83eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e56a00640 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c7e9160 0x55cc3c83eec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7962f082b7fa7e server_cookie=101e97ce5086df61 in_seq=0 out_seq=0 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.401+0000 7f4e4cc00640 10 client.6150 ms_handle_connect on v2:172.21.15.164:6834/1114280106 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e4cc00640 1 -- 172.21.15.164:0/3710260737 <== mds.0 v2:172.21.15.164:6834/1114280106 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc3c5cea50 con 0x55cc3c7e9160 2024-05-08T04:40:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e4cc00640 10 client.6150 handle_command_reply: tid=0 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 client.6150 shutdown 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 msgr2=0x55cc3c834e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4e50097c40 tx=0x7f4e5011f1e0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 shutdown_connections 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c7e9160 0x55cc3c83eec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c82e240 0x55cc3c7e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c8353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 --2- 172.21.15.164:0/3710260737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c834e60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e4cc00640 0 client.6150 ms_handle_reset on v2:172.21.15.164:6834/1114280106 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 >> 172.21.15.164:0/3710260737 conn(0x55cc3c7ea2a0 msgr2=0x55cc3c7e29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 shutdown_connections 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 1 -- 172.21.15.164:0/3710260737 wait complete. 2024-05-08T04:40:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.405+0000 7f4e69651000 20 client.6150 trim_cache size 0 max 16384 2024-05-08T04:40:41.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 Processor -- start 2024-05-08T04:40:41.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- start start 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7e3f30 0x55cc3c889ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c88a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e602023d0 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600c60b0 con 0x55cc3c82d920 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60201e90 con 0x55cc3c7e3f30 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46286/0 (socket says 172.21.15.164:46286) 2024-05-08T04:40:41.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 -- 172.21.15.164:0/1162223677 learned_addr learned my addr 172.21.15.164:0/1162223677 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 -- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7e3f30 msgr2=0x55cc3c889ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7e3f30 0x55cc3c889ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 -- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 msgr2=0x55cc3c88a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c88a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 -- 172.21.15.164:0/1162223677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc3c6b4600 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e5f400640 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4e5001c8d0 tx=0x7f4e500f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6c8a47a225f158 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e4f400640 1 -- 172.21.15.164:0/1162223677 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500f7eb0 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e4f400640 1 -- 172.21.15.164:0/1162223677 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500a4e80 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e4f400640 1 -- 172.21.15.164:0/1162223677 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500efa00 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- 172.21.15.164:0/1162223677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 msgr2=0x55cc3c889580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4e5001c8d0 tx=0x7f4e500f3000 comp rx=0 tx=0).stop 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- 172.21.15.164:0/1162223677 shutdown_connections 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7e3f30 0x55cc3c889ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c88a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 --2- 172.21.15.164:0/1162223677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7ea2a0 0x55cc3c889580 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- 172.21.15.164:0/1162223677 >> 172.21.15.164:0/1162223677 conn(0x55cc3c834e80 msgr2=0x55cc3c7e7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- 172.21.15.164:0/1162223677 shutdown_connections 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- 172.21.15.164:0/1162223677 wait complete. 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 -- start start 2024-05-08T04:40:41.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.421+0000 7f4e69651000 1 Processor -- start 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 0x55cc3c871810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c871d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c6373d0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c6eed50 con 0x55cc3c82d920 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc3c5cd980 con 0x55cc3c7ea2a0 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 10 client.0 _ll_register_callbacks cb 0x7f4e58009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46294/0 (socket says 172.21.15.164:46294) 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 -- 172.21.15.164:0/882767359 learned_addr learned my addr 172.21.15.164:0/882767359 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 -- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 msgr2=0x55cc3c871810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e56a00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 0x55cc3c871810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c871d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 0x55cc3c871810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 -- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 msgr2=0x55cc3c871d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c871d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 -- 172.21.15.164:0/882767359 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc3c5c79b0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e56a00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 0x55cc3c871810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5fe00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c871d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e5f400640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4e500ebc00 tx=0x7f4e5001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1ee17ec453a6c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500ef570 con 0x55cc3c7e3f30 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e500f7c20 con 0x55cc3c7e3f30 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e500a4e80 con 0x55cc3c7e3f30 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e50107100 con 0x55cc3c7e3f30 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 10 client.6156 fetch_fsmap learned FSMap version 34 2024-05-08T04:40:41.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55cc3c5da0b0 con 0x55cc3c7e3f30 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500a4e80 con 0x55cc3c7e3f30 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e5011db70 con 0x55cc3c7e3f30 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 10 client.6156 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 10 client.6156 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 20 client.6156 populate_metadata read hostname 'smithi164' 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c82e240 0x55cc3c8726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 4 client.6156 mds_command: new command op to 6018 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 --> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc3c6402e0 con 0x55cc3c82e240 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e56a00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c82e240 0x55cc3c8726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e56a00640 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c82e240 0x55cc3c8726b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a46fc024b8de9c server_cookie=61439393320e8d81 in_seq=0 out_seq=0 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 10 client.6156 ms_handle_connect on v2:172.21.15.164:6834/1114280106 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 1 -- 172.21.15.164:0/882767359 <== mds.0 v2:172.21.15.164:6834/1114280106 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cc3c6402e0 con 0x55cc3c82e240 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 10 client.6156 handle_command_reply: tid=0 2024-05-08T04:40:41.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 client.6156 shutdown 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 msgr2=0x55cc3c83a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4e500ebc00 tx=0x7f4e5001c360 comp rx=0 tx=0).stop 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 shutdown_connections 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:6834/1114280106,v1:172.21.15.164:6837/1114280106] conn(0x55cc3c82e240 0x55cc3c8726b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cc3c7ea2a0 0x55cc3c871810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc3c82d920 0x55cc3c871d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 --2- 172.21.15.164:0/882767359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc3c7e3f30 0x55cc3c83a300 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e4cc00640 0 client.6156 ms_handle_reset on v2:172.21.15.164:6834/1114280106 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 >> 172.21.15.164:0/882767359 conn(0x55cc3c834e80 msgr2=0x55cc3c835a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 shutdown_connections 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 1 -- 172.21.15.164:0/882767359 wait complete. 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.425+0000 7f4e69651000 20 client.6156 trim_cache size 0 max 16384 2024-05-08T04:40:41.435 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 6018, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 5.8396969260000002, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 34, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 60, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 60, 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 11.745801453 2024-05-08T04:40:41.436 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 msgr2=0x7f4e3c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 0x7f4e3c069690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e6015c580 tx=0x7f4e48009040 comp rx=0 tx=0).stop 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 msgr2=0x7f4e6014ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005040 tx=0x7f4e44005840 comp rx=0 tx=0).stop 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 shutdown_connections 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e3c0671d0 0x7f4e3c069690 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e600b1190 0x7f4e600b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e60163be0 0x7f4e60164060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 --2- 172.21.15.164:0/912534721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e440039f0 0x7f4e6014ab70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 >> 172.21.15.164:0/912534721 conn(0x7f4e6009aa20 msgr2=0x7f4e600af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 shutdown_connections 2024-05-08T04:40:41.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.429+0000 7f4e64c00640 1 -- 172.21.15.164:0/912534721 wait complete. 2024-05-08T04:40:41.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:40:41.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.581+0000 7f957da00640 1 Processor -- start 2024-05-08T04:40:41.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.581+0000 7f957da00640 1 -- start start 2024-05-08T04:40:41.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f957da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f9578072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f957da00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9578072640 con 0x7f95780cfda0 2024-05-08T04:40:41.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f957da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957806c770 con 0x7f95780b26c0 2024-05-08T04:40:41.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f957da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957806ca50 con 0x7f957815ab60 2024-05-08T04:40:41.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9577400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f95780cfda0 0x7f95780cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37030/0 (socket says 172.21.15.164:37030) 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9577400640 1 -- 172.21.15.164:0/3052291040 learned_addr learned my addr 172.21.15.164:0/3052291040 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576a00640 1 --2- 172.21.15.164:0/3052291040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f9578072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96981779 0 0) 0x7f9578072640 con 0x7f95780cfda0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f955c003530 con 0x7f95780cfda0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132866573 0 0) 0x7f957806c770 con 0x7f95780b26c0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9578072640 con 0x7f95780b26c0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468783348 0 0) 0x7f955c003530 con 0x7f95780cfda0 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576a00640 1 -- 172.21.15.164:0/3052291040 >> v1:172.21.15.172:6789/0 conn(0x7f95780b26c0 legacy=0x7f95780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576a00640 1 -- 172.21.15.164:0/3052291040 >> v1:172.21.15.164:6789/0 conn(0x7f95780cfda0 legacy=0x7f95780cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.585+0000 7f9576a00640 1 -- 172.21.15.164:0/3052291040 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957806cd30 con 0x7f957815ab60 2024-05-08T04:40:41.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f9576a00640 1 --2- 172.21.15.164:0/3052291040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f9578072100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104b7b7055fc9263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570001680 con 0x7f957815ab60 2024-05-08T04:40:41.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95680a6500 con 0x7f957815ab60 2024-05-08T04:40:41.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f9576000640 1 -- 172.21.15.164:0/3052291040 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680a68d0 con 0x7f957815ab60 2024-05-08T04:40:41.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- 172.21.15.164:0/3052291040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 msgr2=0x7f9578072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 --2- 172.21.15.164:0/3052291040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f9578072100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956801c8b0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- 172.21.15.164:0/3052291040 shutdown_connections 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 --2- 172.21.15.164:0/3052291040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f9578072100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- 172.21.15.164:0/3052291040 >> 172.21.15.164:0/3052291040 conn(0x7f957806ac90 msgr2=0x7f957806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- 172.21.15.164:0/3052291040 shutdown_connections 2024-05-08T04:40:41.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- 172.21.15.164:0/3052291040 wait complete. 2024-05-08T04:40:41.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 Processor -- start 2024-05-08T04:40:41.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.589+0000 7f957da00640 1 -- start start 2024-05-08T04:40:41.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780b26c0 0x7f9578152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f957814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780aeeb0 con 0x7f95780cfda0 2024-05-08T04:40:41.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780aed30 con 0x7f95780b26c0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9578168980 con 0x7f957815ab60 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46308/0 (socket says 172.21.15.164:46308) 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 -- 172.21.15.164:0/3069497673 learned_addr learned my addr 172.21.15.164:0/3069497673 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f9577400640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f957814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f9576a00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780b26c0 0x7f9578152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 -- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 msgr2=0x7f957814f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f957814f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 -- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780b26c0 msgr2=0x7f9578152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780b26c0 0x7f9578152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 -- 172.21.15.164:0/3069497673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957815cb60 con 0x7f95780cfda0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956e000640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f956000c500 tx=0x7f956000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95600052d0 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560005490 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957815ce40 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560012850 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957815d400 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9578069d60 con 0x7f95780cfda0 2024-05-08T04:40:41.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.593+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f95600129f0 con 0x7f95780cfda0 2024-05-08T04:40:41.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.597+0000 7f956f400640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 0x7f9554069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:41.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.597+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9560085080 con 0x7f95780cfda0 2024-05-08T04:40:41.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.597+0000 7f9576a00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 0x7f9554069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:41.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.597+0000 7f9576a00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 0x7f9554069690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f95680af000 tx=0x7f95680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:41.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.601+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9560055630 con 0x7f95780cfda0 2024-05-08T04:40:41.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f957806d9a0 con 0x7f95780cfda0 2024-05-08T04:40:41.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f956f400640 1 -- 172.21.15.164:0/3069497673 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f95600596d0 con 0x7f95780cfda0 2024-05-08T04:40:41.794 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 msgr2=0x7f9554069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 0x7f9554069690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f95680af000 tx=0x7f95680a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 msgr2=0x7f957815c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f956000c500 tx=0x7f956000c9c0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 shutdown_connections 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95540671d0 0x7f9554069690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f957815ab60 0x7f957814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95780cfda0 0x7f957815c0a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 --2- 172.21.15.164:0/3069497673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780b26c0 0x7f9578152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:41.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.789+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 >> 172.21.15.164:0/3069497673 conn(0x7f957806ac90 msgr2=0x7f95780cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:41.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.793+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 shutdown_connections 2024-05-08T04:40:41.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:41.793+0000 7f957da00640 1 -- 172.21.15.164:0/3069497673 wait complete. 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:40:41.816 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:41.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:41.816 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:40:41.825 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:40:41.825 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:40:41.825 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:40:41.825 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.826 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.827 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:40:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:41.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:41.831 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:40:41.888 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:40:41.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:41.891 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:40:41.941 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:40:41.941 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:40:41.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:40:41.943 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:40:42.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.034 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:40:42.043 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:40:42.043 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:40:42.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:42.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:40:42.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:42.111 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:42.163 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:40:42.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:42.163 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:42.214 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:40:42.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:40:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:40:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:40:42.330 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:40:42.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.331 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:42.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.391 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:40:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:client6165 2024-05-08T04:40:42.451 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:42.451 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:42.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:40:42.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 Processor -- start 2024-05-08T04:40:42.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 -- start start 2024-05-08T04:40:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f375c15ba70 0x7f375c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f375c0cfda0 0x7f375c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0cd9f0 con 0x7f375c15be50 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c15fe70 con 0x7f375c0cfda0 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.681+0000 7f3764200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c0cdcd0 con 0x7f375c15ba70 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f375c0cfda0 0x7f375c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f375c0cfda0 0x7f375c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46314/0 (socket says 172.21.15.164:46314) 2024-05-08T04:40:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 -- 172.21.15.164:0/1009250001 learned_addr learned my addr 172.21.15.164:0/1009250001 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3761800640 1 --2- 172.21.15.164:0/1009250001 >> v2:172.21.15.172:3300/0 conn(0x7f375c15ba70 0x7f375c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 -- 172.21.15.164:0/1009250001 >> v2:172.21.15.172:3300/0 conn(0x7f375c15ba70 msgr2=0x7f375c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 --2- 172.21.15.164:0/1009250001 >> v2:172.21.15.172:3300/0 conn(0x7f375c15ba70 0x7f375c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 -- 172.21.15.164:0/1009250001 >> v1:172.21.15.172:6789/0 conn(0x7f375c15be50 legacy=0x7f375c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 -- 172.21.15.164:0/1009250001 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c15f820 con 0x7f375c0cfda0 2024-05-08T04:40:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f3762200640 1 --2- 172.21.15.164:0/1009250001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c0cfda0 0x7f375c0b31a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f375c06bc70 tx=0x7f375401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66394a735daa0f57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f375be00640 1 -- 172.21.15.164:0/1009250001 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37540a5750 con 0x7f375c0cfda0 2024-05-08T04:40:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f375be00640 1 -- 172.21.15.164:0/1009250001 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37540a58f0 con 0x7f375c0cfda0 2024-05-08T04:40:42.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.685+0000 7f375be00640 1 -- 172.21.15.164:0/1009250001 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37540a5cc0 con 0x7f375c0cfda0 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- 172.21.15.164:0/1009250001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c0cfda0 msgr2=0x7f375c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- 172.21.15.164:0/1009250001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c0cfda0 0x7f375c0b31a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f375c06bc70 tx=0x7f375401c980 comp rx=0 tx=0).stop 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- 172.21.15.164:0/1009250001 shutdown_connections 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- 172.21.15.164:0/1009250001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c0cfda0 0x7f375c0b31a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- 172.21.15.164:0/1009250001 >> v2:172.21.15.172:3300/0 conn(0x7f375c15ba70 0x7f375c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- 172.21.15.164:0/1009250001 >> 172.21.15.164:0/1009250001 conn(0x7f375c06ac90 msgr2=0x7f375c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- 172.21.15.164:0/1009250001 shutdown_connections 2024-05-08T04:40:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- 172.21.15.164:0/1009250001 wait complete. 2024-05-08T04:40:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 Processor -- start 2024-05-08T04:40:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- start start 2024-05-08T04:40:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f375c0cfda0 0x7f375c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f375c15ba70 0x7f375c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c168d40 con 0x7f375c15be50 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c168bc0 con 0x7f375c0cfda0 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3764200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c168ec0 con 0x7f375c15ba70 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3762200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3762200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46328/0 (socket says 172.21.15.164:46328) 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3762200640 1 -- 172.21.15.164:0/1341939750 learned_addr learned my addr 172.21.15.164:0/1341939750 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3760e00640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f375c15ba70 0x7f375c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3761800640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f375c0cfda0 0x7f375c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3762200640 1 -- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f375c15ba70 msgr2=0x7f375c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.689+0000 7f3762200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f375c15ba70 0x7f375c154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3762200640 1 -- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f375c0cfda0 msgr2=0x7f375c0afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3762200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f375c0cfda0 0x7f375c0afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3762200640 1 -- 172.21.15.164:0/1341939750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c15f0e0 con 0x7f375c15be50 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3762200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f375401b030 tx=0x7f3754019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37540bc020 con 0x7f375c15be50 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37540b8670 con 0x7f375c15be50 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37540b8a40 con 0x7f375c15be50 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c0af340 con 0x7f375c15be50 2024-05-08T04:40:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c0af5e0 con 0x7f375c15be50 2024-05-08T04:40:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c069d60 con 0x7f375c15be50 2024-05-08T04:40:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f37540bf020 con 0x7f375c15be50 2024-05-08T04:40:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.693+0000 7f3759600640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 0x7f373c069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.697+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3754120410 con 0x7f375c15be50 2024-05-08T04:40:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.697+0000 7f3761800640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 0x7f373c069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:42.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.697+0000 7f3761800640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 0x7f373c069660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3750007a60 tx=0x7f3750008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:42.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.701+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37540c6030 con 0x7f375c15be50 2024-05-08T04:40:42.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.885+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f375c06e210 con 0x7f375c15be50 2024-05-08T04:40:42.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.885+0000 7f3759600640 1 -- 172.21.15.164:0/1341939750 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37540165a0 con 0x7f375c15be50 2024-05-08T04:40:42.891 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 msgr2=0x7f373c069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 0x7f373c069660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3750007a60 tx=0x7f3750008040 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 msgr2=0x7f375c15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f375401b030 tx=0x7f3754019720 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 shutdown_connections 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f373c0671a0 0x7f373c069660 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f375c15ba70 0x7f375c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f375c15be50 0x7f375c15e390 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 --2- 172.21.15.164:0/1341939750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f375c0cfda0 0x7f375c0afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:42.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 >> 172.21.15.164:0/1341939750 conn(0x7f375c06ac90 msgr2=0x7f375c0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:42.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 shutdown_connections 2024-05-08T04:40:42.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:42.889+0000 7f3764200640 1 -- 172.21.15.164:0/1341939750 wait complete. 2024-05-08T04:40:42.911 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:40:42.912 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:42.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.912 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:40:42.922 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:40:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:40:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:40:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42228sec preferred_lft 42228sec 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:40:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:40:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.925 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:40:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:40:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:40:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:40:42.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.926 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:40:42.994 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:40:42.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:42.996 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:40:43.055 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:40:43.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:40:43.056 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:40:43.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.156 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:40:43.165 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:40:43.165 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:40:43.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:43.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:40:43.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:43.226 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:43.274 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:40:43.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:40:43.274 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:43.325 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:40:43.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.325 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:43.416 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:43.417 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:43.417 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:43.432 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:43.432 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:43.432 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:43.432 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:43.432 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:40:43.441 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:40:43.442 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:40:43.445 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:40:43.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:43.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:43.466 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:40:43.525 INFO:teuthology.orchestra.run.smithi172.stdout:client5998 2024-05-08T04:40:43.525 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:43.525 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:43.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:40:43.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 Processor -- start 2024-05-08T04:40:43.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 -- start start 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f940cdb10 0x7f6f9406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f940d0c20 0x7f6f940ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f940cfe60 con 0x7f6f9406f520 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f940d0790 con 0x7f6f940cdb10 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f9a000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f940d0140 con 0x7f6f940d0c20 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f940cdb10 0x7f6f9406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f940cdb10 0x7f6f9406efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46338/0 (socket says 172.21.15.164:46338) 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 -- 172.21.15.164:0/3901261596 learned_addr learned my addr 172.21.15.164:0/3901261596 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f92a00640 1 --2- 172.21.15.164:0/3901261596 >> v2:172.21.15.172:3300/0 conn(0x7f6f940d0c20 0x7f6f940ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 -- 172.21.15.164:0/3901261596 >> v2:172.21.15.172:3300/0 conn(0x7f6f940d0c20 msgr2=0x7f6f940ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 --2- 172.21.15.164:0/3901261596 >> v2:172.21.15.172:3300/0 conn(0x7f6f940d0c20 0x7f6f940ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 -- 172.21.15.164:0/3901261596 >> v1:172.21.15.172:6789/0 conn(0x7f6f9406f520 legacy=0x7f6f94161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:43.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.717+0000 7f6f93400640 1 -- 172.21.15.164:0/3901261596 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f940d0420 con 0x7f6f940cdb10 2024-05-08T04:40:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f93400640 1 --2- 172.21.15.164:0/3901261596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f940cdb10 0x7f6f9406efe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6f9406df50 tx=0x7f6f8401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee296fc20c4a4e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f92000640 1 -- 172.21.15.164:0/3901261596 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840a5730 con 0x7f6f940cdb10 2024-05-08T04:40:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f92000640 1 -- 172.21.15.164:0/3901261596 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840a58d0 con 0x7f6f940cdb10 2024-05-08T04:40:43.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f92000640 1 -- 172.21.15.164:0/3901261596 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840a5ca0 con 0x7f6f940cdb10 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 -- 172.21.15.164:0/3901261596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f940cdb10 msgr2=0x7f6f9406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 --2- 172.21.15.164:0/3901261596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f940cdb10 0x7f6f9406efe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6f9406df50 tx=0x7f6f8401c960 comp rx=0 tx=0).stop 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 -- 172.21.15.164:0/3901261596 shutdown_connections 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 --2- 172.21.15.164:0/3901261596 >> v2:172.21.15.172:3300/0 conn(0x7f6f940d0c20 0x7f6f940ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 --2- 172.21.15.164:0/3901261596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f940cdb10 0x7f6f9406efe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 -- 172.21.15.164:0/3901261596 >> 172.21.15.164:0/3901261596 conn(0x7f6f940b34f0 msgr2=0x7f6f940b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 -- 172.21.15.164:0/3901261596 shutdown_connections 2024-05-08T04:40:43.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.721+0000 7f6f9a000640 1 -- 172.21.15.164:0/3901261596 wait complete. 2024-05-08T04:40:43.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 Processor -- start 2024-05-08T04:40:43.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- start start 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940cdb10 0x7f6f940b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 0x7f6f9415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416b260 con 0x7f6f9406f520 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416b0e0 con 0x7f6f940d0c20 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416b3e0 con 0x7f6f940cdb10 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46352/0 (socket says 172.21.15.164:46352) 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 -- 172.21.15.164:0/360448640 learned_addr learned my addr 172.21.15.164:0/360448640 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f92a00640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940cdb10 0x7f6f940b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93e00640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 0x7f6f9415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 -- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940cdb10 msgr2=0x7f6f940b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940cdb10 0x7f6f940b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 -- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 msgr2=0x7f6f9415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 0x7f6f9415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 -- 172.21.15.164:0/360448640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f941fb1f0 con 0x7f6f9406f520 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93e00640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 0x7f6f9415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f93400640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f84019570 tx=0x7f6f84019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840bc020 con 0x7f6f9406f520 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840b8690 con 0x7f6f9406f520 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b8a80 con 0x7f6f9406f520 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f941fb4d0 con 0x7f6f9406f520 2024-05-08T04:40:43.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f941fba90 con 0x7f6f9406f520 2024-05-08T04:40:43.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.725+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9409fae0 con 0x7f6f9406f520 2024-05-08T04:40:43.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.729+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f6f840bf020 con 0x7f6f9406f520 2024-05-08T04:40:43.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.729+0000 7f6f8b400640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 0x7f6f74069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:43.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.729+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6f841204b0 con 0x7f6f9406f520 2024-05-08T04:40:43.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.729+0000 7f6f92a00640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 0x7f6f74069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:43.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.729+0000 7f6f92a00640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 0x7f6f74069660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6f9415af00 tx=0x7f6f80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:43.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.733+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f840c6030 con 0x7f6f9406f520 2024-05-08T04:40:43.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:43.921+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f9406ef80 con 0x7f6f9406f520 2024-05-08T04:40:44.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.633+0000 7f6f8b400640 1 -- 172.21.15.164:0/360448640 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6f840165e0 con 0x7f6f9406f520 2024-05-08T04:40:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 msgr2=0x7f6f74069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 0x7f6f74069660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6f9415af00 tx=0x7f6f80008040 comp rx=0 tx=0).stop 2024-05-08T04:40:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 msgr2=0x7f6f94156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f84019570 tx=0x7f6f84019700 comp rx=0 tx=0).stop 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 shutdown_connections 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f740671a0 0x7f6f74069660 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940cdb10 0x7f6f940b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940d0c20 0x7f6f9415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 --2- 172.21.15.164:0/360448640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f9406f520 0x7f6f94156d50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 >> 172.21.15.164:0/360448640 conn(0x7f6f940b34f0 msgr2=0x7f6f9415f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 shutdown_connections 2024-05-08T04:40:44.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.637+0000 7f6f9a000640 1 -- 172.21.15.164:0/360448640 wait complete. 2024-05-08T04:40:44.664 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:40:44.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 Processor -- start 2024-05-08T04:40:44.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 -- start start 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffacc15ba70 0x7ffacc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc0cd9f0 con 0x7ffacc0cfda0 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc0cdcd0 con 0x7ffacc15be50 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad4600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc15fe70 con 0x7ffacc15ba70 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad1c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffacc15ba70 0x7ffacc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad2600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ffacc0cfda0 0x7ffacc0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37032/0 (socket says 172.21.15.164:37032) 2024-05-08T04:40:44.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad2600640 1 -- 172.21.15.164:0/3463350028 learned_addr learned my addr 172.21.15.164:0/3463350028 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:44.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad1c00640 1 -- 172.21.15.164:0/3463350028 >> v1:172.21.15.172:6789/0 conn(0x7ffacc15be50 legacy=0x7ffacc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:44.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad1c00640 1 -- 172.21.15.164:0/3463350028 >> v1:172.21.15.164:6789/0 conn(0x7ffacc0cfda0 legacy=0x7ffacc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffad1c00640 1 -- 172.21.15.164:0/3463350028 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc15f820 con 0x7ffacc15ba70 2024-05-08T04:40:44.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.801+0000 7ffacbe00640 1 -- 172.21.15.164:0/3463350028 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226797394 0 0) 0x7ffacc0cd9f0 con 0x7ffacc0cfda0 2024-05-08T04:40:44.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad1c00640 1 --2- 172.21.15.164:0/3463350028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc160300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097c0 tx=0x7ffac001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df60b00b0c0c3b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:44.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffacbe00640 1 -- 172.21.15.164:0/3463350028 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b0070 con 0x7ffacc15ba70 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffacbe00640 1 -- 172.21.15.164:0/3463350028 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00ab600 con 0x7ffacc15ba70 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffacbe00640 1 -- 172.21.15.164:0/3463350028 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00ab9d0 con 0x7ffacc15ba70 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 -- 172.21.15.164:0/3463350028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 msgr2=0x7ffacc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 --2- 172.21.15.164:0/3463350028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc160300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097c0 tx=0x7ffac001c430 comp rx=0 tx=0).stop 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 -- 172.21.15.164:0/3463350028 shutdown_connections 2024-05-08T04:40:44.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 --2- 172.21.15.164:0/3463350028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc160300 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 -- 172.21.15.164:0/3463350028 >> 172.21.15.164:0/3463350028 conn(0x7ffacc06ac90 msgr2=0x7ffacc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:44.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 -- 172.21.15.164:0/3463350028 shutdown_connections 2024-05-08T04:40:44.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.805+0000 7ffad4600640 1 -- 172.21.15.164:0/3463350028 wait complete. 2024-05-08T04:40:44.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 Processor -- start 2024-05-08T04:40:44.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- start start 2024-05-08T04:40:44.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 0x7ffacc071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:44.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:44.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffacc15be50 0x7ffacc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:44.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc169f60 con 0x7ffacc15ba70 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc169de0 con 0x7ffacc0cfda0 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc16a0e0 con 0x7ffacc15be50 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46370/0 (socket says 172.21.15.164:46370) 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 -- 172.21.15.164:0/1629287395 learned_addr learned my addr 172.21.15.164:0/1629287395 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1c00640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 0x7ffacc071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad2600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffacc15be50 0x7ffacc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 -- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffacc15be50 msgr2=0x7ffacc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffacc15be50 0x7ffacc0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 -- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 msgr2=0x7ffacc071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 0x7ffacc071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 -- 172.21.15.164:0/1629287395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc14f170 con 0x7ffacc15ba70 2024-05-08T04:40:44.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1c00640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 0x7ffacc071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad1200640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b440 tx=0x7ffab800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8015020 con 0x7ffacc15ba70 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab8011890 con 0x7ffacc15ba70 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8011c80 con 0x7ffacc15ba70 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc14f450 con 0x7ffacc15ba70 2024-05-08T04:40:44.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc071850 con 0x7ffacc15ba70 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.809+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc069d60 con 0x7ffacc15ba70 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.813+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7ffab8018020 con 0x7ffacc15ba70 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.813+0000 7ffac9600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 0x7ffaac069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.813+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ffab8085380 con 0x7ffacc15ba70 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.813+0000 7ffad1c00640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 0x7ffaac069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:44.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.813+0000 7ffad1c00640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 0x7ffaac069660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffacc15fde0 tx=0x7ffac00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:44.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.817+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffab801f030 con 0x7ffacc15ba70 2024-05-08T04:40:44.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.993+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffacc06c580 con 0x7ffaac0671a0 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.993+0000 7ffac9600640 1 -- 172.21.15.164:0/1629287395 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ffacc06c580 con 0x7ffaac0671a0 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:40:44.998 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 msgr2=0x7ffaac069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 0x7ffaac069660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffacc15fde0 tx=0x7ffac00a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 msgr2=0x7ffacc14e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b440 tx=0x7ffab800b900 comp rx=0 tx=0).stop 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 shutdown_connections 2024-05-08T04:40:45.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffaac0671a0 0x7ffaac069660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffacc15be50 0x7ffacc0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffacc15ba70 0x7ffacc14e740 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 --2- 172.21.15.164:0/1629287395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffacc0cfda0 0x7ffacc071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 >> 172.21.15.164:0/1629287395 conn(0x7ffacc06ac90 msgr2=0x7ffacc1500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 shutdown_connections 2024-05-08T04:40:45.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:44.997+0000 7ffad4600640 1 -- 172.21.15.164:0/1629287395 wait complete. 2024-05-08T04:40:45.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:40:45.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 Processor -- start 2024-05-08T04:40:45.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 -- start start 2024-05-08T04:40:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f093415f200 0x7f09340cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0934160820 con 0x7f09340cebb0 2024-05-08T04:40:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340b2a80 con 0x7f09340b2fb0 2024-05-08T04:40:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f093ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340cce70 con 0x7f093415f200 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f093415f200 0x7f09340cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f093415f200 0x7f09340cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46380/0 (socket says 172.21.15.164:46380) 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0938c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f09340cebb0 0x7f0934160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37040/0 (socket says 172.21.15.164:37040) 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 -- 172.21.15.164:0/1484651078 learned_addr learned my addr 172.21.15.164:0/1484651078 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 -- 172.21.15.164:0/1484651078 >> v1:172.21.15.172:6789/0 conn(0x7f09340b2fb0 legacy=0x7f093415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 -- 172.21.15.164:0/1484651078 >> v1:172.21.15.164:6789/0 conn(0x7f09340cebb0 legacy=0x7f0934160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0933e00640 1 -- 172.21.15.164:0/1484651078 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340cd0c0 con 0x7f093415f200 2024-05-08T04:40:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.153+0000 7f0932a00640 1 -- 172.21.15.164:0/1484651078 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132283372 0 0) 0x7f0934160820 con 0x7f09340cebb0 2024-05-08T04:40:45.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f0933e00640 1 --2- 172.21.15.164:0/1484651078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f093415f200 0x7f09340cf9f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e50 tx=0x7f092401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a2beed46c1d5cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f0932a00640 1 -- 172.21.15.164:0/1484651078 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240a57a0 con 0x7f093415f200 2024-05-08T04:40:45.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f0932a00640 1 -- 172.21.15.164:0/1484651078 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240a5940 con 0x7f093415f200 2024-05-08T04:40:45.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f0932a00640 1 -- 172.21.15.164:0/1484651078 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240a5d10 con 0x7f093415f200 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 -- 172.21.15.164:0/1484651078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f093415f200 msgr2=0x7f09340cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 --2- 172.21.15.164:0/1484651078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f093415f200 0x7f09340cf9f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e50 tx=0x7f092401c8e0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 -- 172.21.15.164:0/1484651078 shutdown_connections 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 --2- 172.21.15.164:0/1484651078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f093415f200 0x7f09340cf9f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 -- 172.21.15.164:0/1484651078 >> 172.21.15.164:0/1484651078 conn(0x7f09340d1980 msgr2=0x7f09340d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 -- 172.21.15.164:0/1484651078 shutdown_connections 2024-05-08T04:40:45.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.157+0000 7f093ac00640 1 -- 172.21.15.164:0/1484651078 wait complete. 2024-05-08T04:40:45.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 Processor -- start 2024-05-08T04:40:45.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- start start 2024-05-08T04:40:45.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09340b2fb0 0x7f093414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f093415f200 0x7f09340c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093415f5e0 con 0x7f09340cebb0 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340d0340 con 0x7f09340b2fb0 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093415f760 con 0x7f093415f200 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46388/0 (socket says 172.21.15.164:46388) 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 -- 172.21.15.164:0/1349012221 learned_addr learned my addr 172.21.15.164:0/1349012221 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933e00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09340b2fb0 0x7f093414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0938c00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f093415f200 0x7f09340c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 -- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f093415f200 msgr2=0x7f09340c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f093415f200 0x7f09340c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 -- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09340b2fb0 msgr2=0x7f093414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09340b2fb0 0x7f093414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 -- 172.21.15.164:0/1349012221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093414cbf0 con 0x7f09340cebb0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f0933400640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f092000c3c0 tx=0x7f092000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920017070 con 0x7f09340cebb0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09200052d0 con 0x7f09340cebb0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920003820 con 0x7f09340cebb0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340b11f0 con 0x7f09340cebb0 2024-05-08T04:40:45.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.161+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340c8b80 con 0x7f09340cebb0 2024-05-08T04:40:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093404f010 con 0x7f09340cebb0 2024-05-08T04:40:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f092001a070 con 0x7f09340cebb0 2024-05-08T04:40:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f092be00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 0x7f0914069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0920084610 con 0x7f09340cebb0 2024-05-08T04:40:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f0933e00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 0x7f0914069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.165+0000 7f0933e00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 0x7f0914069660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09240ae000 tx=0x7f09240a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.169+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0920005480 con 0x7f09340cebb0 2024-05-08T04:40:45.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0934072890 con 0x7f09340cebb0 2024-05-08T04:40:45.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f092be00640 1 -- 172.21.15.164:0/1349012221 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0920054d10 con 0x7f09340cebb0 2024-05-08T04:40:45.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 msgr2=0x7f0914069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 0x7f0914069660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09240ae000 tx=0x7f09240a7070 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 msgr2=0x7f09340b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f092000c3c0 tx=0x7f092000c880 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 shutdown_connections 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f09140671a0 0x7f0914069660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f093415f200 0x7f09340c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09340cebb0 0x7f09340b0810 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 --2- 172.21.15.164:0/1349012221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09340b2fb0 0x7f093414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 >> 172.21.15.164:0/1349012221 conn(0x7f09340d1980 msgr2=0x7f09340cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 shutdown_connections 2024-05-08T04:40:45.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.325+0000 7f093ac00640 1 -- 172.21.15.164:0/1349012221 wait complete. 2024-05-08T04:40:45.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol88810 2024-05-08T04:40:45.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 Processor -- start 2024-05-08T04:40:45.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 -- start start 2024-05-08T04:40:45.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 0x7fc16c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc16c0d1270 0x7fc16c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c06eea0 con 0x7fc16c0b0b90 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c06c490 con 0x7fc16c0d1270 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c06f120 con 0x7fc16c06f6f0 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc16c0d1270 0x7fc16c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc16c0d1270 0x7fc16c0ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46400/0 (socket says 172.21.15.164:46400) 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 -- 172.21.15.164:0/177476014 learned_addr learned my addr 172.21.15.164:0/177476014 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc162000640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 0x7fc16c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 -- 172.21.15.164:0/177476014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 msgr2=0x7fc16c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 0x7fc16c0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 -- 172.21.15.164:0/177476014 >> v1:172.21.15.172:6789/0 conn(0x7fc16c0b0b90 legacy=0x7fc16c06c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc16b400640 1 -- 172.21.15.164:0/177476014 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c06f400 con 0x7fc16c0d1270 2024-05-08T04:40:45.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.485+0000 7fc162000640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 0x7fc16c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:45.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16b400640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c0d1270 0x7fc16c0ce730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc164009870 tx=0x7fc16401c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc825af532400f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16a000640 1 -- 172.21.15.164:0/177476014 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640b0070 con 0x7fc16c0d1270 2024-05-08T04:40:45.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16a000640 1 -- 172.21.15.164:0/177476014 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1640ab600 con 0x7fc16c0d1270 2024-05-08T04:40:45.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16a000640 1 -- 172.21.15.164:0/177476014 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640ab9d0 con 0x7fc16c0d1270 2024-05-08T04:40:45.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 -- 172.21.15.164:0/177476014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c0d1270 msgr2=0x7fc16c0ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c0d1270 0x7fc16c0ce730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc164009870 tx=0x7fc16401c2b0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 -- 172.21.15.164:0/177476014 shutdown_connections 2024-05-08T04:40:45.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c0d1270 0x7fc16c0ce730 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 --2- 172.21.15.164:0/177476014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c06f6f0 0x7fc16c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 -- 172.21.15.164:0/177476014 >> 172.21.15.164:0/177476014 conn(0x7fc16c0b3370 msgr2=0x7fc16c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 -- 172.21.15.164:0/177476014 shutdown_connections 2024-05-08T04:40:45.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.489+0000 7fc16be00640 1 -- 172.21.15.164:0/177476014 wait complete. 2024-05-08T04:40:45.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 Processor -- start 2024-05-08T04:40:45.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- start start 2024-05-08T04:40:45.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 0x7fc16c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c0d1270 0x7fc16c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b10f0 con 0x7fc16c06f6f0 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b0f70 con 0x7fc16c0b0b90 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0d0180 con 0x7fc16c0d1270 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46404/0 (socket says 172.21.15.164:46404) 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 -- 172.21.15.164:0/2495130591 learned_addr learned my addr 172.21.15.164:0/2495130591 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc162000640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 0x7fc16c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16b400640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c0d1270 0x7fc16c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 -- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c0d1270 msgr2=0x7fc16c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c0d1270 0x7fc16c15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 -- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 msgr2=0x7fc16c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 0x7fc16c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 -- 172.21.15.164:0/2495130591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0c5b60 con 0x7fc16c06f6f0 2024-05-08T04:40:45.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc162000640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 0x7fc16c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16aa00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00ba50 tx=0x7fc15c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c019070 con 0x7fc16c06f6f0 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c00cc20 con 0x7fc16c06f6f0 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0c5e40 con 0x7fc16c06f6f0 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c014440 con 0x7fc16c06f6f0 2024-05-08T04:40:45.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.493+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0c6400 con 0x7fc16c06f6f0 2024-05-08T04:40:45.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c04ef30 con 0x7fc16c06f6f0 2024-05-08T04:40:45.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fc15c01c070 con 0x7fc16c06f6f0 2024-05-08T04:40:45.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc163400640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 0x7fc148069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc15c085190 con 0x7fc16c06f6f0 2024-05-08T04:40:45.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc162000640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 0x7fc148069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.497+0000 7fc162000640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 0x7fc148069660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc16c09f4c0 tx=0x7fc154008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.501+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc15c055730 con 0x7fc16c06f6f0 2024-05-08T04:40:45.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.681+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol88810", "target": ["mon-mgr", ""]}) v1 -- 0x7fc16c067720 con 0x7fc1480671a0 2024-05-08T04:40:45.692 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:45.689+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:45.692 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:45.689+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:45.692 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:45.689+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:45.692 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:45.689+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:45.692 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:45.689+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:45.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc163400640 1 -- 172.21.15.164:0/2495130591 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc16c067720 con 0x7fc1480671a0 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 msgr2=0x7fc148069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 0x7fc148069660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc16c09f4c0 tx=0x7fc154008040 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 msgr2=0x7fc16c0c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00ba50 tx=0x7fc15c00bf10 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 shutdown_connections 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc1480671a0 0x7fc148069660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc16c0d1270 0x7fc16c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc16c0b0b90 0x7fc16c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 --2- 172.21.15.164:0/2495130591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc16c06f6f0 0x7fc16c0c5310 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.721+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 >> 172.21.15.164:0/2495130591 conn(0x7fc16c0b3370 msgr2=0x7fc16c0ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.725+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 shutdown_connections 2024-05-08T04:40:45.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.725+0000 7fc16be00640 1 -- 172.21.15.164:0/2495130591 wait complete. 2024-05-08T04:40:45.747 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol88810 subvol_snap59550 2024-05-08T04:40:45.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.881+0000 7fcf3fe00640 1 Processor -- start 2024-05-08T04:40:45.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3fe00640 1 -- start start 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf4006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40070210 con 0x7fcf400b0a20 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4006ef70 con 0x7fcf400cfb70 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006f250 con 0x7fcf400cff50 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcf400b0a20 0x7fcf400aeb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37042/0 (socket says 172.21.15.164:37042) 2024-05-08T04:40:45.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3e000640 1 -- 172.21.15.164:0/2562242384 learned_addr learned my addr 172.21.15.164:0/2562242384 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:45.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf4006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054839952 0 0) 0x7fcf40070210 con 0x7fcf400b0a20 2024-05-08T04:40:45.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003530 con 0x7fcf400b0a20 2024-05-08T04:40:45.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143609357 0 0) 0x7fcf4006ef70 con 0x7fcf400cfb70 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf40070210 con 0x7fcf400cfb70 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289032179 0 0) 0x7fcf24003530 con 0x7fcf400b0a20 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 msgr2=0x7fcf4006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf4006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 >> v1:172.21.15.172:6789/0 conn(0x7fcf400cfb70 legacy=0x7fcf400b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400704f0 con 0x7fcf400b0a20 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fcf2c001600 con 0x7fcf400b0a20 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 >> v1:172.21.15.164:6789/0 conn(0x7fcf400b0a20 legacy=0x7fcf400aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf24003b80 0x7fcf24003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4006f250 con 0x7fcf24004d50 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400bb60 con 0x7fcf24003b80 2024-05-08T04:40:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400bd30 con 0x7fcf400cff50 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3e000640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3ea00640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf24003b80 0x7fcf24003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 -- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 msgr2=0x7fcf24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400cff50 0x7fcf24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 -- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf24003b80 msgr2=0x7fcf24003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf24003b80 0x7fcf24003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3f400640 1 -- 172.21.15.164:0/2562242384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24003530 con 0x7fcf24004d50 2024-05-08T04:40:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.885+0000 7fcf3ea00640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf24003b80 0x7fcf24003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3f400640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf240051d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009b10 tx=0x7fcf3801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790411df90544d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380b2070 con 0x7fcf24004d50 2024-05-08T04:40:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf380a5660 con 0x7fcf24004d50 2024-05-08T04:40:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3d600640 1 -- 172.21.15.164:0/2562242384 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380096b0 con 0x7fcf24004d50 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 -- 172.21.15.164:0/2562242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 msgr2=0x7fcf240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/2562242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf240051d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009b10 tx=0x7fcf3801ca70 comp rx=0 tx=0).stop 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3f400640 1 -- 172.21.15.164:0/2562242384 reap_dead start 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 -- 172.21.15.164:0/2562242384 shutdown_connections 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 -- 172.21.15.164:0/2562242384 >> 172.21.15.164:0/2562242384 conn(0x7fcf4009aa20 msgr2=0x7fcf400b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 -- 172.21.15.164:0/2562242384 shutdown_connections 2024-05-08T04:40:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.889+0000 7fcf3fe00640 1 -- 172.21.15.164:0/2562242384 wait complete. 2024-05-08T04:40:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 Processor -- start 2024-05-08T04:40:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- start start 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 0x7fcf4014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400b7150 0x7fcf40151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400b1040 con 0x7fcf24004d50 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400b0ec0 con 0x7fcf400cb0e0 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400b11c0 con 0x7fcf400b7150 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46430/0 (socket says 172.21.15.164:46430) 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 -- 172.21.15.164:0/79083627 learned_addr learned my addr 172.21.15.164:0/79083627 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3f400640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400b7150 0x7fcf40151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3e000640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 0x7fcf4014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 -- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400b7150 msgr2=0x7fcf40151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400b7150 0x7fcf40151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 -- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 msgr2=0x7fcf4014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 0x7fcf4014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 -- 172.21.15.164:0/79083627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4014fbb0 con 0x7fcf24004d50 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3e000640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 0x7fcf4014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3ea00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf30007be0 tx=0x7fcf3000f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30010040 con 0x7fcf24004d50 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3001a070 con 0x7fcf24004d50 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000fe60 con 0x7fcf24004d50 2024-05-08T04:40:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400b48b0 con 0x7fcf24004d50 2024-05-08T04:40:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400ca8e0 con 0x7fcf24004d50 2024-05-08T04:40:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.893+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4004ef30 con 0x7fcf24004d50 2024-05-08T04:40:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.897+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fcf30015830 con 0x7fcf24004d50 2024-05-08T04:40:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.897+0000 7fcf36a00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 0x7fcf20069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.897+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcf30088ce0 con 0x7fcf24004d50 2024-05-08T04:40:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.897+0000 7fcf3e000640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 0x7fcf20069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.897+0000 7fcf3e000640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 0x7fcf20069690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006de20 tx=0x7fcf2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:45.901+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf300598d0 con 0x7fcf24004d50 2024-05-08T04:40:46.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.081+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol88810", "snap_name": "subvol_snap59550", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf400b0440 con 0x7fcf200671d0 2024-05-08T04:40:46.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.093+0000 7fcf36a00640 1 -- 172.21.15.164:0/79083627 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcf400b0440 con 0x7fcf200671d0 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 msgr2=0x7fcf20069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 0x7fcf20069690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006de20 tx=0x7fcf2c009040 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 msgr2=0x7fcf400b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf30007be0 tx=0x7fcf3000f540 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 shutdown_connections 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcf200671d0 0x7fcf20069690 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf400b7150 0x7fcf40151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf400cb0e0 0x7fcf4014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 --2- 172.21.15.164:0/79083627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf24004d50 0x7fcf400b42b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 >> 172.21.15.164:0/79083627 conn(0x7fcf4009aa20 msgr2=0x7fcf400b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 shutdown_connections 2024-05-08T04:40:46.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.097+0000 7fcf3fe00640 1 -- 172.21.15.164:0/79083627 wait complete. 2024-05-08T04:40:46.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol88810 subvol_snap59550 2024-05-08T04:40:46.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.257+0000 7f9a8ec00640 1 Processor -- start 2024-05-08T04:40:46.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.257+0000 7f9a8ec00640 1 -- start start 2024-05-08T04:40:46.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a880cecc0 0x7f9a880ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8815e870 con 0x7f9a8815ab10 2024-05-08T04:40:46.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880cf130 con 0x7f9a880cecc0 2024-05-08T04:40:46.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8806bb80 con 0x7f9a8806cce0 2024-05-08T04:40:46.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a880cecc0 0x7f9a880ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a7fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a87e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9a8815ab10 0x7f9a880ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46236/0 (socket says 172.21.15.164:46236) 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a87e00640 1 -- 172.21.15.164:0/1591341490 learned_addr learned my addr 172.21.15.164:0/1591341490 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a880cecc0 0x7f9a880ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:60454/0 (socket says 172.21.15.164:60454) 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a87400640 1 -- 172.21.15.164:0/1591341490 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704119603 0 0) 0x7f9a8815e870 con 0x7f9a8815ab10 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a87400640 1 -- 172.21.15.164:0/1591341490 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a6c003530 con 0x7f9a8815ab10 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 msgr2=0x7f9a880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1591341490 >> v1:172.21.15.172:6789/0 conn(0x7f9a8815ab10 legacy=0x7f9a880ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1591341490 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8815eb50 con 0x7f9a880cecc0 2024-05-08T04:40:46.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.261+0000 7f9a7fe00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:46.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8cc00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a880ca6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009d90 tx=0x7f9a8001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8148ec56b006e4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a87400640 1 -- 172.21.15.164:0/1591341490 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800b2070 con 0x7f9a880cecc0 2024-05-08T04:40:46.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a87400640 1 -- 172.21.15.164:0/1591341490 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a800a5880 con 0x7f9a880cecc0 2024-05-08T04:40:46.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a87400640 1 -- 172.21.15.164:0/1591341490 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800a5c50 con 0x7f9a880cecc0 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 msgr2=0x7f9a880ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a880ca6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009d90 tx=0x7f9a8001c8b0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1591341490 shutdown_connections 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1591341490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a880ca6e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1591341490 >> 172.21.15.164:0/1591341490 conn(0x7f9a880cf770 msgr2=0x7f9a880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1591341490 shutdown_connections 2024-05-08T04:40:46.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1591341490 wait complete. 2024-05-08T04:40:46.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 Processor -- start 2024-05-08T04:40:46.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.265+0000 7f9a8ec00640 1 -- start start 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a881f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880aedf0 con 0x7f9a8815ab10 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8815aef0 con 0x7f9a880cecc0 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880aef70 con 0x7f9a8806cce0 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46436/0 (socket says 172.21.15.164:46436) 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1637592872 learned_addr learned my addr 172.21.15.164:0/1637592872 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:46.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a7fe00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a881f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a87e00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 msgr2=0x7f9a880b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 msgr2=0x7f9a881f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a881f81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 -- 172.21.15.164:0/1637592872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880cda50 con 0x7f9a8815ab10 2024-05-08T04:40:46.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8cc00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a8001af60 tx=0x7f9a80019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800b2050 con 0x7f9a8815ab10 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a800be070 con 0x7f9a8815ab10 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880cdd30 con 0x7f9a8815ab10 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800ad850 con 0x7f9a8815ab10 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a880ce2f0 con 0x7f9a8815ab10 2024-05-08T04:40:46.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a88072d30 con 0x7f9a8815ab10 2024-05-08T04:40:46.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.269+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f9a800ad9f0 con 0x7f9a8815ab10 2024-05-08T04:40:46.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.273+0000 7f9a84c00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 0x7f9a64069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.273+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9a801205a0 con 0x7f9a8815ab10 2024-05-08T04:40:46.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.273+0000 7f9a87e00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 0x7f9a64069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.273+0000 7f9a87e00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 0x7f9a64069690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005890 tx=0x7f9a78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.277+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a80016560 con 0x7f9a8815ab10 2024-05-08T04:40:46.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.457+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol88810", "snap_name": "subvol_snap59550", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a881f6fe0 con 0x7f9a640671d0 2024-05-08T04:40:46.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a84c00640 1 -- 172.21.15.164:0/1637592872 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9a78001cd0 con 0x7f9a640671d0 2024-05-08T04:40:46.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 msgr2=0x7f9a64069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 0x7f9a64069690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005890 tx=0x7f9a78009040 comp rx=0 tx=0).stop 2024-05-08T04:40:46.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 msgr2=0x7f9a880ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a8001af60 tx=0x7f9a80019650 comp rx=0 tx=0).stop 2024-05-08T04:40:46.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 shutdown_connections 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9a640671d0 0x7f9a64069690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a8815ab10 0x7f9a880ad890 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a880cecc0 0x7f9a881f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 --2- 172.21.15.164:0/1637592872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a8806cce0 0x7f9a880b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 >> 172.21.15.164:0/1637592872 conn(0x7f9a880cf770 msgr2=0x7f9a880b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 shutdown_connections 2024-05-08T04:40:46.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.481+0000 7f9a8ec00640 1 -- 172.21.15.164:0/1637592872 wait complete. 2024-05-08T04:40:46.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol88810 subvol_snap59550 2024-05-08T04:40:46.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 Processor -- start 2024-05-08T04:40:46.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 -- start start 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f33380b3fe0 0x7f33380d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 0x7f333815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3338070280 con 0x7f333815f2b0 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333815f690 con 0x7f33380b3fe0 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f333e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33380cff80 con 0x7f33380cd560 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f33380b3fe0 0x7f33380d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3337400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f333815f2b0 0x7f33380cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37046/0 (socket says 172.21.15.164:37046) 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3337400640 1 -- 172.21.15.164:0/1004594350 learned_addr learned my addr 172.21.15.164:0/1004594350 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:46.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3337e00640 1 --2- 172.21.15.164:0/1004594350 >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 0x7f333815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336a00640 1 -- 172.21.15.164:0/1004594350 >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 msgr2=0x7f333815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336a00640 1 --2- 172.21.15.164:0/1004594350 >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 0x7f333815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336a00640 1 -- 172.21.15.164:0/1004594350 >> v1:172.21.15.164:6789/0 conn(0x7f333815f2b0 legacy=0x7f33380cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336a00640 1 -- 172.21.15.164:0/1004594350 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338070560 con 0x7f33380b3fe0 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3336000640 1 -- 172.21.15.164:0/1004594350 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773572635 0 0) 0x7f3338070280 con 0x7f333815f2b0 2024-05-08T04:40:46.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.645+0000 7f3337e00640 1 --2- 172.21.15.164:0/1004594350 >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 0x7f333815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:46.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f3336a00640 1 --2- 172.21.15.164:0/1004594350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33380b3fe0 0x7f33380d0410 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3338072340 tx=0x7f332001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad0845514007bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f3336000640 1 -- 172.21.15.164:0/1004594350 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33200ab650 con 0x7f33380b3fe0 2024-05-08T04:40:46.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f3336000640 1 -- 172.21.15.164:0/1004594350 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33200b0070 con 0x7f33380b3fe0 2024-05-08T04:40:46.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f3336000640 1 -- 172.21.15.164:0/1004594350 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33200aba10 con 0x7f33380b3fe0 2024-05-08T04:40:46.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 -- 172.21.15.164:0/1004594350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33380b3fe0 msgr2=0x7f33380d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 --2- 172.21.15.164:0/1004594350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33380b3fe0 0x7f33380d0410 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3338072340 tx=0x7f332001c450 comp rx=0 tx=0).stop 2024-05-08T04:40:46.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 -- 172.21.15.164:0/1004594350 shutdown_connections 2024-05-08T04:40:46.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 --2- 172.21.15.164:0/1004594350 >> v2:172.21.15.172:3300/0 conn(0x7f33380cd560 0x7f333815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 --2- 172.21.15.164:0/1004594350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33380b3fe0 0x7f33380d0410 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 -- 172.21.15.164:0/1004594350 >> 172.21.15.164:0/1004594350 conn(0x7f33380b34f0 msgr2=0x7f33380b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:46.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 -- 172.21.15.164:0/1004594350 shutdown_connections 2024-05-08T04:40:46.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.649+0000 7f333e000640 1 -- 172.21.15.164:0/1004594350 wait complete. 2024-05-08T04:40:46.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 Processor -- start 2024-05-08T04:40:46.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- start start 2024-05-08T04:40:46.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33380b3fe0 0x7f3338156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33380cd560 0x7f33380c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f333815f2b0 0x7f33380aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338168d70 con 0x7f333815f2b0 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338168bf0 con 0x7f33380cd560 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3338168ef0 con 0x7f33380b3fe0 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f333815f2b0 0x7f33380aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f333815f2b0 0x7f33380aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46450/0 (socket says 172.21.15.164:46450) 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 -- 172.21.15.164:0/1081986761 learned_addr learned my addr 172.21.15.164:0/1081986761 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337400640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33380b3fe0 0x7f3338156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3336a00640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33380cd560 0x7f33380c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 -- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33380b3fe0 msgr2=0x7f3338156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33380b3fe0 0x7f3338156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 -- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33380cd560 msgr2=0x7f33380c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:46.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33380cd560 0x7f33380c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:46.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 -- 172.21.15.164:0/1081986761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33380ae900 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f3337e00640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f333815f2b0 0x7f33380aede0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f333000e810 tx=0x7f333000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330017070 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333000c8d0 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- 172.21.15.164:0/1081986761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33380b0630 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333000f040 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- 172.21.15.164:0/1081986761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33380afdf0 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.653+0000 7f333e000640 1 -- 172.21.15.164:0/1081986761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33380aead0 con 0x7f333815f2b0 2024-05-08T04:40:46.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.657+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7f333001a070 con 0x7f333815f2b0 2024-05-08T04:40:46.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.657+0000 7f332f400640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f33140671a0 0x7f3314069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:46.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.657+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f33300850a0 con 0x7f333815f2b0 2024-05-08T04:40:46.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.657+0000 7f3337400640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f33140671a0 0x7f3314069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:46.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.657+0000 7f3337400640 1 --2- 172.21.15.164:0/1081986761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f33140671a0 0x7f3314069660 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33280012f0 tx=0x7f3328001690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:46.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.661+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33300556a0 con 0x7f333815f2b0 2024-05-08T04:40:46.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.849+0000 7f333e000640 1 -- 172.21.15.164:0/1081986761 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol88810", "snap_name": "subvol_snap59550", "target": ["mon-mgr", ""]}) v1 -- 0x7f333809fae0 con 0x7f33140671a0 2024-05-08T04:40:46.864 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:46.861+0000 7ffa56200640 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap59550'' does not exist 2024-05-08T04:40:46.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:46.861+0000 7f332f400640 1 -- 172.21.15.164:0/1081986761 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap59550'' does not exist) v1 ==== 53+0+0 (secure 0 0 0) 0x7f333809fae0 con 0x7f33140671a0 2024-05-08T04:40:46.865 INFO:teuthology.orchestra.run.smithi164.stderr:Error ENOENT: snapshot 'b'subvol_snap59550'' does not exist 2024-05-08T04:40:46.874 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T04:40:46.874 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol88810 2024-05-08T04:40:47.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 Processor -- start 2024-05-08T04:40:47.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 -- start start 2024-05-08T04:40:47.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7806cce0 0x7fdf780ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf780ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7815e870 con 0x7fdf7815ab10 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf780cf130 con 0x7fdf7806cce0 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7806bb80 con 0x7fdf780cecc0 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf6f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7806cce0 0x7fdf780ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf6f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7806cce0 0x7fdf780ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46466/0 (socket says 172.21.15.164:46466) 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf6f400640 1 -- 172.21.15.164:0/4011663033 learned_addr learned my addr 172.21.15.164:0/4011663033 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.009+0000 7fdf7cc00640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf780ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf6f400640 1 -- 172.21.15.164:0/4011663033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 msgr2=0x7fdf780ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:47.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf6f400640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf780ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf6f400640 1 -- 172.21.15.164:0/4011663033 >> v1:172.21.15.172:6789/0 conn(0x7fdf7815ab10 legacy=0x7fdf780ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf6f400640 1 -- 172.21.15.164:0/4011663033 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7815eb50 con 0x7fdf7806cce0 2024-05-08T04:40:47.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf6f400640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7806cce0 0x7fdf780ce780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009870 tx=0x7fdf6001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fc2df01c687525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf77400640 1 -- 172.21.15.164:0/4011663033 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600aa600 con 0x7fdf7806cce0 2024-05-08T04:40:47.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf77400640 1 -- 172.21.15.164:0/4011663033 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf600af070 con 0x7fdf7806cce0 2024-05-08T04:40:47.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf77400640 1 -- 172.21.15.164:0/4011663033 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600aa9c0 con 0x7fdf7806cce0 2024-05-08T04:40:47.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 -- 172.21.15.164:0/4011663033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7806cce0 msgr2=0x7fdf780ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7806cce0 0x7fdf780ce780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009870 tx=0x7fdf6001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 -- 172.21.15.164:0/4011663033 shutdown_connections 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf780ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/4011663033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7806cce0 0x7fdf780ce780 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 -- 172.21.15.164:0/4011663033 >> 172.21.15.164:0/4011663033 conn(0x7fdf780cf770 msgr2=0x7fdf780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.013+0000 7fdf7ec00640 1 -- 172.21.15.164:0/4011663033 shutdown_connections 2024-05-08T04:40:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- 172.21.15.164:0/4011663033 wait complete. 2024-05-08T04:40:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 Processor -- start 2024-05-08T04:40:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- start start 2024-05-08T04:40:47.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7806cce0 0x7fdf78158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf7814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780aedf0 con 0x7fdf7815ab10 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7815aef0 con 0x7fdf7806cce0 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780aef70 con 0x7fdf780cecc0 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46482/0 (socket says 172.21.15.164:46482) 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 -- 172.21.15.164:0/2836940251 learned_addr learned my addr 172.21.15.164:0/2836940251 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf77e00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7806cce0 0x7fdf78158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf6f400640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf7814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 -- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 msgr2=0x7fdf7814de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf7814de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 -- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7806cce0 msgr2=0x7fdf78158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7806cce0 0x7fdf78158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 -- 172.21.15.164:0/2836940251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78152ef0 con 0x7fdf7815ab10 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7cc00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d590 tx=0x7fdf7000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700125f0 con 0x7fdf7815ab10 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70019070 con 0x7fdf7815ab10 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700129f0 con 0x7fdf7815ab10 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.017+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf781581b0 con 0x7fdf7815ab10 2024-05-08T04:40:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf781531d0 con 0x7fdf7815ab10 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7814e990 con 0x7fdf7815ab10 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) v1 ==== 86931+0+0 (secure 0 0 0) 0x7fdf70012b90 con 0x7fdf7815ab10 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf74c00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 0x7fdf54069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdf70085880 con 0x7fdf7815ab10 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf77e00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 0x7fdf54069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.021+0000 7fdf77e00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 0x7fdf54069660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001cb0 tx=0x7fdf68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.025+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf70056500 con 0x7fdf7815ab10 2024-05-08T04:40:47.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.205+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol88810", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf78072d30 con 0x7fdf540671a0 2024-05-08T04:40:47.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf74c00640 1 -- 172.21.15.164:0/2836940251 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdf78072d30 con 0x7fdf540671a0 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 msgr2=0x7fdf54069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 0x7fdf54069660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001cb0 tx=0x7fdf68009040 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 msgr2=0x7fdf780afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d590 tx=0x7fdf7000da50 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 shutdown_connections 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf540671a0 0x7fdf54069660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf780cecc0 0x7fdf7814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7815ab10 0x7fdf780afe90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 --2- 172.21.15.164:0/2836940251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7806cce0 0x7fdf78158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 >> 172.21.15.164:0/2836940251 conn(0x7fdf780cf770 msgr2=0x7fdf780af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:47.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 shutdown_connections 2024-05-08T04:40:47.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.221+0000 7fdf7ec00640 1 -- 172.21.15.164:0/2836940251 wait complete. 2024-05-08T04:40:47.230 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.225+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.230 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.225+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.230 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.225+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.230 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.225+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.230 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.225+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.248 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.245+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.248 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.245+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.248 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.245+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.248 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.245+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.248 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:40:47.245+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:40:47.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:47.253 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:40:47.270 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:40:47.271 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:40:47.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:47.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:40:47.497 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:40:47.498 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:47.498 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:47.507 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:47.507 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:47.558 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:47.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.733+0000 7fd7a6a00640 1 Processor -- start 2024-05-08T04:40:47.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.733+0000 7fd7a6a00640 1 -- start start 2024-05-08T04:40:47.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7a00cfda0 0x7fd7a0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7a015be50 0x7fd7a00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a6a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00cd9f0 con 0x7fd7a015ba70 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a6a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a015fe70 con 0x7fd7a00cfda0 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a6a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a00cdcd0 con 0x7fd7a015be50 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7a00cfda0 0x7fd7a0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7a00cfda0 0x7fd7a0160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46510/0 (socket says 172.21.15.164:46510) 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 -- 172.21.15.164:0/3596912019 learned_addr learned my addr 172.21.15.164:0/3596912019 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd7a4a00640 1 --2- 172.21.15.164:0/3596912019 >> v2:172.21.15.172:3300/0 conn(0x7fd7a015be50 0x7fd7a00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 -- 172.21.15.164:0/3596912019 >> v2:172.21.15.172:3300/0 conn(0x7fd7a015be50 msgr2=0x7fd7a00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 --2- 172.21.15.164:0/3596912019 >> v2:172.21.15.172:3300/0 conn(0x7fd7a015be50 0x7fd7a00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 -- 172.21.15.164:0/3596912019 >> v1:172.21.15.172:6789/0 conn(0x7fd7a015ba70 legacy=0x7fd7a00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:47.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 -- 172.21.15.164:0/3596912019 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a015f820 con 0x7fd7a00cfda0 2024-05-08T04:40:47.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.737+0000 7fd797400640 1 --2- 172.21.15.164:0/3596912019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0160300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd7a015fde0 tx=0x7fd78801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09358e2d1164adf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd79f400640 1 -- 172.21.15.164:0/3596912019 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b1070 con 0x7fd7a00cfda0 2024-05-08T04:40:47.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd79f400640 1 -- 172.21.15.164:0/3596912019 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7880ac620 con 0x7fd7a00cfda0 2024-05-08T04:40:47.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd79f400640 1 -- 172.21.15.164:0/3596912019 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880ac9f0 con 0x7fd7a00cfda0 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3596912019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a00cfda0 msgr2=0x7fd7a0160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3596912019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0160300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd7a015fde0 tx=0x7fd78801c4b0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3596912019 shutdown_connections 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3596912019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0160300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3596912019 >> v2:172.21.15.172:3300/0 conn(0x7fd7a015be50 0x7fd7a00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3596912019 >> 172.21.15.164:0/3596912019 conn(0x7fd7a006ac90 msgr2=0x7fd7a006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3596912019 shutdown_connections 2024-05-08T04:40:47.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3596912019 wait complete. 2024-05-08T04:40:47.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 Processor -- start 2024-05-08T04:40:47.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- start start 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a015be50 0x7fd7a00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0168d40 con 0x7fd7a015ba70 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0168bc0 con 0x7fd7a00cfda0 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd7a6a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0168ec0 con 0x7fd7a015be50 2024-05-08T04:40:47.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd797400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd797400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46518/0 (socket says 172.21.15.164:46518) 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.741+0000 7fd797400640 1 -- 172.21.15.164:0/3724692761 learned_addr learned my addr 172.21.15.164:0/3724692761 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79fe00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd7a4a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a015be50 0x7fd7a00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 -- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a015be50 msgr2=0x7fd7a00717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a015be50 0x7fd7a00717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 -- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a00cfda0 msgr2=0x7fd7a0070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 -- 172.21.15.164:0/3724692761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a0072000 con 0x7fd7a015ba70 2024-05-08T04:40:47.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd797400640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd78801a0e0 tx=0x7fd78801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b1040 con 0x7fd7a015ba70 2024-05-08T04:40:47.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7880bd070 con 0x7fd7a015ba70 2024-05-08T04:40:47.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b7860 con 0x7fd7a015ba70 2024-05-08T04:40:47.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a0070410 con 0x7fd7a015ba70 2024-05-08T04:40:47.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a00709d0 con 0x7fd7a015ba70 2024-05-08T04:40:47.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0069d60 con 0x7fd7a015ba70 2024-05-08T04:40:47.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd7880ac4e0 con 0x7fd7a015ba70 2024-05-08T04:40:47.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 0x7fd77c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:47.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd78811fc70 con 0x7fd7a015ba70 2024-05-08T04:40:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.745+0000 7fd79fe00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 0x7fd77c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.749+0000 7fd79fe00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 0x7fd77c069810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd790007fd0 tx=0x7fd790007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.753+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7880160d0 con 0x7fd7a015ba70 2024-05-08T04:40:47.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.905+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7a006e210 con 0x7fd7a015ba70 2024-05-08T04:40:47.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.905+0000 7fd79cc00640 1 -- 172.21.15.164:0/3724692761 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16768 (secure 0 0 0) 0x7fd7880efee0 con 0x7fd7a015ba70 2024-05-08T04:40:47.911 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:47.911 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":60,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:36.590286+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:47.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 msgr2=0x7fd77c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 0x7fd77c069810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd790007fd0 tx=0x7fd790007e40 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 msgr2=0x7fd7a0146d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd78801a0e0 tx=0x7fd78801acf0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 shutdown_connections 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd77c067350 0x7fd77c069810 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a015be50 0x7fd7a00717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7a015ba70 0x7fd7a0146d70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 --2- 172.21.15.164:0/3724692761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a00cfda0 0x7fd7a0070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 >> 172.21.15.164:0/3724692761 conn(0x7fd7a006ac90 msgr2=0x7fd7a0155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 shutdown_connections 2024-05-08T04:40:47.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:47.909+0000 7fd7a6a00640 1 -- 172.21.15.164:0/3724692761 wait complete. 2024-05-08T04:40:47.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:40:32.669821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:40:33.956608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:47.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:48.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.065+0000 7f98b3200640 1 Processor -- start 2024-05-08T04:40:48.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.065+0000 7f98b3200640 1 -- start start 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98ac15ab10 0x7f98ac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac0cee90 0x7f98ac0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac15e870 con 0x7f98ac0ae6e0 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06bb80 con 0x7f98ac15ab10 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15eb50 con 0x7f98ac0cee90 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98ac15ab10 0x7f98ac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98ac15ab10 0x7f98ac06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46530/0 (socket says 172.21.15.164:46530) 2024-05-08T04:40:48.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 -- 172.21.15.164:0/2909950234 learned_addr learned my addr 172.21.15.164:0/2909950234 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:48.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98b1200640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac0cee90 0x7f98ac0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 -- 172.21.15.164:0/2909950234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac0cee90 msgr2=0x7f98ac0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:48.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac0cee90 0x7f98ac0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 -- 172.21.15.164:0/2909950234 >> v1:172.21.15.172:6789/0 conn(0x7f98ac0ae6e0 legacy=0x7f98ac0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.069+0000 7f98abe00640 1 -- 172.21.15.164:0/2909950234 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac15ee30 con 0x7f98ac15ab10 2024-05-08T04:40:48.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98abe00640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac15ab10 0x7f98ac06cce0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f989c00bb10 tx=0x7f989c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783c36cc1173fe8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98ab400640 1 -- 172.21.15.164:0/2909950234 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0b4070 con 0x7f98ac15ab10 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98ab400640 1 -- 172.21.15.164:0/2909950234 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0a7c00 con 0x7f98ac15ab10 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98ab400640 1 -- 172.21.15.164:0/2909950234 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0af440 con 0x7f98ac15ab10 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- 172.21.15.164:0/2909950234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac15ab10 msgr2=0x7f98ac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac15ab10 0x7f98ac06cce0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f989c00bb10 tx=0x7f989c01eb70 comp rx=0 tx=0).stop 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- 172.21.15.164:0/2909950234 shutdown_connections 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac0cee90 0x7f98ac0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- 172.21.15.164:0/2909950234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac15ab10 0x7f98ac06cce0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- 172.21.15.164:0/2909950234 >> 172.21.15.164:0/2909950234 conn(0x7f98ac0cf770 msgr2=0x7f98ac0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- 172.21.15.164:0/2909950234 shutdown_connections 2024-05-08T04:40:48.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- 172.21.15.164:0/2909950234 wait complete. 2024-05-08T04:40:48.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 Processor -- start 2024-05-08T04:40:48.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- start start 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98ac0ae6e0 0x7f98ac070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac15ab10 0x7f98ac0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0cb070 con 0x7f98ac0cee90 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0caef0 con 0x7f98ac0ae6e0 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.073+0000 7f98b3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0cb1f0 con 0x7f98ac15ab10 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46542/0 (socket says 172.21.15.164:46542) 2024-05-08T04:40:48.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 -- 172.21.15.164:0/1763994168 learned_addr learned my addr 172.21.15.164:0/1763994168 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98b1200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac15ab10 0x7f98ac0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98abe00640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98ac0ae6e0 0x7f98ac070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 -- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac15ab10 msgr2=0x7f98ac0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac15ab10 0x7f98ac0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 -- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98ac0ae6e0 msgr2=0x7f98ac070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98ac0ae6e0 0x7f98ac070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 -- 172.21.15.164:0/1763994168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac149280 con 0x7f98ac0cee90 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a3400640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f989400c420 tx=0x7f989400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940052d0 con 0x7f98ac0cee90 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894005470 con 0x7f98ac0cee90 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac149560 con 0x7f98ac0cee90 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894007260 con 0x7f98ac0cee90 2024-05-08T04:40:48.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac149b20 con 0x7f98ac0cee90 2024-05-08T04:40:48.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac072d30 con 0x7f98ac0cee90 2024-05-08T04:40:48.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f9894007400 con 0x7f98ac0cee90 2024-05-08T04:40:48.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 0x7f9888069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.077+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9894084d00 con 0x7f98ac0cee90 2024-05-08T04:40:48.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.081+0000 7f98abe00640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 0x7f9888069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.081+0000 7f98abe00640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 0x7f9888069810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f989c0b1000 tx=0x7f989c0a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.081+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98940556d0 con 0x7f98ac0cee90 2024-05-08T04:40:48.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.237+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98ac06c600 con 0x7f98ac0cee90 2024-05-08T04:40:48.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.237+0000 7f98a8c00640 1 -- 172.21.15.164:0/1763994168 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9894054fd0 con 0x7f98ac0cee90 2024-05-08T04:40:48.244 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 msgr2=0x7f9888069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 0x7f9888069810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f989c0b1000 tx=0x7f989c0a9070 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 msgr2=0x7f98ac142b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f989400c420 tx=0x7f989400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 shutdown_connections 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9888067350 0x7f9888069810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98ac15ab10 0x7f98ac0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98ac0cee90 0x7f98ac142b30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 --2- 172.21.15.164:0/1763994168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98ac0ae6e0 0x7f98ac070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 >> 172.21.15.164:0/1763994168 conn(0x7f98ac0cf770 msgr2=0x7f98ac14ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 shutdown_connections 2024-05-08T04:40:48.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.241+0000 7f98b3200640 1 -- 172.21.15.164:0/1763994168 wait complete. 2024-05-08T04:40:48.265 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:48.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 Processor -- start 2024-05-08T04:40:48.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 -- start start 2024-05-08T04:40:48.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d7c0b1c00 0x7f4d7c071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d7c0b2060 0x7f4d7c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cf460 con 0x7f4d7c0b2060 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c15b020 con 0x7f4d7c0b1c00 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15b300 con 0x7f4d7c0cc0f0 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d7c0b2060 0x7f4d7c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d7c0b2060 0x7f4d7c0716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46554/0 (socket says 172.21.15.164:46554) 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 -- 172.21.15.164:0/932554929 learned_addr learned my addr 172.21.15.164:0/932554929 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7a000640 1 --2- 172.21.15.164:0/932554929 >> v2:172.21.15.172:3300/0 conn(0x7f4d7c0b1c00 0x7f4d7c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 -- 172.21.15.164:0/932554929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0cc0f0 msgr2=0x7f4d7c0af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 --2- 172.21.15.164:0/932554929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 -- 172.21.15.164:0/932554929 >> v2:172.21.15.172:3300/0 conn(0x7f4d7c0b1c00 msgr2=0x7f4d7c071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 --2- 172.21.15.164:0/932554929 >> v2:172.21.15.172:3300/0 conn(0x7f4d7c0b1c00 0x7f4d7c071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.401+0000 7f4d7b400640 1 -- 172.21.15.164:0/932554929 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c15b5e0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7b400640 1 --2- 172.21.15.164:0/932554929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c0716b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06bc60 tx=0x7f4d7401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc114bcdd6d7740 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d79600640 1 -- 172.21.15.164:0/932554929 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a5860 con 0x7f4d7c0b2060 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d79600640 1 -- 172.21.15.164:0/932554929 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d740a5a00 con 0x7f4d7c0b2060 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d79600640 1 -- 172.21.15.164:0/932554929 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a5dd0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 -- 172.21.15.164:0/932554929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 msgr2=0x7f4d7c0716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 --2- 172.21.15.164:0/932554929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c0716b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06bc60 tx=0x7f4d7401c9c0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 -- 172.21.15.164:0/932554929 shutdown_connections 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 --2- 172.21.15.164:0/932554929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c0716b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 --2- 172.21.15.164:0/932554929 >> v2:172.21.15.172:3300/0 conn(0x7f4d7c0b1c00 0x7f4d7c071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 --2- 172.21.15.164:0/932554929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 -- 172.21.15.164:0/932554929 >> 172.21.15.164:0/932554929 conn(0x7f4d7c06ac90 msgr2=0x7f4d7c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 -- 172.21.15.164:0/932554929 shutdown_connections 2024-05-08T04:40:48.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.405+0000 7f4d7be00640 1 -- 172.21.15.164:0/932554929 wait complete. 2024-05-08T04:40:48.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 Processor -- start 2024-05-08T04:40:48.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- start start 2024-05-08T04:40:48.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0b1c00 0x7f4d7c1f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15ade0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15ac60 con 0x7f4d7c0cc0f0 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15ee40 con 0x7f4d7c0b1c00 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46566/0 (socket says 172.21.15.164:46566) 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 -- 172.21.15.164:0/1747159900 learned_addr learned my addr 172.21.15.164:0/1747159900 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7b400640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7aa00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0b1c00 0x7f4d7c1f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 -- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0b1c00 msgr2=0x7f4d7c1f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0b1c00 0x7f4d7c1f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 -- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d7c0cc0f0 msgr2=0x7f4d7c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 -- 172.21.15.164:0/1747159900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0b3200 con 0x7f4d7c0b2060 2024-05-08T04:40:48.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7a000640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e8b0 tx=0x7f4d6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68017070 con 0x7f4d7c0b2060 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6800c940 con 0x7f4d7c0b2060 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800cd10 con 0x7f4d7c0b2060 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c0b34e0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c1f7690 con 0x7f4d7c0b2060 2024-05-08T04:40:48.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.409+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c0b36b0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.413+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f4d6801a070 con 0x7f4d7c0b2060 2024-05-08T04:40:48.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.413+0000 7f4d72a00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 0x7f4d5c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.413+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4d68085a40 con 0x7f4d7c0b2060 2024-05-08T04:40:48.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.413+0000 7f4d7aa00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 0x7f4d5c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.413+0000 7f4d7aa00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 0x7f4d5c069810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009ac0 tx=0x7f4d6c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.417+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d68055e50 con 0x7f4d7c0b2060 2024-05-08T04:40:48.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.569+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d7c06dc70 con 0x7f4d7c0b2060 2024-05-08T04:40:48.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d72a00640 1 -- 172.21.15.164:0/1747159900 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d68059ef0 con 0x7f4d7c0b2060 2024-05-08T04:40:48.576 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:48.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 msgr2=0x7f4d5c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 0x7f4d5c069810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009ac0 tx=0x7f4d6c009190 comp rx=0 tx=0).stop 2024-05-08T04:40:48.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 msgr2=0x7f4d7c163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e8b0 tx=0x7f4d6800ed70 comp rx=0 tx=0).stop 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 shutdown_connections 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4d5c067350 0x7f4d5c069810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d7c0cc0f0 0x7f4d7c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d7c0b2060 0x7f4d7c163ad0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 --2- 172.21.15.164:0/1747159900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d7c0b1c00 0x7f4d7c1f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 >> 172.21.15.164:0/1747159900 conn(0x7f4d7c06ac90 msgr2=0x7f4d7c15daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 shutdown_connections 2024-05-08T04:40:48.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.573+0000 7f4d7be00640 1 -- 172.21.15.164:0/1747159900 wait complete. 2024-05-08T04:40:48.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:48.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.733+0000 7f5bd5800640 1 Processor -- start 2024-05-08T04:40:48.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 -- start start 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5bd015ab10 0x7f5bd006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 0x7f5bd00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd015e870 con 0x7f5bd00ce780 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd006bb80 con 0x7f5bd015ab10 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bd5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd015eb50 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcf400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5bd00ce780 0x7f5bd00c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37048/0 (socket says 172.21.15.164:37048) 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcf400640 1 -- 172.21.15.164:0/3457309895 learned_addr learned my addr 172.21.15.164:0/3457309895 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3457309895 >> v2:172.21.15.164:3300/0 conn(0x7f5bd015ab10 0x7f5bd006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bc7e00640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 0x7f5bd00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bce000640 1 -- 172.21.15.164:0/3457309895 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539560495 0 0) 0x7f5bd015e870 con 0x7f5bd00ce780 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bce000640 1 -- 172.21.15.164:0/3457309895 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb4005740 con 0x7f5bd00ce780 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcea00640 1 -- 172.21.15.164:0/3457309895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 msgr2=0x7f5bd00aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 0x7f5bd00aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcea00640 1 -- 172.21.15.164:0/3457309895 >> v1:172.21.15.164:6789/0 conn(0x7f5bd00ce780 legacy=0x7f5bd00c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bcea00640 1 -- 172.21.15.164:0/3457309895 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd015ee30 con 0x7f5bd015ab10 2024-05-08T04:40:48.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.737+0000 7f5bc7e00640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 0x7f5bd00aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:48.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd015ab10 0x7f5bd006cce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00097c0 tx=0x7f5bc001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa3cdb825a054d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bce000640 1 -- 172.21.15.164:0/3457309895 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b1070 con 0x7f5bd015ab10 2024-05-08T04:40:48.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bce000640 1 -- 172.21.15.164:0/3457309895 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00a8050 con 0x7f5bd015ab10 2024-05-08T04:40:48.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bce000640 1 -- 172.21.15.164:0/3457309895 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00ac860 con 0x7f5bd015ab10 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- 172.21.15.164:0/3457309895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd015ab10 msgr2=0x7f5bd006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd015ab10 0x7f5bd006cce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00097c0 tx=0x7f5bc001c3e0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- 172.21.15.164:0/3457309895 shutdown_connections 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00ae6e0 0x7f5bd00aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3457309895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd015ab10 0x7f5bd006cce0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- 172.21.15.164:0/3457309895 >> 172.21.15.164:0/3457309895 conn(0x7f5bd00cf770 msgr2=0x7f5bd00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- 172.21.15.164:0/3457309895 shutdown_connections 2024-05-08T04:40:48.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- 172.21.15.164:0/3457309895 wait complete. 2024-05-08T04:40:48.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 Processor -- start 2024-05-08T04:40:48.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.741+0000 7f5bd5800640 1 -- start start 2024-05-08T04:40:48.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd00ce780 0x7f5bd015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd015ab10 0x7f5bd0142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00cefa0 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00cd670 con 0x7f5bd00ce780 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd01426d0 con 0x7f5bd015ab10 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46598/0 (socket says 172.21.15.164:46598) 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 -- 172.21.15.164:0/3877220876 learned_addr learned my addr 172.21.15.164:0/3877220876 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 -- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd015ab10 msgr2=0x7f5bd0142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd015ab10 0x7f5bd0142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 -- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd00ce780 msgr2=0x7f5bd015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd00ce780 0x7f5bd015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 -- 172.21.15.164:0/3877220876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd015cab0 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bcea00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5bc001aa90 tx=0x7f5bc001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00bb020 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00ace10 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd015cd90 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b1040 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd015d350 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0072d30 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.745+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f5bc00be020 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.749+0000 7f5bc6a00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 0x7f5bac069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:48.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.749+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5bc011f8c0 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.749+0000 7f5bc7e00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 0x7f5bac069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:48.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.749+0000 7f5bc7e00640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 0x7f5bac069810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006e90 tx=0x7f5bb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:48.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.753+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bc00c5070 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.905+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bd004f010 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.905+0000 7f5bc6a00640 1 -- 172.21.15.164:0/3877220876 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5bc011f0a0 con 0x7f5bd00ae6e0 2024-05-08T04:40:48.911 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 msgr2=0x7f5bac069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 0x7f5bac069810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006e90 tx=0x7f5bb8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 msgr2=0x7f5bd015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5bc001aa90 tx=0x7f5bc001ac20 comp rx=0 tx=0).stop 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 shutdown_connections 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5bac067350 0x7f5bac069810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd015ab10 0x7f5bd0142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd00ce780 0x7f5bd015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 --2- 172.21.15.164:0/3877220876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd00ae6e0 0x7f5bd015c4b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:48.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 >> 172.21.15.164:0/3877220876 conn(0x7f5bd00cf770 msgr2=0x7f5bd00cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:48.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 shutdown_connections 2024-05-08T04:40:48.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:48.909+0000 7f5bd5800640 1 -- 172.21.15.164:0/3877220876 wait complete. 2024-05-08T04:40:48.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:40:49.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 Processor -- start 2024-05-08T04:40:49.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 -- start start 2024-05-08T04:40:49.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00cd9f0 con 0x7f42e00aeda0 2024-05-08T04:40:49.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e015fe70 con 0x7f42e015be50 2024-05-08T04:40:49.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00cdcd0 con 0x7f42e015ba70 2024-05-08T04:40:49.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42df400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42df400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:60576/0 (socket says 172.21.15.164:60576) 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:55168/0 (socket says 172.21.15.164:55168) 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42df400640 1 -- 172.21.15.164:0/3234054850 learned_addr learned my addr 172.21.15.164:0/3234054850 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42e5200640 1 --1- 172.21.15.164:0/3234054850 >> v1:172.21.15.172:6789/0 conn(0x7f42e00aeda0 0x7f42e00b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46276/0 (socket says 172.21.15.164:46276) 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 -- 172.21.15.164:0/3234054850 >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 msgr2=0x7f42e00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 --2- 172.21.15.164:0/3234054850 >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 -- 172.21.15.164:0/3234054850 >> v1:172.21.15.172:6789/0 conn(0x7f42e00aeda0 legacy=0x7f42e00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42dfe00640 1 -- 172.21.15.164:0/3234054850 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e015f820 con 0x7f42e015ba70 2024-05-08T04:40:49.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.065+0000 7f42df400640 1 --2- 172.21.15.164:0/3234054850 >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:49.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42dfe00640 1 --2- 172.21.15.164:0/3234054850 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f42d000dbe0 tx=0x7f42d0020780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=861253f4a244b73e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42dea00640 1 -- 172.21.15.164:0/3234054850 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000d0d0 con 0x7f42e015ba70 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42dea00640 1 -- 172.21.15.164:0/3234054850 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000d270 con 0x7f42e015ba70 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42dea00640 1 -- 172.21.15.164:0/3234054850 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00a9780 con 0x7f42e015ba70 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 -- 172.21.15.164:0/3234054850 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 msgr2=0x7f42e0160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 --2- 172.21.15.164:0/3234054850 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f42d000dbe0 tx=0x7f42d0020780 comp rx=0 tx=0).stop 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 -- 172.21.15.164:0/3234054850 shutdown_connections 2024-05-08T04:40:49.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 --2- 172.21.15.164:0/3234054850 >> v2:172.21.15.172:3300/0 conn(0x7f42e015be50 0x7f42e00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 --2- 172.21.15.164:0/3234054850 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e0160300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 -- 172.21.15.164:0/3234054850 >> 172.21.15.164:0/3234054850 conn(0x7f42e006ac90 msgr2=0x7f42e006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:49.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 -- 172.21.15.164:0/3234054850 shutdown_connections 2024-05-08T04:40:49.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.069+0000 7f42e7200640 1 -- 172.21.15.164:0/3234054850 wait complete. 2024-05-08T04:40:49.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 Processor -- start 2024-05-08T04:40:49.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- start start 2024-05-08T04:40:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e00b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 0x7f42e00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0166d30 con 0x7f42e00aeda0 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0166bb0 con 0x7f42e015be50 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0166eb0 con 0x7f42e015ba70 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46612/0 (socket says 172.21.15.164:46612) 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 -- 172.21.15.164:0/4097911117 learned_addr learned my addr 172.21.15.164:0/4097911117 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42df400640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e00b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e5200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 0x7f42e00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 -- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 msgr2=0x7f42e00b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e00b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 -- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 msgr2=0x7f42e00b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 0x7f42e00b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 -- 172.21.15.164:0/4097911117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e014a7a0 con 0x7f42e00aeda0 2024-05-08T04:40:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e5200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 0x7f42e00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42dfe00640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f42d000dbe0 tx=0x7f42d001d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000d0d0 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d00c2070 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00bc8d0 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e01507a0 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0150d60 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.073+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0069d60 con 0x7f42e00aeda0 2024-05-08T04:40:49.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.077+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f42d00bca70 con 0x7f42e00aeda0 2024-05-08T04:40:49.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.077+0000 7f42d7e00640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 0x7f42c0069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.077+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f42d0124710 con 0x7f42e00aeda0 2024-05-08T04:40:49.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.077+0000 7f42df400640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 0x7f42c0069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.077+0000 7f42df400640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 0x7f42c0069810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f42e015a8a0 tx=0x7f42cc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.081+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42d001a3a0 con 0x7f42e00aeda0 2024-05-08T04:40:49.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.245+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f42e006e210 con 0x7f42e00aeda0 2024-05-08T04:40:49.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.245+0000 7f42d7e00640 1 -- 172.21.15.164:0/4097911117 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7f42d00f5140 con 0x7f42e00aeda0 2024-05-08T04:40:49.251 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:40:49.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 msgr2=0x7f42c0069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 0x7f42c0069810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f42e015a8a0 tx=0x7f42cc009240 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 msgr2=0x7f42e014a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f42d000dbe0 tx=0x7f42d001d520 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 shutdown_connections 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42c0067350 0x7f42c0069810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42e015ba70 0x7f42e00b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42e015be50 0x7f42e00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 --2- 172.21.15.164:0/4097911117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e00aeda0 0x7f42e014a260 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 >> 172.21.15.164:0/4097911117 conn(0x7f42e006ac90 msgr2=0x7f42e006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 shutdown_connections 2024-05-08T04:40:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.249+0000 7f42e7200640 1 -- 172.21.15.164:0/4097911117 wait complete. 2024-05-08T04:40:49.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:40:49.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:49.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:40:49.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:40:49.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:49.299 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:40:49.352 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:49.352 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:49.361 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:49.362 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:49.411 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:49.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.541+0000 7f7e6f000640 1 Processor -- start 2024-05-08T04:40:49.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.541+0000 7f7e6f000640 1 -- start start 2024-05-08T04:40:49.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 0x7f7e68072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6f000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e68072640 con 0x7f7e680cfda0 2024-05-08T04:40:49.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6f000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6806c770 con 0x7f7e680b26c0 2024-05-08T04:40:49.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6806ca50 con 0x7f7e6815ab60 2024-05-08T04:40:49.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6d000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7e680cfda0 0x7f7e680cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50702/0 (socket says 172.21.15.164:50702) 2024-05-08T04:40:49.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e6d000640 1 -- 172.21.15.164:0/572071995 learned_addr learned my addr 172.21.15.164:0/572071995 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e67e00640 1 --2- 172.21.15.164:0/572071995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 0x7f7e68072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469126517 0 0) 0x7f7e68072640 con 0x7f7e680cfda0 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4c003530 con 0x7f7e680cfda0 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730112134 0 0) 0x7f7e6806c770 con 0x7f7e680b26c0 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e68072640 con 0x7f7e680b26c0 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e67e00640 1 -- 172.21.15.164:0/572071995 >> v1:172.21.15.172:6789/0 conn(0x7f7e680b26c0 legacy=0x7f7e680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e67e00640 1 -- 172.21.15.164:0/572071995 >> v1:172.21.15.164:6789/0 conn(0x7f7e680cfda0 legacy=0x7f7e680cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.545+0000 7f7e67e00640 1 -- 172.21.15.164:0/572071995 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6806cd30 con 0x7f7e6815ab60 2024-05-08T04:40:49.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e67e00640 1 --2- 172.21.15.164:0/572071995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 0x7f7e68072100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009dc0 tx=0x7f7e5801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d791d03b6db8114f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580b2070 con 0x7f7e6815ab60 2024-05-08T04:40:49.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580a5850 con 0x7f7e6815ab60 2024-05-08T04:40:49.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e66a00640 1 -- 172.21.15.164:0/572071995 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5c00 con 0x7f7e6815ab60 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- 172.21.15.164:0/572071995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 msgr2=0x7f7e68072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 --2- 172.21.15.164:0/572071995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 0x7f7e68072100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009dc0 tx=0x7f7e5801c990 comp rx=0 tx=0).stop 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- 172.21.15.164:0/572071995 shutdown_connections 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 --2- 172.21.15.164:0/572071995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6815ab60 0x7f7e68072100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- 172.21.15.164:0/572071995 >> 172.21.15.164:0/572071995 conn(0x7f7e6806ac90 msgr2=0x7f7e6806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- 172.21.15.164:0/572071995 shutdown_connections 2024-05-08T04:40:49.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- 172.21.15.164:0/572071995 wait complete. 2024-05-08T04:40:49.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 Processor -- start 2024-05-08T04:40:49.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.549+0000 7f7e6f000640 1 -- start start 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e680b26c0 0x7f7e6815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e680cfda0 0x7f7e681517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680aeeb0 con 0x7f7e6815ab60 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680aed30 con 0x7f7e680b26c0 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68168980 con 0x7f7e680cfda0 2024-05-08T04:40:49.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59082/0 (socket says 172.21.15.164:59082) 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 -- 172.21.15.164:0/3890152000 learned_addr learned my addr 172.21.15.164:0/3890152000 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e67400640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e680cfda0 0x7f7e681517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e67e00640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e680b26c0 0x7f7e6815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 -- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e680cfda0 msgr2=0x7f7e681517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e680cfda0 0x7f7e681517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 -- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e680b26c0 msgr2=0x7f7e6815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e680b26c0 0x7f7e6815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 -- 172.21.15.164:0/3890152000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680ae8a0 con 0x7f7e6815ab60 2024-05-08T04:40:49.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6d000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000c450 tx=0x7f7e6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60017070 con 0x7f7e6815ab60 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e60005db0 con 0x7f7e6815ab60 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6814f280 con 0x7f7e6815ab60 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60007290 con 0x7f7e6815ab60 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6815ba30 con 0x7f7e6815ab60 2024-05-08T04:40:49.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.553+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e68069d60 con 0x7f7e6815ab60 2024-05-08T04:40:49.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.557+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f7e6001a070 con 0x7f7e6815ab60 2024-05-08T04:40:49.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.557+0000 7f7e5fe00640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 0x7f7e48069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.557+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7e60084ae0 con 0x7f7e6815ab60 2024-05-08T04:40:49.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.557+0000 7f7e67e00640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 0x7f7e48069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.557+0000 7f7e67e00640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 0x7f7e48069810 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e580af000 tx=0x7f7e580a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.561+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e60054e60 con 0x7f7e6815ab60 2024-05-08T04:40:49.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e680b10e0 con 0x7f7e6815ab60 2024-05-08T04:40:49.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e5fe00640 1 -- 172.21.15.164:0/3890152000 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f7e60059e20 con 0x7f7e6815ab60 2024-05-08T04:40:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":60,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:36.590286+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 msgr2=0x7f7e48069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 0x7f7e48069810 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e580af000 tx=0x7f7e580a7070 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 msgr2=0x7f7e6814ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000c450 tx=0x7f7e6000c910 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 shutdown_connections 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e48067350 0x7f7e48069810 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e680cfda0 0x7f7e681517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e6815ab60 0x7f7e6814ed40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 --2- 172.21.15.164:0/3890152000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e680b26c0 0x7f7e6815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 >> 172.21.15.164:0/3890152000 conn(0x7f7e6806ac90 msgr2=0x7f7e6806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 shutdown_connections 2024-05-08T04:40:49.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.721+0000 7f7e6f000640 1 -- 172.21.15.164:0/3890152000 wait complete. 2024-05-08T04:40:49.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:40:32.669821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:40:33.956608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:49.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:49.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 Processor -- start 2024-05-08T04:40:49.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 -- start start 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd930147860 0x7fd9300b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd9300cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd9300d0090 0x7fd930161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9300b4e30 con 0x7fd930147860 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93006c5d0 con 0x7fd9300d0090 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.881+0000 7fd938800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006d150 con 0x7fd9300b49d0 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd930147860 0x7fd9300b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd930147860 0x7fd9300b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59094/0 (socket says 172.21.15.164:59094) 2024-05-08T04:40:49.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 -- 172.21.15.164:0/3986540595 learned_addr learned my addr 172.21.15.164:0/3986540595 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935400640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd9300cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd936800640 1 --2- 172.21.15.164:0/3986540595 >> v2:172.21.15.172:3300/0 conn(0x7fd9300d0090 0x7fd930161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 -- 172.21.15.164:0/3986540595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 msgr2=0x7fd9300cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd9300cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 -- 172.21.15.164:0/3986540595 >> v2:172.21.15.172:3300/0 conn(0x7fd9300d0090 msgr2=0x7fd930161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 --2- 172.21.15.164:0/3986540595 >> v2:172.21.15.172:3300/0 conn(0x7fd9300d0090 0x7fd930161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 -- 172.21.15.164:0/3986540595 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93006c820 con 0x7fd930147860 2024-05-08T04:40:49.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd935e00640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd9300b4490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd93006dce0 tx=0x7fd92401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2fe74b49380115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd934a00640 1 -- 172.21.15.164:0/3986540595 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9240b0070 con 0x7fd930147860 2024-05-08T04:40:49.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd934a00640 1 -- 172.21.15.164:0/3986540595 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9240ab600 con 0x7fd930147860 2024-05-08T04:40:49.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.885+0000 7fd934a00640 1 -- 172.21.15.164:0/3986540595 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9240ab9d0 con 0x7fd930147860 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- 172.21.15.164:0/3986540595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 msgr2=0x7fd9300b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd9300b4490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd93006dce0 tx=0x7fd92401c1d0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- 172.21.15.164:0/3986540595 shutdown_connections 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- 172.21.15.164:0/3986540595 >> v2:172.21.15.172:3300/0 conn(0x7fd9300d0090 0x7fd930161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd9300cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- 172.21.15.164:0/3986540595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd9300b4490 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- 172.21.15.164:0/3986540595 >> 172.21.15.164:0/3986540595 conn(0x7fd93006ac90 msgr2=0x7fd93006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- 172.21.15.164:0/3986540595 shutdown_connections 2024-05-08T04:40:49.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- 172.21.15.164:0/3986540595 wait complete. 2024-05-08T04:40:49.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 Processor -- start 2024-05-08T04:40:49.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- start start 2024-05-08T04:40:49.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd93013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300d0090 0x7fd930160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300d05f0 con 0x7fd930147860 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300d0470 con 0x7fd9300d0090 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd938800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b2c00 con 0x7fd9300b49d0 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59100/0 (socket says 172.21.15.164:59100) 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 -- 172.21.15.164:0/2994611485 learned_addr learned my addr 172.21.15.164:0/2994611485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd935e00640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd93013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd935400640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300d0090 0x7fd930160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 -- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 msgr2=0x7fd93013a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd93013a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 -- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300d0090 msgr2=0x7fd930160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300d0090 0x7fd930160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 -- 172.21.15.164:0/2994611485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930068dc0 con 0x7fd930147860 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.889+0000 7fd936800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd92c007e30 tx=0x7fd92c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c014620 con 0x7fd930147860 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c019070 con 0x7fd930147860 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c014a00 con 0x7fd930147860 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd930160680 con 0x7fd930147860 2024-05-08T04:40:49.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9301378e0 con 0x7fd930147860 2024-05-08T04:40:49.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93009e0d0 con 0x7fd930147860 2024-05-08T04:40:49.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd92c014ba0 con 0x7fd930147860 2024-05-08T04:40:49.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 0x7fd914069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:49.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd92c085e80 con 0x7fd930147860 2024-05-08T04:40:49.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd935e00640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 0x7fd914069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:49.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.893+0000 7fd935e00640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 0x7fd914069840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd9240ad000 tx=0x7fd9240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:49.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:49.897+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd930160680 con 0x7fd930147860 2024-05-08T04:40:50.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.053+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd93006dd10 con 0x7fd930147860 2024-05-08T04:40:50.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.053+0000 7fd92a000640 1 -- 172.21.15.164:0/2994611485 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd92c088020 con 0x7fd930147860 2024-05-08T04:40:50.058 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:50.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 msgr2=0x7fd914069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 0x7fd914069840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd9240ad000 tx=0x7fd9240a6000 comp rx=0 tx=0).stop 2024-05-08T04:40:50.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 msgr2=0x7fd93015eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd92c007e30 tx=0x7fd92c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 shutdown_connections 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd914067380 0x7fd914069840 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd930147860 0x7fd93015eec0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300d0090 0x7fd930160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 --2- 172.21.15.164:0/2994611485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9300b49d0 0x7fd93013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 >> 172.21.15.164:0/2994611485 conn(0x7fd93006ac90 msgr2=0x7fd9300b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 shutdown_connections 2024-05-08T04:40:50.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.057+0000 7fd938800640 1 -- 172.21.15.164:0/2994611485 wait complete. 2024-05-08T04:40:50.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:50.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.213+0000 7fcc39c00640 1 Processor -- start 2024-05-08T04:40:50.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.213+0000 7fcc39c00640 1 -- start start 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3406f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc3406e4c0 0x7fcc3406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3415f4c0 con 0x7fcc3406ee80 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3406f310 con 0x7fcc3406e4c0 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3406fe60 con 0x7fcc3415f0e0 2024-05-08T04:40:50.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcc3406ee80 0x7fcc3415f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50710/0 (socket says 172.21.15.164:50710) 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc39200640 1 -- 172.21.15.164:0/1042490662 learned_addr learned my addr 172.21.15.164:0/1042490662 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1042490662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc2be00640 1 --2- 172.21.15.164:0/1042490662 >> v2:172.21.15.172:3300/0 conn(0x7fcc3406e4c0 0x7fcc3406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33400640 1 -- 172.21.15.164:0/1042490662 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759460121 0 0) 0x7fcc3415f4c0 con 0x7fcc3406ee80 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33400640 1 -- 172.21.15.164:0/1042490662 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc18003530 con 0x7fcc3406ee80 2024-05-08T04:40:50.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33e00640 1 -- 172.21.15.164:0/1042490662 >> v2:172.21.15.172:3300/0 conn(0x7fcc3406e4c0 msgr2=0x7fcc3406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1042490662 >> v2:172.21.15.172:3300/0 conn(0x7fcc3406e4c0 0x7fcc3406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33e00640 1 -- 172.21.15.164:0/1042490662 >> v1:172.21.15.164:6789/0 conn(0x7fcc3406ee80 legacy=0x7fcc3415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.217+0000 7fcc33e00640 1 -- 172.21.15.164:0/1042490662 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340703a0 con 0x7fcc3415f0e0 2024-05-08T04:40:50.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1042490662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3406f740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc240097c0 tx=0x7fcc2401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c9378fc59bd645 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc33400640 1 -- 172.21.15.164:0/1042490662 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240b2070 con 0x7fcc3415f0e0 2024-05-08T04:40:50.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc33400640 1 -- 172.21.15.164:0/1042490662 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc240ad640 con 0x7fcc3415f0e0 2024-05-08T04:40:50.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc33400640 1 -- 172.21.15.164:0/1042490662 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240ada10 con 0x7fcc3415f0e0 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- 172.21.15.164:0/1042490662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 msgr2=0x7fcc3406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1042490662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3406f740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc240097c0 tx=0x7fcc2401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- 172.21.15.164:0/1042490662 shutdown_connections 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1042490662 >> v2:172.21.15.172:3300/0 conn(0x7fcc3406e4c0 0x7fcc3406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1042490662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3406f740 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- 172.21.15.164:0/1042490662 >> 172.21.15.164:0/1042490662 conn(0x7fcc3409aa20 msgr2=0x7fcc340d17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- 172.21.15.164:0/1042490662 shutdown_connections 2024-05-08T04:40:50.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- 172.21.15.164:0/1042490662 wait complete. 2024-05-08T04:40:50.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 Processor -- start 2024-05-08T04:40:50.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- start start 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3406e4c0 0x7fcc340cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34168bc0 con 0x7fcc3406ee80 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34168a40 con 0x7fcc3406e4c0 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.221+0000 7fcc39c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34168d40 con 0x7fcc3415f0e0 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59106/0 (socket says 172.21.15.164:59106) 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 -- 172.21.15.164:0/1552532537 learned_addr learned my addr 172.21.15.164:0/1552532537 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3406e4c0 0x7fcc340cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc39200640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 -- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 msgr2=0x7fcc3415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 -- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3406e4c0 msgr2=0x7fcc340cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3406e4c0 0x7fcc340cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 -- 172.21.15.164:0/1552532537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3415ba70 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc2be00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e960 tx=0x7fcc1c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c017070 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00ca00 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00cdb0 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3415bd50 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3415c310 con 0x7fcc3406ee80 2024-05-08T04:40:50.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3404ef30 con 0x7fcc3406ee80 2024-05-08T04:40:50.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.225+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fcc1c01a070 con 0x7fcc3406ee80 2024-05-08T04:40:50.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.229+0000 7fcc30c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 0x7fcc10069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.229+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcc1c085f40 con 0x7fcc3406ee80 2024-05-08T04:40:50.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.229+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 0x7fcc10069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.229+0000 7fcc33e00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 0x7fcc10069810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af000 tx=0x7fcc240a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.233+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc1c056340 con 0x7fcc3406ee80 2024-05-08T04:40:50.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.389+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc340c4950 con 0x7fcc3406ee80 2024-05-08T04:40:50.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.389+0000 7fcc30c00640 1 -- 172.21.15.164:0/1552532537 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc1c05a3e0 con 0x7fcc3406ee80 2024-05-08T04:40:50.395 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 msgr2=0x7fcc10069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 0x7fcc10069810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af000 tx=0x7fcc240a7070 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 msgr2=0x7fcc341546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e960 tx=0x7fcc1c00ee20 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 shutdown_connections 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc10067350 0x7fcc10069810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3415f0e0 0x7fcc3415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc3406ee80 0x7fcc341546d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 --2- 172.21.15.164:0/1552532537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3406e4c0 0x7fcc340cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 >> 172.21.15.164:0/1552532537 conn(0x7fcc3409aa20 msgr2=0x7fcc3415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 shutdown_connections 2024-05-08T04:40:50.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.393+0000 7fcc39c00640 1 -- 172.21.15.164:0/1552532537 wait complete. 2024-05-08T04:40:50.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:50.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.549+0000 7fca98c00640 1 Processor -- start 2024-05-08T04:40:50.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.549+0000 7fca98c00640 1 -- start start 2024-05-08T04:40:50.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca98c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca98c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca940ae6e0 0x7fca940aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca98c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9415e870 con 0x7fca940ce780 2024-05-08T04:40:50.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca98c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9406bb80 con 0x7fca940ae6e0 2024-05-08T04:40:50.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca98c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415eb50 con 0x7fca9415ab10 2024-05-08T04:40:50.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:43090/0 (socket says 172.21.15.164:43090) 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca89400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca940ae6e0 0x7fca940aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 -- 172.21.15.164:0/3306103346 learned_addr learned my addr 172.21.15.164:0/3306103346 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 -- 172.21.15.164:0/3306103346 >> v2:172.21.15.172:3300/0 conn(0x7fca940ae6e0 msgr2=0x7fca940aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 --2- 172.21.15.164:0/3306103346 >> v2:172.21.15.172:3300/0 conn(0x7fca940ae6e0 0x7fca940aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 -- 172.21.15.164:0/3306103346 >> v1:172.21.15.172:6789/0 conn(0x7fca940ce780 legacy=0x7fca940c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.553+0000 7fca91e00640 1 -- 172.21.15.164:0/3306103346 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9415ee30 con 0x7fca9415ab10 2024-05-08T04:40:50.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca91e00640 1 --2- 172.21.15.164:0/3306103346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca84009870 tx=0x7fca8401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59bfe4a22378a122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca91400640 1 -- 172.21.15.164:0/3306103346 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840aa600 con 0x7fca9415ab10 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca91400640 1 -- 172.21.15.164:0/3306103346 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca840af070 con 0x7fca9415ab10 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca91400640 1 -- 172.21.15.164:0/3306103346 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840aa9c0 con 0x7fca9415ab10 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- 172.21.15.164:0/3306103346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 msgr2=0x7fca9406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 --2- 172.21.15.164:0/3306103346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca84009870 tx=0x7fca8401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- 172.21.15.164:0/3306103346 shutdown_connections 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 --2- 172.21.15.164:0/3306103346 >> v2:172.21.15.172:3300/0 conn(0x7fca940ae6e0 0x7fca940aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 --2- 172.21.15.164:0/3306103346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca9415ab10 0x7fca9406cce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- 172.21.15.164:0/3306103346 >> 172.21.15.164:0/3306103346 conn(0x7fca940cf770 msgr2=0x7fca940cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- 172.21.15.164:0/3306103346 shutdown_connections 2024-05-08T04:40:50.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- 172.21.15.164:0/3306103346 wait complete. 2024-05-08T04:40:50.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 Processor -- start 2024-05-08T04:40:50.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.557+0000 7fca98c00640 1 -- start start 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca940ae6e0 0x7fca94144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca940ce780 0x7fca940ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca940cefa0 con 0x7fca9415ab10 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca94165ad0 con 0x7fca940ae6e0 2024-05-08T04:40:50.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca94144e40 con 0x7fca940ce780 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59110/0 (socket says 172.21.15.164:59110) 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 -- 172.21.15.164:0/925083312 learned_addr learned my addr 172.21.15.164:0/925083312 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca91e00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca940ae6e0 0x7fca94144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca89400640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca940ce780 0x7fca940ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 -- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca940ce780 msgr2=0x7fca940ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca940ce780 0x7fca940ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 -- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca940ae6e0 msgr2=0x7fca94144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca940ae6e0 0x7fca94144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 -- 172.21.15.164:0/925083312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca941540c0 con 0x7fca9415ab10 2024-05-08T04:40:50.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca92800640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c770 tx=0x7fca8c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c003b80 con 0x7fca9415ab10 2024-05-08T04:40:50.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c003d20 con 0x7fca9415ab10 2024-05-08T04:40:50.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00aa90 con 0x7fca9415ab10 2024-05-08T04:40:50.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca941543a0 con 0x7fca9415ab10 2024-05-08T04:40:50.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca94154960 con 0x7fca9415ab10 2024-05-08T04:40:50.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.561+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca94072d30 con 0x7fca9415ab10 2024-05-08T04:40:50.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.565+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fca8c00ac30 con 0x7fca9415ab10 2024-05-08T04:40:50.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.565+0000 7fca8aa00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 0x7fca70069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.565+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fca8c0855c0 con 0x7fca9415ab10 2024-05-08T04:40:50.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.565+0000 7fca91e00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 0x7fca70069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.565+0000 7fca91e00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 0x7fca70069810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca840ae000 tx=0x7fca840a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.569+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca8c055940 con 0x7fca9415ab10 2024-05-08T04:40:50.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.733+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca9404f010 con 0x7fca9415ab10 2024-05-08T04:40:50.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.733+0000 7fca8aa00640 1 -- 172.21.15.164:0/925083312 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca8c0599e0 con 0x7fca9415ab10 2024-05-08T04:40:50.740 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:50.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 msgr2=0x7fca70069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 0x7fca70069810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca840ae000 tx=0x7fca840a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 msgr2=0x7fca940ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c770 tx=0x7fca8c00cc30 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 shutdown_connections 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fca70067350 0x7fca70069810 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca940ce780 0x7fca940ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca9415ab10 0x7fca940ade00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 --2- 172.21.15.164:0/925083312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca940ae6e0 0x7fca94144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 >> 172.21.15.164:0/925083312 conn(0x7fca940cf770 msgr2=0x7fca940c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 shutdown_connections 2024-05-08T04:40:50.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.737+0000 7fca98c00640 1 -- 172.21.15.164:0/925083312 wait complete. 2024-05-08T04:40:50.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:40:50.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 Processor -- start 2024-05-08T04:40:50.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 -- start start 2024-05-08T04:40:50.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00200cdfa0 0x7f00200cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 0x7f002006c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0020071980 con 0x7f00200af480 2024-05-08T04:40:50.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0020161810 con 0x7f00200cdfa0 2024-05-08T04:40:50.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.893+0000 7f0027000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002006c650 con 0x7f002006bca0 2024-05-08T04:40:50.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00200cdfa0 0x7f00200cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f0025000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 0x7f002006c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001fe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f00200af480 0x7f002006cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:36526/0 (socket says 172.21.15.164:36526) 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f0025000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 0x7f002006c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:43104/0 (socket says 172.21.15.164:43104) 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001fe00640 1 -- 172.21.15.164:0/4020779131 learned_addr learned my addr 172.21.15.164:0/4020779131 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00200cdfa0 0x7f00200cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:59240/0 (socket says 172.21.15.164:59240) 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 -- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 msgr2=0x7f002006c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 --2- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 0x7f002006c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 -- 172.21.15.164:0/4020779131 >> v1:172.21.15.172:6789/0 conn(0x7f00200af480 legacy=0x7f002006cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 -- 172.21.15.164:0/4020779131 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0020071c00 con 0x7f00200cdfa0 2024-05-08T04:40:50.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001f400640 1 --2- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00200cdfa0 0x7f00200cf470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f000c0097c0 tx=0x7f000c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f94a1970fa95978 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001ea00640 1 -- 172.21.15.164:0/4020779131 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0b1070 con 0x7f00200cdfa0 2024-05-08T04:40:50.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001ea00640 1 -- 172.21.15.164:0/4020779131 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c0ac410 con 0x7f00200cdfa0 2024-05-08T04:40:50.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.897+0000 7f001ea00640 1 -- 172.21.15.164:0/4020779131 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0ac7e0 con 0x7f00200cdfa0 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00200cdfa0 msgr2=0x7f00200cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00200cdfa0 0x7f00200cf470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f000c0097c0 tx=0x7f000c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- 172.21.15.164:0/4020779131 shutdown_connections 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002006bca0 0x7f002006c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- 172.21.15.164:0/4020779131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00200cdfa0 0x7f00200cf470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- 172.21.15.164:0/4020779131 >> 172.21.15.164:0/4020779131 conn(0x7f00200ccab0 msgr2=0x7f00200ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- 172.21.15.164:0/4020779131 shutdown_connections 2024-05-08T04:40:50.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- 172.21.15.164:0/4020779131 wait complete. 2024-05-08T04:40:50.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 Processor -- start 2024-05-08T04:40:50.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- start start 2024-05-08T04:40:50.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002006bca0 0x7f00200c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 0x7f00201592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020071f50 con 0x7f00200cdfa0 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002006eed0 con 0x7f002006bca0 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0027000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0020072bd0 con 0x7f00200af480 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59112/0 (socket says 172.21.15.164:59112) 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 -- 172.21.15.164:0/4028387035 learned_addr learned my addr 172.21.15.164:0/4028387035 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f001f400640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 0x7f00201592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f001fe00640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002006bca0 0x7f00200c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 -- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 msgr2=0x7f00201592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 0x7f00201592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 -- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002006bca0 msgr2=0x7f00200c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:50.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002006bca0 0x7f00200c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f0025000640 1 -- 172.21.15.164:0/4028387035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200ca0c0 con 0x7f00200cdfa0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.901+0000 7f001f400640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 0x7f00201592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0025000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f001800e930 tx=0x7f001800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018017070 con 0x7f00200cdfa0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001800c9a0 con 0x7f00200cdfa0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001800cd70 con 0x7f00200cdfa0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00200c9850 con 0x7f00200cdfa0 2024-05-08T04:40:50.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00200c9290 con 0x7f00200cdfa0 2024-05-08T04:40:50.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0020069d60 con 0x7f00200cdfa0 2024-05-08T04:40:50.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f001801a070 con 0x7f00200cdfa0 2024-05-08T04:40:50.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 0x7f0000069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:50.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f00180856f0 con 0x7f00200cdfa0 2024-05-08T04:40:50.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.905+0000 7f001fe00640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 0x7f0000069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:50.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.909+0000 7f001fe00640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 0x7f0000069810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f00100012f0 tx=0x7f0010001a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:50.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:50.909+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00200ca0c0 con 0x7f00200cdfa0 2024-05-08T04:40:51.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.085+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0020066df0 con 0x7f00200cdfa0 2024-05-08T04:40:51.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.085+0000 7f0017e00640 1 -- 172.21.15.164:0/4028387035 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0018055b80 con 0x7f00200cdfa0 2024-05-08T04:40:51.090 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 msgr2=0x7f0000069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 0x7f0000069810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f00100012f0 tx=0x7f0010001a40 comp rx=0 tx=0).stop 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 msgr2=0x7f00200c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f001800e930 tx=0x7f001800edf0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 shutdown_connections 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0000067350 0x7f0000069810 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00200af480 0x7f00201592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00200cdfa0 0x7f00200c9b80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 --2- 172.21.15.164:0/4028387035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002006bca0 0x7f00200c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 >> 172.21.15.164:0/4028387035 conn(0x7f00200ccab0 msgr2=0x7f00200cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:51.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 shutdown_connections 2024-05-08T04:40:51.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.089+0000 7f0027000640 1 -- 172.21.15.164:0/4028387035 wait complete. 2024-05-08T04:40:51.111 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:40:51.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:51.111 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:40:51.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:40:51.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:40:51.142 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:40:51.191 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:40:51.191 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:51.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:40:51.200 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:40:51.200 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:40:51.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:40:51.242 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 Processor -- start 2024-05-08T04:40:51.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 -- start start 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe02406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0240cf150 0x7fe02406c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02406e0e0 con 0x7fe0240aee30 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02415e8f0 con 0x7fe0240cf150 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe029800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415a8c0 con 0x7fe02415b390 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe022000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe0240aee30 0x7fe0240cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50720/0 (socket says 172.21.15.164:50720) 2024-05-08T04:40:51.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe022000640 1 -- 172.21.15.164:0/1369928946 learned_addr learned my addr 172.21.15.164:0/1369928946 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:51.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe023400640 1 --2- 172.21.15.164:0/1369928946 >> v2:172.21.15.172:3300/0 conn(0x7fe0240cf150 0x7fe02406c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe022a00640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe02406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18239338 0 0) 0x7fe02406e0e0 con 0x7fe0240aee30 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.377+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe008003450 con 0x7fe0240aee30 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707723591 0 0) 0x7fe008003450 con 0x7fe0240aee30 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 msgr2=0x7fe02406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe02406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 >> v2:172.21.15.172:3300/0 conn(0x7fe0240cf150 msgr2=0x7fe02406c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 --2- 172.21.15.164:0/1369928946 >> v2:172.21.15.172:3300/0 conn(0x7fe0240cf150 0x7fe02406c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02415ed80 con 0x7fe0240aee30 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fe010001680 con 0x7fe0240aee30 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe023400640 1 --2- 172.21.15.164:0/1369928946 >> v2:172.21.15.172:3300/0 conn(0x7fe0240cf150 0x7fe02406c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 >> v1:172.21.15.164:6789/0 conn(0x7fe0240aee30 legacy=0x7fe0240cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008003ab0 0x7fe008003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008008860 0x7fe00800acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe00800d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415e8f0 con 0x7fe008003ab0 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415a8c0 con 0x7fe008008860 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00800db90 con 0x7fe02415b390 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008003ab0 0x7fe008003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022000640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008008860 0x7fe00800acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe023400640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe00800d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 -- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 msgr2=0x7fe00800d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe00800d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 -- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008008860 msgr2=0x7fe00800acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:51.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008008860 0x7fe00800acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 -- 172.21.15.164:0/1369928946 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008003450 con 0x7fe008003ab0 2024-05-08T04:40:51.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe023400640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02415b390 0x7fe00800d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe022a00640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008003ab0 0x7fe008003ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fc0 tx=0x7fe01401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e715bc886bc61821 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140b2070 con 0x7fe008003ab0 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0140a5b00 con 0x7fe008003ab0 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.381+0000 7fe021600640 1 -- 172.21.15.164:0/1369928946 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014009c10 con 0x7fe008003ab0 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- 172.21.15.164:0/1369928946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008003ab0 msgr2=0x7fe008003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 --2- 172.21.15.164:0/1369928946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe008003ab0 0x7fe008003ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fc0 tx=0x7fe01401cc80 comp rx=0 tx=0).stop 2024-05-08T04:40:51.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe023400640 1 -- 172.21.15.164:0/1369928946 reap_dead start 2024-05-08T04:40:51.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- 172.21.15.164:0/1369928946 shutdown_connections 2024-05-08T04:40:51.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- 172.21.15.164:0/1369928946 >> 172.21.15.164:0/1369928946 conn(0x7fe02406ac90 msgr2=0x7fe02406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:51.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- 172.21.15.164:0/1369928946 shutdown_connections 2024-05-08T04:40:51.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- 172.21.15.164:0/1369928946 wait complete. 2024-05-08T04:40:51.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 Processor -- start 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- start start 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008003ab0 0x7fe0241ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe024147150 0x7fe0241e8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02406dcc0 con 0x7fe0241eb210 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415b770 con 0x7fe008003ab0 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe029800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02406de40 con 0x7fe024147150 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe022000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe022000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59134/0 (socket says 172.21.15.164:59134) 2024-05-08T04:40:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe022000640 1 -- 172.21.15.164:0/3897213218 learned_addr learned my addr 172.21.15.164:0/3897213218 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe022a00640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008003ab0 0x7fe0241ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.385+0000 7fe023400640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe024147150 0x7fe0241e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 -- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe024147150 msgr2=0x7fe0241e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe024147150 0x7fe0241e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 -- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008003ab0 msgr2=0x7fe0241ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008003ab0 0x7fe0241ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 -- 172.21.15.164:0/3897213218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0241f7c80 con 0x7fe0241eb210 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022000640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe010001060 tx=0x7fe01000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000c690 con 0x7fe0241eb210 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010018070 con 0x7fe0241eb210 2024-05-08T04:40:51.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0241eb720 con 0x7fe0241eb210 2024-05-08T04:40:51.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000ca70 con 0x7fe0241eb210 2024-05-08T04:40:51.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0241ebce0 con 0x7fe0241eb210 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe024154590 con 0x7fe0241eb210 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fe01000cc10 con 0x7fe0241eb210 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 0x7fe004069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe0100854b0 con 0x7fe0241eb210 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.389+0000 7fe022a00640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 0x7fe004069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.393+0000 7fe022a00640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 0x7fe004069810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0140af000 tx=0x7fe014004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.393+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0241f7c80 con 0x7fe0241eb210 2024-05-08T04:40:51.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.581+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe024069d60 con 0x7fe0241eb210 2024-05-08T04:40:51.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.581+0000 7fe01aa00640 1 -- 172.21.15.164:0/3897213218 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fe010055f10 con 0x7fe0241eb210 2024-05-08T04:40:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:51.588 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:51.588 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 35 2024-05-08T04:40:51.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 msgr2=0x7fe004069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 0x7fe004069810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0140af000 tx=0x7fe014004ac0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 msgr2=0x7fe0241f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe010001060 tx=0x7fe01000eb50 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 shutdown_connections 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe004067350 0x7fe004069810 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe024147150 0x7fe0241e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0241eb210 0x7fe0241f7740 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 --2- 172.21.15.164:0/3897213218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe008003ab0 0x7fe0241ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 >> 172.21.15.164:0/3897213218 conn(0x7fe02406ac90 msgr2=0x7fe02406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 shutdown_connections 2024-05-08T04:40:51.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.585+0000 7fe029800640 1 -- 172.21.15.164:0/3897213218 wait complete. 2024-05-08T04:40:51.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:51.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 Processor -- start 2024-05-08T04:40:51.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 -- start start 2024-05-08T04:40:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 0x7f21fc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0cd9f0 con 0x7f21fc0cfda0 2024-05-08T04:40:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0cdcd0 con 0x7f21fc15ba70 2024-05-08T04:40:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.745+0000 7f2201a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc15fe70 con 0x7f21fc15be50 2024-05-08T04:40:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21f2000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f21fc0cfda0 0x7f21fc0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50726/0 (socket says 172.21.15.164:50726) 2024-05-08T04:40:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21f2000640 1 -- 172.21.15.164:0/3023481823 learned_addr learned my addr 172.21.15.164:0/3023481823 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fb400640 1 --2- 172.21.15.164:0/3023481823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 0x7f21fc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843194571 0 0) 0x7f21fc0cd9f0 con 0x7f21fc0cfda0 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0003530 con 0x7f21fc0cfda0 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626395892 0 0) 0x7f21fc0cdcd0 con 0x7f21fc15ba70 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21fc0cd9f0 con 0x7f21fc15ba70 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1103115561 0 0) 0x7f21e0003530 con 0x7f21fc0cfda0 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fb400640 1 -- 172.21.15.164:0/3023481823 >> v1:172.21.15.172:6789/0 conn(0x7f21fc15ba70 legacy=0x7f21fc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fb400640 1 -- 172.21.15.164:0/3023481823 >> v1:172.21.15.164:6789/0 conn(0x7f21fc0cfda0 legacy=0x7f21fc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fb400640 1 -- 172.21.15.164:0/3023481823 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc15f820 con 0x7f21fc15be50 2024-05-08T04:40:51.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.749+0000 7f21fb400640 1 --2- 172.21.15.164:0/3023481823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 0x7f21fc160300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21fc15fff0 tx=0x7f21f401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f089faf82515cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40b0070 con 0x7f21fc15be50 2024-05-08T04:40:51.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40ab400 con 0x7f21fc15be50 2024-05-08T04:40:51.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fa000640 1 -- 172.21.15.164:0/3023481823 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4001680 con 0x7f21fc15be50 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- 172.21.15.164:0/3023481823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 msgr2=0x7f21fc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 --2- 172.21.15.164:0/3023481823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 0x7f21fc160300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21fc15fff0 tx=0x7f21f401c3b0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- 172.21.15.164:0/3023481823 shutdown_connections 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 --2- 172.21.15.164:0/3023481823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc15be50 0x7f21fc160300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- 172.21.15.164:0/3023481823 >> 172.21.15.164:0/3023481823 conn(0x7f21fc06ac90 msgr2=0x7f21fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- 172.21.15.164:0/3023481823 shutdown_connections 2024-05-08T04:40:51.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- 172.21.15.164:0/3023481823 wait complete. 2024-05-08T04:40:51.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 Processor -- start 2024-05-08T04:40:51.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- start start 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 0x7f21fc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21fc15ba70 0x7f21fc13c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc169f60 con 0x7f21fc15be50 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc169de0 con 0x7f21fc15ba70 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f2201a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc16a0e0 con 0x7f21fc0cfda0 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59146/0 (socket says 172.21.15.164:59146) 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.753+0000 7f21fb400640 1 -- 172.21.15.164:0/1470299572 learned_addr learned my addr 172.21.15.164:0/1470299572 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f2000640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21fc15ba70 0x7f21fc13c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21faa00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 0x7f21fc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 -- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 msgr2=0x7f21fc0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 0x7f21fc0703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 -- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21fc15ba70 msgr2=0x7f21fc13c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21fc15ba70 0x7f21fc13c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 -- 172.21.15.164:0/1470299572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0b0760 con 0x7f21fc15be50 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21faa00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 0x7f21fc0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21fb400640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21f40ae040 tx=0x7f21f4019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40bb070 con 0x7f21fc15be50 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40abe10 con 0x7f21fc15be50 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc0b0a40 con 0x7f21fc15be50 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40b0040 con 0x7f21fc15be50 2024-05-08T04:40:51.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc0b1000 con 0x7f21fc15be50 2024-05-08T04:40:51.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc069d60 con 0x7f21fc15be50 2024-05-08T04:40:51.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f21f40be070 con 0x7f21fc15be50 2024-05-08T04:40:51.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 0x7f21d8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:51.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f21f411e700 con 0x7f21fc15be50 2024-05-08T04:40:51.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.757+0000 7f21faa00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 0x7f21d8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:51.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.761+0000 7f21faa00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 0x7f21d8069810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f21ec001570 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:51.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.761+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21f4015fd0 con 0x7f21fc15be50 2024-05-08T04:40:51.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.917+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21fc06e210 con 0x7f21fc15be50 2024-05-08T04:40:51.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f21f3400640 1 -- 172.21.15.164:0/1470299572 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f21f40eefc0 con 0x7f21fc15be50 2024-05-08T04:40:51.924 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:51.924 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":61,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:50.659862+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:51.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 msgr2=0x7f21d8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 0x7f21d8069810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f21ec001570 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 msgr2=0x7f21fc071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21f40ae040 tx=0x7f21f4019150 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 shutdown_connections 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f21d8067350 0x7f21d8069810 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21fc15be50 0x7f21fc071260 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21fc15ba70 0x7f21fc13c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 --2- 172.21.15.164:0/1470299572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21fc0cfda0 0x7f21fc0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 >> 172.21.15.164:0/1470299572 conn(0x7f21fc06ac90 msgr2=0x7f21fc0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 shutdown_connections 2024-05-08T04:40:51.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:51.921+0000 7f2201a00640 1 -- 172.21.15.164:0/1470299572 wait complete. 2024-05-08T04:40:51.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:40:32.669821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:40:33.956608+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:51.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:52.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.077+0000 7fbcd4a00640 1 Processor -- start 2024-05-08T04:40:52.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.077+0000 7fbcd4a00640 1 -- start start 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcd4a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbcd015ba20 0x7fbcd0160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcd4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcd4a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd00c6600 con 0x7fbcd006de50 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcd4a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd015fbe0 con 0x7fbcd015ba20 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcd4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00c6040 con 0x7fbcd00aedf0 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcc5400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbcd006de50 0x7fbcd00ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50728/0 (socket says 172.21.15.164:50728) 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcc5400640 1 -- 172.21.15.164:0/2427384278 learned_addr learned my addr 172.21.15.164:0/2427384278 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccde00640 1 --2- 172.21.15.164:0/2427384278 >> v2:172.21.15.172:3300/0 conn(0x7fbcd015ba20 0x7fbcd0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbcce800640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccd400640 1 -- 172.21.15.164:0/2427384278 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891877694 0 0) 0x7fbcd00c6600 con 0x7fbcd006de50 2024-05-08T04:40:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccd400640 1 -- 172.21.15.164:0/2427384278 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb4003530 con 0x7fbcd006de50 2024-05-08T04:40:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccde00640 1 -- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 msgr2=0x7fbcd006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccde00640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd006e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccde00640 1 -- 172.21.15.164:0/2427384278 >> v1:172.21.15.164:6789/0 conn(0x7fbcd006de50 legacy=0x7fbcd00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.081+0000 7fbccde00640 1 -- 172.21.15.164:0/2427384278 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00c6320 con 0x7fbcd015ba20 2024-05-08T04:40:52.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbccde00640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd015ba20 0x7fbcd0160070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097c0 tx=0x7fbcc001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6121501d9fa1ece7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbccd400640 1 -- 172.21.15.164:0/2427384278 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00ab610 con 0x7fbcd015ba20 2024-05-08T04:40:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbccd400640 1 -- 172.21.15.164:0/2427384278 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc00b0070 con 0x7fbcd015ba20 2024-05-08T04:40:52.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbccd400640 1 -- 172.21.15.164:0/2427384278 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00ab9d0 con 0x7fbcd015ba20 2024-05-08T04:40:52.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd015ba20 msgr2=0x7fbcd0160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd015ba20 0x7fbcd0160070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097c0 tx=0x7fbcc001c470 comp rx=0 tx=0).stop 2024-05-08T04:40:52.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2427384278 shutdown_connections 2024-05-08T04:40:52.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd006e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2427384278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd015ba20 0x7fbcd0160070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2427384278 >> 172.21.15.164:0/2427384278 conn(0x7fbcd00cf770 msgr2=0x7fbcd00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2427384278 shutdown_connections 2024-05-08T04:40:52.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2427384278 wait complete. 2024-05-08T04:40:52.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 Processor -- start 2024-05-08T04:40:52.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.085+0000 7fbcd4a00640 1 -- start start 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 0x7fbcd0150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd0070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00ce000 con 0x7fbcd015ba20 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd00cde80 con 0x7fbcd006de50 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0168b70 con 0x7fbcd00aedf0 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59150/0 (socket says 172.21.15.164:59150) 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 -- 172.21.15.164:0/2582418106 learned_addr learned my addr 172.21.15.164:0/2582418106 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbccde00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 0x7fbcd0150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcc5400640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 -- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 msgr2=0x7fbcd0070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd0070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 -- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 msgr2=0x7fbcd0150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 0x7fbcd0150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 -- 172.21.15.164:0/2582418106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0154550 con 0x7fbcd015ba20 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbccde00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 0x7fbcd0150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcce800640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800efc0 tx=0x7fbcc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8018020 con 0x7fbcd015ba20 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8014480 con 0x7fbcd015ba20 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8014870 con 0x7fbcd015ba20 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0151340 con 0x7fbcd015ba20 2024-05-08T04:40:52.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd014fef0 con 0x7fbcd015ba20 2024-05-08T04:40:52.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.089+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0072d30 con 0x7fbcd015ba20 2024-05-08T04:40:52.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.093+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fbcc801b020 con 0x7fbcd015ba20 2024-05-08T04:40:52.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.093+0000 7fbcc6a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 0x7fbcac069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.093+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fbcc80861b0 con 0x7fbcd015ba20 2024-05-08T04:40:52.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.093+0000 7fbccde00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 0x7fbcac069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.093+0000 7fbccde00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 0x7fbcac069810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00af000 tx=0x7fbcc00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.097+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbcc8022070 con 0x7fbcd015ba20 2024-05-08T04:40:52.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.249+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcd00c52f0 con 0x7fbcd015ba20 2024-05-08T04:40:52.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.249+0000 7fbcc6a00640 1 -- 172.21.15.164:0/2582418106 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcc8010040 con 0x7fbcd015ba20 2024-05-08T04:40:52.255 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 msgr2=0x7fbcac069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 0x7fbcac069810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00af000 tx=0x7fbcc00a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 msgr2=0x7fbcd014f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800efc0 tx=0x7fbcc800c490 comp rx=0 tx=0).stop 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 shutdown_connections 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbcac067350 0x7fbcac069810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcd00aedf0 0x7fbcd0070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd015ba20 0x7fbcd014f9b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 --2- 172.21.15.164:0/2582418106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcd006de50 0x7fbcd0150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 >> 172.21.15.164:0/2582418106 conn(0x7fbcd00cf770 msgr2=0x7fbcd015d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 shutdown_connections 2024-05-08T04:40:52.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.253+0000 7fbcd4a00640 1 -- 172.21.15.164:0/2582418106 wait complete. 2024-05-08T04:40:52.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:40:52.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.409+0000 7f1154a00640 1 Processor -- start 2024-05-08T04:40:52.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.409+0000 7f1154a00640 1 -- start start 2024-05-08T04:40:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1154a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11500ae6e0 0x7f11500aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1154a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11500c6600 con 0x7f115015ba20 2024-05-08T04:40:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1154a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11500c6040 con 0x7f115015ce10 2024-05-08T04:40:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1154a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115015fbe0 con 0x7f11500ae6e0 2024-05-08T04:40:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f115015ba20 0x7f1150160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50740/0 (socket says 172.21.15.164:50740) 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114de00640 1 -- 172.21.15.164:0/336808189 learned_addr learned my addr 172.21.15.164:0/336808189 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114e800640 1 --2- 172.21.15.164:0/336808189 >> v2:172.21.15.172:3300/0 conn(0x7f11500ae6e0 0x7f11500aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755936393 0 0) 0x7f11500c6600 con 0x7f115015ba20 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1134003530 con 0x7f115015ba20 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188679461 0 0) 0x7f1134003530 con 0x7f115015ba20 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 >> v2:172.21.15.172:3300/0 conn(0x7f11500ae6e0 msgr2=0x7f11500aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 --2- 172.21.15.164:0/336808189 >> v2:172.21.15.172:3300/0 conn(0x7f11500ae6e0 0x7f11500aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 >> v1:172.21.15.172:6789/0 conn(0x7f115015ce10 legacy=0x7f115006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500c6320 con 0x7f115015ba20 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1140001680 con 0x7f115015ba20 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 >> v1:172.21.15.164:6789/0 conn(0x7f115015ba20 legacy=0x7f1150160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11340039c0 0x7f1134003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115015fbe0 con 0x7f1134004810 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113400c070 con 0x7f11340051d0 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113400c270 con 0x7f11340039c0 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114de00640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11340039c0 0x7f1134003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114e800640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 -- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11340039c0 msgr2=0x7f1134003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11340039c0 0x7f1134003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 -- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11340051d0 msgr2=0x7f113400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f1145400640 1 -- 172.21.15.164:0/336808189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134003530 con 0x7f1134004810 2024-05-08T04:40:52.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.413+0000 7f114e800640 1 -- 172.21.15.164:0/336808189 reap_dead start 2024-05-08T04:40:52.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1145400640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f1134004c90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1138001780 tx=0x7f113801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f666aafbc3302fba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380013a0 con 0x7f1134004810 2024-05-08T04:40:52.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380052f0 con 0x7f1134004810 2024-05-08T04:40:52.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f114d400640 1 -- 172.21.15.164:0/336808189 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138003840 con 0x7f1134004810 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 -- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 msgr2=0x7f1134004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f1134004c90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1138001780 tx=0x7f113801c400 comp rx=0 tx=0).stop 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 -- 172.21.15.164:0/336808189 shutdown_connections 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 --2- 172.21.15.164:0/336808189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 -- 172.21.15.164:0/336808189 >> 172.21.15.164:0/336808189 conn(0x7f11500cf770 msgr2=0x7f11500cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 -- 172.21.15.164:0/336808189 shutdown_connections 2024-05-08T04:40:52.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 -- 172.21.15.164:0/336808189 wait complete. 2024-05-08T04:40:52.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.417+0000 7f1154a00640 1 Processor -- start 2024-05-08T04:40:52.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- start start 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11500af820 0x7f11501f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11500afc80 0x7f11501f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500aeb20 con 0x7f1134004810 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500cf410 con 0x7f11500af820 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500aeca0 con 0x7f11500afc80 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59170/0 (socket says 172.21.15.164:59170) 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 -- 172.21.15.164:0/3411913722 learned_addr learned my addr 172.21.15.164:0/3411913722 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:52.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1145400640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11500af820 0x7f11501f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114e800640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11500afc80 0x7f11501f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 -- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11500afc80 msgr2=0x7f11501f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11500afc80 0x7f11501f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 -- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11500af820 msgr2=0x7f11501f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11500af820 0x7f11501f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 -- 172.21.15.164:0/3411913722 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500c76a0 con 0x7f1134004810 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f114de00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f11400017f0 tx=0x7f1140012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114001c070 con 0x7f1134004810 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140014750 con 0x7f1134004810 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140014b20 con 0x7f1134004810 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11500c7940 con 0x7f1134004810 2024-05-08T04:40:52.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11500c7f00 con 0x7f1134004810 2024-05-08T04:40:52.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.421+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150072d30 con 0x7f1134004810 2024-05-08T04:40:52.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.425+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f1140014cc0 con 0x7f1134004810 2024-05-08T04:40:52.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.425+0000 7f1146a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 0x7f112c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.425+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1140087e80 con 0x7f1134004810 2024-05-08T04:40:52.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.425+0000 7f1145400640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 0x7f112c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.425+0000 7f1145400640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 0x7f112c069810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1138001750 tx=0x7f11380011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.429+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11400589b0 con 0x7f1134004810 2024-05-08T04:40:52.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11500cea50 con 0x7f1134004810 2024-05-08T04:40:52.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1146a00640 1 -- 172.21.15.164:0/3411913722 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11400582b0 con 0x7f1134004810 2024-05-08T04:40:52.589 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 msgr2=0x7f112c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 0x7f112c069810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1138001750 tx=0x7f11380011a0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 msgr2=0x7f11501500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f11400017f0 tx=0x7f1140012920 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 shutdown_connections 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f112c067350 0x7f112c069810 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11500afc80 0x7f11501f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11500af820 0x7f11501f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 --2- 172.21.15.164:0/3411913722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1134004810 0x7f11501500b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 >> 172.21.15.164:0/3411913722 conn(0x7f11500cf770 msgr2=0x7f115015d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 shutdown_connections 2024-05-08T04:40:52.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.585+0000 7f1154a00640 1 -- 172.21.15.164:0/3411913722 wait complete. 2024-05-08T04:40:52.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:40:52.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 Processor -- start 2024-05-08T04:40:52.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 -- start start 2024-05-08T04:40:52.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16700aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700c6600 con 0x7f167015ba20 2024-05-08T04:40:52.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700c6040 con 0x7f167015ce10 2024-05-08T04:40:52.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f1676c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167015fbe0 con 0x7f16700ae6e0 2024-05-08T04:40:52.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f166fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f167015ba20 0x7f1670160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50744/0 (socket says 172.21.15.164:50744) 2024-05-08T04:40:52.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.745+0000 7f166fe00640 1 -- 172.21.15.164:0/2957914623 learned_addr learned my addr 172.21.15.164:0/2957914623 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:52.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1674c00640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16700aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281067233 0 0) 0x7f16700c6600 con 0x7f167015ba20 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1654003530 con 0x7f167015ba20 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005256751 0 0) 0x7f16700c6040 con 0x7f167015ce10 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16700c6600 con 0x7f167015ce10 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996932273 0 0) 0x7f1654003530 con 0x7f167015ba20 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 msgr2=0x7f16700aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16700aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 >> v1:172.21.15.172:6789/0 conn(0x7f167015ce10 legacy=0x7f167006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700c6320 con 0x7f167015ba20 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1660001680 con 0x7f167015ba20 2024-05-08T04:40:52.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 >> v1:172.21.15.164:6789/0 conn(0x7f167015ba20 legacy=0x7f1670160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1654003b80 0x7f1654003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1654004810 0x7f1654004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167015fbe0 con 0x7f1654004810 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165400ba80 con 0x7f1654003b80 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165400bc80 con 0x7f16700ae6e0 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1654004810 0x7f1654004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f166fe00640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1654003b80 0x7f1654003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1674c00640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 -- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 msgr2=0x7f16540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16700ae6e0 0x7f16540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 -- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1654003b80 msgr2=0x7f1654003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:52.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1654003b80 0x7f1654003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.749+0000 7f1667400640 1 -- 172.21.15.164:0/2957914623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654003530 con 0x7f1654004810 2024-05-08T04:40:52.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1667400640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1654004810 0x7f1654004c90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1658007fd0 tx=0x7f165801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6312d147c489c309 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658001520 con 0x7f1654004810 2024-05-08T04:40:52.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16580052f0 con 0x7f1654004810 2024-05-08T04:40:52.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f166f400640 1 -- 172.21.15.164:0/2957914623 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580a8910 con 0x7f1654004810 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- 172.21.15.164:0/2957914623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1654004810 msgr2=0x7f1654004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 --2- 172.21.15.164:0/2957914623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1654004810 0x7f1654004c90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1658007fd0 tx=0x7f165801ecb0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1674c00640 1 -- 172.21.15.164:0/2957914623 reap_dead start 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- 172.21.15.164:0/2957914623 shutdown_connections 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- 172.21.15.164:0/2957914623 >> 172.21.15.164:0/2957914623 conn(0x7f16700cf770 msgr2=0x7f16700cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- 172.21.15.164:0/2957914623 shutdown_connections 2024-05-08T04:40:52.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- 172.21.15.164:0/2957914623 wait complete. 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 Processor -- start 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- start start 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1654004810 0x7f16701f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16701f4980 0x7f16701f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167006e560 con 0x7f16701f1e20 2024-05-08T04:40:52.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16700cf410 con 0x7f16701f4980 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1676c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167006e6e0 con 0x7f1654004810 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.753+0000 7f1674c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59184/0 (socket says 172.21.15.164:59184) 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 -- 172.21.15.164:0/1285196615 learned_addr learned my addr 172.21.15.164:0/1285196615 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1667400640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16701f4980 0x7f16701f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f166fe00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1654004810 0x7f16701f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 -- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1654004810 msgr2=0x7f16701f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1654004810 0x7f16701f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 -- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16701f4980 msgr2=0x7f16701f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16701f4980 0x7f16701f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 -- 172.21.15.164:0/1285196615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700b3bb0 con 0x7f16701f1e20 2024-05-08T04:40:52.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1674c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f16700cee70 tx=0x7f166800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011e60 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166801d070 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16700c6ae0 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668015650 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16701f2780 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1670072d30 con 0x7f16701f1e20 2024-05-08T04:40:52.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.757+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f16680157f0 con 0x7f16701f1e20 2024-05-08T04:40:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.761+0000 7f166cc00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 0x7f164c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.761+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1668086070 con 0x7f16701f1e20 2024-05-08T04:40:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.761+0000 7f166fe00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 0x7f164c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:52.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.761+0000 7f166fe00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 0x7f164c069810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1660001060 tx=0x7f1660001e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.765+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1668010040 con 0x7f16701f1e20 2024-05-08T04:40:52.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.917+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f167004f010 con 0x7f16701f1e20 2024-05-08T04:40:52.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.917+0000 7f166cc00640 1 -- 172.21.15.164:0/1285196615 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16680568f0 con 0x7f16701f1e20 2024-05-08T04:40:52.922 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:40:52.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 msgr2=0x7f164c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 0x7f164c069810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1660001060 tx=0x7f1660001e30 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 msgr2=0x7f16700afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f16700cee70 tx=0x7f166800f580 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 shutdown_connections 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f164c067350 0x7f164c069810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16701f1e20 0x7f16700afb70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16701f4980 0x7f16701f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 --2- 172.21.15.164:0/1285196615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1654004810 0x7f16701f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 >> 172.21.15.164:0/1285196615 conn(0x7f16700cf770 msgr2=0x7f16700c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 shutdown_connections 2024-05-08T04:40:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:52.921+0000 7f1676c00640 1 -- 172.21.15.164:0/1285196615 wait complete. 2024-05-08T04:40:52.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:53.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 Processor -- start 2024-05-08T04:40:53.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 -- start start 2024-05-08T04:40:53.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86d015ab60 0x7f86d0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00b26c0 0x7f86d00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d0072640 con 0x7f86d00cfbd0 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00d0060 con 0x7f86d015ab60 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.073+0000 7f86d6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d006c770 con 0x7f86d00b26c0 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86d015ab60 0x7f86d0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f86d00cfbd0 0x7f86d00cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50746/0 (socket says 172.21.15.164:50746) 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cfe00640 1 -- 172.21.15.164:0/1963367970 learned_addr learned my addr 172.21.15.164:0/1963367970 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:53.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86c6a00640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00b26c0 0x7f86d00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 -- 172.21.15.164:0/1963367970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00b26c0 msgr2=0x7f86d00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:53.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00b26c0 0x7f86d00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 -- 172.21.15.164:0/1963367970 >> v1:172.21.15.164:6789/0 conn(0x7f86d00cfbd0 legacy=0x7f86d00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 -- 172.21.15.164:0/1963367970 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d006ca50 con 0x7f86d015ab60 2024-05-08T04:40:53.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cea00640 1 -- 172.21.15.164:0/1963367970 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481335265 0 0) 0x7f86d0072640 con 0x7f86d00cfbd0 2024-05-08T04:40:53.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cf400640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015ab60 0x7f86d0072100 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009df0 tx=0x7f86c001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a9e69fcc234dae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cea00640 1 -- 172.21.15.164:0/1963367970 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00b4070 con 0x7f86d015ab60 2024-05-08T04:40:53.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cea00640 1 -- 172.21.15.164:0/1963367970 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c00a7a50 con 0x7f86d015ab60 2024-05-08T04:40:53.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.077+0000 7f86cea00640 1 -- 172.21.15.164:0/1963367970 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00a7e20 con 0x7f86d015ab60 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- 172.21.15.164:0/1963367970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015ab60 msgr2=0x7f86d0072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015ab60 0x7f86d0072100 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009df0 tx=0x7f86c001eb80 comp rx=0 tx=0).stop 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- 172.21.15.164:0/1963367970 shutdown_connections 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00b26c0 0x7f86d00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- 172.21.15.164:0/1963367970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015ab60 0x7f86d0072100 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- 172.21.15.164:0/1963367970 >> 172.21.15.164:0/1963367970 conn(0x7f86d006ac90 msgr2=0x7f86d006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- 172.21.15.164:0/1963367970 shutdown_connections 2024-05-08T04:40:53.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- 172.21.15.164:0/1963367970 wait complete. 2024-05-08T04:40:53.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 Processor -- start 2024-05-08T04:40:53.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- start start 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d00cfbd0 0x7f86d0149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00ad890 0x7f86d00add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d006c550 con 0x7f86d00b26c0 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d006c3d0 con 0x7f86d00cfbd0 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86d6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00aed30 con 0x7f86d00ad890 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59212/0 (socket says 172.21.15.164:59212) 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 -- 172.21.15.164:0/579865854 learned_addr learned my addr 172.21.15.164:0/579865854 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86c6a00640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d00cfbd0 0x7f86d0149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cfe00640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00ad890 0x7f86d00add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 -- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00ad890 msgr2=0x7f86d00add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00ad890 0x7f86d00add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 -- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d00cfbd0 msgr2=0x7f86d0149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d00cfbd0 0x7f86d0149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.081+0000 7f86cf400640 1 -- 172.21.15.164:0/579865854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d014a630 con 0x7f86d00b26c0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86cf400640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86c00b2040 tx=0x7f86c001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00bf070 con 0x7f86d00b26c0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c000e0c0 con 0x7f86d00b26c0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00b4040 con 0x7f86d00b26c0 2024-05-08T04:40:53.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d014a910 con 0x7f86d00b26c0 2024-05-08T04:40:53.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d014aed0 con 0x7f86d00b26c0 2024-05-08T04:40:53.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d0069d60 con 0x7f86d00b26c0 2024-05-08T04:40:53.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f86c00c2070 con 0x7f86d00b26c0 2024-05-08T04:40:53.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 0x7f86ac069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f86c011fbe0 con 0x7f86d00b26c0 2024-05-08T04:40:53.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.085+0000 7f86c6a00640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 0x7f86ac069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.089+0000 7f86c6a00640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 0x7f86ac069810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86d0165930 tx=0x7f86b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.089+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86c0018770 con 0x7f86d00b26c0 2024-05-08T04:40:53.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86d015b880 con 0x7f86d00b26c0 2024-05-08T04:40:53.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86c7e00640 1 -- 172.21.15.164:0/579865854 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f86c00efe90 con 0x7f86d00b26c0 2024-05-08T04:40:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:53.281 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:53.281 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 35 2024-05-08T04:40:53.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 msgr2=0x7f86ac069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 0x7f86ac069810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86d0165930 tx=0x7f86b8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 msgr2=0x7f86d015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86c00b2040 tx=0x7f86c001b920 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 shutdown_connections 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067350 0x7f86ac069810 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d00ad890 0x7f86d00add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d00cfbd0 0x7f86d0149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 --2- 172.21.15.164:0/579865854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d00b26c0 0x7f86d015d8f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 >> 172.21.15.164:0/579865854 conn(0x7f86d006ac90 msgr2=0x7f86d00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 shutdown_connections 2024-05-08T04:40:53.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.277+0000 7f86d6600640 1 -- 172.21.15.164:0/579865854 wait complete. 2024-05-08T04:40:53.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 Processor -- start 2024-05-08T04:40:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 -- start start 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8adc0af480 0x7f8adc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8adc06bca0 0x7f8adc06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc071980 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc161810 con 0x7f8adc06bca0 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ae1800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc06eed0 con 0x7f8adc0af480 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ad2000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8adc0cdfa0 0x7f8adc06cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50758/0 (socket says 172.21.15.164:50758) 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8ad2000640 1 -- 172.21.15.164:0/3670818608 learned_addr learned my addr 172.21.15.164:0/3670818608 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8adb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8adc06bca0 0x7f8adc06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8adaa00640 1 --2- 172.21.15.164:0/3670818608 >> v2:172.21.15.172:3300/0 conn(0x7f8adc0af480 0x7f8adc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8adb400640 1 -- 172.21.15.164:0/3670818608 >> v2:172.21.15.172:3300/0 conn(0x7f8adc0af480 msgr2=0x7f8adc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8adb400640 1 --2- 172.21.15.164:0/3670818608 >> v2:172.21.15.172:3300/0 conn(0x7f8adc0af480 0x7f8adc0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.433+0000 7f8adb400640 1 -- 172.21.15.164:0/3670818608 >> v1:172.21.15.164:6789/0 conn(0x7f8adc0cdfa0 legacy=0x7f8adc06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8adb400640 1 -- 172.21.15.164:0/3670818608 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc071c00 con 0x7f8adc06bca0 2024-05-08T04:40:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8adb400640 1 --2- 172.21.15.164:0/3670818608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc06bca0 0x7f8adc06c0d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8adc070780 tx=0x7f8ad401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e010291f89bab993 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ada000640 1 -- 172.21.15.164:0/3670818608 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40b1070 con 0x7f8adc06bca0 2024-05-08T04:40:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ada000640 1 -- 172.21.15.164:0/3670818608 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad40ac5f0 con 0x7f8adc06bca0 2024-05-08T04:40:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ada000640 1 -- 172.21.15.164:0/3670818608 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40ac9c0 con 0x7f8adc06bca0 2024-05-08T04:40:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 -- 172.21.15.164:0/3670818608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc06bca0 msgr2=0x7f8adc06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 --2- 172.21.15.164:0/3670818608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc06bca0 0x7f8adc06c0d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8adc070780 tx=0x7f8ad401c480 comp rx=0 tx=0).stop 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 -- 172.21.15.164:0/3670818608 shutdown_connections 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 --2- 172.21.15.164:0/3670818608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc06bca0 0x7f8adc06c0d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 --2- 172.21.15.164:0/3670818608 >> v2:172.21.15.172:3300/0 conn(0x7f8adc0af480 0x7f8adc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 -- 172.21.15.164:0/3670818608 >> 172.21.15.164:0/3670818608 conn(0x7f8adc0ccab0 msgr2=0x7f8adc0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 -- 172.21.15.164:0/3670818608 shutdown_connections 2024-05-08T04:40:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.437+0000 7f8ae1800640 1 -- 172.21.15.164:0/3670818608 wait complete. 2024-05-08T04:40:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 Processor -- start 2024-05-08T04:40:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 -- start start 2024-05-08T04:40:53.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8adc06bca0 0x7f8adc0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 0x7f8adc09d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06c610 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc071f50 con 0x7f8adc0af480 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc072bd0 con 0x7f8adc06bca0 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59232/0 (socket says 172.21.15.164:59232) 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 -- 172.21.15.164:0/1086954715 learned_addr learned my addr 172.21.15.164:0/1086954715 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adaa00640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8adc06bca0 0x7f8adc0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ad2000640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 0x7f8adc09d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 -- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8adc06bca0 msgr2=0x7f8adc0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8adc06bca0 0x7f8adc0b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 -- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 msgr2=0x7f8adc09d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 0x7f8adc09d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 -- 172.21.15.164:0/1086954715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc09c000 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ad2000640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 0x7f8adc09d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8adb400640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8ad401ab30 tx=0x7f8ad4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40b1040 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad40bb070 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc09c2e0 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.441+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40b5410 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc09cb80 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc069d60 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f8ad40b55b0 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8ad3400640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 0x7f8ab8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8ad411f420 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8adaa00640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 0x7f8ab8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.445+0000 7f8adaa00640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 0x7f8ab8069810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007480 tx=0x7f8acc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.449+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ad40a9070 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.605+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8adc066df0 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.605+0000 7f8ad3400640 1 -- 172.21.15.164:0/1086954715 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f8ad4016070 con 0x7f8adc0cdfa0 2024-05-08T04:40:53.611 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:53.611 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":61,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:50.659862+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:53.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 msgr2=0x7f8ab8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 0x7f8ab8069810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007480 tx=0x7f8acc00b040 comp rx=0 tx=0).stop 2024-05-08T04:40:53.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 msgr2=0x7f8adc0c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8ad401ab30 tx=0x7f8ad4019220 comp rx=0 tx=0).stop 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 shutdown_connections 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8ab8067350 0x7f8ab8069810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8adc0cdfa0 0x7f8adc0c3b30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8adc0af480 0x7f8adc09d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 --2- 172.21.15.164:0/1086954715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8adc06bca0 0x7f8adc0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 >> 172.21.15.164:0/1086954715 conn(0x7f8adc0ccab0 msgr2=0x7f8adc06dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 shutdown_connections 2024-05-08T04:40:53.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.609+0000 7f8ae1800640 1 -- 172.21.15.164:0/1086954715 wait complete. 2024-05-08T04:40:53.632 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:40:53.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.765+0000 7fedd8600640 1 Processor -- start 2024-05-08T04:40:53.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.765+0000 7fedd8600640 1 -- start start 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd8600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedd00b26c0 0x7fedd00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd8600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 0x7fedd00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd8600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd006c770 con 0x7fedd015ab60 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd8600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00d0060 con 0x7fedd00b26c0 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd8600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd006ca50 con 0x7fedd00cfbd0 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedd00b26c0 0x7fedd00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedd00b26c0 0x7fedd00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59234/0 (socket says 172.21.15.164:59234) 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 -- 172.21.15.164:0/179713069 learned_addr learned my addr 172.21.15.164:0/179713069 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd6600640 1 --2- 172.21.15.164:0/179713069 >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 0x7fedd00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 -- 172.21.15.164:0/179713069 >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 msgr2=0x7fedd00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:53.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 --2- 172.21.15.164:0/179713069 >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 0x7fedd00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 -- 172.21.15.164:0/179713069 >> v1:172.21.15.172:6789/0 conn(0x7fedd015ab60 legacy=0x7fedd0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:53.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 -- 172.21.15.164:0/179713069 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd006cd30 con 0x7fedd00b26c0 2024-05-08T04:40:53.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd6600640 1 --2- 172.21.15.164:0/179713069 >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 0x7fedd00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.769+0000 7fedd5200640 1 --2- 172.21.15.164:0/179713069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd00b26c0 0x7fedd00cf690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fedd00d06c0 tx=0x7fedbc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18605a2c5a27b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedcfe00640 1 -- 172.21.15.164:0/179713069 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0b0070 con 0x7fedd00b26c0 2024-05-08T04:40:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedcfe00640 1 -- 172.21.15.164:0/179713069 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0ab600 con 0x7fedd00b26c0 2024-05-08T04:40:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedcfe00640 1 -- 172.21.15.164:0/179713069 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0ab9d0 con 0x7fedd00b26c0 2024-05-08T04:40:53.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- 172.21.15.164:0/179713069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd00b26c0 msgr2=0x7fedd00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- 172.21.15.164:0/179713069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd00b26c0 0x7fedd00cf690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fedd00d06c0 tx=0x7fedbc01c4e0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- 172.21.15.164:0/179713069 shutdown_connections 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- 172.21.15.164:0/179713069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd00b26c0 0x7fedd00cf690 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- 172.21.15.164:0/179713069 >> v2:172.21.15.172:3300/0 conn(0x7fedd00cfbd0 0x7fedd00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- 172.21.15.164:0/179713069 >> 172.21.15.164:0/179713069 conn(0x7fedd006ac90 msgr2=0x7fedd006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- 172.21.15.164:0/179713069 shutdown_connections 2024-05-08T04:40:53.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- 172.21.15.164:0/179713069 wait complete. 2024-05-08T04:40:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 Processor -- start 2024-05-08T04:40:53.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- start start 2024-05-08T04:40:53.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 0x7fedd0140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedd00cfbd0 0x7fedd01f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00aeeb0 con 0x7fedd015ab60 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00aed30 con 0x7fedd00b26c0 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd8600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0167970 con 0x7fedd00cfbd0 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59246/0 (socket says 172.21.15.164:59246) 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.773+0000 7fedd6600640 1 -- 172.21.15.164:0/1345133577 learned_addr learned my addr 172.21.15.164:0/1345133577 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 -- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedd00cfbd0 msgr2=0x7fedd01f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedd00cfbd0 0x7fedd01f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd5c00640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 0x7fedd0140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 -- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 msgr2=0x7fedd0140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 0x7fedd0140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 -- 172.21.15.164:0/1345133577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0152710 con 0x7fedd015ab60 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd5c00640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 0x7fedd0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd6600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fedc800cef0 tx=0x7fedc800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc801a070 con 0x7fedd015ab60 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc800d860 con 0x7fedd015ab60 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc800dc50 con 0x7fedd015ab60 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd01529f0 con 0x7fedd015ab60 2024-05-08T04:40:53.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd01414e0 con 0x7fedd015ab60 2024-05-08T04:40:53.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd0069d60 con 0x7fedd015ab60 2024-05-08T04:40:53.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.777+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fedc801d070 con 0x7fedd015ab60 2024-05-08T04:40:53.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.781+0000 7fedcd600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 0x7fedb0069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:53.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.781+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fedc8086030 con 0x7fedd015ab60 2024-05-08T04:40:53.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.781+0000 7fedd5c00640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 0x7fedb0069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:53.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.781+0000 7fedd5c00640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 0x7fedb0069810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fedc400a850 tx=0x7fedc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:53.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.785+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fedc8056b10 con 0x7fedd015ab60 2024-05-08T04:40:53.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedd006d9a0 con 0x7fedd015ab60 2024-05-08T04:40:53.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedcd600640 1 -- 172.21.15.164:0/1345133577 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fedc8056410 con 0x7fedd015ab60 2024-05-08T04:40:53.974 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:53.974 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:53.974 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 35 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 msgr2=0x7fedb0069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 0x7fedb0069810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fedc400a850 tx=0x7fedc4008040 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 msgr2=0x7fedd015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fedc800cef0 tx=0x7fedc800a710 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 shutdown_connections 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fedb0067350 0x7fedb0069810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedd00cfbd0 0x7fedd01f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd015ab60 0x7fedd015b1a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 --2- 172.21.15.164:0/1345133577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00b26c0 0x7fedd0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 >> 172.21.15.164:0/1345133577 conn(0x7fedd006ac90 msgr2=0x7fedd006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 shutdown_connections 2024-05-08T04:40:53.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:53.969+0000 7fedd8600640 1 -- 172.21.15.164:0/1345133577 wait complete. 2024-05-08T04:40:53.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:40:54.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.125+0000 7f88bda00640 1 Processor -- start 2024-05-08T04:40:54.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.125+0000 7f88bda00640 1 -- start start 2024-05-08T04:40:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88bda00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88b815ab60 0x7f88b8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88bda00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b8072640 con 0x7f88b80cfda0 2024-05-08T04:40:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88bda00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b806c770 con 0x7f88b80b26c0 2024-05-08T04:40:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88bda00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b806ca50 con 0x7f88b815ab60 2024-05-08T04:40:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b7400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f88b80cfda0 0x7f88b80cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50772/0 (socket says 172.21.15.164:50772) 2024-05-08T04:40:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b7400640 1 -- 172.21.15.164:0/1476622343 learned_addr learned my addr 172.21.15.164:0/1476622343 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:54.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1476622343 >> v2:172.21.15.172:3300/0 conn(0x7f88b815ab60 0x7f88b8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002434564 0 0) 0x7f88b8072640 con 0x7f88b80cfda0 2024-05-08T04:40:54.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f889c003530 con 0x7f88b80cfda0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934353466 0 0) 0x7f889c003530 con 0x7f88b80cfda0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 >> v2:172.21.15.172:3300/0 conn(0x7f88b815ab60 msgr2=0x7f88b8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 --2- 172.21.15.164:0/1476622343 >> v2:172.21.15.172:3300/0 conn(0x7f88b815ab60 0x7f88b8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 >> v1:172.21.15.172:6789/0 conn(0x7f88b80b26c0 legacy=0x7f88b80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b806cd30 con 0x7f88b80cfda0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f88b0001a00 con 0x7f88b80cfda0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 >> v1:172.21.15.164:6789/0 conn(0x7f88b80cfda0 legacy=0x7f88b80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f889c0039c0 0x7f889c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 0x7f889c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c005090 0x7f889c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b806ca50 con 0x7f889c0043c0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c00bff0 con 0x7f889c0039c0 2024-05-08T04:40:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c00e400 con 0x7f889c005090 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 0x7f889c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f889c0039c0 0x7f889c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b7400640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c005090 0x7f889c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 -- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c005090 msgr2=0x7f889c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c005090 0x7f889c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 -- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f889c0039c0 msgr2=0x7f889c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f889c0039c0 0x7f889c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b6000640 1 -- 172.21.15.164:0/1476622343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c003530 con 0x7f889c0043c0 2024-05-08T04:40:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.129+0000 7f88b7400640 1 -- 172.21.15.164:0/1476622343 reap_dead start 2024-05-08T04:40:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88b6000640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 0x7f889c004890 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001910 tx=0x7f88a401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae905bda54be1ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40b0070 con 0x7f889c0043c0 2024-05-08T04:40:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a40ab620 con 0x7f889c0043c0 2024-05-08T04:40:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88b5600640 1 -- 172.21.15.164:0/1476622343 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4001510 con 0x7f889c0043c0 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 -- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 msgr2=0x7f889c004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 0x7f889c004890 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001910 tx=0x7f88a401c540 comp rx=0 tx=0).stop 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 -- 172.21.15.164:0/1476622343 shutdown_connections 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 --2- 172.21.15.164:0/1476622343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f889c0043c0 0x7f889c004890 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 -- 172.21.15.164:0/1476622343 >> 172.21.15.164:0/1476622343 conn(0x7f88b806ac90 msgr2=0x7f88b806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 -- 172.21.15.164:0/1476622343 shutdown_connections 2024-05-08T04:40:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.133+0000 7f88bda00640 1 -- 172.21.15.164:0/1476622343 wait complete. 2024-05-08T04:40:54.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 Processor -- start 2024-05-08T04:40:54.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- start start 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 0x7f88b8147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88b8157ae0 0x7f88b8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80cfc20 con 0x7f88b81471c0 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80cfaa0 con 0x7f88b8157ae0 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80aed30 con 0x7f889c0043c0 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59274/0 (socket says 172.21.15.164:59274) 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 -- 172.21.15.164:0/1951676168 learned_addr learned my addr 172.21.15.164:0/1951676168 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b6000640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88b8157ae0 0x7f88b8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 0x7f88b8147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 -- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 msgr2=0x7f88b8147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 0x7f88b8147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 -- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88b8157ae0 msgr2=0x7f88b8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88b8157ae0 0x7f88b8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 -- 172.21.15.164:0/1951676168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b81573e0 con 0x7f88b81471c0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 0x7f88b8147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88b7400640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f88b0002240 tx=0x7f88b0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b000abf0 con 0x7f88b81471c0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0018070 con 0x7f88b81471c0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00133f0 con 0x7f88b81471c0 2024-05-08T04:40:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b8157f60 con 0x7f88b81471c0 2024-05-08T04:40:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b8151bb0 con 0x7f88b81471c0 2024-05-08T04:40:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.137+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b8069d60 con 0x7f88b81471c0 2024-05-08T04:40:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.141+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f88b0013590 con 0x7f88b81471c0 2024-05-08T04:40:54.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.141+0000 7f88aea00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 0x7f8898069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.141+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f88b0085c10 con 0x7f88b81471c0 2024-05-08T04:40:54.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.141+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 0x7f8898069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.141+0000 7f88b6a00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 0x7f8898069840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f88b80736e0 tx=0x7f88a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.145+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88b0056670 con 0x7f88b81471c0 2024-05-08T04:40:54.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88b806d9a0 con 0x7f88b81471c0 2024-05-08T04:40:54.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88aea00640 1 -- 172.21.15.164:0/1951676168 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f88b0055f70 con 0x7f88b81471c0 2024-05-08T04:40:54.341 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:54.341 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:40:34.960304+0000","modified":"2024-05-08T04:40:37.971888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6018},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6018":{"gid":6018,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1114280106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1114280106},{"type":"v1","addr":"172.21.15.164:6837","nonce":1114280106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-08T04:40:54.342 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 35 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 msgr2=0x7f8898069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 0x7f8898069840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f88b80736e0 tx=0x7f88a8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 msgr2=0x7f88b81594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f88b0002240 tx=0x7f88b0007bd0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 shutdown_connections 2024-05-08T04:40:54.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8898067380 0x7f8898069840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88b81471c0 0x7f88b81594d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88b8157ae0 0x7f88b8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 --2- 172.21.15.164:0/1951676168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889c0043c0 0x7f88b8147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 >> 172.21.15.164:0/1951676168 conn(0x7f88b806ac90 msgr2=0x7f88b80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 shutdown_connections 2024-05-08T04:40:54.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.337+0000 7f88bda00640 1 -- 172.21.15.164:0/1951676168 wait complete. 2024-05-08T04:40:54.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 Processor -- start 2024-05-08T04:40:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 -- start start 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f51dc15bc50 0x7f51dc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51dc0ccfe0 0x7f51dc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc0b42a0 con 0x7f51dc0b4d50 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cd4e0 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e3800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cd7c0 con 0x7f51dc15bc50 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51dc0ccfe0 0x7f51dc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51dc0ccfe0 0x7f51dc0d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59276/0 (socket says 172.21.15.164:59276) 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 -- 172.21.15.164:0/3963390367 learned_addr learned my addr 172.21.15.164:0/3963390367 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e1800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f51dc0b4d50 0x7f51dc0ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50782/0 (socket says 172.21.15.164:50782) 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51e0e00640 1 --2- 172.21.15.164:0/3963390367 >> v2:172.21.15.172:3300/0 conn(0x7f51dc15bc50 0x7f51dc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 -- 172.21.15.164:0/3963390367 >> v2:172.21.15.172:3300/0 conn(0x7f51dc15bc50 msgr2=0x7f51dc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 --2- 172.21.15.164:0/3963390367 >> v2:172.21.15.172:3300/0 conn(0x7f51dc15bc50 0x7f51dc0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 -- 172.21.15.164:0/3963390367 >> v1:172.21.15.164:6789/0 conn(0x7f51dc0b4d50 legacy=0x7f51dc0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51dbe00640 1 -- 172.21.15.164:0/3963390367 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0cf590 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.497+0000 7f51db400640 1 -- 172.21.15.164:0/3963390367 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8968857 0 0) 0x7f51dc0b42a0 con 0x7f51dc0b4d50 2024-05-08T04:40:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51dbe00640 1 --2- 172.21.15.164:0/3963390367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0ccfe0 0x7f51dc0d0070 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f51dc071a00 tx=0x7f51c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491975ce28a5c920 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51db400640 1 -- 172.21.15.164:0/3963390367 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80b0070 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51db400640 1 -- 172.21.15.164:0/3963390367 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80ab600 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51db400640 1 -- 172.21.15.164:0/3963390367 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80ab9d0 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- 172.21.15.164:0/3963390367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0ccfe0 msgr2=0x7f51dc0d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 --2- 172.21.15.164:0/3963390367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0ccfe0 0x7f51dc0d0070 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f51dc071a00 tx=0x7f51c801c460 comp rx=0 tx=0).stop 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- 172.21.15.164:0/3963390367 shutdown_connections 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 --2- 172.21.15.164:0/3963390367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0ccfe0 0x7f51dc0d0070 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 --2- 172.21.15.164:0/3963390367 >> v2:172.21.15.172:3300/0 conn(0x7f51dc15bc50 0x7f51dc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- 172.21.15.164:0/3963390367 >> 172.21.15.164:0/3963390367 conn(0x7f51dc0b34f0 msgr2=0x7f51dc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- 172.21.15.164:0/3963390367 shutdown_connections 2024-05-08T04:40:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- 172.21.15.164:0/3963390367 wait complete. 2024-05-08T04:40:54.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 Processor -- start 2024-05-08T04:40:54.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.501+0000 7f51e3800640 1 -- start start 2024-05-08T04:40:54.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51dc0ccfe0 0x7f51dc1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 0x7f51dc159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc168c60 con 0x7f51dc0b4d50 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc168ae0 con 0x7f51dc15bc50 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc168de0 con 0x7f51dc0ccfe0 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59292/0 (socket says 172.21.15.164:59292) 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 -- 172.21.15.164:0/470953256 learned_addr learned my addr 172.21.15.164:0/470953256 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e1800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 0x7f51dc159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51dbe00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51dc0ccfe0 0x7f51dc1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 -- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51dc0ccfe0 msgr2=0x7f51dc1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51dc0ccfe0 0x7f51dc1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 -- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 msgr2=0x7f51dc159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 0x7f51dc159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 -- 172.21.15.164:0/470953256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0c8560 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e1800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 0x7f51dc159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e0e00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f51d000d620 tx=0x7f51d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000b620 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0019070 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc158080 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000b9e0 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc158640 con 0x7f51dc0b4d50 2024-05-08T04:40:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc069d60 con 0x7f51dc0b4d50 2024-05-08T04:40:54.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.505+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f51d000bb80 con 0x7f51dc0b4d50 2024-05-08T04:40:54.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.509+0000 7f51d8c00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 0x7f51bc069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.509+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f51d0085be0 con 0x7f51dc0b4d50 2024-05-08T04:40:54.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.509+0000 7f51dbe00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 0x7f51bc069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.509+0000 7f51dbe00640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 0x7f51bc069810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ad000 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.513+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51d0056590 con 0x7f51dc0b4d50 2024-05-08T04:40:54.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.669+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51dc0b7940 con 0x7f51dc0b4d50 2024-05-08T04:40:54.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51d8c00640 1 -- 172.21.15.164:0/470953256 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f51d0055e90 con 0x7f51dc0b4d50 2024-05-08T04:40:54.677 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:54.677 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":62,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:54.169985+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:40:32.669821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-08T04:40:33.956608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 msgr2=0x7f51bc069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 0x7f51bc069810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ad000 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 msgr2=0x7f51dc151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f51d000d620 tx=0x7f51d000dae0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 shutdown_connections 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51bc067350 0x7f51bc069810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51dc0ccfe0 0x7f51dc1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51dc15bc50 0x7f51dc159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 --2- 172.21.15.164:0/470953256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0b4d50 0x7f51dc151b70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 >> 172.21.15.164:0/470953256 conn(0x7f51dc0b34f0 msgr2=0x7f51dc0cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 shutdown_connections 2024-05-08T04:40:54.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.673+0000 7f51e3800640 1 -- 172.21.15.164:0/470953256 wait complete. 2024-05-08T04:40:54.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:40:54.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 Processor -- start 2024-05-08T04:40:54.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 -- start start 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 0x7f4dd40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4dd406bca0 0x7f4dd406c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd4071980 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd4161810 con 0x7f4dd406bca0 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4ddae00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd406eed0 con 0x7f4dd40af480 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4dd406bca0 0x7f4dd406c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4dd406bca0 0x7f4dd406c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59306/0 (socket says 172.21.15.164:59306) 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 -- 172.21.15.164:0/3176668662 learned_addr learned my addr 172.21.15.164:0/3176668662 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:54.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd3400640 1 --1- 172.21.15.164:0/3176668662 >> v1:172.21.15.164:6789/0 conn(0x7f4dd40cdfa0 0x7f4dd406cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50792/0 (socket says 172.21.15.164:50792) 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd3e00640 1 --2- 172.21.15.164:0/3176668662 >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 0x7f4dd40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 -- 172.21.15.164:0/3176668662 >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 msgr2=0x7f4dd40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 --2- 172.21.15.164:0/3176668662 >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 0x7f4dd40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 -- 172.21.15.164:0/3176668662 >> v1:172.21.15.164:6789/0 conn(0x7f4dd40cdfa0 legacy=0x7f4dd406cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd8e00640 1 -- 172.21.15.164:0/3176668662 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4071c00 con 0x7f4dd406bca0 2024-05-08T04:40:54.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.833+0000 7f4dd3e00640 1 --2- 172.21.15.164:0/3176668662 >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 0x7f4dd40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:54.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4dd8e00640 1 --2- 172.21.15.164:0/3176668662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd406bca0 0x7f4dd406c0d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4070780 tx=0x7f4dcc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0482db484538e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4dd2a00640 1 -- 172.21.15.164:0/3176668662 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0a57d0 con 0x7f4dd406bca0 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4dd2a00640 1 -- 172.21.15.164:0/3176668662 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0a5970 con 0x7f4dd406bca0 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4dd2a00640 1 -- 172.21.15.164:0/3176668662 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0a5d40 con 0x7f4dd406bca0 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- 172.21.15.164:0/3176668662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd406bca0 msgr2=0x7f4dd406c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 --2- 172.21.15.164:0/3176668662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd406bca0 0x7f4dd406c0d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4070780 tx=0x7f4dcc01c9f0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- 172.21.15.164:0/3176668662 shutdown_connections 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 --2- 172.21.15.164:0/3176668662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd406bca0 0x7f4dd406c0d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 --2- 172.21.15.164:0/3176668662 >> v2:172.21.15.172:3300/0 conn(0x7f4dd40af480 0x7f4dd40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- 172.21.15.164:0/3176668662 >> 172.21.15.164:0/3176668662 conn(0x7f4dd40ccab0 msgr2=0x7f4dd40ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- 172.21.15.164:0/3176668662 shutdown_connections 2024-05-08T04:40:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- 172.21.15.164:0/3176668662 wait complete. 2024-05-08T04:40:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 Processor -- start 2024-05-08T04:40:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.837+0000 7f4ddae00640 1 -- start start 2024-05-08T04:40:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd406bca0 0x7f4dd406cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 0x7f4dd40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd406c610 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4071f50 con 0x7f4dd406bca0 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4072bd0 con 0x7f4dd40af480 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd406bca0 0x7f4dd406cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59318/0 (socket says 172.21.15.164:59318) 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 -- 172.21.15.164:0/1220066744 learned_addr learned my addr 172.21.15.164:0/1220066744 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd3400640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 0x7f4dd40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 -- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 msgr2=0x7f4dd40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 0x7f4dd40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 -- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd406bca0 msgr2=0x7f4dd406cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd406bca0 0x7f4dd406cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 -- 172.21.15.164:0/1220066744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40ca050 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd3400640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 0x7f4dd40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dd8e00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc019600 tx=0x7f4dcc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0bc020 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0b8670 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40c4920 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0b8a40 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd40c4ee0 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd4069d60 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.841+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f4dcc0bf020 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.845+0000 7f4dcbe00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 0x7f4db4069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:54.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.845+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4dcc120080 con 0x7f4dd40cdfa0 2024-05-08T04:40:54.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.845+0000 7f4dd3e00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 0x7f4db4069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:54.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.845+0000 7f4dd3e00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 0x7f4db4069840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009950 tx=0x7f4dc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:54.849+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4dcc0c7020 con 0x7f4dd40cdfa0 2024-05-08T04:40:55.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.033+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4dd4066df0 con 0x7f4dd40cdfa0 2024-05-08T04:40:55.040 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:40:55.033+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:40:55.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.037+0000 7f4dcbe00640 1 -- 172.21.15.164:0/1220066744 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4dcc0165e0 con 0x7f4dd40cdfa0 2024-05-08T04:40:55.041 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 msgr2=0x7f4db4069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 0x7f4db4069840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009950 tx=0x7f4dc4008040 comp rx=0 tx=0).stop 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 msgr2=0x7f4dd409e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc019600 tx=0x7f4dcc019790 comp rx=0 tx=0).stop 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 shutdown_connections 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4db4067380 0x7f4db4069840 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009950 tx=0x7f4dc4008040 comp rx=0 tx=0).stop 2024-05-08T04:40:55.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40af480 0x7f4dd40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dd40cdfa0 0x7f4dd409e3c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 --2- 172.21.15.164:0/1220066744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd406bca0 0x7f4dd406cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 >> 172.21.15.164:0/1220066744 conn(0x7f4dd40ccab0 msgr2=0x7f4dd40ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:55.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 shutdown_connections 2024-05-08T04:40:55.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.041+0000 7f4ddae00640 1 -- 172.21.15.164:0/1220066744 wait complete. 2024-05-08T04:40:55.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:40:55.111 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:40:55.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 Processor -- start 2024-05-08T04:40:55.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 -- start start 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80c6600 con 0x7f1da806e390 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80c6040 con 0x7f1da815ba20 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1dace00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815fbe0 con 0x7f1da815ce10 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da6800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1da806e390 0x7f1da80ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50808/0 (socket says 172.21.15.164:50808) 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da6800640 1 -- 172.21.15.164:0/4102833355 learned_addr learned my addr 172.21.15.164:0/4102833355 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1d9d400640 1 --2- 172.21.15.164:0/4102833355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:55.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407703282 0 0) 0x7f1da80c6600 con 0x7f1da806e390 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d8c003530 con 0x7f1da806e390 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217031893 0 0) 0x7f1da80c6040 con 0x7f1da815ba20 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da80c6600 con 0x7f1da815ba20 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1d9d400640 1 -- 172.21.15.164:0/4102833355 >> v1:172.21.15.172:6789/0 conn(0x7f1da815ba20 legacy=0x7f1da8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003874327 0 0) 0x7f1d8c003530 con 0x7f1da806e390 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1d9d400640 1 -- 172.21.15.164:0/4102833355 >> v1:172.21.15.164:6789/0 conn(0x7f1da806e390 legacy=0x7f1da80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.205+0000 7f1d9d400640 1 -- 172.21.15.164:0/4102833355 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80c6320 con 0x7f1da815ce10 2024-05-08T04:40:55.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1d9d400640 1 --2- 172.21.15.164:0/4102833355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da806de50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d900097f0 tx=0x7f1d9001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b0de979f07f48ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:55.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d900b0070 con 0x7f1da815ce10 2024-05-08T04:40:55.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d900ab400 con 0x7f1da815ce10 2024-05-08T04:40:55.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1da5400640 1 -- 172.21.15.164:0/4102833355 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0001a00 con 0x7f1da815ce10 2024-05-08T04:40:55.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 -- 172.21.15.164:0/4102833355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 msgr2=0x7f1da806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 --2- 172.21.15.164:0/4102833355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da806de50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d900097f0 tx=0x7f1d9001c370 comp rx=0 tx=0).stop 2024-05-08T04:40:55.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 -- 172.21.15.164:0/4102833355 shutdown_connections 2024-05-08T04:40:55.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 --2- 172.21.15.164:0/4102833355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da806de50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 -- 172.21.15.164:0/4102833355 >> 172.21.15.164:0/4102833355 conn(0x7f1da80cf770 msgr2=0x7f1da80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:55.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 -- 172.21.15.164:0/4102833355 shutdown_connections 2024-05-08T04:40:55.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.209+0000 7f1dace00640 1 -- 172.21.15.164:0/4102833355 wait complete. 2024-05-08T04:40:55.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 Processor -- start 2024-05-08T04:40:55.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- start start 2024-05-08T04:40:55.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:55.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da815ba20 0x7f1da80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:55.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da80b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:55.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80cde80 con 0x7f1da806e390 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80af000 con 0x7f1da815ba20 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80ce000 con 0x7f1da815ce10 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59356/0 (socket says 172.21.15.164:59356) 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 -- 172.21.15.164:0/3925743752 learned_addr learned my addr 172.21.15.164:0/3925743752 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da6800640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da80b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1d9d400640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da815ba20 0x7f1da80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 -- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 msgr2=0x7f1da80b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da80b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 -- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da815ba20 msgr2=0x7f1da80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da815ba20 0x7f1da80b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 -- 172.21.15.164:0/3925743752 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80b3770 con 0x7f1da806e390 2024-05-08T04:40:55.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1da5e00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c490 tx=0x7f1d9800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:55.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980052d0 con 0x7f1da806e390 2024-05-08T04:40:55.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98005470 con 0x7f1da806e390 2024-05-08T04:40:55.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98012870 con 0x7f1da806e390 2024-05-08T04:40:55.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80afc20 con 0x7f1da806e390 2024-05-08T04:40:55.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da80b01e0 con 0x7f1da806e390 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.213+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da8072d30 con 0x7f1da806e390 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.217+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f1d98012a10 con 0x7f1da806e390 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.217+0000 7f1d9ea00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 0x7f1d84069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.217+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f1d98085310 con 0x7f1da806e390 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.217+0000 7f1d9d400640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 0x7f1d84069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:55.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.217+0000 7f1d9d400640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 0x7f1d84069810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1d900ad000 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:55.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.221+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d98055640 con 0x7f1da806e390 2024-05-08T04:40:55.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:55.405+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1da8167950 con 0x7f1da806e390 2024-05-08T04:40:56.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.041+0000 7f1d9ea00640 1 -- 172.21.15.164:0/3925743752 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d980596e0 con 0x7f1da806e390 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 msgr2=0x7f1d84069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 0x7f1d84069810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1d900ad000 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 msgr2=0x7f1da80c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c490 tx=0x7f1d9800c950 comp rx=0 tx=0).stop 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 shutdown_connections 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1d84067350 0x7f1d84069810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1da815ce10 0x7f1da80b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da815ba20 0x7f1da80b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 --2- 172.21.15.164:0/3925743752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da806e390 0x7f1da80c5680 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 >> 172.21.15.164:0/3925743752 conn(0x7f1da80cf770 msgr2=0x7f1da815e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 shutdown_connections 2024-05-08T04:40:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.045+0000 7f1dace00640 1 -- 172.21.15.164:0/3925743752 wait complete. 2024-05-08T04:40:56.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:40:56.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 Processor -- start 2024-05-08T04:40:56.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 -- start start 2024-05-08T04:40:56.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f6015d6c0 0x7f3f60143600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f60070bd0 0x7f3f60071060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f60143cc0 con 0x7f3f600b43d0 2024-05-08T04:40:56.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6006ac90 con 0x7f3f60070bd0 2024-05-08T04:40:56.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f68200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6015daa0 con 0x7f3f6015d6c0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f64e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3f600b43d0 0x7f3f6015a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50824/0 (socket says 172.21.15.164:50824) 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f64e00640 1 -- 172.21.15.164:0/803856913 learned_addr learned my addr 172.21.15.164:0/803856913 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f66200640 1 --2- 172.21.15.164:0/803856913 >> v2:172.21.15.172:3300/0 conn(0x7f3f60070bd0 0x7f3f60071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f65800640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f6015d6c0 0x7f3f60143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956562493 0 0) 0x7f3f60143cc0 con 0x7f3f600b43d0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f48003530 con 0x7f3f600b43d0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f66200640 1 -- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f6015d6c0 msgr2=0x7f3f60143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f66200640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f6015d6c0 0x7f3f60143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f66200640 1 -- 172.21.15.164:0/803856913 >> v1:172.21.15.164:6789/0 conn(0x7f3f600b43d0 legacy=0x7f3f6015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833561978 0 0) 0x7f3f48003530 con 0x7f3f600b43d0 2024-05-08T04:40:56.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.257+0000 7f3f66200640 1 -- 172.21.15.164:0/803856913 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6006aee0 con 0x7f3f60070bd0 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f66200640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f60070bd0 0x7f3f60071060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009d90 tx=0x7f3f5801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f3146027c206 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f580ac070 con 0x7f3f60070bd0 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580a6480 con 0x7f3f60070bd0 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f5fe00640 1 -- 172.21.15.164:0/803856913 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c001680 con 0x7f3f60070bd0 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 -- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f60070bd0 msgr2=0x7f3f60071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f60070bd0 0x7f3f60071060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009d90 tx=0x7f3f5801c800 comp rx=0 tx=0).stop 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 -- 172.21.15.164:0/803856913 shutdown_connections 2024-05-08T04:40:56.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f60070bd0 0x7f3f60071060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 --2- 172.21.15.164:0/803856913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f6015d6c0 0x7f3f60143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 -- 172.21.15.164:0/803856913 >> 172.21.15.164:0/803856913 conn(0x7f3f6006cca0 msgr2=0x7f3f6006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 -- 172.21.15.164:0/803856913 shutdown_connections 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.261+0000 7f3f68200640 1 -- 172.21.15.164:0/803856913 wait complete. 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 Processor -- start 2024-05-08T04:40:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- start start 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f60070bd0 0x7f3f600cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 0x7f3f6015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6006bb40 con 0x7f3f600b43d0 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6006b9c0 con 0x7f3f6015d6c0 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6006bcc0 con 0x7f3f60070bd0 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:56.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59366/0 (socket says 172.21.15.164:59366) 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 -- 172.21.15.164:0/2555541130 learned_addr learned my addr 172.21.15.164:0/2555541130 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f66200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 0x7f3f6015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 -- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f60070bd0 msgr2=0x7f3f600cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f60070bd0 0x7f3f600cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 -- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 msgr2=0x7f3f6015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 0x7f3f6015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 -- 172.21.15.164:0/2555541130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600b30c0 con 0x7f3f600b43d0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f66200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 0x7f3f6015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f64e00640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c780 tx=0x7f3f4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c005ae0 con 0x7f3f600b43d0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c005ca0 con 0x7f3f600b43d0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0072b0 con 0x7f3f600b43d0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6015a010 con 0x7f3f600b43d0 2024-05-08T04:40:56.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f600cba70 con 0x7f3f600b43d0 2024-05-08T04:40:56.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.265+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6006ddf0 con 0x7f3f600b43d0 2024-05-08T04:40:56.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.269+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f3f4c007450 con 0x7f3f600b43d0 2024-05-08T04:40:56.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.269+0000 7f3f5d600640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 0x7f3f40069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:56.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.269+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f3f4c0856e0 con 0x7f3f600b43d0 2024-05-08T04:40:56.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.269+0000 7f3f65800640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 0x7f3f40069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:56.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.269+0000 7f3f65800640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 0x7f3f40069810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006aa0 tx=0x7f3f54006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:56.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.273+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f4c0560f0 con 0x7f3f600b43d0 2024-05-08T04:40:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:56.429+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f6004f010 con 0x7f3f600b43d0 2024-05-08T04:40:57.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.045+0000 7f3f5d600640 1 -- 172.21.15.164:0/2555541130 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3f4c0559f0 con 0x7f3f600b43d0 2024-05-08T04:40:57.050 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:40:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 msgr2=0x7f3f40069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 0x7f3f40069810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006aa0 tx=0x7f3f54006a30 comp rx=0 tx=0).stop 2024-05-08T04:40:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 msgr2=0x7f3f60159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c780 tx=0x7f3f4c00cc40 comp rx=0 tx=0).stop 2024-05-08T04:40:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 shutdown_connections 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f40067350 0x7f3f40069810 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f6015d6c0 0x7f3f6015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f600b43d0 0x7f3f60159ad0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 --2- 172.21.15.164:0/2555541130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f60070bd0 0x7f3f600cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 >> 172.21.15.164:0/2555541130 conn(0x7f3f6006cca0 msgr2=0x7f3f60142920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 shutdown_connections 2024-05-08T04:40:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.049+0000 7f3f68200640 1 -- 172.21.15.164:0/2555541130 wait complete. 2024-05-08T04:40:57.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:40:57.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 Processor -- start 2024-05-08T04:40:57.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 -- start start 2024-05-08T04:40:57.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f8410160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f841006de50 0x7f841006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100c6600 con 0x7f84100ae6e0 2024-05-08T04:40:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841015fbe0 con 0x7f841006de50 2024-05-08T04:40:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f8415e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c6040 con 0x7f841015ba20 2024-05-08T04:40:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f84100ae6e0 0x7f84100cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50834/0 (socket says 172.21.15.164:50834) 2024-05-08T04:40:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 -- 172.21.15.164:0/3780992031 learned_addr learned my addr 172.21.15.164:0/3780992031 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201837067 0 0) 0x7f84100c6600 con 0x7f84100ae6e0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f4003530 con 0x7f84100ae6e0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840ea00640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f8410160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840e000640 1 --2- 172.21.15.164:0/3780992031 >> v2:172.21.15.172:3300/0 conn(0x7f841006de50 0x7f841006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089096941 0 0) 0x7f83f4003530 con 0x7f84100ae6e0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 msgr2=0x7f8410160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f8410160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 >> v2:172.21.15.172:3300/0 conn(0x7f841006de50 msgr2=0x7f841006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 --2- 172.21.15.164:0/3780992031 >> v2:172.21.15.172:3300/0 conn(0x7f841006de50 0x7f841006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100c6320 con 0x7f84100ae6e0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f8408001680 con 0x7f84100ae6e0 2024-05-08T04:40:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 >> v1:172.21.15.164:6789/0 conn(0x7f84100ae6e0 legacy=0x7f84100cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f83f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f4004010 0x7f83f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f83f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841015fbe0 con 0x7f83f4004d50 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c6040 con 0x7f83f4004010 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f400bb00 con 0x7f841015ba20 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f83f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840ea00640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f83f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840e000640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f4004010 0x7f83f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 -- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 msgr2=0x7f83f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841015ba20 0x7f83f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 -- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f4004010 msgr2=0x7f83f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f4004010 0x7f83f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.213+0000 7f840f400640 1 -- 172.21.15.164:0/3780992031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4003530 con 0x7f83f4004d50 2024-05-08T04:40:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f840f400640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f83f40051d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f840800c2c0 tx=0x7f840801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7749e7f1cd0fa48f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408007d60 con 0x7f83f4004d50 2024-05-08T04:40:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84080a9020 con 0x7f83f4004d50 2024-05-08T04:40:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f840d600640 1 -- 172.21.15.164:0/3780992031 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080a5bb0 con 0x7f83f4004d50 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 -- 172.21.15.164:0/3780992031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 msgr2=0x7f83f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 --2- 172.21.15.164:0/3780992031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f83f40051d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f840800c2c0 tx=0x7f840801cd40 comp rx=0 tx=0).stop 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f840f400640 1 -- 172.21.15.164:0/3780992031 reap_dead start 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 -- 172.21.15.164:0/3780992031 shutdown_connections 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 -- 172.21.15.164:0/3780992031 >> 172.21.15.164:0/3780992031 conn(0x7f84100cf770 msgr2=0x7f84100cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 -- 172.21.15.164:0/3780992031 shutdown_connections 2024-05-08T04:40:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.217+0000 7f8415e00640 1 -- 172.21.15.164:0/3780992031 wait complete. 2024-05-08T04:40:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 Processor -- start 2024-05-08T04:40:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- start start 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 0x7f84100ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84100ca860 0x7f84100addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841015ce10 con 0x7f83f4004d50 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100cf410 con 0x7f84100ca3e0 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841015cf90 con 0x7f84100ca860 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59380/0 (socket says 172.21.15.164:59380) 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 -- 172.21.15.164:0/1622994028 learned_addr learned my addr 172.21.15.164:0/1622994028 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840f400640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84100ca860 0x7f84100addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840e000640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 0x7f84100ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 -- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84100ca860 msgr2=0x7f84100addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84100ca860 0x7f84100addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 -- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 msgr2=0x7f84100ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 0x7f84100ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 -- 172.21.15.164:0/1622994028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100cbdf0 con 0x7f83f4004d50 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840e000640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 0x7f84100ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f840ea00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84000046d0 tx=0x7f8400007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400015420 con 0x7f83f4004d50 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840001a070 con 0x7f83f4004d50 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400015800 con 0x7f83f4004d50 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100cad50 con 0x7f83f4004d50 2024-05-08T04:40:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.221+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100cb310 con 0x7f83f4004d50 2024-05-08T04:40:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8410072d30 con 0x7f83f4004d50 2024-05-08T04:40:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f84000159a0 con 0x7f83f4004d50 2024-05-08T04:40:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f8407400640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 0x7f83f0069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5385+0+0 (secure 0 0 0) 0x7f8400085f50 con 0x7f83f4004d50 2024-05-08T04:40:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f840e000640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 0x7f83f0069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:57.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.225+0000 7f840e000640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 0x7f83f0069810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8410156160 tx=0x7f83fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:57.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.229+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84100cbdf0 con 0x7f83f4004d50 2024-05-08T04:40:57.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:57.385+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84100cea50 con 0x7f83f4004d50 2024-05-08T04:40:58.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.049+0000 7f8407400640 1 -- 172.21.15.164:0/1622994028 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8400056a80 con 0x7f83f4004d50 2024-05-08T04:40:58.056 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 msgr2=0x7f83f0069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 0x7f83f0069810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8410156160 tx=0x7f83fc008040 comp rx=0 tx=0).stop 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 msgr2=0x7f8410155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84000046d0 tx=0x7f8400007700 comp rx=0 tx=0).stop 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 shutdown_connections 2024-05-08T04:40:58.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f83f0067350 0x7f83f0069810 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84100ca860 0x7f84100addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100ca3e0 0x7f84100ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 --2- 172.21.15.164:0/1622994028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f4004d50 0x7f8410155130 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 >> 172.21.15.164:0/1622994028 conn(0x7f84100cf770 msgr2=0x7f841015e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 shutdown_connections 2024-05-08T04:40:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.057+0000 7f8415e00640 1 -- 172.21.15.164:0/1622994028 wait complete. 2024-05-08T04:40:58.086 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:40:58.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 Processor -- start 2024-05-08T04:40:58.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 -- start start 2024-05-08T04:40:58.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 0x7f63001617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f630006b3a0 0x7f63000cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000cf530 con 0x7f63000aeda0 2024-05-08T04:40:58.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000ce9b0 con 0x7f630006b3a0 2024-05-08T04:40:58.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f6305200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000cec90 con 0x7f6300071c80 2024-05-08T04:40:58.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.229+0000 7f62ffe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f630006b3a0 0x7f63000cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62f6a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f63000aeda0 0x7f630006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50846/0 (socket says 172.21.15.164:50846) 2024-05-08T04:40:58.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62f6a00640 1 -- 172.21.15.164:0/4255006386 learned_addr learned my addr 172.21.15.164:0/4255006386 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:58.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ff400640 1 --2- 172.21.15.164:0/4255006386 >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 0x7f63001617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ffe00640 1 -- 172.21.15.164:0/4255006386 >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 msgr2=0x7f63001617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ffe00640 1 --2- 172.21.15.164:0/4255006386 >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 0x7f63001617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ffe00640 1 -- 172.21.15.164:0/4255006386 >> v1:172.21.15.164:6789/0 conn(0x7f63000aeda0 legacy=0x7f630006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ffe00640 1 -- 172.21.15.164:0/4255006386 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000cef70 con 0x7f630006b3a0 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ff400640 1 --2- 172.21.15.164:0/4255006386 >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 0x7f63001617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:40:58.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62fea00640 1 -- 172.21.15.164:0/4255006386 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087464351 0 0) 0x7f63000cf530 con 0x7f63000aeda0 2024-05-08T04:40:58.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62ffe00640 1 --2- 172.21.15.164:0/4255006386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63000cc450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f63000b2340 tx=0x7f62f801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80df595f0b4dc232 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62fea00640 1 -- 172.21.15.164:0/4255006386 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80b2070 con 0x7f630006b3a0 2024-05-08T04:40:58.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62fea00640 1 -- 172.21.15.164:0/4255006386 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f80a5930 con 0x7f630006b3a0 2024-05-08T04:40:58.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.233+0000 7f62fea00640 1 -- 172.21.15.164:0/4255006386 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80a5d00 con 0x7f630006b3a0 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- 172.21.15.164:0/4255006386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 msgr2=0x7f63000cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- 172.21.15.164:0/4255006386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63000cc450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f63000b2340 tx=0x7f62f801c870 comp rx=0 tx=0).stop 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- 172.21.15.164:0/4255006386 shutdown_connections 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- 172.21.15.164:0/4255006386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63000cc450 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- 172.21.15.164:0/4255006386 >> v2:172.21.15.172:3300/0 conn(0x7f6300071c80 0x7f63001617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- 172.21.15.164:0/4255006386 >> 172.21.15.164:0/4255006386 conn(0x7f63000b1450 msgr2=0x7f63000b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- 172.21.15.164:0/4255006386 shutdown_connections 2024-05-08T04:40:58.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- 172.21.15.164:0/4255006386 wait complete. 2024-05-08T04:40:58.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 Processor -- start 2024-05-08T04:40:58.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- start start 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 0x7f63001f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 0x7f630015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630016ad10 con 0x7f630006b3a0 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630016ab90 con 0x7f6300071c80 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f6305200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630016ae90 con 0x7f63000aeda0 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59404/0 (socket says 172.21.15.164:59404) 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 -- 172.21.15.164:0/4131319119 learned_addr learned my addr 172.21.15.164:0/4131319119 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62f6a00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 0x7f63001f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ffe00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 0x7f630015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 -- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 msgr2=0x7f630015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 0x7f630015f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 -- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 msgr2=0x7f63001f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 0x7f63001f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 -- 172.21.15.164:0/4131319119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630015f8c0 con 0x7f630006b3a0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62f6a00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 0x7f63001f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ffe00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 0x7f630015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.237+0000 7f62ff400640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c7f0 tx=0x7f62f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0007a30 con 0x7f630006b3a0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0007bd0 con 0x7f630006b3a0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f00113f0 con 0x7f630006b3a0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63001590b0 con 0x7f630006b3a0 2024-05-08T04:40:58.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6300159670 con 0x7f630006b3a0 2024-05-08T04:40:58.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630004ef30 con 0x7f630006b3a0 2024-05-08T04:40:58.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f62f0011590 con 0x7f630006b3a0 2024-05-08T04:40:58.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 0x7f62dc069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f62f0085710 con 0x7f630006b3a0 2024-05-08T04:40:58.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f6a00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 0x7f62dc069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.241+0000 7f62f6a00640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 0x7f62dc069840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f63000cc520 tx=0x7f62e8007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.245+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62f0015020 con 0x7f630006b3a0 2024-05-08T04:40:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.405+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6300159840 con 0x7f630006b3a0 2024-05-08T04:40:58.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f62f7e00640 1 -- 172.21.15.164:0/4131319119 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62f0056360 con 0x7f630006b3a0 2024-05-08T04:40:58.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 msgr2=0x7f62dc069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 0x7f62dc069840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f63000cc520 tx=0x7f62e8007cb0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 msgr2=0x7f63001578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c7f0 tx=0x7f62f000ccb0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 shutdown_connections 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62dc067380 0x7f62dc069840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63000aeda0 0x7f630015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6300071c80 0x7f63001f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 --2- 172.21.15.164:0/4131319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006b3a0 0x7f63001578b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 >> 172.21.15.164:0/4131319119 conn(0x7f63000b1450 msgr2=0x7f6300162000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 shutdown_connections 2024-05-08T04:40:58.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.409+0000 7f6305200640 1 -- 172.21.15.164:0/4131319119 wait complete. 2024-05-08T04:40:58.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2024-05-08T04:40:58.437 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:40:58.441 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:40:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 Processor -- start 2024-05-08T04:40:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 -- start start 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55a415b9f0 0x7f55a406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a40cfa20 0x7f55a406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a415f400 con 0x7f55a406ef60 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a415f6e0 con 0x7f55a415b9f0 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a415f9c0 con 0x7f55a40cfa20 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f5599400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f55a406ef60 0x7f55a406d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50854/0 (socket says 172.21.15.164:50854) 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55a415b9f0 0x7f55a406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f5599400640 1 -- 172.21.15.164:0/2547103117 learned_addr learned my addr 172.21.15.164:0/2547103117 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a2800640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a40cfa20 0x7f55a406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1e00640 1 -- 172.21.15.164:0/2547103117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a40cfa20 msgr2=0x7f55a406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:58.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1e00640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a40cfa20 0x7f55a406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1e00640 1 -- 172.21.15.164:0/2547103117 >> v1:172.21.15.164:6789/0 conn(0x7f55a406ef60 legacy=0x7f55a406d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1e00640 1 -- 172.21.15.164:0/2547103117 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a415fca0 con 0x7f55a415b9f0 2024-05-08T04:40:58.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.569+0000 7f55a1400640 1 -- 172.21.15.164:0/2547103117 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297487002 0 0) 0x7f55a415f400 con 0x7f55a406ef60 2024-05-08T04:40:58.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a1e00640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a415b9f0 0x7f55a406ac90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f55940097c0 tx=0x7f559401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c011a0bb0a5b7c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a1400640 1 -- 172.21.15.164:0/2547103117 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940aa600 con 0x7f55a415b9f0 2024-05-08T04:40:58.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a1400640 1 -- 172.21.15.164:0/2547103117 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55940af070 con 0x7f55a415b9f0 2024-05-08T04:40:58.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a1400640 1 -- 172.21.15.164:0/2547103117 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940aa9c0 con 0x7f55a415b9f0 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 -- 172.21.15.164:0/2547103117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a415b9f0 msgr2=0x7f55a406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a415b9f0 0x7f55a406ac90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f55940097c0 tx=0x7f559401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 -- 172.21.15.164:0/2547103117 shutdown_connections 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a40cfa20 0x7f55a406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 --2- 172.21.15.164:0/2547103117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a415b9f0 0x7f55a406ac90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 -- 172.21.15.164:0/2547103117 >> 172.21.15.164:0/2547103117 conn(0x7f55a4066fa0 msgr2=0x7f55a406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:58.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 -- 172.21.15.164:0/2547103117 shutdown_connections 2024-05-08T04:40:58.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.573+0000 7f55a3200640 1 -- 172.21.15.164:0/2547103117 wait complete. 2024-05-08T04:40:58.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 Processor -- start 2024-05-08T04:40:58.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- start start 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55a406ef60 0x7f55a41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a415b9f0 0x7f55a4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4168b30 con 0x7f55a40cfa20 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41689b0 con 0x7f55a406ef60 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4168cb0 con 0x7f55a415b9f0 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59418/0 (socket says 172.21.15.164:59418) 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 -- 172.21.15.164:0/3927114499 learned_addr learned my addr 172.21.15.164:0/3927114499 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a2800640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a415b9f0 0x7f55a4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a1e00640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55a406ef60 0x7f55a41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 -- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a415b9f0 msgr2=0x7f55a4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a415b9f0 0x7f55a4159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 -- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55a406ef60 msgr2=0x7f55a41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:58.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55a406ef60 0x7f55a41574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:58.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 -- 172.21.15.164:0/3927114499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a414e760 con 0x7f55a40cfa20 2024-05-08T04:40:58.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f5599400640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f558c00c4e0 tx=0x7f558c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c018070 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c007500 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a4152b10 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c004a30 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a41530d0 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.577+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a404ef30 con 0x7f55a40cfa20 2024-05-08T04:40:58.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.581+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f558c01b070 con 0x7f55a40cfa20 2024-05-08T04:40:58.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.581+0000 7f559aa00640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 0x7f5580069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:58.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.581+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f558c0847b0 con 0x7f55a40cfa20 2024-05-08T04:40:58.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.581+0000 7f55a1e00640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 0x7f5580069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:58.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.581+0000 7f55a1e00640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 0x7f5580069810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f55940ae000 tx=0x7f55940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:58.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.585+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f558c055540 con 0x7f55a40cfa20 2024-05-08T04:40:58.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:58.793+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f55a40725c0 con 0x7f55a40cfa20 2024-05-08T04:40:59.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.053+0000 7f559aa00640 1 -- 172.21.15.164:0/3927114499 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v167) v1 ==== 166+0+0 (secure 0 0 0) 0x7f558c054e40 con 0x7f55a40cfa20 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.053+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 msgr2=0x7f5580069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.053+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 0x7f5580069810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f55940ae000 tx=0x7f55940a7000 comp rx=0 tx=0).stop 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 msgr2=0x7f55a414d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f558c00c4e0 tx=0x7f558c00c9a0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 shutdown_connections 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5580067350 0x7f5580069810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55a415b9f0 0x7f55a4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55a40cfa20 0x7f55a414d8a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 --2- 172.21.15.164:0/3927114499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55a406ef60 0x7f55a41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 >> 172.21.15.164:0/3927114499 conn(0x7f55a4066fa0 msgr2=0x7f55a40b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 shutdown_connections 2024-05-08T04:40:59.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.057+0000 7f55a3200640 1 -- 172.21.15.164:0/3927114499 wait complete. 2024-05-08T04:40:59.080 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2024-05-08T04:40:59.081 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2024-05-08T04:40:59.081 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:40:59.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:40:59.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.213+0000 7f0b55c00640 1 Processor -- start 2024-05-08T04:40:59.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.213+0000 7f0b55c00640 1 -- start start 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.213+0000 7f0b55c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.213+0000 7f0b55c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b500cfbd0 0x7f0b500cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b55c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5006c770 con 0x7f0b5015ab60 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b55c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b500d0060 con 0x7f0b500cfbd0 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b55c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5006ca50 con 0x7f0b500b26c0 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4ea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0b5015ab60 0x7f0b50072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39974/0 (socket says 172.21.15.164:39974) 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4ea00640 1 -- 172.21.15.164:0/3668156976 learned_addr learned my addr 172.21.15.164:0/3668156976 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b46000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 --2- 172.21.15.164:0/3668156976 >> v2:172.21.15.172:3300/0 conn(0x7f0b500cfbd0 0x7f0b500cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249153085 0 0) 0x7f0b5006c770 con 0x7f0b5015ab60 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b34003530 con 0x7f0b5015ab60 2024-05-08T04:40:59.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832033044 0 0) 0x7f0b34003530 con 0x7f0b5015ab60 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 msgr2=0x7f0b500cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b500cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 >> v2:172.21.15.172:3300/0 conn(0x7f0b500cfbd0 msgr2=0x7f0b500cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 --2- 172.21.15.164:0/3668156976 >> v2:172.21.15.172:3300/0 conn(0x7f0b500cfbd0 0x7f0b500cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5006cd30 con 0x7f0b5015ab60 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f0b40001680 con 0x7f0b5015ab60 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 >> v1:172.21.15.164:6789/0 conn(0x7f0b5015ab60 legacy=0x7f0b50072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b340039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b34003f90 0x7f0b34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b34004d50 0x7f0b340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5006ca50 con 0x7f0b34004d50 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3400b9d0 con 0x7f0b34003f90 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3400bb50 con 0x7f0b500b26c0 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b34004d50 0x7f0b340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4ea00640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b340039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b46000640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b34003f90 0x7f0b34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 -- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 msgr2=0x7f0b340039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b500b26c0 0x7f0b340039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 -- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b34003f90 msgr2=0x7f0b34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b34003f90 0x7f0b34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.217+0000 7f0b4f400640 1 -- 172.21.15.164:0/3668156976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34003530 con 0x7f0b34004d50 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b4f400640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b34004d50 0x7f0b340051d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e170 tx=0x7f0b4801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab78c0005cdb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480b2070 con 0x7f0b34004d50 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b480a5cb0 con 0x7f0b34004d50 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b4e000640 1 -- 172.21.15.164:0/3668156976 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48009c10 con 0x7f0b34004d50 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- 172.21.15.164:0/3668156976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b34004d50 msgr2=0x7f0b340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 --2- 172.21.15.164:0/3668156976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b34004d50 0x7f0b340051d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e170 tx=0x7f0b4801cca0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b4f400640 1 -- 172.21.15.164:0/3668156976 reap_dead start 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- 172.21.15.164:0/3668156976 shutdown_connections 2024-05-08T04:40:59.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- 172.21.15.164:0/3668156976 >> 172.21.15.164:0/3668156976 conn(0x7f0b5006ac90 msgr2=0x7f0b5006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- 172.21.15.164:0/3668156976 shutdown_connections 2024-05-08T04:40:59.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- 172.21.15.164:0/3668156976 wait complete. 2024-05-08T04:40:59.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 Processor -- start 2024-05-08T04:40:59.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.221+0000 7f0b55c00640 1 -- start start 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b34004d50 0x7f0b50159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b5015ba70 0x7f0b500ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5006c3d0 con 0x7f0b5014c660 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5006d2f0 con 0x7f0b5015ba70 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5006c550 con 0x7f0b34004d50 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57708/0 (socket says 172.21.15.164:57708) 2024-05-08T04:40:59.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 -- 172.21.15.164:0/860668971 learned_addr learned my addr 172.21.15.164:0/860668971 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b46000640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b5015ba70 0x7f0b500ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4ea00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b34004d50 0x7f0b50159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 -- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b34004d50 msgr2=0x7f0b50159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b34004d50 0x7f0b50159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 -- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b5015ba70 msgr2=0x7f0b500ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b5015ba70 0x7f0b500ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 -- 172.21.15.164:0/860668971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5015d0f0 con 0x7f0b5014c660 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b4f400640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e170 tx=0x7f0b48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480bb020 con 0x7f0b5014c660 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b480a5ab0 con 0x7f0b5014c660 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5015d390 con 0x7f0b5014c660 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480b2040 con 0x7f0b5014c660 2024-05-08T04:40:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5015d950 con 0x7f0b5014c660 2024-05-08T04:40:59.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50069d60 con 0x7f0b5014c660 2024-05-08T04:40:59.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.225+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f0b480b6410 con 0x7f0b5014c660 2024-05-08T04:40:59.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.229+0000 7f0b47400640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 0x7f0b2c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.229+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f0b4811f970 con 0x7f0b5014c660 2024-05-08T04:40:59.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.229+0000 7f0b4ea00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 0x7f0b2c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.229+0000 7f0b4ea00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 0x7f0b2c069810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b40001060 tx=0x7f0b40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.233+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b480c5060 con 0x7f0b5014c660 2024-05-08T04:40:59.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.385+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b500cf940 con 0x7f0b5014c660 2024-05-08T04:40:59.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b47400640 1 -- 172.21.15.164:0/860668971 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13125 (secure 0 0 0) 0x7f0b48016a30 con 0x7f0b5014c660 2024-05-08T04:40:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:40:59.392 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":65,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:40:58.050503+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/1114280106":"2024-05-09T04:40:55.039742+0000","172.21.15.164:6834/1114280106":"2024-05-09T04:40:55.039742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:40:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 msgr2=0x7f0b2c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 0x7f0b2c069810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b40001060 tx=0x7f0b40009040 comp rx=0 tx=0).stop 2024-05-08T04:40:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 msgr2=0x7f0b500addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e170 tx=0x7f0b48009750 comp rx=0 tx=0).stop 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 shutdown_connections 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b2c067350 0x7f0b2c069810 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b5014c660 0x7f0b500addd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b5015ba70 0x7f0b500ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 --2- 172.21.15.164:0/860668971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b34004d50 0x7f0b50159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 >> 172.21.15.164:0/860668971 conn(0x7f0b5006ac90 msgr2=0x7f0b5006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 shutdown_connections 2024-05-08T04:40:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.389+0000 7f0b55c00640 1 -- 172.21.15.164:0/860668971 wait complete. 2024-05-08T04:40:59.413 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:40:59.414 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:40:59.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 Processor -- start 2024-05-08T04:40:59.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 -- start start 2024-05-08T04:40:59.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc15c900 con 0x7fd1cc0b1960 2024-05-08T04:40:59.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc15cec0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.549+0000 7fd1cbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15cbe0 con 0x7fd1cc0cec40 2024-05-08T04:40:59.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1c2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42570/0 (socket says 172.21.15.164:42570) 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1caa00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd1cc0b1960 0x7fd1cc071aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:55136/0 (socket says 172.21.15.164:55136) 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 -- 172.21.15.164:0/3237114744 learned_addr learned my addr 172.21.15.164:0/3237114744 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 -- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 msgr2=0x7fd1cc15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 --2- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc15bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 -- 172.21.15.164:0/3237114744 >> v1:172.21.15.172:6789/0 conn(0x7fd1cc0b1960 legacy=0x7fd1cc071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.553+0000 7fd1cb400640 1 -- 172.21.15.164:0/3237114744 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0712a0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cb400640 1 --2- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009870 tx=0x7fd1c401cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5273a5cd03bca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1ca000640 1 -- 172.21.15.164:0/3237114744 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40ab660 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1ca000640 1 -- 172.21.15.164:0/3237114744 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c40b0070 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1ca000640 1 -- 172.21.15.164:0/3237114744 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40aba20 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1d40 msgr2=0x7fd1cc0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009870 tx=0x7fd1c401cea0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3237114744 shutdown_connections 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3237114744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc0ccca0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3237114744 >> 172.21.15.164:0/3237114744 conn(0x7fd1cc06ac90 msgr2=0x7fd1cc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3237114744 shutdown_connections 2024-05-08T04:40:59.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3237114744 wait complete. 2024-05-08T04:40:59.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 Processor -- start 2024-05-08T04:40:59.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- start start 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 0x7fd1cc1f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc1f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b3d70 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b3bf0 con 0x7fd1cc0b1960 2024-05-08T04:40:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.557+0000 7fd1cbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b3ef0 con 0x7fd1cc0cec40 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57724/0 (socket says 172.21.15.164:57724) 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 -- 172.21.15.164:0/3984863233 learned_addr learned my addr 172.21.15.164:0/3984863233 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1caa00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 0x7fd1cc1f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1cb400640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc1f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 -- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 msgr2=0x7fd1cc1f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc1f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 -- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 msgr2=0x7fd1cc1f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 0x7fd1cc1f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 -- 172.21.15.164:0/3984863233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc159ba0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1caa00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 0x7fd1cc1f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:59.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c2000640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e840 tx=0x7fd1b400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400ee90 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400cb80 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc1517e0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1527a0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc04ef30 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd1b401a070 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 0x7fd1a8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.561+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd1b40853b0 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.565+0000 7fd1caa00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 0x7fd1a8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.565+0000 7fd1caa00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 0x7fd1a8069810 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc005ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.565+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1b4056080 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.725+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1cc0b2c50 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.725+0000 7fd1c3400640 1 -- 172.21.15.164:0/3984863233 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1b4055980 con 0x7fd1cc0b1d40 2024-05-08T04:40:59.732 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 msgr2=0x7fd1a8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 0x7fd1a8069810 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc005ba0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 msgr2=0x7fd1cc1512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e840 tx=0x7fd1b400ec40 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 shutdown_connections 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd1a8067350 0x7fd1a8069810 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1cc0cec40 0x7fd1cc1f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1cc0b1d40 0x7fd1cc1512a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 --2- 172.21.15.164:0/3984863233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1cc0b1960 0x7fd1cc1f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 >> 172.21.15.164:0/3984863233 conn(0x7fd1cc06ac90 msgr2=0x7fd1cc06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 shutdown_connections 2024-05-08T04:40:59.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.729+0000 7fd1cbe00640 1 -- 172.21.15.164:0/3984863233 wait complete. 2024-05-08T04:40:59.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2024-05-08T04:40:59.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 Processor -- start 2024-05-08T04:40:59.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 -- start start 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f232415ba70 0x7f2324160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f23240cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240cd9f0 con 0x7f23240aeda0 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f232415fe70 con 0x7f232415ba70 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2328c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23240cdcd0 con 0x7f232415be50 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f232415ba70 0x7f2324160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f232415ba70 0x7f2324160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57732/0 (socket says 172.21.15.164:57732) 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 -- 172.21.15.164:0/2790984967 learned_addr learned my addr 172.21.15.164:0/2790984967 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:59.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321400640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f23240cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 -- 172.21.15.164:0/2790984967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 msgr2=0x7f23240cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:40:59.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f23240cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 -- 172.21.15.164:0/2790984967 >> v1:172.21.15.172:6789/0 conn(0x7f23240aeda0 legacy=0x7f23240b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.885+0000 7f2321e00640 1 -- 172.21.15.164:0/2790984967 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232415f820 con 0x7f232415ba70 2024-05-08T04:40:59.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2321e00640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f2324160300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2314009e50 tx=0x7f231401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2456b822e17afc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2320a00640 1 -- 172.21.15.164:0/2790984967 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140a57e0 con 0x7f232415ba70 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2320a00640 1 -- 172.21.15.164:0/2790984967 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23140a5980 con 0x7f232415ba70 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2320a00640 1 -- 172.21.15.164:0/2790984967 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140a5d50 con 0x7f232415ba70 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 -- 172.21.15.164:0/2790984967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 msgr2=0x7f2324160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f2324160300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2314009e50 tx=0x7f231401ca00 comp rx=0 tx=0).stop 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 -- 172.21.15.164:0/2790984967 shutdown_connections 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f23240cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 --2- 172.21.15.164:0/2790984967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f2324160300 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 -- 172.21.15.164:0/2790984967 >> 172.21.15.164:0/2790984967 conn(0x7f232406ac90 msgr2=0x7f232406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:40:59.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 -- 172.21.15.164:0/2790984967 shutdown_connections 2024-05-08T04:40:59.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.889+0000 7f2328c00640 1 -- 172.21.15.164:0/2790984967 wait complete. 2024-05-08T04:40:59.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 Processor -- start 2024-05-08T04:40:59.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- start start 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23240aeda0 0x7f2324153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f2324148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324168d40 con 0x7f232415ba70 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324168bc0 con 0x7f23240aeda0 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2324168ec0 con 0x7f232415be50 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57736/0 (socket says 172.21.15.164:57736) 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 -- 172.21.15.164:0/1498722930 learned_addr learned my addr 172.21.15.164:0/1498722930 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321e00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23240aeda0 0x7f2324153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2322800640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f2324148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 -- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 msgr2=0x7f2324148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f2324148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 -- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23240aeda0 msgr2=0x7f2324153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23240aeda0 0x7f2324153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 -- 172.21.15.164:0/1498722930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232415df40 con 0x7f232415ba70 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2322800640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f2324148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2321400640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f231000e840 tx=0x7f231000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310017070 con 0x7f232415ba70 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231000ee90 con 0x7f232415ba70 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000cb80 con 0x7f232415ba70 2024-05-08T04:40:59.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232415e220 con 0x7f232415ba70 2024-05-08T04:40:59.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232415eac0 con 0x7f232415ba70 2024-05-08T04:40:59.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.893+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232415e3f0 con 0x7f232415ba70 2024-05-08T04:40:59.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.897+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f231001a070 con 0x7f232415ba70 2024-05-08T04:40:59.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.897+0000 7f231a000640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 0x7f2304069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:40:59.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.897+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2310085660 con 0x7f232415ba70 2024-05-08T04:40:59.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.897+0000 7f2321e00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 0x7f2304069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:40:59.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.897+0000 7f2321e00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 0x7f2304069810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23140ae000 tx=0x7f23140a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:40:59.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:40:59.901+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2310055ce0 con 0x7f232415ba70 2024-05-08T04:41:00.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:00.109+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f2324069d60 con 0x7f232415ba70 2024-05-08T04:41:01.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f231a000640 1 -- 172.21.15.164:0/1498722930 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v169) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2310059d80 con 0x7f232415ba70 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 msgr2=0x7f2304069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 0x7f2304069810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23140ae000 tx=0x7f23140a7070 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 msgr2=0x7f232415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f231000e840 tx=0x7f231000ec40 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 shutdown_connections 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2304067350 0x7f2304069810 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f232415be50 0x7f2324148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f232415ba70 0x7f232415d480 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 --2- 172.21.15.164:0/1498722930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23240aeda0 0x7f2324153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 >> 172.21.15.164:0/1498722930 conn(0x7f232406ac90 msgr2=0x7f2324155230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 shutdown_connections 2024-05-08T04:41:01.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.061+0000 7f2328c00640 1 -- 172.21.15.164:0/1498722930 wait complete. 2024-05-08T04:41:01.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:41:01.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 Processor -- start 2024-05-08T04:41:01.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 -- start start 2024-05-08T04:41:01.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f537415ab10 0x7f537406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 0x7f53740aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 0x7f53740cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740cf130 con 0x7f537415ab10 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537406bb80 con 0x7f53740ae6e0 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5379400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537415e870 con 0x7f53740ce780 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f537415ab10 0x7f537406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f537415ab10 0x7f537406cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57740/0 (socket says 172.21.15.164:57740) 2024-05-08T04:41:01.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 -- 172.21.15.164:0/77207697 learned_addr learned my addr 172.21.15.164:0/77207697 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5373200640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 0x7f53740cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 -- 172.21.15.164:0/77207697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 msgr2=0x7f53740cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5371e00640 1 --2- 172.21.15.164:0/77207697 >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 0x7f53740aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 0x7f53740cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 -- 172.21.15.164:0/77207697 >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 msgr2=0x7f53740aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 --2- 172.21.15.164:0/77207697 >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 0x7f53740aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5372800640 1 -- 172.21.15.164:0/77207697 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537415eb50 con 0x7f537415ab10 2024-05-08T04:41:01.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5373200640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 0x7f53740cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:01.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.229+0000 7f5371e00640 1 --2- 172.21.15.164:0/77207697 >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 0x7f53740aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:01.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5372800640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f537415ab10 0x7f537406cce0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f53740cf3c0 tx=0x7f536401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cdd64bca2a38af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5371400640 1 -- 172.21.15.164:0/77207697 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640ab620 con 0x7f537415ab10 2024-05-08T04:41:01.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5371400640 1 -- 172.21.15.164:0/77207697 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53640b0070 con 0x7f537415ab10 2024-05-08T04:41:01.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5371400640 1 -- 172.21.15.164:0/77207697 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640ab9e0 con 0x7f537415ab10 2024-05-08T04:41:01.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 -- 172.21.15.164:0/77207697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f537415ab10 msgr2=0x7f537406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f537415ab10 0x7f537406cce0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f53740cf3c0 tx=0x7f536401c460 comp rx=0 tx=0).stop 2024-05-08T04:41:01.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 -- 172.21.15.164:0/77207697 shutdown_connections 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53740ce780 0x7f53740cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 --2- 172.21.15.164:0/77207697 >> v2:172.21.15.172:3300/0 conn(0x7f53740ae6e0 0x7f53740aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 --2- 172.21.15.164:0/77207697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f537415ab10 0x7f537406cce0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 -- 172.21.15.164:0/77207697 >> 172.21.15.164:0/77207697 conn(0x7f53740cf770 msgr2=0x7f53740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 -- 172.21.15.164:0/77207697 shutdown_connections 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.233+0000 7f5379400640 1 -- 172.21.15.164:0/77207697 wait complete. 2024-05-08T04:41:01.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 Processor -- start 2024-05-08T04:41:01.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- start start 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 0x7f537415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f537415ab10 0x7f537414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c5b70 con 0x7f53740ae6e0 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537415aef0 con 0x7f53740ce780 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c5cf0 con 0x7f537415ab10 2024-05-08T04:41:01.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57742/0 (socket says 172.21.15.164:57742) 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 -- 172.21.15.164:0/454339432 learned_addr learned my addr 172.21.15.164:0/454339432 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5371e00640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 0x7f537415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5373200640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f537415ab10 0x7f537414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 -- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f537415ab10 msgr2=0x7f537414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f537415ab10 0x7f537414f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 -- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 msgr2=0x7f537415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 0x7f537415b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 -- 172.21.15.164:0/454339432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374137640 con 0x7f53740ae6e0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5371e00640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 0x7f537415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5372800640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f53640ad040 tx=0x7f536401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640abd80 con 0x7f53740ae6e0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53640be3f0 con 0x7f53740ae6e0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640be7e0 con 0x7f53740ae6e0 2024-05-08T04:41:01.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537414c0a0 con 0x7f53740ae6e0 2024-05-08T04:41:01.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537414dc70 con 0x7f53740ae6e0 2024-05-08T04:41:01.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.237+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374147650 con 0x7f53740ae6e0 2024-05-08T04:41:01.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.241+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f53640c7440 con 0x7f53740ae6e0 2024-05-08T04:41:01.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.241+0000 7f536aa00640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 0x7f5354069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.241+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f536411fbc0 con 0x7f53740ae6e0 2024-05-08T04:41:01.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.241+0000 7f5371e00640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 0x7f5354069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.241+0000 7f5371e00640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 0x7f5354069810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5360006fd0 tx=0x7f5360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.245+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5364016110 con 0x7f53740ae6e0 2024-05-08T04:41:01.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.397+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f53740ce560 con 0x7f53740ae6e0 2024-05-08T04:41:01.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.401+0000 7f536aa00640 1 -- 172.21.15.164:0/454339432 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53640f0950 con 0x7f53740ae6e0 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 msgr2=0x7f5354069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 0x7f5354069810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5360006fd0 tx=0x7f5360008040 comp rx=0 tx=0).stop 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 msgr2=0x7f5374137100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f53640ad040 tx=0x7f536401aca0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 shutdown_connections 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5354067350 0x7f5354069810 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f537415ab10 0x7f537414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53740ce780 0x7f537415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 --2- 172.21.15.164:0/454339432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53740ae6e0 0x7f5374137100 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 >> 172.21.15.164:0/454339432 conn(0x7f53740cf770 msgr2=0x7f537414c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:01.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 shutdown_connections 2024-05-08T04:41:01.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.405+0000 7f5379400640 1 -- 172.21.15.164:0/454339432 wait complete. 2024-05-08T04:41:01.429 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:41:01.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:01.490 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:01.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:41:01.590 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:01.590 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:01.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:41:01.599 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:01.599 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:01.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:41:01.609 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:01.609 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:01.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:41:01.651 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:01.651 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:01.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:41:01.660 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:41:01.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:01.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.829+0000 7fd792600640 1 Processor -- start 2024-05-08T04:41:01.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.829+0000 7fd792600640 1 -- start start 2024-05-08T04:41:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd792600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd78c0cdfa0 0x7fd78c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd792600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c06c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd792600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c071980 con 0x7fd78c0af480 2024-05-08T04:41:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd792600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c161810 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd792600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c06c650 con 0x7fd78c06bca0 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd78c0cdfa0 0x7fd78c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd78be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd78c0cdfa0 0x7fd78c0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42624/0 (socket says 172.21.15.164:42624) 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 -- 172.21.15.164:0/2819087352 learned_addr learned my addr 172.21.15.164:0/2819087352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd78b400640 1 --1- 172.21.15.164:0/2819087352 >> v1:172.21.15.172:6789/0 conn(0x7fd78c0af480 0x7fd78c06cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:55158/0 (socket says 172.21.15.164:55158) 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 -- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 msgr2=0x7fd78c06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 --2- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c06c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 -- 172.21.15.164:0/2819087352 >> v1:172.21.15.172:6789/0 conn(0x7fd78c0af480 legacy=0x7fd78c06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd783e00640 1 -- 172.21.15.164:0/2819087352 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c071c00 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.833+0000 7fd78aa00640 1 -- 172.21.15.164:0/2819087352 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001966366 0 0) 0x7fd77c001120 con 0x7fd78c0af480 2024-05-08T04:41:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd783e00640 1 --2- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0cf470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd774009870 tx=0x7fd77401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59701950b75d26c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd78aa00640 1 -- 172.21.15.164:0/2819087352 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7740a7070 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd78aa00640 1 -- 172.21.15.164:0/2819087352 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7740b0070 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd78aa00640 1 -- 172.21.15.164:0/2819087352 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7740ab890 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0cdfa0 msgr2=0x7fd78c0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 --2- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0cf470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd774009870 tx=0x7fd77401c560 comp rx=0 tx=0).stop 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- 172.21.15.164:0/2819087352 shutdown_connections 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 --2- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 --2- 172.21.15.164:0/2819087352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0cf470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- 172.21.15.164:0/2819087352 >> 172.21.15.164:0/2819087352 conn(0x7fd78c0ccab0 msgr2=0x7fd78c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:01.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- 172.21.15.164:0/2819087352 shutdown_connections 2024-05-08T04:41:01.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- 172.21.15.164:0/2819087352 wait complete. 2024-05-08T04:41:01.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 Processor -- start 2024-05-08T04:41:01.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.837+0000 7fd792600640 1 -- start start 2024-05-08T04:41:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c0c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0af480 0x7fd78c15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c071f50 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c06eed0 con 0x7fd78c0af480 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c072bf0 con 0x7fd78c06bca0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57758/0 (socket says 172.21.15.164:57758) 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 -- 172.21.15.164:0/3371169685 learned_addr learned my addr 172.21.15.164:0/3371169685 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd783e00640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0af480 0x7fd78c15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78b400640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c0c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 -- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 msgr2=0x7fd78c0c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c0c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 -- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0af480 msgr2=0x7fd78c15de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0af480 0x7fd78c15de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 -- 172.21.15.164:0/3371169685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0c53c0 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd78be00640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd784009dd0 tx=0x7fd78400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784018020 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784014480 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784014870 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c0c56a0 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c0c5c60 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c069d60 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.841+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd78401b020 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.845+0000 7fd783400640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 0x7fd768069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.845+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd784085f60 con 0x7fd78c0cdfa0 2024-05-08T04:41:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.845+0000 7fd78b400640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 0x7fd768069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.845+0000 7fd78b400640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 0x7fd768069810 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000f70 tx=0x7fd77c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:01.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:01.849+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd784022070 con 0x7fd78c0cdfa0 2024-05-08T04:41:02.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd78c066df0 con 0x7fd78c0cdfa0 2024-05-08T04:41:02.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd783400640 1 -- 172.21.15.164:0/3371169685 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fd784010040 con 0x7fd78c0cdfa0 2024-05-08T04:41:02.040 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:02.041 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/776122260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":776122260},{"type":"v1","addr":"172.21.15.172:6834","nonce":776122260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2881096781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2881096781},{"type":"v1","addr":"172.21.15.172:6835","nonce":2881096781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3978567499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3978567499},{"type":"v1","addr":"172.21.15.164:6836","nonce":3978567499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2241833370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2241833370},{"type":"v1","addr":"172.21.15.164:6837","nonce":2241833370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2024-05-08T04:41:02.041 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 37 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 msgr2=0x7fd768069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 0x7fd768069810 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000f70 tx=0x7fd77c007b60 comp rx=0 tx=0).stop 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 msgr2=0x7fd78c0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd784009dd0 tx=0x7fd78400c490 comp rx=0 tx=0).stop 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 shutdown_connections 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd768067350 0x7fd768069810 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd78c0cdfa0 0x7fd78c0c9900 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd78c0af480 0x7fd78c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 --2- 172.21.15.164:0/3371169685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd78c06bca0 0x7fd78c0c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 >> 172.21.15.164:0/3371169685 conn(0x7fd78c0ccab0 msgr2=0x7fd78c160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:02.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 shutdown_connections 2024-05-08T04:41:02.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.037+0000 7fd792600640 1 -- 172.21.15.164:0/3371169685 wait complete. 2024-05-08T04:41:02.062 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:41:02.062 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:41:02.063 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:41:02.063 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:02.063 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:41:02.063 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:41:02.064 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:41:02.064 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:02.064 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:41:02.064 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:41:02.064 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:41:02.064 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:02.065 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:41:02.065 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:41:02.065 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:41:02.065 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:02.077 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:41:02.078 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:41:02.078 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:41:02.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:41:02.080 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:41:02.080 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:41:02.080 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:41:02.081 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:41:02.122 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:41:02.123 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:02.124 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:41:02.124 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:02.125 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:41:02.126 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:02.127 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:41:02.127 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:02.127 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:41:02.209 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:41:02.209 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:41:02.216 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:41:02.216 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:41:02.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 Processor -- start 2024-05-08T04:41:02.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 -- start start 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d7c0cdc10 0x7f2d7c0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c0702f0 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0ae5e0 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d83200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c06f970 con 0x7f2d7c0cdc10 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57796/0 (socket says 172.21.15.164:57796) 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 -- 172.21.15.164:0/303738783 learned_addr learned my addr 172.21.15.164:0/303738783 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:02.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81400640 1 --1- 172.21.15.164:0/303738783 >> v1:172.21.15.164:6789/0 conn(0x7f2d7c0b3a80 0x7f2d7c06ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39990/0 (socket says 172.21.15.164:39990) 2024-05-08T04:41:02.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d82800640 1 --2- 172.21.15.164:0/303738783 >> v2:172.21.15.172:3300/0 conn(0x7f2d7c0cdc10 0x7f2d7c0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:02.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 -- 172.21.15.164:0/303738783 >> v2:172.21.15.172:3300/0 conn(0x7f2d7c0cdc10 msgr2=0x7f2d7c0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:02.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 --2- 172.21.15.164:0/303738783 >> v2:172.21.15.172:3300/0 conn(0x7f2d7c0cdc10 0x7f2d7c0ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 -- 172.21.15.164:0/303738783 >> v1:172.21.15.164:6789/0 conn(0x7f2d7c0b3a80 legacy=0x7f2d7c06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.269+0000 7f2d81e00640 1 -- 172.21.15.164:0/303738783 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0cce50 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d81e00640 1 --2- 172.21.15.164:0/303738783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0cf610 tx=0x7f2d7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553288af7c8eaf38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:02.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d80a00640 1 -- 172.21.15.164:0/303738783 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700ab660 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d80a00640 1 -- 172.21.15.164:0/303738783 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700b0070 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d80a00640 1 -- 172.21.15.164:0/303738783 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700aba20 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 -- 172.21.15.164:0/303738783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c15f0f0 msgr2=0x7f2d7c06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 --2- 172.21.15.164:0/303738783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0cf610 tx=0x7f2d7001c460 comp rx=0 tx=0).stop 2024-05-08T04:41:02.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 -- 172.21.15.164:0/303738783 shutdown_connections 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 --2- 172.21.15.164:0/303738783 >> v2:172.21.15.172:3300/0 conn(0x7f2d7c0cdc10 0x7f2d7c0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 --2- 172.21.15.164:0/303738783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c15f0f0 0x7f2d7c06e4c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 -- 172.21.15.164:0/303738783 >> 172.21.15.164:0/303738783 conn(0x7f2d7c09aa20 msgr2=0x7f2d7c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 -- 172.21.15.164:0/303738783 shutdown_connections 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.273+0000 7f2d83200640 1 -- 172.21.15.164:0/303738783 wait complete. 2024-05-08T04:41:02.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 Processor -- start 2024-05-08T04:41:02.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- start start 2024-05-08T04:41:02.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d7c0cdc10 0x7f2d7c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d7c15f0f0 0x7f2d7c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15f4d0 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c06f660 con 0x7f2d7c0cdc10 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15f650 con 0x7f2d7c15f0f0 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57808/0 (socket says 172.21.15.164:57808) 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 -- 172.21.15.164:0/935849885 learned_addr learned my addr 172.21.15.164:0/935849885 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 -- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d7c15f0f0 msgr2=0x7f2d7c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d7c15f0f0 0x7f2d7c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 -- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d7c0cdc10 msgr2=0x7f2d7c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d7c0cdc10 0x7f2d7c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:02.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 -- 172.21.15.164:0/935849885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0b5180 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d81e00640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2d70019070 tx=0x7f2d7001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:02.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700be5f0 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700be790 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700beb80 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c0b5460 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c0b5a20 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.277+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c04ef30 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.281+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f2d700b43f0 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.281+0000 7f2d76000640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 0x7f2d60069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:02.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.281+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2d70120140 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.281+0000 7f2d81400640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 0x7f2d60069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:02.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.281+0000 7f2d81400640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 0x7f2d60069810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000f70 tx=0x7f2d6c007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:02.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.285+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d70016110 con 0x7f2d7c0b3a80 2024-05-08T04:41:02.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:02.453+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2d7c0d0170 con 0x7f2d7c0b3a80 2024-05-08T04:41:03.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.425+0000 7f2d76000640 1 -- 172.21.15.164:0/935849885 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2d700f0eb0 con 0x7f2d7c0b3a80 2024-05-08T04:41:03.429 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.425+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 msgr2=0x7f2d60069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.425+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 0x7f2d60069810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000f70 tx=0x7f2d6c007a10 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.425+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 msgr2=0x7f2d7c151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.425+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2d70019070 tx=0x7f2d7001aca0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 shutdown_connections 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d60067350 0x7f2d60069810 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d7c15f0f0 0x7f2d7c0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d7c0cdc10 0x7f2d7c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 --2- 172.21.15.164:0/935849885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7c0b3a80 0x7f2d7c151220 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 >> 172.21.15.164:0/935849885 conn(0x7f2d7c09aa20 msgr2=0x7f2d7c15df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:03.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 shutdown_connections 2024-05-08T04:41:03.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.429+0000 7f2d83200640 1 -- 172.21.15.164:0/935849885 wait complete. 2024-05-08T04:41:03.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:41:03.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 Processor -- start 2024-05-08T04:41:03.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 -- start start 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febb40ccdf0 0x7febb40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7febb40addd0 0x7febb406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb4072320 con 0x7febb406b1d0 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb406d4e0 con 0x7febb40ccdf0 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.593+0000 7febbb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb4072600 con 0x7febb40addd0 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febb40ccdf0 0x7febb40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febb40ccdf0 0x7febb40ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57812/0 (socket says 172.21.15.164:57812) 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 -- 172.21.15.164:0/1148391754 learned_addr learned my addr 172.21.15.164:0/1148391754 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:03.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3400640 1 --2- 172.21.15.164:0/1148391754 >> v2:172.21.15.172:3300/0 conn(0x7febb40addd0 0x7febb406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 -- 172.21.15.164:0/1148391754 >> v2:172.21.15.172:3300/0 conn(0x7febb40addd0 msgr2=0x7febb406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:03.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 --2- 172.21.15.164:0/1148391754 >> v2:172.21.15.172:3300/0 conn(0x7febb40addd0 0x7febb406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 -- 172.21.15.164:0/1148391754 >> v1:172.21.15.172:6789/0 conn(0x7febb406b1d0 legacy=0x7febb406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 -- 172.21.15.164:0/1148391754 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb406d730 con 0x7febb40ccdf0 2024-05-08T04:41:03.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb3e00640 1 --2- 172.21.15.164:0/1148391754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb40ccdf0 0x7febb40ad890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feba4009870 tx=0x7feba401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4916a81d3894c0f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb2a00640 1 -- 172.21.15.164:0/1148391754 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40b0070 con 0x7febb40ccdf0 2024-05-08T04:41:03.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb2a00640 1 -- 172.21.15.164:0/1148391754 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40ab600 con 0x7febb40ccdf0 2024-05-08T04:41:03.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.597+0000 7febb2a00640 1 -- 172.21.15.164:0/1148391754 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40ab9d0 con 0x7febb40ccdf0 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- 172.21.15.164:0/1148391754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb40ccdf0 msgr2=0x7febb40ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- 172.21.15.164:0/1148391754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb40ccdf0 0x7febb40ad890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feba4009870 tx=0x7feba401c490 comp rx=0 tx=0).stop 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- 172.21.15.164:0/1148391754 shutdown_connections 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- 172.21.15.164:0/1148391754 >> v2:172.21.15.172:3300/0 conn(0x7febb40addd0 0x7febb406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- 172.21.15.164:0/1148391754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb40ccdf0 0x7febb40ad890 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- 172.21.15.164:0/1148391754 >> 172.21.15.164:0/1148391754 conn(0x7febb4066fa0 msgr2=0x7febb406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- 172.21.15.164:0/1148391754 shutdown_connections 2024-05-08T04:41:03.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- 172.21.15.164:0/1148391754 wait complete. 2024-05-08T04:41:03.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 Processor -- start 2024-05-08T04:41:03.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- start start 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb40addd0 0x7febb40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febb40ccdf0 0x7febb40b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4157af0 con 0x7febb406b1d0 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4071ff0 con 0x7febb40addd0 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febbb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb4157c70 con 0x7febb40ccdf0 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57826/0 (socket says 172.21.15.164:57826) 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 -- 172.21.15.164:0/3144818337 learned_addr learned my addr 172.21.15.164:0/3144818337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3400640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb40addd0 0x7febb40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb9200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febb40ccdf0 0x7febb40b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 -- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febb40ccdf0 msgr2=0x7febb40b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febb40ccdf0 0x7febb40b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 -- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb40addd0 msgr2=0x7febb40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb40addd0 0x7febb40b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.601+0000 7febb3e00640 1 -- 172.21.15.164:0/3144818337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40cf8a0 con 0x7febb406b1d0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febb3e00640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feba40ae040 tx=0x7feba401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40ba020 con 0x7febb406b1d0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40abe10 con 0x7febb406b1d0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb40cfb80 con 0x7febb406b1d0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40b0040 con 0x7febb406b1d0 2024-05-08T04:41:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb40d0140 con 0x7febb406b1d0 2024-05-08T04:41:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb4069d60 con 0x7febb406b1d0 2024-05-08T04:41:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7feba40bd020 con 0x7febb406b1d0 2024-05-08T04:41:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 0x7feb94069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4885+0+0 (secure 0 0 0) 0x7feba411f840 con 0x7febb406b1d0 2024-05-08T04:41:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.605+0000 7febb3400640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 0x7feb94069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.609+0000 7febb3400640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 0x7feb94069810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7febb406add0 tx=0x7feba0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.613+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feba40c4030 con 0x7febb406b1d0 2024-05-08T04:41:03.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.817+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7febb4160330 con 0x7febb406b1d0 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.817+0000 7febabe00640 1 -- 172.21.15.164:0/3144818337 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7feba4016110 con 0x7febb406b1d0 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:41:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:41:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:41:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:41:03.836 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 msgr2=0x7feb94069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 0x7feb94069810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7febb406add0 tx=0x7feba0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 msgr2=0x7febb40cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feba40ae040 tx=0x7feba401acd0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 shutdown_connections 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7feb94067350 0x7feb94069810 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febb40ccdf0 0x7febb40b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb40addd0 0x7febb40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 --2- 172.21.15.164:0/3144818337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febb406b1d0 0x7febb40cef70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 >> 172.21.15.164:0/3144818337 conn(0x7febb4066fa0 msgr2=0x7febb4160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:03.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 shutdown_connections 2024-05-08T04:41:03.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.821+0000 7febbb200640 1 -- 172.21.15.164:0/3144818337 wait complete. 2024-05-08T04:41:03.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:03.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.973+0000 7f246ea00640 1 Processor -- start 2024-05-08T04:41:03.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.973+0000 7f246ea00640 1 -- start start 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f24680ca020 0x7f24680cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f24680c4420 0x7f2468070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2468071b20 con 0x7f24680cef80 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246ea00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24680718d0 con 0x7f24680c4420 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468071e00 con 0x7f24680ca020 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f24680c4420 0x7f2468070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f24680c4420 0x7f2468070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57834/0 (socket says 172.21.15.164:57834) 2024-05-08T04:41:03.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 -- 172.21.15.164:0/1282652626 learned_addr learned my addr 172.21.15.164:0/1282652626 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:03.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 -- 172.21.15.164:0/1282652626 >> v2:172.21.15.172:3300/0 conn(0x7f24680ca020 msgr2=0x7f24680cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:03.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 --2- 172.21.15.164:0/1282652626 >> v2:172.21.15.172:3300/0 conn(0x7f24680ca020 0x7f24680cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 -- 172.21.15.164:0/1282652626 >> v1:172.21.15.172:6789/0 conn(0x7f24680cef80 legacy=0x7f2468070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:03.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 -- 172.21.15.164:0/1282652626 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680720e0 con 0x7f24680c4420 2024-05-08T04:41:03.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f246cc00640 1 --2- 172.21.15.164:0/1282652626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680c4420 0x7f2468070810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1ec0 tx=0x7f245401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931a0362d543060d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.977+0000 7f2467e00640 1 -- 172.21.15.164:0/1282652626 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24540b5070 con 0x7f24680c4420 2024-05-08T04:41:03.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f2467e00640 1 -- 172.21.15.164:0/1282652626 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24540a89d0 con 0x7f24680c4420 2024-05-08T04:41:03.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f2467e00640 1 -- 172.21.15.164:0/1282652626 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24540a8da0 con 0x7f24680c4420 2024-05-08T04:41:03.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- 172.21.15.164:0/1282652626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680c4420 msgr2=0x7f2468070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- 172.21.15.164:0/1282652626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680c4420 0x7f2468070810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1ec0 tx=0x7f245401ecd0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- 172.21.15.164:0/1282652626 shutdown_connections 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- 172.21.15.164:0/1282652626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680c4420 0x7f2468070810 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- 172.21.15.164:0/1282652626 >> v2:172.21.15.172:3300/0 conn(0x7f24680ca020 0x7f24680cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- 172.21.15.164:0/1282652626 >> 172.21.15.164:0/1282652626 conn(0x7f246806b670 msgr2=0x7f24680b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- 172.21.15.164:0/1282652626 shutdown_connections 2024-05-08T04:41:03.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- 172.21.15.164:0/1282652626 wait complete. 2024-05-08T04:41:03.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 Processor -- start 2024-05-08T04:41:03.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- start start 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680c4420 0x7f24681f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680ca020 0x7f246815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468071460 con 0x7f24680cef80 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680ce3d0 con 0x7f24680c4420 2024-05-08T04:41:03.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680715e0 con 0x7f24680ca020 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57850/0 (socket says 172.21.15.164:57850) 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246e000640 1 -- 172.21.15.164:0/856750685 learned_addr learned my addr 172.21.15.164:0/856750685 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246cc00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680ca020 0x7f246815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.981+0000 7f246d600640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680c4420 0x7f24681f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 -- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680ca020 msgr2=0x7f246815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680ca020 0x7f246815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 -- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680c4420 msgr2=0x7f24681f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680c4420 0x7f24681f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 -- 172.21.15.164:0/856750685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24681f3b60 con 0x7f24680cef80 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246e000640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f246000b990 tx=0x7f246000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460019070 con 0x7f24680cef80 2024-05-08T04:41:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246000cba0 con 0x7f24680cef80 2024-05-08T04:41:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460014440 con 0x7f24680cef80 2024-05-08T04:41:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24681563e0 con 0x7f24680cef80 2024-05-08T04:41:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681569a0 con 0x7f24680cef80 2024-05-08T04:41:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246804ef30 con 0x7f24680cef80 2024-05-08T04:41:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f246001c070 con 0x7f24680cef80 2024-05-08T04:41:03.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 0x7f2448069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:03.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.985+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f2460085150 con 0x7f24680cef80 2024-05-08T04:41:03.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.989+0000 7f246d600640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 0x7f2448069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:03.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.989+0000 7f246d600640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 0x7f2448069810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f245c009990 tx=0x7f245c009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:03.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:03.989+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24681f3b60 con 0x7f24680cef80 2024-05-08T04:41:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.145+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24680b1150 con 0x7f24680cef80 2024-05-08T04:41:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f2465600640 1 -- 172.21.15.164:0/856750685 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+12994 (secure 0 0 0) 0x7f2460055820 con 0x7f24680cef80 2024-05-08T04:41:04.152 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:04.152 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":70,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:03.426378+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 msgr2=0x7f2448069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 0x7f2448069810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f245c009990 tx=0x7f245c009110 comp rx=0 tx=0).stop 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 msgr2=0x7f246815b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f246000b990 tx=0x7f246000be50 comp rx=0 tx=0).stop 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 shutdown_connections 2024-05-08T04:41:04.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2448067350 0x7f2448069810 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680ca020 0x7f246815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24680cef80 0x7f246815b1e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 --2- 172.21.15.164:0/856750685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680c4420 0x7f24681f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 >> 172.21.15.164:0/856750685 conn(0x7f246806b670 msgr2=0x7f24680c4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 shutdown_connections 2024-05-08T04:41:04.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.149+0000 7f246ea00640 1 -- 172.21.15.164:0/856750685 wait complete. 2024-05-08T04:41:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:04.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:04.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.305+0000 7fd3e6000640 1 Processor -- start 2024-05-08T04:41:04.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 -- start start 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e006cce0 0x7fd3e00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e015e870 con 0x7fd3e015ab10 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e00cf130 con 0x7fd3e00cecc0 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3e6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e006bb80 con 0x7fd3e006cce0 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57852/0 (socket says 172.21.15.164:57852) 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 -- 172.21.15.164:0/2478976588 learned_addr learned my addr 172.21.15.164:0/2478976588 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:04.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dea00640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e006cce0 0x7fd3e00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 -- 172.21.15.164:0/2478976588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e006cce0 msgr2=0x7fd3e00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e006cce0 0x7fd3e00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 -- 172.21.15.164:0/2478976588 >> v1:172.21.15.172:6789/0 conn(0x7fd3e015ab10 legacy=0x7fd3e00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.309+0000 7fd3dfe00640 1 -- 172.21.15.164:0/2478976588 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e015eb50 con 0x7fd3e00cecc0 2024-05-08T04:41:04.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3dfe00640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009e50 tx=0x7fd3d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ccd3d731b9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3de000640 1 -- 172.21.15.164:0/2478976588 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80a5740 con 0x7fd3e00cecc0 2024-05-08T04:41:04.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3de000640 1 -- 172.21.15.164:0/2478976588 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80a58e0 con 0x7fd3e00cecc0 2024-05-08T04:41:04.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3de000640 1 -- 172.21.15.164:0/2478976588 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80a5cb0 con 0x7fd3e00cecc0 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- 172.21.15.164:0/2478976588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e00cecc0 msgr2=0x7fd3e00ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009e50 tx=0x7fd3d801c970 comp rx=0 tx=0).stop 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- 172.21.15.164:0/2478976588 shutdown_connections 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e00cecc0 0x7fd3e00ca6e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2478976588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e006cce0 0x7fd3e00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- 172.21.15.164:0/2478976588 >> 172.21.15.164:0/2478976588 conn(0x7fd3e00cf770 msgr2=0x7fd3e00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- 172.21.15.164:0/2478976588 shutdown_connections 2024-05-08T04:41:04.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- 172.21.15.164:0/2478976588 wait complete. 2024-05-08T04:41:04.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 Processor -- start 2024-05-08T04:41:04.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.313+0000 7fd3e6000640 1 -- start start 2024-05-08T04:41:04.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 0x7fd3e0070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 0x7fd3e0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00aedf0 con 0x7fd3e006cce0 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e015aef0 con 0x7fd3e00cecc0 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00aef70 con 0x7fd3e015ab10 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57854/0 (socket says 172.21.15.164:57854) 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 -- 172.21.15.164:0/2053106485 learned_addr learned my addr 172.21.15.164:0/2053106485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3dea00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 0x7fd3e0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3dfe00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 0x7fd3e0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 -- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 msgr2=0x7fd3e0158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 0x7fd3e0158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 -- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 msgr2=0x7fd3e0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 0x7fd3e0070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 -- 172.21.15.164:0/2053106485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0071060 con 0x7fd3e006cce0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3dea00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 0x7fd3e0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3dfe00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 0x7fd3e0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3df400640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800c370 tx=0x7fd3c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80052d0 con 0x7fd3e006cce0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c8005470 con 0x7fd3e006cce0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8007260 con 0x7fd3e006cce0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e0071340 con 0x7fd3e006cce0 2024-05-08T04:41:04.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e0071900 con 0x7fd3e006cce0 2024-05-08T04:41:04.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e0072d30 con 0x7fd3e006cce0 2024-05-08T04:41:04.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.317+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd3c8007400 con 0x7fd3e006cce0 2024-05-08T04:41:04.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.321+0000 7fd3d7400640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 0x7fd3bc069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.321+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd3c8084810 con 0x7fd3e006cce0 2024-05-08T04:41:04.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.321+0000 7fd3dea00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 0x7fd3bc069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.321+0000 7fd3dea00640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 0x7fd3bc069810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0154980 tx=0x7fd3d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.325+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3c8055600 con 0x7fd3e006cce0 2024-05-08T04:41:04.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3e015de80 con 0x7fd3e006cce0 2024-05-08T04:41:04.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3d7400640 1 -- 172.21.15.164:0/2053106485 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3c8054f00 con 0x7fd3e006cce0 2024-05-08T04:41:04.485 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 msgr2=0x7fd3bc069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 0x7fd3bc069810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0154980 tx=0x7fd3d0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 msgr2=0x7fd3e015d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800c370 tx=0x7fd3c800c830 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 shutdown_connections 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd3bc067350 0x7fd3bc069810 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3e015ab10 0x7fd3e0158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e00cecc0 0x7fd3e0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 --2- 172.21.15.164:0/2053106485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e006cce0 0x7fd3e015d920 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 >> 172.21.15.164:0/2053106485 conn(0x7fd3e00cf770 msgr2=0x7fd3e014c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 shutdown_connections 2024-05-08T04:41:04.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.481+0000 7fd3e6000640 1 -- 172.21.15.164:0/2053106485 wait complete. 2024-05-08T04:41:04.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:41:04.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 Processor -- start 2024-05-08T04:41:04.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 -- start start 2024-05-08T04:41:04.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42c00b26c0 0x7f42c00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00cfbd0 0x7f42c00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c006c770 con 0x7f42c015ab60 2024-05-08T04:41:04.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c00d0060 con 0x7f42c00b26c0 2024-05-08T04:41:04.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c006ca50 con 0x7f42c00cfbd0 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42bfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42c00b26c0 0x7f42c00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42bfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42c00b26c0 0x7f42c00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57864/0 (socket says 172.21.15.164:57864) 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42bfe00640 1 -- 172.21.15.164:0/3443933350 learned_addr learned my addr 172.21.15.164:0/3443933350 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42c5600640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00cfbd0 0x7f42c00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42bfe00640 1 -- 172.21.15.164:0/3443933350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00cfbd0 msgr2=0x7f42c00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.637+0000 7f42bfe00640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00cfbd0 0x7f42c00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bfe00640 1 -- 172.21.15.164:0/3443933350 >> v1:172.21.15.172:6789/0 conn(0x7f42c015ab60 legacy=0x7f42c0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bfe00640 1 -- 172.21.15.164:0/3443933350 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c006cd30 con 0x7f42c00b26c0 2024-05-08T04:41:04.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bfe00640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c00b26c0 0x7f42c00cf690 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009870 tx=0x7f42ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1bf6848f3ed420 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bf400640 1 -- 172.21.15.164:0/3443933350 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0b2070 con 0x7f42c00b26c0 2024-05-08T04:41:04.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bf400640 1 -- 172.21.15.164:0/3443933350 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac0a9070 con 0x7f42c00b26c0 2024-05-08T04:41:04.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42bf400640 1 -- 172.21.15.164:0/3443933350 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0ad880 con 0x7f42c00b26c0 2024-05-08T04:41:04.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 -- 172.21.15.164:0/3443933350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c00b26c0 msgr2=0x7f42c00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c00b26c0 0x7f42c00cf690 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009870 tx=0x7f42ac01c560 comp rx=0 tx=0).stop 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 -- 172.21.15.164:0/3443933350 shutdown_connections 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00cfbd0 0x7f42c00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 --2- 172.21.15.164:0/3443933350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c00b26c0 0x7f42c00cf690 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 -- 172.21.15.164:0/3443933350 >> 172.21.15.164:0/3443933350 conn(0x7f42c006ac90 msgr2=0x7f42c006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 -- 172.21.15.164:0/3443933350 shutdown_connections 2024-05-08T04:41:04.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.641+0000 7f42c7600640 1 -- 172.21.15.164:0/3443933350 wait complete. 2024-05-08T04:41:04.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 Processor -- start 2024-05-08T04:41:04.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- start start 2024-05-08T04:41:04.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00b26c0 0x7f42c01509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c00cfbd0 0x7f42c014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00aeeb0 con 0x7f42c015ab60 2024-05-08T04:41:04.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00aed30 con 0x7f42c00cfbd0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c0167970 con 0x7f42c00b26c0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57874/0 (socket says 172.21.15.164:57874) 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 -- 172.21.15.164:0/823484614 learned_addr learned my addr 172.21.15.164:0/823484614 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42bfe00640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c00cfbd0 0x7f42c014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c4c00640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00b26c0 0x7f42c01509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 -- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00b26c0 msgr2=0x7f42c01509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00b26c0 0x7f42c01509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 -- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c00cfbd0 msgr2=0x7f42c014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c00cfbd0 0x7f42c014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 -- 172.21.15.164:0/823484614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0150f10 con 0x7f42c015ab60 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c5600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d830 tx=0x7f42b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800b6c0 con 0x7f42c015ab60 2024-05-08T04:41:04.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8019070 con 0x7f42c015ab60 2024-05-08T04:41:04.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800baa0 con 0x7f42c015ab60 2024-05-08T04:41:04.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c0158740 con 0x7f42c015ab60 2024-05-08T04:41:04.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.645+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c0158d00 con 0x7f42c015ab60 2024-05-08T04:41:04.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c01593c0 con 0x7f42c015ab60 2024-05-08T04:41:04.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f42b801e460 con 0x7f42c015ab60 2024-05-08T04:41:04.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42bcc00640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 0x7f42a0069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f42b8085a80 con 0x7f42c015ab60 2024-05-08T04:41:04.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42c4c00640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 0x7f42a0069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.649+0000 7f42c4c00640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 0x7f42a0069840 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42b0001cb0 tx=0x7f42b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.653+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42c01593c0 con 0x7f42c015ab60 2024-05-08T04:41:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.809+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f42c00ceb90 con 0x7f42c015ab60 2024-05-08T04:41:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.809+0000 7f42bcc00640 1 -- 172.21.15.164:0/823484614 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7f42c0150f10 con 0x7f42c015ab60 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:04.816 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 msgr2=0x7f42a0069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 0x7f42a0069840 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42b0001cb0 tx=0x7f42b0009040 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 msgr2=0x7f42c00ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d830 tx=0x7f42b800dcf0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 shutdown_connections 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f42a0067380 0x7f42a0069840 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c015ab60 0x7f42c00ad890 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42c00cfbd0 0x7f42c014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 --2- 172.21.15.164:0/823484614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42c00b26c0 0x7f42c01509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 >> 172.21.15.164:0/823484614 conn(0x7f42c006ac90 msgr2=0x7f42c00b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 shutdown_connections 2024-05-08T04:41:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.813+0000 7f42c7600640 1 -- 172.21.15.164:0/823484614 wait complete. 2024-05-08T04:41:04.837 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:41:04.837 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:41:04.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 Processor -- start 2024-05-08T04:41:04.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 -- start start 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe0780c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe07806ac90 0x7fe07806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0780adc50 con 0x7fe0781619d0 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780ae860 con 0x7fe07806ac90 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe078144740 con 0x7fe078144360 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ce00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe0781619d0 0x7fe078161e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39994/0 (socket says 172.21.15.164:39994) 2024-05-08T04:41:04.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ce00640 1 -- 172.21.15.164:0/976116729 learned_addr learned my addr 172.21.15.164:0/976116729 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077e00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe0780c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe06f400640 1 --2- 172.21.15.164:0/976116729 >> v2:172.21.15.172:3300/0 conn(0x7fe07806ac90 0x7fe07806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151926558 0 0) 0x7fe0780adc50 con 0x7fe0781619d0 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe05c003530 con 0x7fe0781619d0 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180645317 0 0) 0x7fe05c003530 con 0x7fe0781619d0 2024-05-08T04:41:04.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 msgr2=0x7fe0780c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe0780c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 >> v2:172.21.15.172:3300/0 conn(0x7fe07806ac90 msgr2=0x7fe07806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 --2- 172.21.15.164:0/976116729 >> v2:172.21.15.172:3300/0 conn(0x7fe07806ac90 0x7fe07806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780adf30 con 0x7fe0781619d0 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fe070001a00 con 0x7fe0781619d0 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 >> v1:172.21.15.164:6789/0 conn(0x7fe0781619d0 legacy=0x7fe078161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe05c0039c0 0x7fe05c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe05c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe05c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780ae860 con 0x7fe05c004d50 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe078144740 con 0x7fe05c0039c0 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c00b9d0 con 0x7fe078144360 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe07ce00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe05c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe06f400640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe05c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.973+0000 7fe077e00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe05c0039c0 0x7fe05c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 -- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 msgr2=0x7fe05c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe078144360 0x7fe05c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 -- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe05c0039c0 msgr2=0x7fe05c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:04.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe05c0039c0 0x7fe05c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 -- 172.21.15.164:0/976116729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c003530 con 0x7fe05c004d50 2024-05-08T04:41:04.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe05c0051d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe07000a880 tx=0x7fe07001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be891de53e0cd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07000a4a0 con 0x7fe05c004d50 2024-05-08T04:41:04.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0700ad630 con 0x7fe05c004d50 2024-05-08T04:41:04.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe077400640 1 -- 172.21.15.164:0/976116729 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700ada60 con 0x7fe05c004d50 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ee00640 1 -- 172.21.15.164:0/976116729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 msgr2=0x7fe05c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ee00640 1 --2- 172.21.15.164:0/976116729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe05c0051d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe07000a880 tx=0x7fe07001d3a0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ce00640 1 -- 172.21.15.164:0/976116729 reap_dead start 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ee00640 1 -- 172.21.15.164:0/976116729 shutdown_connections 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.977+0000 7fe07ee00640 1 -- 172.21.15.164:0/976116729 >> 172.21.15.164:0/976116729 conn(0x7fe0780cf770 msgr2=0x7fe0780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- 172.21.15.164:0/976116729 shutdown_connections 2024-05-08T04:41:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- 172.21.15.164:0/976116729 wait complete. 2024-05-08T04:41:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 Processor -- start 2024-05-08T04:41:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- start start 2024-05-08T04:41:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0781427f0 0x7fe07815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07806d880 0x7fe07806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780ad890 con 0x7fe05c004d50 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780ae4f0 con 0x7fe0781427f0 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780ada10 con 0x7fe07806d880 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57890/0 (socket says 172.21.15.164:57890) 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 -- 172.21.15.164:0/535133528 learned_addr learned my addr 172.21.15.164:0/535133528 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe06f400640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0781427f0 0x7fe07815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ce00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07806d880 0x7fe07806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 -- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07806d880 msgr2=0x7fe07806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07806d880 0x7fe07806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 -- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0781427f0 msgr2=0x7fe07815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0781427f0 0x7fe07815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 -- 172.21.15.164:0/535133528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780b4ab0 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe077e00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe0680119c0 tx=0x7fe068011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800faf0 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06800fcb0 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07813d570 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.981+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068014680 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07813db30 con 0x7fe05c004d50 2024-05-08T04:41:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe078072d30 con 0x7fe05c004d50 2024-05-08T04:41:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fe068012040 con 0x7fe05c004d50 2024-05-08T04:41:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe074c00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 0x7fe054069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe0680859f0 con 0x7fe05c004d50 2024-05-08T04:41:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe06f400640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 0x7fe054069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.985+0000 7fe06f400640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 0x7fe054069840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe078071fb0 tx=0x7fe060009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:04.989+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe068019030 con 0x7fe05c004d50 2024-05-08T04:41:05.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:05.145+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe07804f010 con 0x7fe05c004d50 2024-05-08T04:41:06.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.089+0000 7fe074c00640 1 -- 172.21.15.164:0/535133528 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe06800fe60 con 0x7fe05c004d50 2024-05-08T04:41:06.092 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 msgr2=0x7fe054069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 0x7fe054069840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe078071fb0 tx=0x7fe060009190 comp rx=0 tx=0).stop 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 msgr2=0x7fe0780b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe0680119c0 tx=0x7fe068011dc0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 shutdown_connections 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe054067380 0x7fe054069840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07806d880 0x7fe07806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0781427f0 0x7fe07815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 --2- 172.21.15.164:0/535133528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe05c004d50 0x7fe0780b41d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 >> 172.21.15.164:0/535133528 conn(0x7fe0780cf770 msgr2=0x7fe0780706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:06.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 shutdown_connections 2024-05-08T04:41:06.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.093+0000 7fe07ee00640 1 -- 172.21.15.164:0/535133528 wait complete. 2024-05-08T04:41:06.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:41:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.261+0000 7f3593c00640 1 Processor -- start 2024-05-08T04:41:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.261+0000 7f3593c00640 1 -- start start 2024-05-08T04:41:06.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3593c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f358c15ba20 0x7f358c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3593c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f358c0aedf0 0x7f358c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3593c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0c6600 con 0x7f358c06de50 2024-05-08T04:41:06.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3593c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c15fbe0 con 0x7f358c0aedf0 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3593c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0c6040 con 0x7f358c15ba20 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f358c0aedf0 0x7f358c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f358c0aedf0 0x7f358c06e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57900/0 (socket says 172.21.15.164:57900) 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 -- 172.21.15.164:0/525206504 learned_addr learned my addr 172.21.15.164:0/525206504 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591200640 1 --2- 172.21.15.164:0/525206504 >> v2:172.21.15.172:3300/0 conn(0x7f358c15ba20 0x7f358c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 -- 172.21.15.164:0/525206504 >> v2:172.21.15.172:3300/0 conn(0x7f358c15ba20 msgr2=0x7f358c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 --2- 172.21.15.164:0/525206504 >> v2:172.21.15.172:3300/0 conn(0x7f358c15ba20 0x7f358c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 -- 172.21.15.164:0/525206504 >> v1:172.21.15.172:6789/0 conn(0x7f358c06de50 legacy=0x7f358c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.265+0000 7f3591c00640 1 -- 172.21.15.164:0/525206504 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0c6320 con 0x7f358c0aedf0 2024-05-08T04:41:06.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3591c00640 1 --2- 172.21.15.164:0/525206504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c06e2c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cc0e0 tx=0x7f358401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dc732cc8acc1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:06.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f358be00640 1 -- 172.21.15.164:0/525206504 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840a5820 con 0x7f358c0aedf0 2024-05-08T04:41:06.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f358be00640 1 -- 172.21.15.164:0/525206504 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840a59c0 con 0x7f358c0aedf0 2024-05-08T04:41:06.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f358be00640 1 -- 172.21.15.164:0/525206504 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840a5d90 con 0x7f358c0aedf0 2024-05-08T04:41:06.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- 172.21.15.164:0/525206504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 msgr2=0x7f358c06e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:06.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 --2- 172.21.15.164:0/525206504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c06e2c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f358c0cc0e0 tx=0x7f358401c9c0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- 172.21.15.164:0/525206504 shutdown_connections 2024-05-08T04:41:06.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 --2- 172.21.15.164:0/525206504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c06e2c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 --2- 172.21.15.164:0/525206504 >> v2:172.21.15.172:3300/0 conn(0x7f358c15ba20 0x7f358c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- 172.21.15.164:0/525206504 >> 172.21.15.164:0/525206504 conn(0x7f358c0cf770 msgr2=0x7f358c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- 172.21.15.164:0/525206504 shutdown_connections 2024-05-08T04:41:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- 172.21.15.164:0/525206504 wait complete. 2024-05-08T04:41:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 Processor -- start 2024-05-08T04:41:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.269+0000 7f3593c00640 1 -- start start 2024-05-08T04:41:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f358c06de50 0x7f358c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0ce000 con 0x7f358c0aedf0 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0cde80 con 0x7f358c15ba20 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c168b70 con 0x7f358c06de50 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3591c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3591200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f358c06de50 0x7f358c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57902/0 (socket says 172.21.15.164:57902) 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3591c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42798/0 (socket says 172.21.15.164:42798) 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 -- 172.21.15.164:0/204810883 learned_addr learned my addr 172.21.15.164:0/204810883 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 -- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f358c06de50 msgr2=0x7f358c14dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f358c06de50 0x7f358c14dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 -- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 msgr2=0x7f358c159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 -- 172.21.15.164:0/204810883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0c9f60 con 0x7f358c0aedf0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3591c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3583e00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f357400c420 tx=0x7f357400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740052d0 con 0x7f358c0aedf0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3574005470 con 0x7f358c0aedf0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574007260 con 0x7f358c0aedf0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c0ca200 con 0x7f358c0aedf0 2024-05-08T04:41:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0ca7c0 con 0x7f358c0aedf0 2024-05-08T04:41:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.273+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c072d30 con 0x7f358c0aedf0 2024-05-08T04:41:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.277+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f3574007400 con 0x7f358c0aedf0 2024-05-08T04:41:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.277+0000 7f3589600640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 0x7f3568069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.277+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f35740849f0 con 0x7f358c0aedf0 2024-05-08T04:41:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.277+0000 7f3591200640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 0x7f3568069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:06.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.277+0000 7f3591200640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 0x7f3568069840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f357c0079c0 tx=0x7f357c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:06.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.281+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3574054f50 con 0x7f358c0aedf0 2024-05-08T04:41:06.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:06.437+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f358c0ca990 con 0x7f358c0aedf0 2024-05-08T04:41:07.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.093+0000 7f3589600640 1 -- 172.21.15.164:0/204810883 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3574058ff0 con 0x7f358c0aedf0 2024-05-08T04:41:07.099 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:41:07.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 msgr2=0x7f3568069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 0x7f3568069840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f357c0079c0 tx=0x7f357c008040 comp rx=0 tx=0).stop 2024-05-08T04:41:07.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 msgr2=0x7f358c159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f357400c420 tx=0x7f357400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 shutdown_connections 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3568067380 0x7f3568069840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358c15ba20 0x7f358c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f358c0aedf0 0x7f358c159a60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 --2- 172.21.15.164:0/204810883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f358c06de50 0x7f358c14dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 >> 172.21.15.164:0/204810883 conn(0x7f358c0cf770 msgr2=0x7f358c15e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 shutdown_connections 2024-05-08T04:41:07.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.097+0000 7f3593c00640 1 -- 172.21.15.164:0/204810883 wait complete. 2024-05-08T04:41:07.136 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:41:07.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 Processor -- start 2024-05-08T04:41:07.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 -- start start 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4efc15ab10 0x7f4efc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc15e870 con 0x7f4efc0cee90 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc15eb50 con 0x7f4efc0ae6e0 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4f02000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4efc06bb80 con 0x7f4efc15ab10 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efbe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4efc0cee90 0x7f4efc0ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40006/0 (socket says 172.21.15.164:40006) 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efbe00640 1 -- 172.21.15.164:0/2282573364 learned_addr learned my addr 172.21.15.164:0/2282573364 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efb400640 1 --2- 172.21.15.164:0/2282573364 >> v2:172.21.15.172:3300/0 conn(0x7f4efc15ab10 0x7f4efc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907038075 0 0) 0x7f4efc15e870 con 0x7f4efc0cee90 2024-05-08T04:41:07.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee0005740 con 0x7f4efc0cee90 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145764730 0 0) 0x7f4ee0005740 con 0x7f4efc0cee90 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 >> v2:172.21.15.172:3300/0 conn(0x7f4efc15ab10 msgr2=0x7f4efc06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 --2- 172.21.15.164:0/2282573364 >> v2:172.21.15.172:3300/0 conn(0x7f4efc15ab10 0x7f4efc06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 >> v1:172.21.15.172:6789/0 conn(0x7f4efc0ae6e0 legacy=0x7f4efc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc15ee30 con 0x7f4efc0cee90 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.273+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f4ef4001a20 con 0x7f4efc0cee90 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 >> v1:172.21.15.164:6789/0 conn(0x7f4efc0cee90 legacy=0x7f4efc0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 0x7f4ee0005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee00065d0 0x7f4ee0006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee0007340 0x7f4ee000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc06bb80 con 0x7f4ee0005bd0 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee000e280 con 0x7f4ee0007340 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee000e480 con 0x7f4ee00065d0 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 0x7f4ee0005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efbe00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee0007340 0x7f4ee000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4ef2a00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee00065d0 0x7f4ee0006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 -- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee00065d0 msgr2=0x7f4ee0006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee00065d0 0x7f4ee0006aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 -- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee0007340 msgr2=0x7f4ee000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee0007340 0x7f4ee000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efbe00640 1 -- 172.21.15.164:0/2282573364 reap_dead start 2024-05-08T04:41:07.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 -- 172.21.15.164:0/2282573364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0005740 con 0x7f4ee0005bd0 2024-05-08T04:41:07.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4ef2a00640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee00065d0 0x7f4ee0006aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:07.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efb400640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 0x7f4ee0005fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009ae0 tx=0x7f4eec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800d0681ae1a74d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:07.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0b2070 con 0x7f4ee0005bd0 2024-05-08T04:41:07.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eec0a5520 con 0x7f4ee0005bd0 2024-05-08T04:41:07.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.277+0000 7f4efaa00640 1 -- 172.21.15.164:0/2282573364 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0096b0 con 0x7f4ee0005bd0 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 msgr2=0x7f4ee0005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 0x7f4ee0005fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009ae0 tx=0x7f4eec01ca40 comp rx=0 tx=0).stop 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- 172.21.15.164:0/2282573364 shutdown_connections 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 --2- 172.21.15.164:0/2282573364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ee0005bd0 0x7f4ee0005fb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- 172.21.15.164:0/2282573364 >> 172.21.15.164:0/2282573364 conn(0x7f4efc0cf770 msgr2=0x7f4efc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- 172.21.15.164:0/2282573364 shutdown_connections 2024-05-08T04:41:07.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- 172.21.15.164:0/2282573364 wait complete. 2024-05-08T04:41:07.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 Processor -- start 2024-05-08T04:41:07.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- start start 2024-05-08T04:41:07.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee0005bd0 0x7f4efc14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4efc0caed0 0x7f4efc0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0c5b70 con 0x7f4efc158da0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc15aef0 con 0x7f4efc0caed0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4f02000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0c5cf0 con 0x7f4ee0005bd0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57920/0 (socket says 172.21.15.164:57920) 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 -- 172.21.15.164:0/3399397119 learned_addr learned my addr 172.21.15.164:0/3399397119 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4efbe00640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4efc0caed0 0x7f4efc0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4efb400640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee0005bd0 0x7f4efc14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 -- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee0005bd0 msgr2=0x7f4efc14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee0005bd0 0x7f4efc14a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 -- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4efc0caed0 msgr2=0x7f4efc0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4efc0caed0 0x7f4efc0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:07.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.281+0000 7f4ef2a00640 1 -- 172.21.15.164:0/3399397119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc14ac20 con 0x7f4efc158da0 2024-05-08T04:41:07.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef2a00640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e570 tx=0x7f4ee400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:07.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40052e0 con 0x7f4efc158da0 2024-05-08T04:41:07.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4005480 con 0x7f4efc158da0 2024-05-08T04:41:07.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc14af00 con 0x7f4efc158da0 2024-05-08T04:41:07.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007020 con 0x7f4efc158da0 2024-05-08T04:41:07.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc14b4c0 con 0x7f4efc158da0 2024-05-08T04:41:07.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc072d30 con 0x7f4efc158da0 2024-05-08T04:41:07.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f4ee401a020 con 0x7f4efc158da0 2024-05-08T04:41:07.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 0x7f4ed8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:07.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f4ee40858d0 con 0x7f4efc158da0 2024-05-08T04:41:07.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.285+0000 7f4efb400640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 0x7f4ed8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:07.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.289+0000 7f4efb400640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 0x7f4ed8069810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0af000 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:07.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.289+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ee401b370 con 0x7f4efc158da0 2024-05-08T04:41:07.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:07.477+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4efc04f010 con 0x7f4efc158da0 2024-05-08T04:41:08.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.097+0000 7f4ef3e00640 1 -- 172.21.15.164:0/3399397119 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4ee4056390 con 0x7f4efc158da0 2024-05-08T04:41:08.102 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 10 and data pool 11 2024-05-08T04:41:08.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 msgr2=0x7f4ed8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 0x7f4ed8069810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0af000 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-05-08T04:41:08.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 msgr2=0x7f4efc159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e570 tx=0x7f4ee400ea30 comp rx=0 tx=0).stop 2024-05-08T04:41:08.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 shutdown_connections 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4ed8067350 0x7f4ed8069810 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4efc0caed0 0x7f4efc0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4efc158da0 0x7f4efc159220 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 --2- 172.21.15.164:0/3399397119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ee0005bd0 0x7f4efc14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 >> 172.21.15.164:0/3399397119 conn(0x7f4efc0cf770 msgr2=0x7f4efc071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 shutdown_connections 2024-05-08T04:41:08.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.101+0000 7f4f02000640 1 -- 172.21.15.164:0/3399397119 wait complete. 2024-05-08T04:41:08.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:08.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 Processor -- start 2024-05-08T04:41:08.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 -- start start 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 0x7fee8c168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8c06cc70 con 0x7fee8c15dd70 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8c06cf50 con 0x7fee8c06c7b0 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee91600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0b1570 con 0x7fee8c0b2d80 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fee8c15dd70 0x7fee8c1619a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40014/0 (socket says 172.21.15.164:40014) 2024-05-08T04:41:08.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8aa00640 1 -- 172.21.15.164:0/3460846788 learned_addr learned my addr 172.21.15.164:0/3460846788 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8b400640 1 --2- 172.21.15.164:0/3460846788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 0x7fee8c168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105793315 0 0) 0x7fee8c06cc70 con 0x7fee8c15dd70 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee70003530 con 0x7fee8c15dd70 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013597998 0 0) 0x7fee70003530 con 0x7fee8c15dd70 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8b400640 1 -- 172.21.15.164:0/3460846788 >> v1:172.21.15.172:6789/0 conn(0x7fee8c06c7b0 legacy=0x7fee8c1620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8b400640 1 -- 172.21.15.164:0/3460846788 >> v1:172.21.15.164:6789/0 conn(0x7fee8c15dd70 legacy=0x7fee8c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.277+0000 7fee8b400640 1 -- 172.21.15.164:0/3460846788 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c0b1850 con 0x7fee8c0b2d80 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee8b400640 1 --2- 172.21.15.164:0/3460846788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 0x7fee8c168cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee84009dc0 tx=0x7fee8401c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c937f637ce1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840b2070 con 0x7fee8c0b2d80 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee840a5650 con 0x7fee8c0b2d80 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee8a000640 1 -- 172.21.15.164:0/3460846788 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c001680 con 0x7fee8c0b2d80 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 -- 172.21.15.164:0/3460846788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 msgr2=0x7fee8c168cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 --2- 172.21.15.164:0/3460846788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 0x7fee8c168cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee84009dc0 tx=0x7fee8401c7f0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 -- 172.21.15.164:0/3460846788 shutdown_connections 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 --2- 172.21.15.164:0/3460846788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c0b2d80 0x7fee8c168cd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 -- 172.21.15.164:0/3460846788 >> 172.21.15.164:0/3460846788 conn(0x7fee8c0b34f0 msgr2=0x7fee8c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 -- 172.21.15.164:0/3460846788 shutdown_connections 2024-05-08T04:41:08.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.281+0000 7fee91600640 1 -- 172.21.15.164:0/3460846788 wait complete. 2024-05-08T04:41:08.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 Processor -- start 2024-05-08T04:41:08.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- start start 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee8c0b2d80 0x7fee8c0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c15dd70 0x7fee8c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c15e150 con 0x7fee8c06c7b0 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0b4230 con 0x7fee8c0b2d80 2024-05-08T04:41:08.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c15e2d0 con 0x7fee8c15dd70 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57934/0 (socket says 172.21.15.164:57934) 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 -- 172.21.15.164:0/187693040 learned_addr learned my addr 172.21.15.164:0/187693040 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8b400640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c15dd70 0x7fee8c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee82000640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee8c0b2d80 0x7fee8c0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 -- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c15dd70 msgr2=0x7fee8c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c15dd70 0x7fee8c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 -- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee8c0b2d80 msgr2=0x7fee8c0af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee8c0b2d80 0x7fee8c0af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 -- 172.21.15.164:0/187693040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c0b6fb0 con 0x7fee8c06c7b0 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee8aa00640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c510 tx=0x7fee7c00c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c018070 con 0x7fee8c06c7b0 2024-05-08T04:41:08.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c007500 con 0x7fee8c06c7b0 2024-05-08T04:41:08.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c0b6710 con 0x7fee8c06c7b0 2024-05-08T04:41:08.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004a30 con 0x7fee8c06c7b0 2024-05-08T04:41:08.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.285+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8c0b6cd0 con 0x7fee8c06c7b0 2024-05-08T04:41:08.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8c069d60 con 0x7fee8c06c7b0 2024-05-08T04:41:08.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fee7c01b070 con 0x7fee8c06c7b0 2024-05-08T04:41:08.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee83400640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 0x7fee68069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fee7c084b40 con 0x7fee8c06c7b0 2024-05-08T04:41:08.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee82000640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 0x7fee68069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.289+0000 7fee82000640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 0x7fee68069810 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee74001ec0 tx=0x7fee7400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.293+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee7c0555a0 con 0x7fee8c06c7b0 2024-05-08T04:41:08.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.449+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee8c0b7940 con 0x7fee8c06c7b0 2024-05-08T04:41:08.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.449+0000 7fee83400640 1 -- 172.21.15.164:0/187693040 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16784 (secure 0 0 0) 0x7fee7c054ea0 con 0x7fee8c06c7b0 2024-05-08T04:41:08.454 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:08.454 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:07.483188+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.449+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 msgr2=0x7fee68069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.449+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 0x7fee68069810 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee74001ec0 tx=0x7fee7400b040 comp rx=0 tx=0).stop 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 msgr2=0x7fee8c0b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c510 tx=0x7fee7c00c9d0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 shutdown_connections 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee68067350 0x7fee68069810 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee8c15dd70 0x7fee8c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee8c0b2d80 0x7fee8c0af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 --2- 172.21.15.164:0/187693040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee8c06c7b0 0x7fee8c0b5c30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 >> 172.21.15.164:0/187693040 conn(0x7fee8c0b34f0 msgr2=0x7fee8c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 shutdown_connections 2024-05-08T04:41:08.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.453+0000 7fee91600640 1 -- 172.21.15.164:0/187693040 wait complete. 2024-05-08T04:41:08.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 Processor -- start 2024-05-08T04:41:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 -- start start 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f39a40cc5e0 0x7f39a40c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a40ccc40 con 0x7f39a415aab0 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a40ce170 con 0x7f39a40c5530 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a8a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a415ae90 con 0x7f39a40cc5e0 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f39a415aab0 0x7f39a40cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40022/0 (socket says 172.21.15.164:40022) 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3400640 1 -- 172.21.15.164:0/674833727 learned_addr learned my addr 172.21.15.164:0/674833727 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3e00640 1 --2- 172.21.15.164:0/674833727 >> v2:172.21.15.172:3300/0 conn(0x7f39a40cc5e0 0x7f39a40c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772666659 0 0) 0x7f39a40ccc40 con 0x7f39a415aab0 2024-05-08T04:41:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3988003530 con 0x7f39a415aab0 2024-05-08T04:41:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448611121 0 0) 0x7f39a40ce170 con 0x7f39a40c5530 2024-05-08T04:41:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a40ccc40 con 0x7f39a40c5530 2024-05-08T04:41:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3e00640 1 -- 172.21.15.164:0/674833727 >> v1:172.21.15.172:6789/0 conn(0x7f39a40c5530 legacy=0x7f39a40cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3e00640 1 -- 172.21.15.164:0/674833727 >> v1:172.21.15.164:6789/0 conn(0x7f39a415aab0 legacy=0x7f39a40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.609+0000 7f39a3e00640 1 -- 172.21.15.164:0/674833727 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40ce450 con 0x7f39a40cc5e0 2024-05-08T04:41:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a3e00640 1 --2- 172.21.15.164:0/674833727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40cc5e0 0x7f39a40c5990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097c0 tx=0x7f399c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402c6890c65973b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0b0070 con 0x7f39a40cc5e0 2024-05-08T04:41:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c0ab600 con 0x7f39a40cc5e0 2024-05-08T04:41:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a2000640 1 -- 172.21.15.164:0/674833727 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0ab9d0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 -- 172.21.15.164:0/674833727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40cc5e0 msgr2=0x7f39a40c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 --2- 172.21.15.164:0/674833727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40cc5e0 0x7f39a40c5990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097c0 tx=0x7f399c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 -- 172.21.15.164:0/674833727 shutdown_connections 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 --2- 172.21.15.164:0/674833727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40cc5e0 0x7f39a40c5990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 -- 172.21.15.164:0/674833727 >> 172.21.15.164:0/674833727 conn(0x7f39a406b670 msgr2=0x7f39a40b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 -- 172.21.15.164:0/674833727 shutdown_connections 2024-05-08T04:41:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.613+0000 7f39a8a00640 1 -- 172.21.15.164:0/674833727 wait complete. 2024-05-08T04:41:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 Processor -- start 2024-05-08T04:41:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- start start 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40c5530 0x7f39a406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39a415aab0 0x7f39a40c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40713f0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40724b0 con 0x7f39a40c5530 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4154bc0 con 0x7f39a415aab0 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57938/0 (socket says 172.21.15.164:57938) 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 -- 172.21.15.164:0/4165868059 learned_addr learned my addr 172.21.15.164:0/4165868059 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a3e00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39a415aab0 0x7f39a40c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a3400640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40c5530 0x7f39a406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 -- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39a415aab0 msgr2=0x7f39a40c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39a415aab0 0x7f39a40c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 -- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40c5530 msgr2=0x7f39a406dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40c5530 0x7f39a406dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 -- 172.21.15.164:0/4165868059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40c6dd0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a2a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f399000c500 tx=0x7f399000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39900052d0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990005490 con 0x7f39a40cc5e0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a40c70b0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990012850 con 0x7f39a40cc5e0 2024-05-08T04:41:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.617+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a40c7670 con 0x7f39a40cc5e0 2024-05-08T04:41:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a404ef30 con 0x7f39a40cc5e0 2024-05-08T04:41:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f39900129f0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f399b400640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 0x7f3984069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f39900853f0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f39a3400640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 0x7f3984069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.621+0000 7f39a3400640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 0x7f3984069810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3994001570 tx=0x7f3994001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.625+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3990055780 con 0x7f39a40cc5e0 2024-05-08T04:41:08.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.777+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a406e4a0 con 0x7f39a40cc5e0 2024-05-08T04:41:08.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f399b400640 1 -- 172.21.15.164:0/4165868059 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f3990059820 con 0x7f39a40cc5e0 2024-05-08T04:41:08.785 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:08.785 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:07.483188+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:08.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 msgr2=0x7f3984069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 0x7f3984069810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3994001570 tx=0x7f3994001a00 comp rx=0 tx=0).stop 2024-05-08T04:41:08.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 msgr2=0x7f39a4154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f399000c500 tx=0x7f399000c9c0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 shutdown_connections 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3984067350 0x7f3984069810 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39a415aab0 0x7f39a40c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39a40cc5e0 0x7f39a4154680 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 --2- 172.21.15.164:0/4165868059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a40c5530 0x7f39a406dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 >> 172.21.15.164:0/4165868059 conn(0x7f39a406b670 msgr2=0x7f39a40adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 shutdown_connections 2024-05-08T04:41:08.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.781+0000 7f39a8a00640 1 -- 172.21.15.164:0/4165868059 wait complete. 2024-05-08T04:41:08.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:41:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.937+0000 7fe490000640 1 Processor -- start 2024-05-08T04:41:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.937+0000 7fe490000640 1 -- start start 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe490000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe4880ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe490000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4880cecc0 0x7fe4880ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe490000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48815e870 con 0x7fe48815ab10 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe490000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880cf130 con 0x7fe4880cecc0 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe490000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48806bb80 con 0x7fe48806cce0 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4880cecc0 0x7fe4880ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4880cecc0 0x7fe4880ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57946/0 (socket says 172.21.15.164:57946) 2024-05-08T04:41:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 -- 172.21.15.164:0/3963623416 learned_addr learned my addr 172.21.15.164:0/3963623416 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 -- 172.21.15.164:0/3963623416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 msgr2=0x7fe4880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48cc00640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe4880ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe4880ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 -- 172.21.15.164:0/3963623416 >> v1:172.21.15.164:6789/0 conn(0x7fe48815ab10 legacy=0x7fe4880ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48e000640 1 -- 172.21.15.164:0/3963623416 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48815eb50 con 0x7fe4880cecc0 2024-05-08T04:41:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.941+0000 7fe48cc00640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe4880ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe48e000640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe4880ca6e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe480009870 tx=0x7fe48001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2d1cb724f407a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe487e00640 1 -- 172.21.15.164:0/3963623416 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800b1070 con 0x7fe4880cecc0 2024-05-08T04:41:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe487e00640 1 -- 172.21.15.164:0/3963623416 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800ac410 con 0x7fe4880cecc0 2024-05-08T04:41:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe487e00640 1 -- 172.21.15.164:0/3963623416 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800ac7e0 con 0x7fe4880cecc0 2024-05-08T04:41:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- 172.21.15.164:0/3963623416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 msgr2=0x7fe4880ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe4880ca6e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe480009870 tx=0x7fe48001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- 172.21.15.164:0/3963623416 shutdown_connections 2024-05-08T04:41:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe4880ca6e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 --2- 172.21.15.164:0/3963623416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe4880ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- 172.21.15.164:0/3963623416 >> 172.21.15.164:0/3963623416 conn(0x7fe4880cf770 msgr2=0x7fe4880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- 172.21.15.164:0/3963623416 shutdown_connections 2024-05-08T04:41:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- 172.21.15.164:0/3963623416 wait complete. 2024-05-08T04:41:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 Processor -- start 2024-05-08T04:41:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.945+0000 7fe490000640 1 -- start start 2024-05-08T04:41:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe48815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe48815ab10 0x7fe488142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880aedf0 con 0x7fe4880cecc0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815aef0 con 0x7fe48815ab10 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880aef70 con 0x7fe48806cce0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57950/0 (socket says 172.21.15.164:57950) 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 -- 172.21.15.164:0/2709298965 learned_addr learned my addr 172.21.15.164:0/2709298965 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48d600640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe48815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48e000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe48815ab10 0x7fe488142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 -- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 msgr2=0x7fe48815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe48815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 -- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe48815ab10 msgr2=0x7fe488142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe48815ab10 0x7fe488142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 -- 172.21.15.164:0/2709298965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488142b90 con 0x7fe4880cecc0 2024-05-08T04:41:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe48cc00640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe47400c7f0 tx=0x7fe47400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474015070 con 0x7fe4880cecc0 2024-05-08T04:41:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474007820 con 0x7fe4880cecc0 2024-05-08T04:41:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474007c10 con 0x7fe4880cecc0 2024-05-08T04:41:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe488142e70 con 0x7fe4880cecc0 2024-05-08T04:41:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488143430 con 0x7fe4880cecc0 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488072d30 con 0x7fe4880cecc0 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.949+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fe474018070 con 0x7fe4880cecc0 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.953+0000 7fe485600640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 0x7fe468069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.953+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe474085910 con 0x7fe4880cecc0 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.953+0000 7fe48d600640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 0x7fe468069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.953+0000 7fe48d600640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 0x7fe468069810 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe47c001e00 tx=0x7fe47c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:08.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:08.957+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe474055c10 con 0x7fe4880cecc0 2024-05-08T04:41:09.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:09.149+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe4880ce950 con 0x7fe4880cecc0 2024-05-08T04:41:10.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.124+0000 7fe485600640 1 -- 172.21.15.164:0/2709298965 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe474059cb0 con 0x7fe4880cecc0 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 msgr2=0x7fe468069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 0x7fe468069810 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe47c001e00 tx=0x7fe47c009040 comp rx=0 tx=0).stop 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 msgr2=0x7fe48815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe47400c7f0 tx=0x7fe47400ccb0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 shutdown_connections 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe468067350 0x7fe468069810 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe48815ab10 0x7fe488142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4880cecc0 0x7fe48815da20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 --2- 172.21.15.164:0/2709298965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe48806cce0 0x7fe48815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 >> 172.21.15.164:0/2709298965 conn(0x7fe4880cf770 msgr2=0x7fe4880ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 shutdown_connections 2024-05-08T04:41:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.128+0000 7fe490000640 1 -- 172.21.15.164:0/2709298965 wait complete. 2024-05-08T04:41:10.156 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:41:10.156 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:41:10.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 Processor -- start 2024-05-08T04:41:10.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 -- start start 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4500af6d0 0x7fd4500b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500b4dc0 0x7fd4500ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45015e300 con 0x7fd4500addd0 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45006bca0 con 0x7fd4500af6d0 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd457000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45015e570 con 0x7fd4500b4dc0 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4500af6d0 0x7fd4500b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4500af6d0 0x7fd4500b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41704/0 (socket says 172.21.15.164:41704) 2024-05-08T04:41:10.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 -- 172.21.15.164:0/1618900506 learned_addr learned my addr 172.21.15.164:0/1618900506 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:10.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 -- 172.21.15.164:0/1618900506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500b4dc0 msgr2=0x7fd4500ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:10.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 --2- 172.21.15.164:0/1618900506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500b4dc0 0x7fd4500ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 -- 172.21.15.164:0/1618900506 >> v1:172.21.15.172:6789/0 conn(0x7fd4500addd0 legacy=0x7fd4501617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:10.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.292+0000 7fd44fe00640 1 -- 172.21.15.164:0/1618900506 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45015e850 con 0x7fd4500af6d0 2024-05-08T04:41:10.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd44fe00640 1 --2- 172.21.15.164:0/1618900506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500af6d0 0x7fd4500b2ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e50 tx=0x7fd44001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c8d6a55fe88576 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:10.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd44f400640 1 -- 172.21.15.164:0/1618900506 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400a6510 con 0x7fd4500af6d0 2024-05-08T04:41:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd44f400640 1 -- 172.21.15.164:0/1618900506 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400a66b0 con 0x7fd4500af6d0 2024-05-08T04:41:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd44f400640 1 -- 172.21.15.164:0/1618900506 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400a6a80 con 0x7fd4500af6d0 2024-05-08T04:41:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- 172.21.15.164:0/1618900506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500af6d0 msgr2=0x7fd4500b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 --2- 172.21.15.164:0/1618900506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500af6d0 0x7fd4500b2ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e50 tx=0x7fd44001c960 comp rx=0 tx=0).stop 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- 172.21.15.164:0/1618900506 shutdown_connections 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 --2- 172.21.15.164:0/1618900506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500b4dc0 0x7fd4500ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 --2- 172.21.15.164:0/1618900506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500af6d0 0x7fd4500b2ff0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- 172.21.15.164:0/1618900506 >> 172.21.15.164:0/1618900506 conn(0x7fd45006cca0 msgr2=0x7fd45006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- 172.21.15.164:0/1618900506 shutdown_connections 2024-05-08T04:41:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- 172.21.15.164:0/1618900506 wait complete. 2024-05-08T04:41:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 Processor -- start 2024-05-08T04:41:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.296+0000 7fd457000640 1 -- start start 2024-05-08T04:41:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500addd0 0x7fd450150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 0x7fd450152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016aad0 con 0x7fd4500b4dc0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016a950 con 0x7fd4500af6d0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45016ac50 con 0x7fd4500addd0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41710/0 (socket says 172.21.15.164:41710) 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 -- 172.21.15.164:0/4089239203 learned_addr learned my addr 172.21.15.164:0/4089239203 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd447400640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 0x7fd450152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd44fe00640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500addd0 0x7fd450150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 -- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500addd0 msgr2=0x7fd450150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500addd0 0x7fd450150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 -- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 msgr2=0x7fd450152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 0x7fd450152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 -- 172.21.15.164:0/4089239203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450071060 con 0x7fd4500b4dc0 2024-05-08T04:41:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd447400640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 0x7fd450152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd455000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c410 tx=0x7fd44800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448017070 con 0x7fd4500b4dc0 2024-05-08T04:41:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448003600 con 0x7fd4500b4dc0 2024-05-08T04:41:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd450071340 con 0x7fd4500b4dc0 2024-05-08T04:41:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448003a30 con 0x7fd4500b4dc0 2024-05-08T04:41:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd450071900 con 0x7fd4500b4dc0 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45006ef70 con 0x7fd4500b4dc0 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.300+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd44801a070 con 0x7fd4500b4dc0 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.304+0000 7fd44cc00640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 0x7fd42c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.304+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd448084ac0 con 0x7fd4500b4dc0 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.304+0000 7fd44fe00640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 0x7fd42c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.304+0000 7fd44fe00640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 0x7fd42c069810 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4400ae000 tx=0x7fd4400b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:10.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.308+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd448055520 con 0x7fd4500b4dc0 2024-05-08T04:41:10.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:10.500+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd45006c5b0 con 0x7fd4500b4dc0 2024-05-08T04:41:11.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.132+0000 7fd44cc00640 1 -- 172.21.15.164:0/4089239203 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd448054e20 con 0x7fd4500b4dc0 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 msgr2=0x7fd42c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 0x7fd42c069810 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4400ae000 tx=0x7fd4400b1040 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 msgr2=0x7fd4501481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c410 tx=0x7fd44800c8d0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 shutdown_connections 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd42c067350 0x7fd42c069810 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4500b4dc0 0x7fd4501481a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4500af6d0 0x7fd450152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 --2- 172.21.15.164:0/4089239203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4500addd0 0x7fd450150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 >> 172.21.15.164:0/4089239203 conn(0x7fd45006cca0 msgr2=0x7fd45015ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:11.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 shutdown_connections 2024-05-08T04:41:11.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.136+0000 7fd457000640 1 -- 172.21.15.164:0/4089239203 wait complete. 2024-05-08T04:41:11.161 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:41:11.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:11.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 Processor -- start 2024-05-08T04:41:11.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 -- start start 2024-05-08T04:41:11.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 0x7f5518072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55180b26c0 0x7f55180cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55180cfbd0 0x7f55180cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180d0060 con 0x7f55180cfbd0 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5518072640 con 0x7f551815ab60 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.292+0000 7f551e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551806c770 con 0x7f55180b26c0 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55180cfbd0 0x7f55180cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55180cfbd0 0x7f55180cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41726/0 (socket says 172.21.15.164:41726) 2024-05-08T04:41:11.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 -- 172.21.15.164:0/650026491 learned_addr learned my addr 172.21.15.164:0/650026491 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 -- 172.21.15.164:0/650026491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55180b26c0 msgr2=0x7f55180cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 --2- 172.21.15.164:0/650026491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55180b26c0 0x7f55180cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517400640 1 --2- 172.21.15.164:0/650026491 >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 0x7f5518072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 -- 172.21.15.164:0/650026491 >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 msgr2=0x7f5518072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 --2- 172.21.15.164:0/650026491 >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 0x7f5518072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 -- 172.21.15.164:0/650026491 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551806ca50 con 0x7f55180cfbd0 2024-05-08T04:41:11.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517400640 1 --2- 172.21.15.164:0/650026491 >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 0x7f5518072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:11.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5517e00640 1 --2- 172.21.15.164:0/650026491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180cfbd0 0x7f55180cd910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f55180cd420 tx=0x7f551001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a683cc8113c1d848 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5516a00640 1 -- 172.21.15.164:0/650026491 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100a5670 con 0x7f55180cfbd0 2024-05-08T04:41:11.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5516a00640 1 -- 172.21.15.164:0/650026491 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55100a5810 con 0x7f55180cfbd0 2024-05-08T04:41:11.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.296+0000 7f5516a00640 1 -- 172.21.15.164:0/650026491 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100a5be0 con 0x7f55180cfbd0 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- 172.21.15.164:0/650026491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180cfbd0 msgr2=0x7f55180cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- 172.21.15.164:0/650026491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180cfbd0 0x7f55180cd910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f55180cd420 tx=0x7f551001c9f0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- 172.21.15.164:0/650026491 shutdown_connections 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- 172.21.15.164:0/650026491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180cfbd0 0x7f55180cd910 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- 172.21.15.164:0/650026491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55180b26c0 0x7f55180cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- 172.21.15.164:0/650026491 >> v2:172.21.15.172:3300/0 conn(0x7f551815ab60 0x7f5518072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- 172.21.15.164:0/650026491 >> 172.21.15.164:0/650026491 conn(0x7f551806ac90 msgr2=0x7f551806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:11.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- 172.21.15.164:0/650026491 shutdown_connections 2024-05-08T04:41:11.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- 172.21.15.164:0/650026491 wait complete. 2024-05-08T04:41:11.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 Processor -- start 2024-05-08T04:41:11.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- start start 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 0x7f5518158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f551815ab60 0x7f55181501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551806c550 con 0x7f55180b26c0 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551806c3d0 con 0x7f55180cfbd0 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f551e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180aed30 con 0x7f551815ab60 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41728/0 (socket says 172.21.15.164:41728) 2024-05-08T04:41:11.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 -- 172.21.15.164:0/1409074025 learned_addr learned my addr 172.21.15.164:0/1409074025 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517e00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f551815ab60 0x7f55181501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f550ea00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 0x7f5518158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 -- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f551815ab60 msgr2=0x7f55181501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f551815ab60 0x7f55181501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 -- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 msgr2=0x7f5518158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 0x7f5518158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 -- 172.21.15.164:0/1409074025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518159930 con 0x7f55180b26c0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f550ea00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 0x7f5518158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.300+0000 7f5517400640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f550800c7f0 tx=0x7f550800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508015070 con 0x7f55180b26c0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508007820 con 0x7f55180b26c0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508007c10 con 0x7f55180b26c0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5518159c10 con 0x7f55180b26c0 2024-05-08T04:41:11.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551815a1d0 con 0x7f55180b26c0 2024-05-08T04:41:11.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518069d60 con 0x7f55180b26c0 2024-05-08T04:41:11.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f5508018070 con 0x7f55180b26c0 2024-05-08T04:41:11.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 0x7f54f4069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5508085900 con 0x7f55180b26c0 2024-05-08T04:41:11.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.304+0000 7f550ea00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 0x7f54f4069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.308+0000 7f550ea00640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 0x7f54f4069810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5518155270 tx=0x7f5500008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.308+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5508055bb0 con 0x7f55180b26c0 2024-05-08T04:41:11.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f551806d9a0 con 0x7f55180b26c0 2024-05-08T04:41:11.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f550fe00640 1 -- 172.21.15.164:0/1409074025 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5508059c50 con 0x7f55180b26c0 2024-05-08T04:41:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:11.506 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:11.506 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 43 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 msgr2=0x7f54f4069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 0x7f54f4069810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5518155270 tx=0x7f5500008040 comp rx=0 tx=0).stop 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 msgr2=0x7f5518147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f550800c7f0 tx=0x7f550800ccb0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 shutdown_connections 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f54f4067350 0x7f54f4069810 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f551815ab60 0x7f55181501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55180cfbd0 0x7f5518158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 --2- 172.21.15.164:0/1409074025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55180b26c0 0x7f5518147810 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 >> 172.21.15.164:0/1409074025 conn(0x7f551806ac90 msgr2=0x7f5518070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:11.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 shutdown_connections 2024-05-08T04:41:11.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.500+0000 7f551e800640 1 -- 172.21.15.164:0/1409074025 wait complete. 2024-05-08T04:41:11.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:11.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 Processor -- start 2024-05-08T04:41:11.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 -- start start 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6300cfda0 0x7fa630160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa63015be50 0x7fa6300b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6300cd9f0 con 0x7fa63015ba70 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63015fe70 con 0x7fa6300cfda0 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa637800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6300cdcd0 con 0x7fa63015be50 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa62fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6300cfda0 0x7fa630160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa62fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6300cfda0 0x7fa630160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41740/0 (socket says 172.21.15.164:41740) 2024-05-08T04:41:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa62fe00640 1 -- 172.21.15.164:0/2803925726 learned_addr learned my addr 172.21.15.164:0/2803925726 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.656+0000 7fa635800640 1 --2- 172.21.15.164:0/2803925726 >> v2:172.21.15.172:3300/0 conn(0x7fa63015be50 0x7fa6300b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62fe00640 1 -- 172.21.15.164:0/2803925726 >> v2:172.21.15.172:3300/0 conn(0x7fa63015be50 msgr2=0x7fa6300b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62fe00640 1 --2- 172.21.15.164:0/2803925726 >> v2:172.21.15.172:3300/0 conn(0x7fa63015be50 0x7fa6300b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62fe00640 1 -- 172.21.15.164:0/2803925726 >> v1:172.21.15.172:6789/0 conn(0x7fa63015ba70 legacy=0x7fa6300cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62fe00640 1 -- 172.21.15.164:0/2803925726 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63015f820 con 0x7fa6300cfda0 2024-05-08T04:41:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62fe00640 1 --2- 172.21.15.164:0/2803925726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6300cfda0 0x7fa630160300 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa63015fde0 tx=0x7fa61c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baad4eeda57f6779 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62f400640 1 -- 172.21.15.164:0/2803925726 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0ab600 con 0x7fa6300cfda0 2024-05-08T04:41:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62f400640 1 -- 172.21.15.164:0/2803925726 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c0b0070 con 0x7fa6300cfda0 2024-05-08T04:41:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa62f400640 1 -- 172.21.15.164:0/2803925726 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0ab9c0 con 0x7fa6300cfda0 2024-05-08T04:41:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 -- 172.21.15.164:0/2803925726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6300cfda0 msgr2=0x7fa630160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 --2- 172.21.15.164:0/2803925726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6300cfda0 0x7fa630160300 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa63015fde0 tx=0x7fa61c01c4f0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 -- 172.21.15.164:0/2803925726 shutdown_connections 2024-05-08T04:41:11.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 --2- 172.21.15.164:0/2803925726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6300cfda0 0x7fa630160300 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 --2- 172.21.15.164:0/2803925726 >> v2:172.21.15.172:3300/0 conn(0x7fa63015be50 0x7fa6300b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 -- 172.21.15.164:0/2803925726 >> 172.21.15.164:0/2803925726 conn(0x7fa63006ac90 msgr2=0x7fa63006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:11.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 -- 172.21.15.164:0/2803925726 shutdown_connections 2024-05-08T04:41:11.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.660+0000 7fa637800640 1 -- 172.21.15.164:0/2803925726 wait complete. 2024-05-08T04:41:11.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 Processor -- start 2024-05-08T04:41:11.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 -- start start 2024-05-08T04:41:11.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 0x7fa6300b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 0x7fa630154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168d40 con 0x7fa63015be50 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168bc0 con 0x7fa6300cfda0 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168ec0 con 0x7fa63015ba70 2024-05-08T04:41:11.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa634e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 0x7fa6300b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41746/0 (socket says 172.21.15.164:41746) 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 -- 172.21.15.164:0/3302633559 learned_addr learned my addr 172.21.15.164:0/3302633559 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa634e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 0x7fa6300b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:43406/0 (socket says 172.21.15.164:43406) 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa62fe00640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 0x7fa630154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 -- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 msgr2=0x7fa630154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 0x7fa630154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 -- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 msgr2=0x7fa6300b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 0x7fa6300b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 -- 172.21.15.164:0/3302633559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630071a00 con 0x7fa63015be50 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa62fe00640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 0x7fa630154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa635800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d560 tx=0x7fa62800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280125f0 con 0x7fa63015be50 2024-05-08T04:41:11.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa628019070 con 0x7fa63015be50 2024-05-08T04:41:11.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa630070960 con 0x7fa63015be50 2024-05-08T04:41:11.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.664+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280129f0 con 0x7fa63015be50 2024-05-08T04:41:11.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa630070f20 con 0x7fa63015be50 2024-05-08T04:41:11.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa630067820 con 0x7fa63015be50 2024-05-08T04:41:11.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fa628012b90 con 0x7fa63015be50 2024-05-08T04:41:11.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa62cc00640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 0x7fa610069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa628085ad0 con 0x7fa63015be50 2024-05-08T04:41:11.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa634e00640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 0x7fa610069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.668+0000 7fa634e00640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 0x7fa610069840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa620001ea0 tx=0x7fa620009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:11.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.672+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa628087e20 con 0x7fa63015be50 2024-05-08T04:41:11.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.832+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa630069d60 con 0x7fa63015be50 2024-05-08T04:41:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.832+0000 7fa62cc00640 1 -- 172.21.15.164:0/3302633559 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16771 (secure 0 0 0) 0x7fa6280564a0 con 0x7fa63015be50 2024-05-08T04:41:11.838 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:11.838 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":74,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:09.101885+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 msgr2=0x7fa610069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 0x7fa610069840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa620001ea0 tx=0x7fa620009040 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 msgr2=0x7fa630154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d560 tx=0x7fa62800da20 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 shutdown_connections 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa610067380 0x7fa610069840 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015ba70 0x7fa630154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa63015be50 0x7fa630154d70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 --2- 172.21.15.164:0/3302633559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300cfda0 0x7fa6300b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 >> 172.21.15.164:0/3302633559 conn(0x7fa63006ac90 msgr2=0x7fa63006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 shutdown_connections 2024-05-08T04:41:11.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.836+0000 7fa637800640 1 -- 172.21.15.164:0/3302633559 wait complete. 2024-05-08T04:41:11.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:41:11.859 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:11.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.988+0000 7f349aa00640 1 Processor -- start 2024-05-08T04:41:11.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.988+0000 7f349aa00640 1 -- start start 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 0x7f34940ad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34940ad850 0x7f34940cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34940cff10 0x7f34940b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940715c0 con 0x7f34940ad850 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349415e9d0 con 0x7f349415acf0 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f349aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940b3fd0 con 0x7f34940cff10 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34940ad850 0x7f34940cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34940ad850 0x7f34940cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41758/0 (socket says 172.21.15.164:41758) 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 -- 172.21.15.164:0/2171589230 learned_addr learned my addr 172.21.15.164:0/2171589230 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493e00640 1 --2- 172.21.15.164:0/2171589230 >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 0x7f34940ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3498a00640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34940cff10 0x7f34940b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 -- 172.21.15.164:0/2171589230 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34940cff10 msgr2=0x7f34940b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34940cff10 0x7f34940b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 -- 172.21.15.164:0/2171589230 >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 msgr2=0x7f34940ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 --2- 172.21.15.164:0/2171589230 >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 0x7f34940ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493400640 1 -- 172.21.15.164:0/2171589230 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494071830 con 0x7f34940ad850 2024-05-08T04:41:11.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.992+0000 7f3493e00640 1 --2- 172.21.15.164:0/2171589230 >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 0x7f34940ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:12.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f3493400640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cf9d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34940d0d60 tx=0x7f348001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02101b64d70b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f3492a00640 1 -- 172.21.15.164:0/2171589230 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800b0070 con 0x7f34940ad850 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f3492a00640 1 -- 172.21.15.164:0/2171589230 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34800ab600 con 0x7f34940ad850 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f3492a00640 1 -- 172.21.15.164:0/2171589230 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800a4e60 con 0x7f34940ad850 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- 172.21.15.164:0/2171589230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 msgr2=0x7f34940cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cf9d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34940d0d60 tx=0x7f348001c520 comp rx=0 tx=0).stop 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- 172.21.15.164:0/2171589230 shutdown_connections 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34940cff10 0x7f34940b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 --2- 172.21.15.164:0/2171589230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cf9d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 --2- 172.21.15.164:0/2171589230 >> v2:172.21.15.172:3300/0 conn(0x7f349415acf0 0x7f34940ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- 172.21.15.164:0/2171589230 >> 172.21.15.164:0/2171589230 conn(0x7f349406ae60 msgr2=0x7f349406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- 172.21.15.164:0/2171589230 shutdown_connections 2024-05-08T04:41:12.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- 172.21.15.164:0/2171589230 wait complete. 2024-05-08T04:41:12.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 Processor -- start 2024-05-08T04:41:12.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:11.996+0000 7f349aa00640 1 -- start start 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34940cff10 0x7f349414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f349415acf0 0x7f34941525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494167bf0 con 0x7f34940ad850 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494072a50 con 0x7f34940cff10 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3494167d70 con 0x7f349415acf0 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41770/0 (socket says 172.21.15.164:41770) 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 -- 172.21.15.164:0/495939031 learned_addr learned my addr 172.21.15.164:0/495939031 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493400640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34940cff10 0x7f349414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3498a00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f349415acf0 0x7f34941525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 -- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f349415acf0 msgr2=0x7f34941525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f349415acf0 0x7f34941525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 -- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34940cff10 msgr2=0x7f349414fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34940cff10 0x7f349414fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 -- 172.21.15.164:0/495939031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940cebf0 con 0x7f34940ad850 2024-05-08T04:41:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f3493e00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3484009f60 tx=0x7f3484009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484018020 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484014680 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484014a50 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3494156320 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3494150b30 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494069d60 con 0x7f34940ad850 2024-05-08T04:41:12.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.000+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f348401b020 con 0x7f34940ad850 2024-05-08T04:41:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.004+0000 7f348be00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 0x7f3474069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.004+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f34840864f0 con 0x7f34940ad850 2024-05-08T04:41:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.004+0000 7f3493400640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 0x7f3474069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.004+0000 7f3493400640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 0x7f3474069810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34800ad000 tx=0x7f3480008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.008+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3484022030 con 0x7f34940ad850 2024-05-08T04:41:12.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.160+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3494071040 con 0x7f3474067350 2024-05-08T04:41:12.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.168+0000 7f348be00640 1 -- 172.21.15.164:0/495939031 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246750 (secure 0 0 0) 0x7f3494071040 con 0x7f3474067350 2024-05-08T04:41:12.174 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:12.176 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.168+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 msgr2=0x7f3474069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.168+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 0x7f3474069810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34800ad000 tx=0x7f3480008040 comp rx=0 tx=0).stop 2024-05-08T04:41:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.168+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 msgr2=0x7f34940cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.168+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3484009f60 tx=0x7f3484009f90 comp rx=0 tx=0).stop 2024-05-08T04:41:12.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 shutdown_connections 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3474067350 0x7f3474069810 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f349415acf0 0x7f34941525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34940cff10 0x7f349414fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 --2- 172.21.15.164:0/495939031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34940ad850 0x7f34940cc670 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 >> 172.21.15.164:0/495939031 conn(0x7f349406ae60 msgr2=0x7f34940aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 shutdown_connections 2024-05-08T04:41:12.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.172+0000 7f349aa00640 1 -- 172.21.15.164:0/495939031 wait complete. 2024-05-08T04:41:12.195 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-05-08T04:41:11.462834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244972,"kb_used_data":7984,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729748,"statfs":{"total":773094113280,"available":772843261952,"internally_reserved":0,"allocated":8175616,"data_stored":4133594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.369323"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110592+0000","last_change":"2024-05-08T04:41:07.110592+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110592+0000","last_undegraded":"2024-05-08T04:41:07.110592+0000","last_fullsized":"2024-05-08T04:41:07.110592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.116717+0000","last_change":"2024-05-08T04:41:07.116717+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.116717+0000","last_undegraded":"2024-05-08T04:41:07.116717+0000","last_fullsized":"2024-05-08T04:41:07.116717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111388+0000","last_change":"2024-05-08T04:41:07.111388+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111388+0000","last_undegraded":"2024-05-08T04:41:07.111388+0000","last_fullsized":"2024-05-08T04:41:07.111388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105294+0000","last_change":"2024-05-08T04:41:07.105294+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105294+0000","last_undegraded":"2024-05-08T04:41:07.105294+0000","last_fullsized":"2024-05-08T04:41:07.105294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108963+0000","last_change":"2024-05-08T04:41:07.108963+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108963+0000","last_undegraded":"2024-05-08T04:41:07.108963+0000","last_fullsized":"2024-05-08T04:41:07.108963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115911+0000","last_change":"2024-05-08T04:41:07.115911+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115911+0000","last_undegraded":"2024-05-08T04:41:07.115911+0000","last_fullsized":"2024-05-08T04:41:07.115911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104906+0000","last_change":"2024-05-08T04:41:07.104906+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104906+0000","last_undegraded":"2024-05-08T04:41:07.104906+0000","last_fullsized":"2024-05-08T04:41:07.104906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115370+0000","last_change":"2024-05-08T04:41:07.115370+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115370+0000","last_undegraded":"2024-05-08T04:41:07.115370+0000","last_fullsized":"2024-05-08T04:41:07.115370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111074+0000","last_change":"2024-05-08T04:41:07.111074+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111074+0000","last_undegraded":"2024-05-08T04:41:07.111074+0000","last_fullsized":"2024-05-08T04:41:07.111074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107667+0000","last_change":"2024-05-08T04:41:07.107667+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107667+0000","last_undegraded":"2024-05-08T04:41:07.107667+0000","last_fullsized":"2024-05-08T04:41:07.107667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111665+0000","last_change":"2024-05-08T04:41:07.111665+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111665+0000","last_undegraded":"2024-05-08T04:41:07.111665+0000","last_fullsized":"2024-05-08T04:41:07.111665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104932+0000","last_change":"2024-05-08T04:41:07.104932+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104932+0000","last_undegraded":"2024-05-08T04:41:07.104932+0000","last_fullsized":"2024-05-08T04:41:07.104932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106039+0000","last_change":"2024-05-08T04:41:07.106039+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106039+0000","last_undegraded":"2024-05-08T04:41:07.106039+0000","last_fullsized":"2024-05-08T04:41:07.106039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107095+0000","last_change":"2024-05-08T04:41:07.107095+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107095+0000","last_undegraded":"2024-05-08T04:41:07.107095+0000","last_fullsized":"2024-05-08T04:41:07.107095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106245+0000","last_change":"2024-05-08T04:41:07.106245+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106245+0000","last_undegraded":"2024-05-08T04:41:07.106245+0000","last_fullsized":"2024-05-08T04:41:07.106245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106478+0000","last_change":"2024-05-08T04:41:07.106478+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106478+0000","last_undegraded":"2024-05-08T04:41:07.106478+0000","last_fullsized":"2024-05-08T04:41:07.106478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110595+0000","last_change":"2024-05-08T04:41:07.110595+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110595+0000","last_undegraded":"2024-05-08T04:41:07.110595+0000","last_fullsized":"2024-05-08T04:41:07.110595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104441+0000","last_change":"2024-05-08T04:41:07.104441+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104441+0000","last_undegraded":"2024-05-08T04:41:07.104441+0000","last_fullsized":"2024-05-08T04:41:07.104441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114231+0000","last_change":"2024-05-08T04:41:07.114231+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114231+0000","last_undegraded":"2024-05-08T04:41:07.114231+0000","last_fullsized":"2024-05-08T04:41:07.114231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105925+0000","last_change":"2024-05-08T04:41:07.105925+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105925+0000","last_undegraded":"2024-05-08T04:41:07.105925+0000","last_fullsized":"2024-05-08T04:41:07.105925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105897+0000","last_change":"2024-05-08T04:41:07.105897+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105897+0000","last_undegraded":"2024-05-08T04:41:07.105897+0000","last_fullsized":"2024-05-08T04:41:07.105897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106157+0000","last_change":"2024-05-08T04:41:07.106157+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106157+0000","last_undegraded":"2024-05-08T04:41:07.106157+0000","last_fullsized":"2024-05-08T04:41:07.106157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112836+0000","last_change":"2024-05-08T04:41:07.112836+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112836+0000","last_undegraded":"2024-05-08T04:41:07.112836+0000","last_fullsized":"2024-05-08T04:41:07.112836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107394+0000","last_change":"2024-05-08T04:41:07.107394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107394+0000","last_undegraded":"2024-05-08T04:41:07.107394+0000","last_fullsized":"2024-05-08T04:41:07.107394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106637+0000","last_change":"2024-05-08T04:41:07.106637+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106637+0000","last_undegraded":"2024-05-08T04:41:07.106637+0000","last_fullsized":"2024-05-08T04:41:07.106637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104164+0000","last_change":"2024-05-08T04:41:07.104164+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104164+0000","last_undegraded":"2024-05-08T04:41:07.104164+0000","last_fullsized":"2024-05-08T04:41:07.104164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108566+0000","last_change":"2024-05-08T04:41:07.108566+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108566+0000","last_undegraded":"2024-05-08T04:41:07.108566+0000","last_fullsized":"2024-05-08T04:41:07.108566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108233+0000","last_change":"2024-05-08T04:41:07.108233+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108233+0000","last_undegraded":"2024-05-08T04:41:07.108233+0000","last_fullsized":"2024-05-08T04:41:07.108233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.113288+0000","last_change":"2024-05-08T04:41:07.113288+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.113288+0000","last_undegraded":"2024-05-08T04:41:07.113288+0000","last_fullsized":"2024-05-08T04:41:07.113288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104159+0000","last_change":"2024-05-08T04:41:07.104159+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104159+0000","last_undegraded":"2024-05-08T04:41:07.104159+0000","last_fullsized":"2024-05-08T04:41:07.104159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103467+0000","last_change":"2024-05-08T04:41:07.103467+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103467+0000","last_undegraded":"2024-05-08T04:41:07.103467+0000","last_fullsized":"2024-05-08T04:41:07.103467+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114887+0000","last_change":"2024-05-08T04:41:07.114887+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114887+0000","last_undegraded":"2024-05-08T04:41:07.114887+0000","last_fullsized":"2024-05-08T04:41:07.114887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109849+0000","last_change":"2024-05-08T04:41:07.109849+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109849+0000","last_undegraded":"2024-05-08T04:41:07.109849+0000","last_fullsized":"2024-05-08T04:41:07.109849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.129995+0000","last_change":"2024-05-08T04:41:07.123950+0000","last_active":"2024-05-08T04:41:07.129995+0000","last_peered":"2024-05-08T04:41:07.129995+0000","last_clean":"2024-05-08T04:41:07.129995+0000","last_became_active":"2024-05-08T04:41:07.123577+0000","last_became_peered":"2024-05-08T04:41:07.123577+0000","last_unstale":"2024-05-08T04:41:07.129995+0000","last_undegraded":"2024-05-08T04:41:07.129995+0000","last_fullsized":"2024-05-08T04:41:07.129995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:28.418132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110185+0000","last_change":"2024-05-08T04:41:07.110185+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110185+0000","last_undegraded":"2024-05-08T04:41:07.110185+0000","last_fullsized":"2024-05-08T04:41:07.110185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125062+0000","last_change":"2024-05-08T04:41:07.122634+0000","last_active":"2024-05-08T04:41:07.125062+0000","last_peered":"2024-05-08T04:41:07.125062+0000","last_clean":"2024-05-08T04:41:07.125062+0000","last_became_active":"2024-05-08T04:41:07.122307+0000","last_became_peered":"2024-05-08T04:41:07.122307+0000","last_unstale":"2024-05-08T04:41:07.125062+0000","last_undegraded":"2024-05-08T04:41:07.125062+0000","last_fullsized":"2024-05-08T04:41:07.125062+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:18.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110634+0000","last_change":"2024-05-08T04:41:07.110634+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110634+0000","last_undegraded":"2024-05-08T04:41:07.110634+0000","last_fullsized":"2024-05-08T04:41:07.110634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128519+0000","last_change":"2024-05-08T04:41:07.124366+0000","last_active":"2024-05-08T04:41:07.128519+0000","last_peered":"2024-05-08T04:41:07.128519+0000","last_clean":"2024-05-08T04:41:07.128519+0000","last_became_active":"2024-05-08T04:41:07.124039+0000","last_became_peered":"2024-05-08T04:41:07.124039+0000","last_unstale":"2024-05-08T04:41:07.128519+0000","last_undegraded":"2024-05-08T04:41:07.128519+0000","last_fullsized":"2024-05-08T04:41:07.128519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:24.785925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111227+0000","last_change":"2024-05-08T04:41:07.111227+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111227+0000","last_undegraded":"2024-05-08T04:41:07.111227+0000","last_fullsized":"2024-05-08T04:41:07.111227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128296+0000","last_change":"2024-05-08T04:41:07.126964+0000","last_active":"2024-05-08T04:41:07.128296+0000","last_peered":"2024-05-08T04:41:07.128296+0000","last_clean":"2024-05-08T04:41:07.128296+0000","last_became_active":"2024-05-08T04:41:07.126433+0000","last_became_peered":"2024-05-08T04:41:07.126433+0000","last_unstale":"2024-05-08T04:41:07.128296+0000","last_undegraded":"2024-05-08T04:41:07.128296+0000","last_fullsized":"2024-05-08T04:41:07.128296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103412+0000","last_change":"2024-05-08T04:41:07.103412+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103412+0000","last_undegraded":"2024-05-08T04:41:07.103412+0000","last_fullsized":"2024-05-08T04:41:07.103412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122741+0000","last_change":"2024-05-08T04:41:07.118893+0000","last_active":"2024-05-08T04:41:07.122741+0000","last_peered":"2024-05-08T04:41:07.122741+0000","last_clean":"2024-05-08T04:41:07.122741+0000","last_became_active":"2024-05-08T04:41:07.117180+0000","last_became_peered":"2024-05-08T04:41:07.117180+0000","last_unstale":"2024-05-08T04:41:07.122741+0000","last_undegraded":"2024-05-08T04:41:07.122741+0000","last_fullsized":"2024-05-08T04:41:07.122741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:45.295908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105256+0000","last_change":"2024-05-08T04:41:07.105256+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105256+0000","last_undegraded":"2024-05-08T04:41:07.105256+0000","last_fullsized":"2024-05-08T04:41:07.105256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124504+0000","last_change":"2024-05-08T04:41:07.122025+0000","last_active":"2024-05-08T04:41:07.124504+0000","last_peered":"2024-05-08T04:41:07.124504+0000","last_clean":"2024-05-08T04:41:07.124504+0000","last_became_active":"2024-05-08T04:41:07.120512+0000","last_became_peered":"2024-05-08T04:41:07.120512+0000","last_unstale":"2024-05-08T04:41:07.124504+0000","last_undegraded":"2024-05-08T04:41:07.124504+0000","last_fullsized":"2024-05-08T04:41:07.124504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:27.054889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110394+0000","last_change":"2024-05-08T04:41:07.110394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110394+0000","last_undegraded":"2024-05-08T04:41:07.110394+0000","last_fullsized":"2024-05-08T04:41:07.110394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121508+0000","last_change":"2024-05-08T04:41:07.115029+0000","last_active":"2024-05-08T04:41:07.121508+0000","last_peered":"2024-05-08T04:41:07.121508+0000","last_clean":"2024-05-08T04:41:07.121508+0000","last_became_active":"2024-05-08T04:41:07.114753+0000","last_became_peered":"2024-05-08T04:41:07.114753+0000","last_unstale":"2024-05-08T04:41:07.121508+0000","last_undegraded":"2024-05-08T04:41:07.121508+0000","last_fullsized":"2024-05-08T04:41:07.121508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105796+0000","last_change":"2024-05-08T04:41:07.105796+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105796+0000","last_undegraded":"2024-05-08T04:41:07.105796+0000","last_fullsized":"2024-05-08T04:41:07.105796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116667+0000","last_change":"2024-05-08T04:41:07.113389+0000","last_active":"2024-05-08T04:41:07.116667+0000","last_peered":"2024-05-08T04:41:07.116667+0000","last_clean":"2024-05-08T04:41:07.116667+0000","last_became_active":"2024-05-08T04:41:07.113100+0000","last_became_peered":"2024-05-08T04:41:07.113100+0000","last_unstale":"2024-05-08T04:41:07.116667+0000","last_undegraded":"2024-05-08T04:41:07.116667+0000","last_fullsized":"2024-05-08T04:41:07.116667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:13.388255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103521+0000","last_change":"2024-05-08T04:41:07.103521+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103521+0000","last_undegraded":"2024-05-08T04:41:07.103521+0000","last_fullsized":"2024-05-08T04:41:07.103521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116039+0000","last_change":"2024-05-08T04:41:07.113877+0000","last_active":"2024-05-08T04:41:07.116039+0000","last_peered":"2024-05-08T04:41:07.116039+0000","last_clean":"2024-05-08T04:41:07.116039+0000","last_became_active":"2024-05-08T04:41:07.112924+0000","last_became_peered":"2024-05-08T04:41:07.112924+0000","last_unstale":"2024-05-08T04:41:07.116039+0000","last_undegraded":"2024-05-08T04:41:07.116039+0000","last_fullsized":"2024-05-08T04:41:07.116039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:33.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110219+0000","last_change":"2024-05-08T04:41:07.110219+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110219+0000","last_undegraded":"2024-05-08T04:41:07.110219+0000","last_fullsized":"2024-05-08T04:41:07.110219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130088+0000","last_change":"2024-05-08T04:41:07.127217+0000","last_active":"2024-05-08T04:41:07.130088+0000","last_peered":"2024-05-08T04:41:07.130088+0000","last_clean":"2024-05-08T04:41:07.130088+0000","last_became_active":"2024-05-08T04:41:07.126634+0000","last_became_peered":"2024-05-08T04:41:07.126634+0000","last_unstale":"2024-05-08T04:41:07.130088+0000","last_undegraded":"2024-05-08T04:41:07.130088+0000","last_fullsized":"2024-05-08T04:41:07.130088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110012+0000","last_change":"2024-05-08T04:41:07.110012+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110012+0000","last_undegraded":"2024-05-08T04:41:07.110012+0000","last_fullsized":"2024-05-08T04:41:07.110012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124534+0000","last_change":"2024-05-08T04:41:07.120447+0000","last_active":"2024-05-08T04:41:07.124534+0000","last_peered":"2024-05-08T04:41:07.124534+0000","last_clean":"2024-05-08T04:41:07.124534+0000","last_became_active":"2024-05-08T04:41:07.120185+0000","last_became_peered":"2024-05-08T04:41:07.120185+0000","last_unstale":"2024-05-08T04:41:07.124534+0000","last_undegraded":"2024-05-08T04:41:07.124534+0000","last_fullsized":"2024-05-08T04:41:07.124534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:22.105771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109383+0000","last_change":"2024-05-08T04:41:07.109383+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109383+0000","last_undegraded":"2024-05-08T04:41:07.109383+0000","last_fullsized":"2024-05-08T04:41:07.109383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.120864+0000","last_change":"2024-05-08T04:41:07.114558+0000","last_active":"2024-05-08T04:41:07.120864+0000","last_peered":"2024-05-08T04:41:07.120864+0000","last_clean":"2024-05-08T04:41:07.120864+0000","last_became_active":"2024-05-08T04:41:07.114273+0000","last_became_peered":"2024-05-08T04:41:07.114273+0000","last_unstale":"2024-05-08T04:41:07.120864+0000","last_undegraded":"2024-05-08T04:41:07.120864+0000","last_fullsized":"2024-05-08T04:41:07.120864+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112472+0000","last_change":"2024-05-08T04:41:07.112472+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112472+0000","last_undegraded":"2024-05-08T04:41:07.112472+0000","last_fullsized":"2024-05-08T04:41:07.112472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122504+0000","last_change":"2024-05-08T04:41:07.119807+0000","last_active":"2024-05-08T04:41:07.122504+0000","last_peered":"2024-05-08T04:41:07.122504+0000","last_clean":"2024-05-08T04:41:07.122504+0000","last_became_active":"2024-05-08T04:41:07.119549+0000","last_became_peered":"2024-05-08T04:41:07.119549+0000","last_unstale":"2024-05-08T04:41:07.122504+0000","last_undegraded":"2024-05-08T04:41:07.122504+0000","last_fullsized":"2024-05-08T04:41:07.122504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104512+0000","last_change":"2024-05-08T04:41:07.104512+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104512+0000","last_undegraded":"2024-05-08T04:41:07.104512+0000","last_fullsized":"2024-05-08T04:41:07.104512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122180+0000","last_change":"2024-05-08T04:41:07.118440+0000","last_active":"2024-05-08T04:41:07.122180+0000","last_peered":"2024-05-08T04:41:07.122180+0000","last_clean":"2024-05-08T04:41:07.122180+0000","last_became_active":"2024-05-08T04:41:07.118074+0000","last_became_peered":"2024-05-08T04:41:07.118074+0000","last_unstale":"2024-05-08T04:41:07.122180+0000","last_undegraded":"2024-05-08T04:41:07.122180+0000","last_fullsized":"2024-05-08T04:41:07.122180+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:05.318932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105487+0000","last_change":"2024-05-08T04:41:07.105487+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105487+0000","last_undegraded":"2024-05-08T04:41:07.105487+0000","last_fullsized":"2024-05-08T04:41:07.105487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117862+0000","last_change":"2024-05-08T04:41:07.111672+0000","last_active":"2024-05-08T04:41:07.117862+0000","last_peered":"2024-05-08T04:41:07.117862+0000","last_clean":"2024-05-08T04:41:07.117862+0000","last_became_active":"2024-05-08T04:41:07.110877+0000","last_became_peered":"2024-05-08T04:41:07.110877+0000","last_unstale":"2024-05-08T04:41:07.117862+0000","last_undegraded":"2024-05-08T04:41:07.117862+0000","last_fullsized":"2024-05-08T04:41:07.117862+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:58.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107314+0000","last_change":"2024-05-08T04:41:07.107314+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107314+0000","last_undegraded":"2024-05-08T04:41:07.107314+0000","last_fullsized":"2024-05-08T04:41:07.107314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125413+0000","last_change":"2024-05-08T04:41:07.120313+0000","last_active":"2024-05-08T04:41:07.125413+0000","last_peered":"2024-05-08T04:41:07.125413+0000","last_clean":"2024-05-08T04:41:07.125413+0000","last_became_active":"2024-05-08T04:41:07.120022+0000","last_became_peered":"2024-05-08T04:41:07.120022+0000","last_unstale":"2024-05-08T04:41:07.125413+0000","last_undegraded":"2024-05-08T04:41:07.125413+0000","last_fullsized":"2024-05-08T04:41:07.125413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:13.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107179+0000","last_change":"2024-05-08T04:41:07.107179+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107179+0000","last_undegraded":"2024-05-08T04:41:07.107179+0000","last_fullsized":"2024-05-08T04:41:07.107179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130756+0000","last_change":"2024-05-08T04:41:07.128722+0000","last_active":"2024-05-08T04:41:07.130756+0000","last_peered":"2024-05-08T04:41:07.130756+0000","last_clean":"2024-05-08T04:41:07.130756+0000","last_became_active":"2024-05-08T04:41:07.128152+0000","last_became_peered":"2024-05-08T04:41:07.128152+0000","last_unstale":"2024-05-08T04:41:07.130756+0000","last_undegraded":"2024-05-08T04:41:07.130756+0000","last_fullsized":"2024-05-08T04:41:07.130756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109605+0000","last_change":"2024-05-08T04:41:07.109605+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109605+0000","last_undegraded":"2024-05-08T04:41:07.109605+0000","last_fullsized":"2024-05-08T04:41:07.109605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128615+0000","last_change":"2024-05-08T04:41:07.124431+0000","last_active":"2024-05-08T04:41:07.128615+0000","last_peered":"2024-05-08T04:41:07.128615+0000","last_clean":"2024-05-08T04:41:07.128615+0000","last_became_active":"2024-05-08T04:41:07.124047+0000","last_became_peered":"2024-05-08T04:41:07.124047+0000","last_unstale":"2024-05-08T04:41:07.128615+0000","last_undegraded":"2024-05-08T04:41:07.128615+0000","last_fullsized":"2024-05-08T04:41:07.128615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:44.583544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104470+0000","last_change":"2024-05-08T04:41:07.104470+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104470+0000","last_undegraded":"2024-05-08T04:41:07.104470+0000","last_fullsized":"2024-05-08T04:41:07.104470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128375+0000","last_change":"2024-05-08T04:41:07.126275+0000","last_active":"2024-05-08T04:41:07.128375+0000","last_peered":"2024-05-08T04:41:07.128375+0000","last_clean":"2024-05-08T04:41:07.128375+0000","last_became_active":"2024-05-08T04:41:07.125938+0000","last_became_peered":"2024-05-08T04:41:07.125938+0000","last_unstale":"2024-05-08T04:41:07.128375+0000","last_undegraded":"2024-05-08T04:41:07.128375+0000","last_fullsized":"2024-05-08T04:41:07.128375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108987+0000","last_change":"2024-05-08T04:41:07.108987+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108987+0000","last_undegraded":"2024-05-08T04:41:07.108987+0000","last_fullsized":"2024-05-08T04:41:07.108987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130586+0000","last_change":"2024-05-08T04:41:07.129166+0000","last_active":"2024-05-08T04:41:07.130586+0000","last_peered":"2024-05-08T04:41:07.130586+0000","last_clean":"2024-05-08T04:41:07.130586+0000","last_became_active":"2024-05-08T04:41:07.128785+0000","last_became_peered":"2024-05-08T04:41:07.128785+0000","last_unstale":"2024-05-08T04:41:07.130586+0000","last_undegraded":"2024-05-08T04:41:07.130586+0000","last_fullsized":"2024-05-08T04:41:07.130586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:53.881464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111401+0000","last_change":"2024-05-08T04:41:07.111401+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111401+0000","last_undegraded":"2024-05-08T04:41:07.111401+0000","last_fullsized":"2024-05-08T04:41:07.111401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124524+0000","last_change":"2024-05-08T04:41:07.122473+0000","last_active":"2024-05-08T04:41:07.124524+0000","last_peered":"2024-05-08T04:41:07.124524+0000","last_clean":"2024-05-08T04:41:07.124524+0000","last_became_active":"2024-05-08T04:41:07.122179+0000","last_became_peered":"2024-05-08T04:41:07.122179+0000","last_unstale":"2024-05-08T04:41:07.124524+0000","last_undegraded":"2024-05-08T04:41:07.124524+0000","last_fullsized":"2024-05-08T04:41:07.124524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:58.656913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107497+0000","last_change":"2024-05-08T04:41:07.107497+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107497+0000","last_undegraded":"2024-05-08T04:41:07.107497+0000","last_fullsized":"2024-05-08T04:41:07.107497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.123393+0000","last_change":"2024-05-08T04:41:07.119592+0000","last_active":"2024-05-08T04:41:07.123393+0000","last_peered":"2024-05-08T04:41:07.123393+0000","last_clean":"2024-05-08T04:41:07.123393+0000","last_became_active":"2024-05-08T04:41:07.119253+0000","last_became_peered":"2024-05-08T04:41:07.119253+0000","last_unstale":"2024-05-08T04:41:07.123393+0000","last_undegraded":"2024-05-08T04:41:07.123393+0000","last_fullsized":"2024-05-08T04:41:07.123393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108207+0000","last_change":"2024-05-08T04:41:07.108207+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108207+0000","last_undegraded":"2024-05-08T04:41:07.108207+0000","last_fullsized":"2024-05-08T04:41:07.108207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.119190+0000","last_change":"2024-05-08T04:41:07.118353+0000","last_active":"2024-05-08T04:41:07.119190+0000","last_peered":"2024-05-08T04:41:07.119190+0000","last_clean":"2024-05-08T04:41:07.119190+0000","last_became_active":"2024-05-08T04:41:07.118022+0000","last_became_peered":"2024-05-08T04:41:07.118022+0000","last_unstale":"2024-05-08T04:41:07.119190+0000","last_undegraded":"2024-05-08T04:41:07.119190+0000","last_fullsized":"2024-05-08T04:41:07.119190+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:57.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106673+0000","last_change":"2024-05-08T04:41:07.106673+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106673+0000","last_undegraded":"2024-05-08T04:41:07.106673+0000","last_fullsized":"2024-05-08T04:41:07.106673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128901+0000","last_change":"2024-05-08T04:41:07.125972+0000","last_active":"2024-05-08T04:41:07.128901+0000","last_peered":"2024-05-08T04:41:07.128901+0000","last_clean":"2024-05-08T04:41:07.128901+0000","last_became_active":"2024-05-08T04:41:07.125631+0000","last_became_peered":"2024-05-08T04:41:07.125631+0000","last_unstale":"2024-05-08T04:41:07.128901+0000","last_undegraded":"2024-05-08T04:41:07.128901+0000","last_fullsized":"2024-05-08T04:41:07.128901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:26.436957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104255+0000","last_change":"2024-05-08T04:41:07.104255+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104255+0000","last_undegraded":"2024-05-08T04:41:07.104255+0000","last_fullsized":"2024-05-08T04:41:07.104255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117933+0000","last_change":"2024-05-08T04:41:07.115163+0000","last_active":"2024-05-08T04:41:07.117933+0000","last_peered":"2024-05-08T04:41:07.117933+0000","last_clean":"2024-05-08T04:41:07.117933+0000","last_became_active":"2024-05-08T04:41:07.114516+0000","last_became_peered":"2024-05-08T04:41:07.114516+0000","last_unstale":"2024-05-08T04:41:07.117933+0000","last_undegraded":"2024-05-08T04:41:07.117933+0000","last_fullsized":"2024-05-08T04:41:07.117933+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:55.546865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108488+0000","last_change":"2024-05-08T04:41:07.108488+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108488+0000","last_undegraded":"2024-05-08T04:41:07.108488+0000","last_fullsized":"2024-05-08T04:41:07.108488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127601+0000","last_change":"2024-05-08T04:41:07.123483+0000","last_active":"2024-05-08T04:41:07.127601+0000","last_peered":"2024-05-08T04:41:07.127601+0000","last_clean":"2024-05-08T04:41:07.127601+0000","last_became_active":"2024-05-08T04:41:07.121115+0000","last_became_peered":"2024-05-08T04:41:07.121115+0000","last_unstale":"2024-05-08T04:41:07.127601+0000","last_undegraded":"2024-05-08T04:41:07.127601+0000","last_fullsized":"2024-05-08T04:41:07.127601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:24.593215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109534+0000","last_change":"2024-05-08T04:41:07.109534+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109534+0000","last_undegraded":"2024-05-08T04:41:07.109534+0000","last_fullsized":"2024-05-08T04:41:07.109534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121479+0000","last_change":"2024-05-08T04:41:07.120042+0000","last_active":"2024-05-08T04:41:07.121479+0000","last_peered":"2024-05-08T04:41:07.121479+0000","last_clean":"2024-05-08T04:41:07.121479+0000","last_became_active":"2024-05-08T04:41:07.119795+0000","last_became_peered":"2024-05-08T04:41:07.119795+0000","last_unstale":"2024-05-08T04:41:07.121479+0000","last_undegraded":"2024-05-08T04:41:07.121479+0000","last_fullsized":"2024-05-08T04:41:07.121479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:16.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105368+0000","last_change":"2024-05-08T04:41:07.105368+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105368+0000","last_undegraded":"2024-05-08T04:41:07.105368+0000","last_fullsized":"2024-05-08T04:41:07.105368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125386+0000","last_change":"2024-05-08T04:41:07.122490+0000","last_active":"2024-05-08T04:41:07.125386+0000","last_peered":"2024-05-08T04:41:07.125386+0000","last_clean":"2024-05-08T04:41:07.125386+0000","last_became_active":"2024-05-08T04:41:07.122220+0000","last_became_peered":"2024-05-08T04:41:07.122220+0000","last_unstale":"2024-05-08T04:41:07.125386+0000","last_undegraded":"2024-05-08T04:41:07.125386+0000","last_fullsized":"2024-05-08T04:41:07.125386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:01:04.274102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110538+0000","last_change":"2024-05-08T04:41:07.110538+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110538+0000","last_undegraded":"2024-05-08T04:41:07.110538+0000","last_fullsized":"2024-05-08T04:41:07.110538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121376+0000","last_change":"2024-05-08T04:41:07.114940+0000","last_active":"2024-05-08T04:41:07.121376+0000","last_peered":"2024-05-08T04:41:07.121376+0000","last_clean":"2024-05-08T04:41:07.121376+0000","last_became_active":"2024-05-08T04:41:07.114663+0000","last_became_peered":"2024-05-08T04:41:07.114663+0000","last_unstale":"2024-05-08T04:41:07.121376+0000","last_undegraded":"2024-05-08T04:41:07.121376+0000","last_fullsized":"2024-05-08T04:41:07.121376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105714+0000","last_change":"2024-05-08T04:41:07.105714+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105714+0000","last_undegraded":"2024-05-08T04:41:07.105714+0000","last_fullsized":"2024-05-08T04:41:07.105714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125304+0000","last_change":"2024-05-08T04:41:07.121852+0000","last_active":"2024-05-08T04:41:07.125304+0000","last_peered":"2024-05-08T04:41:07.125304+0000","last_clean":"2024-05-08T04:41:07.125304+0000","last_became_active":"2024-05-08T04:41:07.121549+0000","last_became_peered":"2024-05-08T04:41:07.121549+0000","last_unstale":"2024-05-08T04:41:07.125304+0000","last_undegraded":"2024-05-08T04:41:07.125304+0000","last_fullsized":"2024-05-08T04:41:07.125304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:20.842180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110425+0000","last_change":"2024-05-08T04:41:07.110425+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110425+0000","last_undegraded":"2024-05-08T04:41:07.110425+0000","last_fullsized":"2024-05-08T04:41:07.110425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127812+0000","last_change":"2024-05-08T04:41:07.125169+0000","last_active":"2024-05-08T04:41:07.127812+0000","last_peered":"2024-05-08T04:41:07.127812+0000","last_clean":"2024-05-08T04:41:07.127812+0000","last_became_active":"2024-05-08T04:41:07.124870+0000","last_became_peered":"2024-05-08T04:41:07.124870+0000","last_unstale":"2024-05-08T04:41:07.127812+0000","last_undegraded":"2024-05-08T04:41:07.127812+0000","last_fullsized":"2024-05-08T04:41:07.127812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:33.327374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109871+0000","last_change":"2024-05-08T04:41:07.109871+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109871+0000","last_undegraded":"2024-05-08T04:41:07.109871+0000","last_fullsized":"2024-05-08T04:41:07.109871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124263+0000","last_change":"2024-05-08T04:41:07.121887+0000","last_active":"2024-05-08T04:41:07.124263+0000","last_peered":"2024-05-08T04:41:07.124263+0000","last_clean":"2024-05-08T04:41:07.124263+0000","last_became_active":"2024-05-08T04:41:07.121407+0000","last_became_peered":"2024-05-08T04:41:07.121407+0000","last_unstale":"2024-05-08T04:41:07.124263+0000","last_undegraded":"2024-05-08T04:41:07.124263+0000","last_fullsized":"2024-05-08T04:41:07.124263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:09.012810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.113889+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:07.113889+0000","last_peered":"2024-05-08T04:41:07.113889+0000","last_clean":"2024-05-08T04:41:07.113889+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:07.113889+0000","last_undegraded":"2024-05-08T04:41:07.113889+0000","last_fullsized":"2024-05-08T04:41:07.113889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:12.200 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:12.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.368+0000 7fd4ece00640 1 Processor -- start 2024-05-08T04:41:12.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.368+0000 7fd4ece00640 1 -- start start 2024-05-08T04:41:12.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.368+0000 7fd4ece00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4e815ab60 0x7fd4e8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4ece00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4e80cfda0 0x7fd4e80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4ece00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e8072640 con 0x7fd4e80b26c0 2024-05-08T04:41:12.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4ece00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e806c770 con 0x7fd4e80cfda0 2024-05-08T04:41:12.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4ece00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e806ca50 con 0x7fd4e815ab60 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4e80cfda0 0x7fd4e80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4e80cfda0 0x7fd4e80cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41780/0 (socket says 172.21.15.164:41780) 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 -- 172.21.15.164:0/4294341927 learned_addr learned my addr 172.21.15.164:0/4294341927 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e5e00640 1 --2- 172.21.15.164:0/4294341927 >> v2:172.21.15.172:3300/0 conn(0x7fd4e815ab60 0x7fd4e8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 -- 172.21.15.164:0/4294341927 >> v2:172.21.15.172:3300/0 conn(0x7fd4e815ab60 msgr2=0x7fd4e8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 --2- 172.21.15.164:0/4294341927 >> v2:172.21.15.172:3300/0 conn(0x7fd4e815ab60 0x7fd4e8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 -- 172.21.15.164:0/4294341927 >> v1:172.21.15.164:6789/0 conn(0x7fd4e80b26c0 legacy=0x7fd4e80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 -- 172.21.15.164:0/4294341927 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e806cd30 con 0x7fd4e80cfda0 2024-05-08T04:41:12.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.372+0000 7fd4e6800640 1 --2- 172.21.15.164:0/4294341927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80cfda0 0x7fd4e80cd910 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cd420 tx=0x7fd4e001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d15f741c608b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e4a00640 1 -- 172.21.15.164:0/4294341927 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00b4070 con 0x7fd4e80cfda0 2024-05-08T04:41:12.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e4a00640 1 -- 172.21.15.164:0/4294341927 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e00a7ac0 con 0x7fd4e80cfda0 2024-05-08T04:41:12.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e4a00640 1 -- 172.21.15.164:0/4294341927 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00af440 con 0x7fd4e80cfda0 2024-05-08T04:41:12.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- 172.21.15.164:0/4294341927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80cfda0 msgr2=0x7fd4e80cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4294341927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80cfda0 0x7fd4e80cd910 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cd420 tx=0x7fd4e001ebf0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- 172.21.15.164:0/4294341927 shutdown_connections 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4294341927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80cfda0 0x7fd4e80cd910 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4294341927 >> v2:172.21.15.172:3300/0 conn(0x7fd4e815ab60 0x7fd4e8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- 172.21.15.164:0/4294341927 >> 172.21.15.164:0/4294341927 conn(0x7fd4e806ac90 msgr2=0x7fd4e806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- 172.21.15.164:0/4294341927 shutdown_connections 2024-05-08T04:41:12.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- 172.21.15.164:0/4294341927 wait complete. 2024-05-08T04:41:12.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 Processor -- start 2024-05-08T04:41:12.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- start start 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 0x7fd4e80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4e815ab60 0x7fd4e80addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80aeeb0 con 0x7fd4e80b26c0 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80aed30 con 0x7fd4e815ab60 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4ece00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80cc1b0 con 0x7fd4e80cfda0 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41794/0 (socket says 172.21.15.164:41794) 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.376+0000 7fd4e5e00640 1 -- 172.21.15.164:0/4208064439 learned_addr learned my addr 172.21.15.164:0/4208064439 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e6800640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4e815ab60 0x7fd4e80addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5400640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 0x7fd4e80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 -- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 msgr2=0x7fd4e80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 0x7fd4e80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 -- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4e815ab60 msgr2=0x7fd4e80addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4e815ab60 0x7fd4e80addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 -- 172.21.15.164:0/4208064439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e81560e0 con 0x7fd4e80b26c0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5400640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 0x7fd4e80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5e00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800efc0 tx=0x7fd4d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8018020 con 0x7fd4e80b26c0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d8014480 con 0x7fd4e80b26c0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e81563c0 con 0x7fd4e80b26c0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8014850 con 0x7fd4e80b26c0 2024-05-08T04:41:12.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e8149820 con 0x7fd4e80b26c0 2024-05-08T04:41:12.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8069d60 con 0x7fd4e80b26c0 2024-05-08T04:41:12.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd4d801b020 con 0x7fd4e80b26c0 2024-05-08T04:41:12.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 0x7fd4c8069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd4d8086220 con 0x7fd4e80b26c0 2024-05-08T04:41:12.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.380+0000 7fd4e5400640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 0x7fd4c8069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.384+0000 7fd4e5400640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 0x7fd4c8069860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4e815ca50 tx=0x7fd4d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.384+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4d8022070 con 0x7fd4e80b26c0 2024-05-08T04:41:12.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.536+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4e815a210 con 0x7fd4c80673a0 2024-05-08T04:41:12.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.544+0000 7fd4de000640 1 -- 172.21.15.164:0/4208064439 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246750 (secure 0 0 0) 0x7fd4e815a210 con 0x7fd4c80673a0 2024-05-08T04:41:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:12.553 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:12.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 msgr2=0x7fd4c8069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 0x7fd4c8069860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4e815ca50 tx=0x7fd4d4009040 comp rx=0 tx=0).stop 2024-05-08T04:41:12.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 msgr2=0x7fd4e81492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800efc0 tx=0x7fd4d800c490 comp rx=0 tx=0).stop 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 shutdown_connections 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4c80673a0 0x7fd4c8069860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4e80cfda0 0x7fd4e80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4e815ab60 0x7fd4e80addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 --2- 172.21.15.164:0/4208064439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e80b26c0 0x7fd4e81492e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 >> 172.21.15.164:0/4208064439 conn(0x7fd4e806ac90 msgr2=0x7fd4e80af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 shutdown_connections 2024-05-08T04:41:12.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.548+0000 7fd4ece00640 1 -- 172.21.15.164:0/4208064439 wait complete. 2024-05-08T04:41:12.572 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-05-08T04:41:11.462834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244972,"kb_used_data":7984,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729748,"statfs":{"total":773094113280,"available":772843261952,"internally_reserved":0,"allocated":8175616,"data_stored":4133594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.369323"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110592+0000","last_change":"2024-05-08T04:41:07.110592+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110592+0000","last_undegraded":"2024-05-08T04:41:07.110592+0000","last_fullsized":"2024-05-08T04:41:07.110592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.116717+0000","last_change":"2024-05-08T04:41:07.116717+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.116717+0000","last_undegraded":"2024-05-08T04:41:07.116717+0000","last_fullsized":"2024-05-08T04:41:07.116717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111388+0000","last_change":"2024-05-08T04:41:07.111388+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111388+0000","last_undegraded":"2024-05-08T04:41:07.111388+0000","last_fullsized":"2024-05-08T04:41:07.111388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105294+0000","last_change":"2024-05-08T04:41:07.105294+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105294+0000","last_undegraded":"2024-05-08T04:41:07.105294+0000","last_fullsized":"2024-05-08T04:41:07.105294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108963+0000","last_change":"2024-05-08T04:41:07.108963+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108963+0000","last_undegraded":"2024-05-08T04:41:07.108963+0000","last_fullsized":"2024-05-08T04:41:07.108963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115911+0000","last_change":"2024-05-08T04:41:07.115911+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115911+0000","last_undegraded":"2024-05-08T04:41:07.115911+0000","last_fullsized":"2024-05-08T04:41:07.115911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104906+0000","last_change":"2024-05-08T04:41:07.104906+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104906+0000","last_undegraded":"2024-05-08T04:41:07.104906+0000","last_fullsized":"2024-05-08T04:41:07.104906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115370+0000","last_change":"2024-05-08T04:41:07.115370+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115370+0000","last_undegraded":"2024-05-08T04:41:07.115370+0000","last_fullsized":"2024-05-08T04:41:07.115370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111074+0000","last_change":"2024-05-08T04:41:07.111074+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111074+0000","last_undegraded":"2024-05-08T04:41:07.111074+0000","last_fullsized":"2024-05-08T04:41:07.111074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107667+0000","last_change":"2024-05-08T04:41:07.107667+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107667+0000","last_undegraded":"2024-05-08T04:41:07.107667+0000","last_fullsized":"2024-05-08T04:41:07.107667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111665+0000","last_change":"2024-05-08T04:41:07.111665+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111665+0000","last_undegraded":"2024-05-08T04:41:07.111665+0000","last_fullsized":"2024-05-08T04:41:07.111665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104932+0000","last_change":"2024-05-08T04:41:07.104932+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104932+0000","last_undegraded":"2024-05-08T04:41:07.104932+0000","last_fullsized":"2024-05-08T04:41:07.104932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106039+0000","last_change":"2024-05-08T04:41:07.106039+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106039+0000","last_undegraded":"2024-05-08T04:41:07.106039+0000","last_fullsized":"2024-05-08T04:41:07.106039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107095+0000","last_change":"2024-05-08T04:41:07.107095+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107095+0000","last_undegraded":"2024-05-08T04:41:07.107095+0000","last_fullsized":"2024-05-08T04:41:07.107095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106245+0000","last_change":"2024-05-08T04:41:07.106245+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106245+0000","last_undegraded":"2024-05-08T04:41:07.106245+0000","last_fullsized":"2024-05-08T04:41:07.106245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106478+0000","last_change":"2024-05-08T04:41:07.106478+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106478+0000","last_undegraded":"2024-05-08T04:41:07.106478+0000","last_fullsized":"2024-05-08T04:41:07.106478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110595+0000","last_change":"2024-05-08T04:41:07.110595+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110595+0000","last_undegraded":"2024-05-08T04:41:07.110595+0000","last_fullsized":"2024-05-08T04:41:07.110595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104441+0000","last_change":"2024-05-08T04:41:07.104441+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104441+0000","last_undegraded":"2024-05-08T04:41:07.104441+0000","last_fullsized":"2024-05-08T04:41:07.104441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114231+0000","last_change":"2024-05-08T04:41:07.114231+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114231+0000","last_undegraded":"2024-05-08T04:41:07.114231+0000","last_fullsized":"2024-05-08T04:41:07.114231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105925+0000","last_change":"2024-05-08T04:41:07.105925+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105925+0000","last_undegraded":"2024-05-08T04:41:07.105925+0000","last_fullsized":"2024-05-08T04:41:07.105925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105897+0000","last_change":"2024-05-08T04:41:07.105897+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105897+0000","last_undegraded":"2024-05-08T04:41:07.105897+0000","last_fullsized":"2024-05-08T04:41:07.105897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106157+0000","last_change":"2024-05-08T04:41:07.106157+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106157+0000","last_undegraded":"2024-05-08T04:41:07.106157+0000","last_fullsized":"2024-05-08T04:41:07.106157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112836+0000","last_change":"2024-05-08T04:41:07.112836+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112836+0000","last_undegraded":"2024-05-08T04:41:07.112836+0000","last_fullsized":"2024-05-08T04:41:07.112836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107394+0000","last_change":"2024-05-08T04:41:07.107394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107394+0000","last_undegraded":"2024-05-08T04:41:07.107394+0000","last_fullsized":"2024-05-08T04:41:07.107394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106637+0000","last_change":"2024-05-08T04:41:07.106637+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106637+0000","last_undegraded":"2024-05-08T04:41:07.106637+0000","last_fullsized":"2024-05-08T04:41:07.106637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104164+0000","last_change":"2024-05-08T04:41:07.104164+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104164+0000","last_undegraded":"2024-05-08T04:41:07.104164+0000","last_fullsized":"2024-05-08T04:41:07.104164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108566+0000","last_change":"2024-05-08T04:41:07.108566+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108566+0000","last_undegraded":"2024-05-08T04:41:07.108566+0000","last_fullsized":"2024-05-08T04:41:07.108566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108233+0000","last_change":"2024-05-08T04:41:07.108233+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108233+0000","last_undegraded":"2024-05-08T04:41:07.108233+0000","last_fullsized":"2024-05-08T04:41:07.108233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.113288+0000","last_change":"2024-05-08T04:41:07.113288+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.113288+0000","last_undegraded":"2024-05-08T04:41:07.113288+0000","last_fullsized":"2024-05-08T04:41:07.113288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104159+0000","last_change":"2024-05-08T04:41:07.104159+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104159+0000","last_undegraded":"2024-05-08T04:41:07.104159+0000","last_fullsized":"2024-05-08T04:41:07.104159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103467+0000","last_change":"2024-05-08T04:41:07.103467+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103467+0000","last_undegraded":"2024-05-08T04:41:07.103467+0000","last_fullsized":"2024-05-08T04:41:07.103467+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114887+0000","last_change":"2024-05-08T04:41:07.114887+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114887+0000","last_undegraded":"2024-05-08T04:41:07.114887+0000","last_fullsized":"2024-05-08T04:41:07.114887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109849+0000","last_change":"2024-05-08T04:41:07.109849+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109849+0000","last_undegraded":"2024-05-08T04:41:07.109849+0000","last_fullsized":"2024-05-08T04:41:07.109849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.129995+0000","last_change":"2024-05-08T04:41:07.123950+0000","last_active":"2024-05-08T04:41:07.129995+0000","last_peered":"2024-05-08T04:41:07.129995+0000","last_clean":"2024-05-08T04:41:07.129995+0000","last_became_active":"2024-05-08T04:41:07.123577+0000","last_became_peered":"2024-05-08T04:41:07.123577+0000","last_unstale":"2024-05-08T04:41:07.129995+0000","last_undegraded":"2024-05-08T04:41:07.129995+0000","last_fullsized":"2024-05-08T04:41:07.129995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:28.418132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110185+0000","last_change":"2024-05-08T04:41:07.110185+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110185+0000","last_undegraded":"2024-05-08T04:41:07.110185+0000","last_fullsized":"2024-05-08T04:41:07.110185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125062+0000","last_change":"2024-05-08T04:41:07.122634+0000","last_active":"2024-05-08T04:41:07.125062+0000","last_peered":"2024-05-08T04:41:07.125062+0000","last_clean":"2024-05-08T04:41:07.125062+0000","last_became_active":"2024-05-08T04:41:07.122307+0000","last_became_peered":"2024-05-08T04:41:07.122307+0000","last_unstale":"2024-05-08T04:41:07.125062+0000","last_undegraded":"2024-05-08T04:41:07.125062+0000","last_fullsized":"2024-05-08T04:41:07.125062+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:18.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110634+0000","last_change":"2024-05-08T04:41:07.110634+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110634+0000","last_undegraded":"2024-05-08T04:41:07.110634+0000","last_fullsized":"2024-05-08T04:41:07.110634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128519+0000","last_change":"2024-05-08T04:41:07.124366+0000","last_active":"2024-05-08T04:41:07.128519+0000","last_peered":"2024-05-08T04:41:07.128519+0000","last_clean":"2024-05-08T04:41:07.128519+0000","last_became_active":"2024-05-08T04:41:07.124039+0000","last_became_peered":"2024-05-08T04:41:07.124039+0000","last_unstale":"2024-05-08T04:41:07.128519+0000","last_undegraded":"2024-05-08T04:41:07.128519+0000","last_fullsized":"2024-05-08T04:41:07.128519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:24.785925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111227+0000","last_change":"2024-05-08T04:41:07.111227+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111227+0000","last_undegraded":"2024-05-08T04:41:07.111227+0000","last_fullsized":"2024-05-08T04:41:07.111227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128296+0000","last_change":"2024-05-08T04:41:07.126964+0000","last_active":"2024-05-08T04:41:07.128296+0000","last_peered":"2024-05-08T04:41:07.128296+0000","last_clean":"2024-05-08T04:41:07.128296+0000","last_became_active":"2024-05-08T04:41:07.126433+0000","last_became_peered":"2024-05-08T04:41:07.126433+0000","last_unstale":"2024-05-08T04:41:07.128296+0000","last_undegraded":"2024-05-08T04:41:07.128296+0000","last_fullsized":"2024-05-08T04:41:07.128296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103412+0000","last_change":"2024-05-08T04:41:07.103412+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103412+0000","last_undegraded":"2024-05-08T04:41:07.103412+0000","last_fullsized":"2024-05-08T04:41:07.103412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122741+0000","last_change":"2024-05-08T04:41:07.118893+0000","last_active":"2024-05-08T04:41:07.122741+0000","last_peered":"2024-05-08T04:41:07.122741+0000","last_clean":"2024-05-08T04:41:07.122741+0000","last_became_active":"2024-05-08T04:41:07.117180+0000","last_became_peered":"2024-05-08T04:41:07.117180+0000","last_unstale":"2024-05-08T04:41:07.122741+0000","last_undegraded":"2024-05-08T04:41:07.122741+0000","last_fullsized":"2024-05-08T04:41:07.122741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:45.295908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105256+0000","last_change":"2024-05-08T04:41:07.105256+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105256+0000","last_undegraded":"2024-05-08T04:41:07.105256+0000","last_fullsized":"2024-05-08T04:41:07.105256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124504+0000","last_change":"2024-05-08T04:41:07.122025+0000","last_active":"2024-05-08T04:41:07.124504+0000","last_peered":"2024-05-08T04:41:07.124504+0000","last_clean":"2024-05-08T04:41:07.124504+0000","last_became_active":"2024-05-08T04:41:07.120512+0000","last_became_peered":"2024-05-08T04:41:07.120512+0000","last_unstale":"2024-05-08T04:41:07.124504+0000","last_undegraded":"2024-05-08T04:41:07.124504+0000","last_fullsized":"2024-05-08T04:41:07.124504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:27.054889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110394+0000","last_change":"2024-05-08T04:41:07.110394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110394+0000","last_undegraded":"2024-05-08T04:41:07.110394+0000","last_fullsized":"2024-05-08T04:41:07.110394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121508+0000","last_change":"2024-05-08T04:41:07.115029+0000","last_active":"2024-05-08T04:41:07.121508+0000","last_peered":"2024-05-08T04:41:07.121508+0000","last_clean":"2024-05-08T04:41:07.121508+0000","last_became_active":"2024-05-08T04:41:07.114753+0000","last_became_peered":"2024-05-08T04:41:07.114753+0000","last_unstale":"2024-05-08T04:41:07.121508+0000","last_undegraded":"2024-05-08T04:41:07.121508+0000","last_fullsized":"2024-05-08T04:41:07.121508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105796+0000","last_change":"2024-05-08T04:41:07.105796+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105796+0000","last_undegraded":"2024-05-08T04:41:07.105796+0000","last_fullsized":"2024-05-08T04:41:07.105796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116667+0000","last_change":"2024-05-08T04:41:07.113389+0000","last_active":"2024-05-08T04:41:07.116667+0000","last_peered":"2024-05-08T04:41:07.116667+0000","last_clean":"2024-05-08T04:41:07.116667+0000","last_became_active":"2024-05-08T04:41:07.113100+0000","last_became_peered":"2024-05-08T04:41:07.113100+0000","last_unstale":"2024-05-08T04:41:07.116667+0000","last_undegraded":"2024-05-08T04:41:07.116667+0000","last_fullsized":"2024-05-08T04:41:07.116667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:13.388255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103521+0000","last_change":"2024-05-08T04:41:07.103521+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103521+0000","last_undegraded":"2024-05-08T04:41:07.103521+0000","last_fullsized":"2024-05-08T04:41:07.103521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116039+0000","last_change":"2024-05-08T04:41:07.113877+0000","last_active":"2024-05-08T04:41:07.116039+0000","last_peered":"2024-05-08T04:41:07.116039+0000","last_clean":"2024-05-08T04:41:07.116039+0000","last_became_active":"2024-05-08T04:41:07.112924+0000","last_became_peered":"2024-05-08T04:41:07.112924+0000","last_unstale":"2024-05-08T04:41:07.116039+0000","last_undegraded":"2024-05-08T04:41:07.116039+0000","last_fullsized":"2024-05-08T04:41:07.116039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:33.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110219+0000","last_change":"2024-05-08T04:41:07.110219+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110219+0000","last_undegraded":"2024-05-08T04:41:07.110219+0000","last_fullsized":"2024-05-08T04:41:07.110219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130088+0000","last_change":"2024-05-08T04:41:07.127217+0000","last_active":"2024-05-08T04:41:07.130088+0000","last_peered":"2024-05-08T04:41:07.130088+0000","last_clean":"2024-05-08T04:41:07.130088+0000","last_became_active":"2024-05-08T04:41:07.126634+0000","last_became_peered":"2024-05-08T04:41:07.126634+0000","last_unstale":"2024-05-08T04:41:07.130088+0000","last_undegraded":"2024-05-08T04:41:07.130088+0000","last_fullsized":"2024-05-08T04:41:07.130088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110012+0000","last_change":"2024-05-08T04:41:07.110012+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110012+0000","last_undegraded":"2024-05-08T04:41:07.110012+0000","last_fullsized":"2024-05-08T04:41:07.110012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124534+0000","last_change":"2024-05-08T04:41:07.120447+0000","last_active":"2024-05-08T04:41:07.124534+0000","last_peered":"2024-05-08T04:41:07.124534+0000","last_clean":"2024-05-08T04:41:07.124534+0000","last_became_active":"2024-05-08T04:41:07.120185+0000","last_became_peered":"2024-05-08T04:41:07.120185+0000","last_unstale":"2024-05-08T04:41:07.124534+0000","last_undegraded":"2024-05-08T04:41:07.124534+0000","last_fullsized":"2024-05-08T04:41:07.124534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:22.105771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109383+0000","last_change":"2024-05-08T04:41:07.109383+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109383+0000","last_undegraded":"2024-05-08T04:41:07.109383+0000","last_fullsized":"2024-05-08T04:41:07.109383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.120864+0000","last_change":"2024-05-08T04:41:07.114558+0000","last_active":"2024-05-08T04:41:07.120864+0000","last_peered":"2024-05-08T04:41:07.120864+0000","last_clean":"2024-05-08T04:41:07.120864+0000","last_became_active":"2024-05-08T04:41:07.114273+0000","last_became_peered":"2024-05-08T04:41:07.114273+0000","last_unstale":"2024-05-08T04:41:07.120864+0000","last_undegraded":"2024-05-08T04:41:07.120864+0000","last_fullsized":"2024-05-08T04:41:07.120864+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112472+0000","last_change":"2024-05-08T04:41:07.112472+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112472+0000","last_undegraded":"2024-05-08T04:41:07.112472+0000","last_fullsized":"2024-05-08T04:41:07.112472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122504+0000","last_change":"2024-05-08T04:41:07.119807+0000","last_active":"2024-05-08T04:41:07.122504+0000","last_peered":"2024-05-08T04:41:07.122504+0000","last_clean":"2024-05-08T04:41:07.122504+0000","last_became_active":"2024-05-08T04:41:07.119549+0000","last_became_peered":"2024-05-08T04:41:07.119549+0000","last_unstale":"2024-05-08T04:41:07.122504+0000","last_undegraded":"2024-05-08T04:41:07.122504+0000","last_fullsized":"2024-05-08T04:41:07.122504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104512+0000","last_change":"2024-05-08T04:41:07.104512+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104512+0000","last_undegraded":"2024-05-08T04:41:07.104512+0000","last_fullsized":"2024-05-08T04:41:07.104512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122180+0000","last_change":"2024-05-08T04:41:07.118440+0000","last_active":"2024-05-08T04:41:07.122180+0000","last_peered":"2024-05-08T04:41:07.122180+0000","last_clean":"2024-05-08T04:41:07.122180+0000","last_became_active":"2024-05-08T04:41:07.118074+0000","last_became_peered":"2024-05-08T04:41:07.118074+0000","last_unstale":"2024-05-08T04:41:07.122180+0000","last_undegraded":"2024-05-08T04:41:07.122180+0000","last_fullsized":"2024-05-08T04:41:07.122180+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:05.318932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105487+0000","last_change":"2024-05-08T04:41:07.105487+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105487+0000","last_undegraded":"2024-05-08T04:41:07.105487+0000","last_fullsized":"2024-05-08T04:41:07.105487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117862+0000","last_change":"2024-05-08T04:41:07.111672+0000","last_active":"2024-05-08T04:41:07.117862+0000","last_peered":"2024-05-08T04:41:07.117862+0000","last_clean":"2024-05-08T04:41:07.117862+0000","last_became_active":"2024-05-08T04:41:07.110877+0000","last_became_peered":"2024-05-08T04:41:07.110877+0000","last_unstale":"2024-05-08T04:41:07.117862+0000","last_undegraded":"2024-05-08T04:41:07.117862+0000","last_fullsized":"2024-05-08T04:41:07.117862+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:58.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107314+0000","last_change":"2024-05-08T04:41:07.107314+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107314+0000","last_undegraded":"2024-05-08T04:41:07.107314+0000","last_fullsized":"2024-05-08T04:41:07.107314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125413+0000","last_change":"2024-05-08T04:41:07.120313+0000","last_active":"2024-05-08T04:41:07.125413+0000","last_peered":"2024-05-08T04:41:07.125413+0000","last_clean":"2024-05-08T04:41:07.125413+0000","last_became_active":"2024-05-08T04:41:07.120022+0000","last_became_peered":"2024-05-08T04:41:07.120022+0000","last_unstale":"2024-05-08T04:41:07.125413+0000","last_undegraded":"2024-05-08T04:41:07.125413+0000","last_fullsized":"2024-05-08T04:41:07.125413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:13.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107179+0000","last_change":"2024-05-08T04:41:07.107179+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107179+0000","last_undegraded":"2024-05-08T04:41:07.107179+0000","last_fullsized":"2024-05-08T04:41:07.107179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130756+0000","last_change":"2024-05-08T04:41:07.128722+0000","last_active":"2024-05-08T04:41:07.130756+0000","last_peered":"2024-05-08T04:41:07.130756+0000","last_clean":"2024-05-08T04:41:07.130756+0000","last_became_active":"2024-05-08T04:41:07.128152+0000","last_became_peered":"2024-05-08T04:41:07.128152+0000","last_unstale":"2024-05-08T04:41:07.130756+0000","last_undegraded":"2024-05-08T04:41:07.130756+0000","last_fullsized":"2024-05-08T04:41:07.130756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109605+0000","last_change":"2024-05-08T04:41:07.109605+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109605+0000","last_undegraded":"2024-05-08T04:41:07.109605+0000","last_fullsized":"2024-05-08T04:41:07.109605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128615+0000","last_change":"2024-05-08T04:41:07.124431+0000","last_active":"2024-05-08T04:41:07.128615+0000","last_peered":"2024-05-08T04:41:07.128615+0000","last_clean":"2024-05-08T04:41:07.128615+0000","last_became_active":"2024-05-08T04:41:07.124047+0000","last_became_peered":"2024-05-08T04:41:07.124047+0000","last_unstale":"2024-05-08T04:41:07.128615+0000","last_undegraded":"2024-05-08T04:41:07.128615+0000","last_fullsized":"2024-05-08T04:41:07.128615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:44.583544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104470+0000","last_change":"2024-05-08T04:41:07.104470+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104470+0000","last_undegraded":"2024-05-08T04:41:07.104470+0000","last_fullsized":"2024-05-08T04:41:07.104470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128375+0000","last_change":"2024-05-08T04:41:07.126275+0000","last_active":"2024-05-08T04:41:07.128375+0000","last_peered":"2024-05-08T04:41:07.128375+0000","last_clean":"2024-05-08T04:41:07.128375+0000","last_became_active":"2024-05-08T04:41:07.125938+0000","last_became_peered":"2024-05-08T04:41:07.125938+0000","last_unstale":"2024-05-08T04:41:07.128375+0000","last_undegraded":"2024-05-08T04:41:07.128375+0000","last_fullsized":"2024-05-08T04:41:07.128375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108987+0000","last_change":"2024-05-08T04:41:07.108987+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108987+0000","last_undegraded":"2024-05-08T04:41:07.108987+0000","last_fullsized":"2024-05-08T04:41:07.108987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130586+0000","last_change":"2024-05-08T04:41:07.129166+0000","last_active":"2024-05-08T04:41:07.130586+0000","last_peered":"2024-05-08T04:41:07.130586+0000","last_clean":"2024-05-08T04:41:07.130586+0000","last_became_active":"2024-05-08T04:41:07.128785+0000","last_became_peered":"2024-05-08T04:41:07.128785+0000","last_unstale":"2024-05-08T04:41:07.130586+0000","last_undegraded":"2024-05-08T04:41:07.130586+0000","last_fullsized":"2024-05-08T04:41:07.130586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:53.881464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111401+0000","last_change":"2024-05-08T04:41:07.111401+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111401+0000","last_undegraded":"2024-05-08T04:41:07.111401+0000","last_fullsized":"2024-05-08T04:41:07.111401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124524+0000","last_change":"2024-05-08T04:41:07.122473+0000","last_active":"2024-05-08T04:41:07.124524+0000","last_peered":"2024-05-08T04:41:07.124524+0000","last_clean":"2024-05-08T04:41:07.124524+0000","last_became_active":"2024-05-08T04:41:07.122179+0000","last_became_peered":"2024-05-08T04:41:07.122179+0000","last_unstale":"2024-05-08T04:41:07.124524+0000","last_undegraded":"2024-05-08T04:41:07.124524+0000","last_fullsized":"2024-05-08T04:41:07.124524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:58.656913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107497+0000","last_change":"2024-05-08T04:41:07.107497+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107497+0000","last_undegraded":"2024-05-08T04:41:07.107497+0000","last_fullsized":"2024-05-08T04:41:07.107497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.123393+0000","last_change":"2024-05-08T04:41:07.119592+0000","last_active":"2024-05-08T04:41:07.123393+0000","last_peered":"2024-05-08T04:41:07.123393+0000","last_clean":"2024-05-08T04:41:07.123393+0000","last_became_active":"2024-05-08T04:41:07.119253+0000","last_became_peered":"2024-05-08T04:41:07.119253+0000","last_unstale":"2024-05-08T04:41:07.123393+0000","last_undegraded":"2024-05-08T04:41:07.123393+0000","last_fullsized":"2024-05-08T04:41:07.123393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108207+0000","last_change":"2024-05-08T04:41:07.108207+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108207+0000","last_undegraded":"2024-05-08T04:41:07.108207+0000","last_fullsized":"2024-05-08T04:41:07.108207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.119190+0000","last_change":"2024-05-08T04:41:07.118353+0000","last_active":"2024-05-08T04:41:07.119190+0000","last_peered":"2024-05-08T04:41:07.119190+0000","last_clean":"2024-05-08T04:41:07.119190+0000","last_became_active":"2024-05-08T04:41:07.118022+0000","last_became_peered":"2024-05-08T04:41:07.118022+0000","last_unstale":"2024-05-08T04:41:07.119190+0000","last_undegraded":"2024-05-08T04:41:07.119190+0000","last_fullsized":"2024-05-08T04:41:07.119190+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:57.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106673+0000","last_change":"2024-05-08T04:41:07.106673+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106673+0000","last_undegraded":"2024-05-08T04:41:07.106673+0000","last_fullsized":"2024-05-08T04:41:07.106673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128901+0000","last_change":"2024-05-08T04:41:07.125972+0000","last_active":"2024-05-08T04:41:07.128901+0000","last_peered":"2024-05-08T04:41:07.128901+0000","last_clean":"2024-05-08T04:41:07.128901+0000","last_became_active":"2024-05-08T04:41:07.125631+0000","last_became_peered":"2024-05-08T04:41:07.125631+0000","last_unstale":"2024-05-08T04:41:07.128901+0000","last_undegraded":"2024-05-08T04:41:07.128901+0000","last_fullsized":"2024-05-08T04:41:07.128901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:26.436957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104255+0000","last_change":"2024-05-08T04:41:07.104255+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104255+0000","last_undegraded":"2024-05-08T04:41:07.104255+0000","last_fullsized":"2024-05-08T04:41:07.104255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117933+0000","last_change":"2024-05-08T04:41:07.115163+0000","last_active":"2024-05-08T04:41:07.117933+0000","last_peered":"2024-05-08T04:41:07.117933+0000","last_clean":"2024-05-08T04:41:07.117933+0000","last_became_active":"2024-05-08T04:41:07.114516+0000","last_became_peered":"2024-05-08T04:41:07.114516+0000","last_unstale":"2024-05-08T04:41:07.117933+0000","last_undegraded":"2024-05-08T04:41:07.117933+0000","last_fullsized":"2024-05-08T04:41:07.117933+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:55.546865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108488+0000","last_change":"2024-05-08T04:41:07.108488+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108488+0000","last_undegraded":"2024-05-08T04:41:07.108488+0000","last_fullsized":"2024-05-08T04:41:07.108488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127601+0000","last_change":"2024-05-08T04:41:07.123483+0000","last_active":"2024-05-08T04:41:07.127601+0000","last_peered":"2024-05-08T04:41:07.127601+0000","last_clean":"2024-05-08T04:41:07.127601+0000","last_became_active":"2024-05-08T04:41:07.121115+0000","last_became_peered":"2024-05-08T04:41:07.121115+0000","last_unstale":"2024-05-08T04:41:07.127601+0000","last_undegraded":"2024-05-08T04:41:07.127601+0000","last_fullsized":"2024-05-08T04:41:07.127601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:24.593215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109534+0000","last_change":"2024-05-08T04:41:07.109534+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109534+0000","last_undegraded":"2024-05-08T04:41:07.109534+0000","last_fullsized":"2024-05-08T04:41:07.109534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121479+0000","last_change":"2024-05-08T04:41:07.120042+0000","last_active":"2024-05-08T04:41:07.121479+0000","last_peered":"2024-05-08T04:41:07.121479+0000","last_clean":"2024-05-08T04:41:07.121479+0000","last_became_active":"2024-05-08T04:41:07.119795+0000","last_became_peered":"2024-05-08T04:41:07.119795+0000","last_unstale":"2024-05-08T04:41:07.121479+0000","last_undegraded":"2024-05-08T04:41:07.121479+0000","last_fullsized":"2024-05-08T04:41:07.121479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:16.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105368+0000","last_change":"2024-05-08T04:41:07.105368+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105368+0000","last_undegraded":"2024-05-08T04:41:07.105368+0000","last_fullsized":"2024-05-08T04:41:07.105368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125386+0000","last_change":"2024-05-08T04:41:07.122490+0000","last_active":"2024-05-08T04:41:07.125386+0000","last_peered":"2024-05-08T04:41:07.125386+0000","last_clean":"2024-05-08T04:41:07.125386+0000","last_became_active":"2024-05-08T04:41:07.122220+0000","last_became_peered":"2024-05-08T04:41:07.122220+0000","last_unstale":"2024-05-08T04:41:07.125386+0000","last_undegraded":"2024-05-08T04:41:07.125386+0000","last_fullsized":"2024-05-08T04:41:07.125386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:01:04.274102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110538+0000","last_change":"2024-05-08T04:41:07.110538+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110538+0000","last_undegraded":"2024-05-08T04:41:07.110538+0000","last_fullsized":"2024-05-08T04:41:07.110538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121376+0000","last_change":"2024-05-08T04:41:07.114940+0000","last_active":"2024-05-08T04:41:07.121376+0000","last_peered":"2024-05-08T04:41:07.121376+0000","last_clean":"2024-05-08T04:41:07.121376+0000","last_became_active":"2024-05-08T04:41:07.114663+0000","last_became_peered":"2024-05-08T04:41:07.114663+0000","last_unstale":"2024-05-08T04:41:07.121376+0000","last_undegraded":"2024-05-08T04:41:07.121376+0000","last_fullsized":"2024-05-08T04:41:07.121376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105714+0000","last_change":"2024-05-08T04:41:07.105714+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105714+0000","last_undegraded":"2024-05-08T04:41:07.105714+0000","last_fullsized":"2024-05-08T04:41:07.105714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125304+0000","last_change":"2024-05-08T04:41:07.121852+0000","last_active":"2024-05-08T04:41:07.125304+0000","last_peered":"2024-05-08T04:41:07.125304+0000","last_clean":"2024-05-08T04:41:07.125304+0000","last_became_active":"2024-05-08T04:41:07.121549+0000","last_became_peered":"2024-05-08T04:41:07.121549+0000","last_unstale":"2024-05-08T04:41:07.125304+0000","last_undegraded":"2024-05-08T04:41:07.125304+0000","last_fullsized":"2024-05-08T04:41:07.125304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:20.842180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110425+0000","last_change":"2024-05-08T04:41:07.110425+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110425+0000","last_undegraded":"2024-05-08T04:41:07.110425+0000","last_fullsized":"2024-05-08T04:41:07.110425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127812+0000","last_change":"2024-05-08T04:41:07.125169+0000","last_active":"2024-05-08T04:41:07.127812+0000","last_peered":"2024-05-08T04:41:07.127812+0000","last_clean":"2024-05-08T04:41:07.127812+0000","last_became_active":"2024-05-08T04:41:07.124870+0000","last_became_peered":"2024-05-08T04:41:07.124870+0000","last_unstale":"2024-05-08T04:41:07.127812+0000","last_undegraded":"2024-05-08T04:41:07.127812+0000","last_fullsized":"2024-05-08T04:41:07.127812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:33.327374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109871+0000","last_change":"2024-05-08T04:41:07.109871+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109871+0000","last_undegraded":"2024-05-08T04:41:07.109871+0000","last_fullsized":"2024-05-08T04:41:07.109871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124263+0000","last_change":"2024-05-08T04:41:07.121887+0000","last_active":"2024-05-08T04:41:07.124263+0000","last_peered":"2024-05-08T04:41:07.124263+0000","last_clean":"2024-05-08T04:41:07.124263+0000","last_became_active":"2024-05-08T04:41:07.121407+0000","last_became_peered":"2024-05-08T04:41:07.121407+0000","last_unstale":"2024-05-08T04:41:07.124263+0000","last_undegraded":"2024-05-08T04:41:07.124263+0000","last_fullsized":"2024-05-08T04:41:07.124263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:09.012810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.113889+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:07.113889+0000","last_peered":"2024-05-08T04:41:07.113889+0000","last_clean":"2024-05-08T04:41:07.113889+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:07.113889+0000","last_undegraded":"2024-05-08T04:41:07.113889+0000","last_fullsized":"2024-05-08T04:41:07.113889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:12.575 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:12.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.744+0000 7f2281600640 1 Processor -- start 2024-05-08T04:41:12.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.744+0000 7f2281600640 1 -- start start 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2281600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f227c0c5530 0x7f227c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2281600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f227c0cbed0 0x7f227c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2281600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227c0ccc40 con 0x7f227c15aab0 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2281600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227c15ae90 con 0x7f227c0c5530 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2281600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227c0c5ed0 con 0x7f227c0cbed0 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f227c0c5530 0x7f227c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f227c0c5530 0x7f227c0c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41806/0 (socket says 172.21.15.164:41806) 2024-05-08T04:41:12.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 -- 172.21.15.164:0/4282331355 learned_addr learned my addr 172.21.15.164:0/4282331355 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:12.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f2280c00640 1 --2- 172.21.15.164:0/4282331355 >> v2:172.21.15.172:3300/0 conn(0x7f227c0cbed0 0x7f227c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 -- 172.21.15.164:0/4282331355 >> v2:172.21.15.172:3300/0 conn(0x7f227c0cbed0 msgr2=0x7f227c0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:12.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 --2- 172.21.15.164:0/4282331355 >> v2:172.21.15.172:3300/0 conn(0x7f227c0cbed0 0x7f227c0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 -- 172.21.15.164:0/4282331355 >> v1:172.21.15.164:6789/0 conn(0x7f227c15aab0 legacy=0x7f227c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.748+0000 7f227b400640 1 -- 172.21.15.164:0/4282331355 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c0ce170 con 0x7f227c0c5530 2024-05-08T04:41:12.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f227b400640 1 --2- 172.21.15.164:0/4282331355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0c5990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f227c0c6210 tx=0x7f226801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ac301f4866c610 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f227aa00640 1 -- 172.21.15.164:0/4282331355 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680b0070 con 0x7f227c0c5530 2024-05-08T04:41:12.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f227aa00640 1 -- 172.21.15.164:0/4282331355 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22680ab600 con 0x7f227c0c5530 2024-05-08T04:41:12.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f227aa00640 1 -- 172.21.15.164:0/4282331355 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680ab9d0 con 0x7f227c0c5530 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- 172.21.15.164:0/4282331355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 msgr2=0x7f227c0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 --2- 172.21.15.164:0/4282331355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0c5990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f227c0c6210 tx=0x7f226801c520 comp rx=0 tx=0).stop 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- 172.21.15.164:0/4282331355 shutdown_connections 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 --2- 172.21.15.164:0/4282331355 >> v2:172.21.15.172:3300/0 conn(0x7f227c0cbed0 0x7f227c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 --2- 172.21.15.164:0/4282331355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0c5990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- 172.21.15.164:0/4282331355 >> 172.21.15.164:0/4282331355 conn(0x7f227c06b670 msgr2=0x7f227c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- 172.21.15.164:0/4282331355 shutdown_connections 2024-05-08T04:41:12.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- 172.21.15.164:0/4282331355 wait complete. 2024-05-08T04:41:12.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 Processor -- start 2024-05-08T04:41:12.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.752+0000 7f2281600640 1 -- start start 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 0x7f227c1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 0x7f227c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0ce7c0 con 0x7f227c0c5530 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0713f0 con 0x7f227c15aab0 2024-05-08T04:41:12.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0cf360 con 0x7f227c0cbed0 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41818/0 (socket says 172.21.15.164:41818) 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 -- 172.21.15.164:0/2616165364 learned_addr learned my addr 172.21.15.164:0/2616165364 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2280c00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 0x7f227c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227b400640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 0x7f227c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 -- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 msgr2=0x7f227c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 0x7f227c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 -- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 msgr2=0x7f227c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 0x7f227c153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 -- 172.21.15.164:0/2616165364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c0adf00 con 0x7f227c0c5530 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2280c00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 0x7f227c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227b400640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 0x7f227c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f227be00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c760 tx=0x7f226c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c016020 con 0x7f227c0c5530 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c006d60 con 0x7f227c0c5530 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c003bc0 con 0x7f227c0c5530 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c0ae1e0 con 0x7f227c0c5530 2024-05-08T04:41:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c15cac0 con 0x7f227c0c5530 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c04ef30 con 0x7f227c0c5530 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.756+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f226c00a850 con 0x7f227c0c5530 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.760+0000 7f2273e00640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 0x7f225c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.760+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f226c0853b0 con 0x7f227c0c5530 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.760+0000 7f227b400640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 0x7f225c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:12.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.760+0000 7f227b400640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 0x7f225c069810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22680ad000 tx=0x7f2268008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.764+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f226c020020 con 0x7f227c0c5530 2024-05-08T04:41:12.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.912+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f227c153e60 con 0x7f225c067350 2024-05-08T04:41:12.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.920+0000 7f2273e00640 1 -- 172.21.15.164:0/2616165364 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246750 (secure 0 0 0) 0x7f227c153e60 con 0x7f225c067350 2024-05-08T04:41:12.927 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:12.928 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:12.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 msgr2=0x7f225c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 0x7f225c069810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22680ad000 tx=0x7f2268008040 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 msgr2=0x7f227c0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c760 tx=0x7f226c00cc20 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 shutdown_connections 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f225c067350 0x7f225c069810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f227c0cbed0 0x7f227c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f227c15aab0 0x7f227c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 --2- 172.21.15.164:0/2616165364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f227c0c5530 0x7f227c0ad890 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 >> 172.21.15.164:0/2616165364 conn(0x7f227c06b670 msgr2=0x7f227c0b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 shutdown_connections 2024-05-08T04:41:12.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:12.924+0000 7f2281600640 1 -- 172.21.15.164:0/2616165364 wait complete. 2024-05-08T04:41:12.947 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-05-08T04:41:11.462834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244972,"kb_used_data":7984,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729748,"statfs":{"total":773094113280,"available":772843261952,"internally_reserved":0,"allocated":8175616,"data_stored":4133594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.369323"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110592+0000","last_change":"2024-05-08T04:41:07.110592+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110592+0000","last_undegraded":"2024-05-08T04:41:07.110592+0000","last_fullsized":"2024-05-08T04:41:07.110592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.116717+0000","last_change":"2024-05-08T04:41:07.116717+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.116717+0000","last_undegraded":"2024-05-08T04:41:07.116717+0000","last_fullsized":"2024-05-08T04:41:07.116717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111388+0000","last_change":"2024-05-08T04:41:07.111388+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111388+0000","last_undegraded":"2024-05-08T04:41:07.111388+0000","last_fullsized":"2024-05-08T04:41:07.111388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105294+0000","last_change":"2024-05-08T04:41:07.105294+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105294+0000","last_undegraded":"2024-05-08T04:41:07.105294+0000","last_fullsized":"2024-05-08T04:41:07.105294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108963+0000","last_change":"2024-05-08T04:41:07.108963+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108963+0000","last_undegraded":"2024-05-08T04:41:07.108963+0000","last_fullsized":"2024-05-08T04:41:07.108963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115911+0000","last_change":"2024-05-08T04:41:07.115911+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115911+0000","last_undegraded":"2024-05-08T04:41:07.115911+0000","last_fullsized":"2024-05-08T04:41:07.115911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104906+0000","last_change":"2024-05-08T04:41:07.104906+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104906+0000","last_undegraded":"2024-05-08T04:41:07.104906+0000","last_fullsized":"2024-05-08T04:41:07.104906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.115370+0000","last_change":"2024-05-08T04:41:07.115370+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.115370+0000","last_undegraded":"2024-05-08T04:41:07.115370+0000","last_fullsized":"2024-05-08T04:41:07.115370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111074+0000","last_change":"2024-05-08T04:41:07.111074+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111074+0000","last_undegraded":"2024-05-08T04:41:07.111074+0000","last_fullsized":"2024-05-08T04:41:07.111074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107667+0000","last_change":"2024-05-08T04:41:07.107667+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107667+0000","last_undegraded":"2024-05-08T04:41:07.107667+0000","last_fullsized":"2024-05-08T04:41:07.107667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111665+0000","last_change":"2024-05-08T04:41:07.111665+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111665+0000","last_undegraded":"2024-05-08T04:41:07.111665+0000","last_fullsized":"2024-05-08T04:41:07.111665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104932+0000","last_change":"2024-05-08T04:41:07.104932+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104932+0000","last_undegraded":"2024-05-08T04:41:07.104932+0000","last_fullsized":"2024-05-08T04:41:07.104932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106039+0000","last_change":"2024-05-08T04:41:07.106039+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106039+0000","last_undegraded":"2024-05-08T04:41:07.106039+0000","last_fullsized":"2024-05-08T04:41:07.106039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107095+0000","last_change":"2024-05-08T04:41:07.107095+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107095+0000","last_undegraded":"2024-05-08T04:41:07.107095+0000","last_fullsized":"2024-05-08T04:41:07.107095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106245+0000","last_change":"2024-05-08T04:41:07.106245+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106245+0000","last_undegraded":"2024-05-08T04:41:07.106245+0000","last_fullsized":"2024-05-08T04:41:07.106245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106478+0000","last_change":"2024-05-08T04:41:07.106478+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106478+0000","last_undegraded":"2024-05-08T04:41:07.106478+0000","last_fullsized":"2024-05-08T04:41:07.106478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110595+0000","last_change":"2024-05-08T04:41:07.110595+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110595+0000","last_undegraded":"2024-05-08T04:41:07.110595+0000","last_fullsized":"2024-05-08T04:41:07.110595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104441+0000","last_change":"2024-05-08T04:41:07.104441+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104441+0000","last_undegraded":"2024-05-08T04:41:07.104441+0000","last_fullsized":"2024-05-08T04:41:07.104441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114231+0000","last_change":"2024-05-08T04:41:07.114231+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114231+0000","last_undegraded":"2024-05-08T04:41:07.114231+0000","last_fullsized":"2024-05-08T04:41:07.114231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105925+0000","last_change":"2024-05-08T04:41:07.105925+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105925+0000","last_undegraded":"2024-05-08T04:41:07.105925+0000","last_fullsized":"2024-05-08T04:41:07.105925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105897+0000","last_change":"2024-05-08T04:41:07.105897+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105897+0000","last_undegraded":"2024-05-08T04:41:07.105897+0000","last_fullsized":"2024-05-08T04:41:07.105897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106157+0000","last_change":"2024-05-08T04:41:07.106157+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106157+0000","last_undegraded":"2024-05-08T04:41:07.106157+0000","last_fullsized":"2024-05-08T04:41:07.106157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112836+0000","last_change":"2024-05-08T04:41:07.112836+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112836+0000","last_undegraded":"2024-05-08T04:41:07.112836+0000","last_fullsized":"2024-05-08T04:41:07.112836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107394+0000","last_change":"2024-05-08T04:41:07.107394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107394+0000","last_undegraded":"2024-05-08T04:41:07.107394+0000","last_fullsized":"2024-05-08T04:41:07.107394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106637+0000","last_change":"2024-05-08T04:41:07.106637+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106637+0000","last_undegraded":"2024-05-08T04:41:07.106637+0000","last_fullsized":"2024-05-08T04:41:07.106637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104164+0000","last_change":"2024-05-08T04:41:07.104164+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104164+0000","last_undegraded":"2024-05-08T04:41:07.104164+0000","last_fullsized":"2024-05-08T04:41:07.104164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108566+0000","last_change":"2024-05-08T04:41:07.108566+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108566+0000","last_undegraded":"2024-05-08T04:41:07.108566+0000","last_fullsized":"2024-05-08T04:41:07.108566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108233+0000","last_change":"2024-05-08T04:41:07.108233+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108233+0000","last_undegraded":"2024-05-08T04:41:07.108233+0000","last_fullsized":"2024-05-08T04:41:07.108233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.113288+0000","last_change":"2024-05-08T04:41:07.113288+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.113288+0000","last_undegraded":"2024-05-08T04:41:07.113288+0000","last_fullsized":"2024-05-08T04:41:07.113288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104159+0000","last_change":"2024-05-08T04:41:07.104159+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104159+0000","last_undegraded":"2024-05-08T04:41:07.104159+0000","last_fullsized":"2024-05-08T04:41:07.104159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103467+0000","last_change":"2024-05-08T04:41:07.103467+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103467+0000","last_undegraded":"2024-05-08T04:41:07.103467+0000","last_fullsized":"2024-05-08T04:41:07.103467+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.114887+0000","last_change":"2024-05-08T04:41:07.114887+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.114887+0000","last_undegraded":"2024-05-08T04:41:07.114887+0000","last_fullsized":"2024-05-08T04:41:07.114887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109849+0000","last_change":"2024-05-08T04:41:07.109849+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109849+0000","last_undegraded":"2024-05-08T04:41:07.109849+0000","last_fullsized":"2024-05-08T04:41:07.109849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.129995+0000","last_change":"2024-05-08T04:41:07.123950+0000","last_active":"2024-05-08T04:41:07.129995+0000","last_peered":"2024-05-08T04:41:07.129995+0000","last_clean":"2024-05-08T04:41:07.129995+0000","last_became_active":"2024-05-08T04:41:07.123577+0000","last_became_peered":"2024-05-08T04:41:07.123577+0000","last_unstale":"2024-05-08T04:41:07.129995+0000","last_undegraded":"2024-05-08T04:41:07.129995+0000","last_fullsized":"2024-05-08T04:41:07.129995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:28.418132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110185+0000","last_change":"2024-05-08T04:41:07.110185+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110185+0000","last_undegraded":"2024-05-08T04:41:07.110185+0000","last_fullsized":"2024-05-08T04:41:07.110185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125062+0000","last_change":"2024-05-08T04:41:07.122634+0000","last_active":"2024-05-08T04:41:07.125062+0000","last_peered":"2024-05-08T04:41:07.125062+0000","last_clean":"2024-05-08T04:41:07.125062+0000","last_became_active":"2024-05-08T04:41:07.122307+0000","last_became_peered":"2024-05-08T04:41:07.122307+0000","last_unstale":"2024-05-08T04:41:07.125062+0000","last_undegraded":"2024-05-08T04:41:07.125062+0000","last_fullsized":"2024-05-08T04:41:07.125062+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:18.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110634+0000","last_change":"2024-05-08T04:41:07.110634+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110634+0000","last_undegraded":"2024-05-08T04:41:07.110634+0000","last_fullsized":"2024-05-08T04:41:07.110634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128519+0000","last_change":"2024-05-08T04:41:07.124366+0000","last_active":"2024-05-08T04:41:07.128519+0000","last_peered":"2024-05-08T04:41:07.128519+0000","last_clean":"2024-05-08T04:41:07.128519+0000","last_became_active":"2024-05-08T04:41:07.124039+0000","last_became_peered":"2024-05-08T04:41:07.124039+0000","last_unstale":"2024-05-08T04:41:07.128519+0000","last_undegraded":"2024-05-08T04:41:07.128519+0000","last_fullsized":"2024-05-08T04:41:07.128519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:24.785925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111227+0000","last_change":"2024-05-08T04:41:07.111227+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111227+0000","last_undegraded":"2024-05-08T04:41:07.111227+0000","last_fullsized":"2024-05-08T04:41:07.111227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128296+0000","last_change":"2024-05-08T04:41:07.126964+0000","last_active":"2024-05-08T04:41:07.128296+0000","last_peered":"2024-05-08T04:41:07.128296+0000","last_clean":"2024-05-08T04:41:07.128296+0000","last_became_active":"2024-05-08T04:41:07.126433+0000","last_became_peered":"2024-05-08T04:41:07.126433+0000","last_unstale":"2024-05-08T04:41:07.128296+0000","last_undegraded":"2024-05-08T04:41:07.128296+0000","last_fullsized":"2024-05-08T04:41:07.128296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103412+0000","last_change":"2024-05-08T04:41:07.103412+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103412+0000","last_undegraded":"2024-05-08T04:41:07.103412+0000","last_fullsized":"2024-05-08T04:41:07.103412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122741+0000","last_change":"2024-05-08T04:41:07.118893+0000","last_active":"2024-05-08T04:41:07.122741+0000","last_peered":"2024-05-08T04:41:07.122741+0000","last_clean":"2024-05-08T04:41:07.122741+0000","last_became_active":"2024-05-08T04:41:07.117180+0000","last_became_peered":"2024-05-08T04:41:07.117180+0000","last_unstale":"2024-05-08T04:41:07.122741+0000","last_undegraded":"2024-05-08T04:41:07.122741+0000","last_fullsized":"2024-05-08T04:41:07.122741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:45.295908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105256+0000","last_change":"2024-05-08T04:41:07.105256+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105256+0000","last_undegraded":"2024-05-08T04:41:07.105256+0000","last_fullsized":"2024-05-08T04:41:07.105256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124504+0000","last_change":"2024-05-08T04:41:07.122025+0000","last_active":"2024-05-08T04:41:07.124504+0000","last_peered":"2024-05-08T04:41:07.124504+0000","last_clean":"2024-05-08T04:41:07.124504+0000","last_became_active":"2024-05-08T04:41:07.120512+0000","last_became_peered":"2024-05-08T04:41:07.120512+0000","last_unstale":"2024-05-08T04:41:07.124504+0000","last_undegraded":"2024-05-08T04:41:07.124504+0000","last_fullsized":"2024-05-08T04:41:07.124504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:27.054889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110394+0000","last_change":"2024-05-08T04:41:07.110394+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110394+0000","last_undegraded":"2024-05-08T04:41:07.110394+0000","last_fullsized":"2024-05-08T04:41:07.110394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121508+0000","last_change":"2024-05-08T04:41:07.115029+0000","last_active":"2024-05-08T04:41:07.121508+0000","last_peered":"2024-05-08T04:41:07.121508+0000","last_clean":"2024-05-08T04:41:07.121508+0000","last_became_active":"2024-05-08T04:41:07.114753+0000","last_became_peered":"2024-05-08T04:41:07.114753+0000","last_unstale":"2024-05-08T04:41:07.121508+0000","last_undegraded":"2024-05-08T04:41:07.121508+0000","last_fullsized":"2024-05-08T04:41:07.121508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105796+0000","last_change":"2024-05-08T04:41:07.105796+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105796+0000","last_undegraded":"2024-05-08T04:41:07.105796+0000","last_fullsized":"2024-05-08T04:41:07.105796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116667+0000","last_change":"2024-05-08T04:41:07.113389+0000","last_active":"2024-05-08T04:41:07.116667+0000","last_peered":"2024-05-08T04:41:07.116667+0000","last_clean":"2024-05-08T04:41:07.116667+0000","last_became_active":"2024-05-08T04:41:07.113100+0000","last_became_peered":"2024-05-08T04:41:07.113100+0000","last_unstale":"2024-05-08T04:41:07.116667+0000","last_undegraded":"2024-05-08T04:41:07.116667+0000","last_fullsized":"2024-05-08T04:41:07.116667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:13.388255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.103521+0000","last_change":"2024-05-08T04:41:07.103521+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.103521+0000","last_undegraded":"2024-05-08T04:41:07.103521+0000","last_fullsized":"2024-05-08T04:41:07.103521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.116039+0000","last_change":"2024-05-08T04:41:07.113877+0000","last_active":"2024-05-08T04:41:07.116039+0000","last_peered":"2024-05-08T04:41:07.116039+0000","last_clean":"2024-05-08T04:41:07.116039+0000","last_became_active":"2024-05-08T04:41:07.112924+0000","last_became_peered":"2024-05-08T04:41:07.112924+0000","last_unstale":"2024-05-08T04:41:07.116039+0000","last_undegraded":"2024-05-08T04:41:07.116039+0000","last_fullsized":"2024-05-08T04:41:07.116039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:33.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110219+0000","last_change":"2024-05-08T04:41:07.110219+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110219+0000","last_undegraded":"2024-05-08T04:41:07.110219+0000","last_fullsized":"2024-05-08T04:41:07.110219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130088+0000","last_change":"2024-05-08T04:41:07.127217+0000","last_active":"2024-05-08T04:41:07.130088+0000","last_peered":"2024-05-08T04:41:07.130088+0000","last_clean":"2024-05-08T04:41:07.130088+0000","last_became_active":"2024-05-08T04:41:07.126634+0000","last_became_peered":"2024-05-08T04:41:07.126634+0000","last_unstale":"2024-05-08T04:41:07.130088+0000","last_undegraded":"2024-05-08T04:41:07.130088+0000","last_fullsized":"2024-05-08T04:41:07.130088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:24.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110012+0000","last_change":"2024-05-08T04:41:07.110012+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110012+0000","last_undegraded":"2024-05-08T04:41:07.110012+0000","last_fullsized":"2024-05-08T04:41:07.110012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124534+0000","last_change":"2024-05-08T04:41:07.120447+0000","last_active":"2024-05-08T04:41:07.124534+0000","last_peered":"2024-05-08T04:41:07.124534+0000","last_clean":"2024-05-08T04:41:07.124534+0000","last_became_active":"2024-05-08T04:41:07.120185+0000","last_became_peered":"2024-05-08T04:41:07.120185+0000","last_unstale":"2024-05-08T04:41:07.124534+0000","last_undegraded":"2024-05-08T04:41:07.124534+0000","last_fullsized":"2024-05-08T04:41:07.124534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:22.105771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109383+0000","last_change":"2024-05-08T04:41:07.109383+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109383+0000","last_undegraded":"2024-05-08T04:41:07.109383+0000","last_fullsized":"2024-05-08T04:41:07.109383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.120864+0000","last_change":"2024-05-08T04:41:07.114558+0000","last_active":"2024-05-08T04:41:07.120864+0000","last_peered":"2024-05-08T04:41:07.120864+0000","last_clean":"2024-05-08T04:41:07.120864+0000","last_became_active":"2024-05-08T04:41:07.114273+0000","last_became_peered":"2024-05-08T04:41:07.114273+0000","last_unstale":"2024-05-08T04:41:07.120864+0000","last_undegraded":"2024-05-08T04:41:07.120864+0000","last_fullsized":"2024-05-08T04:41:07.120864+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.112472+0000","last_change":"2024-05-08T04:41:07.112472+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.112472+0000","last_undegraded":"2024-05-08T04:41:07.112472+0000","last_fullsized":"2024-05-08T04:41:07.112472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122504+0000","last_change":"2024-05-08T04:41:07.119807+0000","last_active":"2024-05-08T04:41:07.122504+0000","last_peered":"2024-05-08T04:41:07.122504+0000","last_clean":"2024-05-08T04:41:07.122504+0000","last_became_active":"2024-05-08T04:41:07.119549+0000","last_became_peered":"2024-05-08T04:41:07.119549+0000","last_unstale":"2024-05-08T04:41:07.122504+0000","last_undegraded":"2024-05-08T04:41:07.122504+0000","last_fullsized":"2024-05-08T04:41:07.122504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104512+0000","last_change":"2024-05-08T04:41:07.104512+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104512+0000","last_undegraded":"2024-05-08T04:41:07.104512+0000","last_fullsized":"2024-05-08T04:41:07.104512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.122180+0000","last_change":"2024-05-08T04:41:07.118440+0000","last_active":"2024-05-08T04:41:07.122180+0000","last_peered":"2024-05-08T04:41:07.122180+0000","last_clean":"2024-05-08T04:41:07.122180+0000","last_became_active":"2024-05-08T04:41:07.118074+0000","last_became_peered":"2024-05-08T04:41:07.118074+0000","last_unstale":"2024-05-08T04:41:07.122180+0000","last_undegraded":"2024-05-08T04:41:07.122180+0000","last_fullsized":"2024-05-08T04:41:07.122180+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:05.318932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105487+0000","last_change":"2024-05-08T04:41:07.105487+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105487+0000","last_undegraded":"2024-05-08T04:41:07.105487+0000","last_fullsized":"2024-05-08T04:41:07.105487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117862+0000","last_change":"2024-05-08T04:41:07.111672+0000","last_active":"2024-05-08T04:41:07.117862+0000","last_peered":"2024-05-08T04:41:07.117862+0000","last_clean":"2024-05-08T04:41:07.117862+0000","last_became_active":"2024-05-08T04:41:07.110877+0000","last_became_peered":"2024-05-08T04:41:07.110877+0000","last_unstale":"2024-05-08T04:41:07.117862+0000","last_undegraded":"2024-05-08T04:41:07.117862+0000","last_fullsized":"2024-05-08T04:41:07.117862+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:58.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107314+0000","last_change":"2024-05-08T04:41:07.107314+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107314+0000","last_undegraded":"2024-05-08T04:41:07.107314+0000","last_fullsized":"2024-05-08T04:41:07.107314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125413+0000","last_change":"2024-05-08T04:41:07.120313+0000","last_active":"2024-05-08T04:41:07.125413+0000","last_peered":"2024-05-08T04:41:07.125413+0000","last_clean":"2024-05-08T04:41:07.125413+0000","last_became_active":"2024-05-08T04:41:07.120022+0000","last_became_peered":"2024-05-08T04:41:07.120022+0000","last_unstale":"2024-05-08T04:41:07.125413+0000","last_undegraded":"2024-05-08T04:41:07.125413+0000","last_fullsized":"2024-05-08T04:41:07.125413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:13.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107179+0000","last_change":"2024-05-08T04:41:07.107179+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107179+0000","last_undegraded":"2024-05-08T04:41:07.107179+0000","last_fullsized":"2024-05-08T04:41:07.107179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130756+0000","last_change":"2024-05-08T04:41:07.128722+0000","last_active":"2024-05-08T04:41:07.130756+0000","last_peered":"2024-05-08T04:41:07.130756+0000","last_clean":"2024-05-08T04:41:07.130756+0000","last_became_active":"2024-05-08T04:41:07.128152+0000","last_became_peered":"2024-05-08T04:41:07.128152+0000","last_unstale":"2024-05-08T04:41:07.130756+0000","last_undegraded":"2024-05-08T04:41:07.130756+0000","last_fullsized":"2024-05-08T04:41:07.130756+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:59.449713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109605+0000","last_change":"2024-05-08T04:41:07.109605+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109605+0000","last_undegraded":"2024-05-08T04:41:07.109605+0000","last_fullsized":"2024-05-08T04:41:07.109605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128615+0000","last_change":"2024-05-08T04:41:07.124431+0000","last_active":"2024-05-08T04:41:07.128615+0000","last_peered":"2024-05-08T04:41:07.128615+0000","last_clean":"2024-05-08T04:41:07.128615+0000","last_became_active":"2024-05-08T04:41:07.124047+0000","last_became_peered":"2024-05-08T04:41:07.124047+0000","last_unstale":"2024-05-08T04:41:07.128615+0000","last_undegraded":"2024-05-08T04:41:07.128615+0000","last_fullsized":"2024-05-08T04:41:07.128615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:44.583544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104470+0000","last_change":"2024-05-08T04:41:07.104470+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104470+0000","last_undegraded":"2024-05-08T04:41:07.104470+0000","last_fullsized":"2024-05-08T04:41:07.104470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128375+0000","last_change":"2024-05-08T04:41:07.126275+0000","last_active":"2024-05-08T04:41:07.128375+0000","last_peered":"2024-05-08T04:41:07.128375+0000","last_clean":"2024-05-08T04:41:07.128375+0000","last_became_active":"2024-05-08T04:41:07.125938+0000","last_became_peered":"2024-05-08T04:41:07.125938+0000","last_unstale":"2024-05-08T04:41:07.128375+0000","last_undegraded":"2024-05-08T04:41:07.128375+0000","last_fullsized":"2024-05-08T04:41:07.128375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:14.213320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108987+0000","last_change":"2024-05-08T04:41:07.108987+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108987+0000","last_undegraded":"2024-05-08T04:41:07.108987+0000","last_fullsized":"2024-05-08T04:41:07.108987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.130586+0000","last_change":"2024-05-08T04:41:07.129166+0000","last_active":"2024-05-08T04:41:07.130586+0000","last_peered":"2024-05-08T04:41:07.130586+0000","last_clean":"2024-05-08T04:41:07.130586+0000","last_became_active":"2024-05-08T04:41:07.128785+0000","last_became_peered":"2024-05-08T04:41:07.128785+0000","last_unstale":"2024-05-08T04:41:07.130586+0000","last_undegraded":"2024-05-08T04:41:07.130586+0000","last_fullsized":"2024-05-08T04:41:07.130586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:53.881464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.111401+0000","last_change":"2024-05-08T04:41:07.111401+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.111401+0000","last_undegraded":"2024-05-08T04:41:07.111401+0000","last_fullsized":"2024-05-08T04:41:07.111401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124524+0000","last_change":"2024-05-08T04:41:07.122473+0000","last_active":"2024-05-08T04:41:07.124524+0000","last_peered":"2024-05-08T04:41:07.124524+0000","last_clean":"2024-05-08T04:41:07.124524+0000","last_became_active":"2024-05-08T04:41:07.122179+0000","last_became_peered":"2024-05-08T04:41:07.122179+0000","last_unstale":"2024-05-08T04:41:07.124524+0000","last_undegraded":"2024-05-08T04:41:07.124524+0000","last_fullsized":"2024-05-08T04:41:07.124524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:58.656913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.107497+0000","last_change":"2024-05-08T04:41:07.107497+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.107497+0000","last_undegraded":"2024-05-08T04:41:07.107497+0000","last_fullsized":"2024-05-08T04:41:07.107497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.123393+0000","last_change":"2024-05-08T04:41:07.119592+0000","last_active":"2024-05-08T04:41:07.123393+0000","last_peered":"2024-05-08T04:41:07.123393+0000","last_clean":"2024-05-08T04:41:07.123393+0000","last_became_active":"2024-05-08T04:41:07.119253+0000","last_became_peered":"2024-05-08T04:41:07.119253+0000","last_unstale":"2024-05-08T04:41:07.123393+0000","last_undegraded":"2024-05-08T04:41:07.123393+0000","last_fullsized":"2024-05-08T04:41:07.123393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108207+0000","last_change":"2024-05-08T04:41:07.108207+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108207+0000","last_undegraded":"2024-05-08T04:41:07.108207+0000","last_fullsized":"2024-05-08T04:41:07.108207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.119190+0000","last_change":"2024-05-08T04:41:07.118353+0000","last_active":"2024-05-08T04:41:07.119190+0000","last_peered":"2024-05-08T04:41:07.119190+0000","last_clean":"2024-05-08T04:41:07.119190+0000","last_became_active":"2024-05-08T04:41:07.118022+0000","last_became_peered":"2024-05-08T04:41:07.118022+0000","last_unstale":"2024-05-08T04:41:07.119190+0000","last_undegraded":"2024-05-08T04:41:07.119190+0000","last_fullsized":"2024-05-08T04:41:07.119190+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:57.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.106673+0000","last_change":"2024-05-08T04:41:07.106673+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.106673+0000","last_undegraded":"2024-05-08T04:41:07.106673+0000","last_fullsized":"2024-05-08T04:41:07.106673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.128901+0000","last_change":"2024-05-08T04:41:07.125972+0000","last_active":"2024-05-08T04:41:07.128901+0000","last_peered":"2024-05-08T04:41:07.128901+0000","last_clean":"2024-05-08T04:41:07.128901+0000","last_became_active":"2024-05-08T04:41:07.125631+0000","last_became_peered":"2024-05-08T04:41:07.125631+0000","last_unstale":"2024-05-08T04:41:07.128901+0000","last_undegraded":"2024-05-08T04:41:07.128901+0000","last_fullsized":"2024-05-08T04:41:07.128901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:26.436957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.104255+0000","last_change":"2024-05-08T04:41:07.104255+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.104255+0000","last_undegraded":"2024-05-08T04:41:07.104255+0000","last_fullsized":"2024-05-08T04:41:07.104255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.117933+0000","last_change":"2024-05-08T04:41:07.115163+0000","last_active":"2024-05-08T04:41:07.117933+0000","last_peered":"2024-05-08T04:41:07.117933+0000","last_clean":"2024-05-08T04:41:07.117933+0000","last_became_active":"2024-05-08T04:41:07.114516+0000","last_became_peered":"2024-05-08T04:41:07.114516+0000","last_unstale":"2024-05-08T04:41:07.117933+0000","last_undegraded":"2024-05-08T04:41:07.117933+0000","last_fullsized":"2024-05-08T04:41:07.117933+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:55.546865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.108488+0000","last_change":"2024-05-08T04:41:07.108488+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.108488+0000","last_undegraded":"2024-05-08T04:41:07.108488+0000","last_fullsized":"2024-05-08T04:41:07.108488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127601+0000","last_change":"2024-05-08T04:41:07.123483+0000","last_active":"2024-05-08T04:41:07.127601+0000","last_peered":"2024-05-08T04:41:07.127601+0000","last_clean":"2024-05-08T04:41:07.127601+0000","last_became_active":"2024-05-08T04:41:07.121115+0000","last_became_peered":"2024-05-08T04:41:07.121115+0000","last_unstale":"2024-05-08T04:41:07.127601+0000","last_undegraded":"2024-05-08T04:41:07.127601+0000","last_fullsized":"2024-05-08T04:41:07.127601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:24.593215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109534+0000","last_change":"2024-05-08T04:41:07.109534+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109534+0000","last_undegraded":"2024-05-08T04:41:07.109534+0000","last_fullsized":"2024-05-08T04:41:07.109534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121479+0000","last_change":"2024-05-08T04:41:07.120042+0000","last_active":"2024-05-08T04:41:07.121479+0000","last_peered":"2024-05-08T04:41:07.121479+0000","last_clean":"2024-05-08T04:41:07.121479+0000","last_became_active":"2024-05-08T04:41:07.119795+0000","last_became_peered":"2024-05-08T04:41:07.119795+0000","last_unstale":"2024-05-08T04:41:07.121479+0000","last_undegraded":"2024-05-08T04:41:07.121479+0000","last_fullsized":"2024-05-08T04:41:07.121479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:16.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105368+0000","last_change":"2024-05-08T04:41:07.105368+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105368+0000","last_undegraded":"2024-05-08T04:41:07.105368+0000","last_fullsized":"2024-05-08T04:41:07.105368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125386+0000","last_change":"2024-05-08T04:41:07.122490+0000","last_active":"2024-05-08T04:41:07.125386+0000","last_peered":"2024-05-08T04:41:07.125386+0000","last_clean":"2024-05-08T04:41:07.125386+0000","last_became_active":"2024-05-08T04:41:07.122220+0000","last_became_peered":"2024-05-08T04:41:07.122220+0000","last_unstale":"2024-05-08T04:41:07.125386+0000","last_undegraded":"2024-05-08T04:41:07.125386+0000","last_fullsized":"2024-05-08T04:41:07.125386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:01:04.274102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110538+0000","last_change":"2024-05-08T04:41:07.110538+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110538+0000","last_undegraded":"2024-05-08T04:41:07.110538+0000","last_fullsized":"2024-05-08T04:41:07.110538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.121376+0000","last_change":"2024-05-08T04:41:07.114940+0000","last_active":"2024-05-08T04:41:07.121376+0000","last_peered":"2024-05-08T04:41:07.121376+0000","last_clean":"2024-05-08T04:41:07.121376+0000","last_became_active":"2024-05-08T04:41:07.114663+0000","last_became_peered":"2024-05-08T04:41:07.114663+0000","last_unstale":"2024-05-08T04:41:07.121376+0000","last_undegraded":"2024-05-08T04:41:07.121376+0000","last_fullsized":"2024-05-08T04:41:07.121376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:30.539334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.105714+0000","last_change":"2024-05-08T04:41:07.105714+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.105714+0000","last_undegraded":"2024-05-08T04:41:07.105714+0000","last_fullsized":"2024-05-08T04:41:07.105714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.125304+0000","last_change":"2024-05-08T04:41:07.121852+0000","last_active":"2024-05-08T04:41:07.125304+0000","last_peered":"2024-05-08T04:41:07.125304+0000","last_clean":"2024-05-08T04:41:07.125304+0000","last_became_active":"2024-05-08T04:41:07.121549+0000","last_became_peered":"2024-05-08T04:41:07.121549+0000","last_unstale":"2024-05-08T04:41:07.125304+0000","last_undegraded":"2024-05-08T04:41:07.125304+0000","last_fullsized":"2024-05-08T04:41:07.125304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:20.842180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.110425+0000","last_change":"2024-05-08T04:41:07.110425+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.110425+0000","last_undegraded":"2024-05-08T04:41:07.110425+0000","last_fullsized":"2024-05-08T04:41:07.110425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.127812+0000","last_change":"2024-05-08T04:41:07.125169+0000","last_active":"2024-05-08T04:41:07.127812+0000","last_peered":"2024-05-08T04:41:07.127812+0000","last_clean":"2024-05-08T04:41:07.127812+0000","last_became_active":"2024-05-08T04:41:07.124870+0000","last_became_peered":"2024-05-08T04:41:07.124870+0000","last_unstale":"2024-05-08T04:41:07.127812+0000","last_undegraded":"2024-05-08T04:41:07.127812+0000","last_fullsized":"2024-05-08T04:41:07.127812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:33.327374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-08T04:41:07.109871+0000","last_change":"2024-05-08T04:41:07.109871+0000","last_active":"2024-05-08T04:41:07.092845+0000","last_peered":"2024-05-08T04:41:07.092845+0000","last_clean":"2024-05-08T04:41:07.092845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:07.109871+0000","last_undegraded":"2024-05-08T04:41:07.109871+0000","last_fullsized":"2024-05-08T04:41:07.109871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.124263+0000","last_change":"2024-05-08T04:41:07.121887+0000","last_active":"2024-05-08T04:41:07.124263+0000","last_peered":"2024-05-08T04:41:07.124263+0000","last_clean":"2024-05-08T04:41:07.124263+0000","last_became_active":"2024-05-08T04:41:07.121407+0000","last_became_peered":"2024-05-08T04:41:07.121407+0000","last_unstale":"2024-05-08T04:41:07.124263+0000","last_undegraded":"2024-05-08T04:41:07.124263+0000","last_fullsized":"2024-05-08T04:41:07.124263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:09.012810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-08T04:41:07.113889+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:07.113889+0000","last_peered":"2024-05-08T04:41:07.113889+0000","last_clean":"2024-05-08T04:41:07.113889+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:07.113889+0000","last_undegraded":"2024-05-08T04:41:07.113889+0000","last_fullsized":"2024-05-08T04:41:07.113889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":370683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":961051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:12.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-05-08T04:41:12.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110592+0000', 'last_change': '2024-05-08T04:41:07.110592+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110592+0000', 'last_undegraded': '2024-05-08T04:41:07.110592+0000', 'last_fullsized': '2024-05-08T04:41:07.110592+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.116717+0000', 'last_change': '2024-05-08T04:41:07.116717+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.116717+0000', 'last_undegraded': '2024-05-08T04:41:07.116717+0000', 'last_fullsized': '2024-05-08T04:41:07.116717+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.111388+0000', 'last_change': '2024-05-08T04:41:07.111388+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.111388+0000', 'last_undegraded': '2024-05-08T04:41:07.111388+0000', 'last_fullsized': '2024-05-08T04:41:07.111388+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105294+0000', 'last_change': '2024-05-08T04:41:07.105294+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105294+0000', 'last_undegraded': '2024-05-08T04:41:07.105294+0000', 'last_fullsized': '2024-05-08T04:41:07.105294+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108963+0000', 'last_change': '2024-05-08T04:41:07.108963+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108963+0000', 'last_undegraded': '2024-05-08T04:41:07.108963+0000', 'last_fullsized': '2024-05-08T04:41:07.108963+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.115911+0000', 'last_change': '2024-05-08T04:41:07.115911+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.115911+0000', 'last_undegraded': '2024-05-08T04:41:07.115911+0000', 'last_fullsized': '2024-05-08T04:41:07.115911+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-05-08T04:41:12.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104906+0000', 'last_change': '2024-05-08T04:41:07.104906+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104906+0000', 'last_undegraded': '2024-05-08T04:41:07.104906+0000', 'last_fullsized': '2024-05-08T04:41:07.104906+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.115370+0000', 'last_change': '2024-05-08T04:41:07.115370+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.115370+0000', 'last_undegraded': '2024-05-08T04:41:07.115370+0000', 'last_fullsized': '2024-05-08T04:41:07.115370+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.111074+0000', 'last_change': '2024-05-08T04:41:07.111074+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.111074+0000', 'last_undegraded': '2024-05-08T04:41:07.111074+0000', 'last_fullsized': '2024-05-08T04:41:07.111074+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107667+0000', 'last_change': '2024-05-08T04:41:07.107667+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107667+0000', 'last_undegraded': '2024-05-08T04:41:07.107667+0000', 'last_fullsized': '2024-05-08T04:41:07.107667+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.111665+0000', 'last_change': '2024-05-08T04:41:07.111665+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.111665+0000', 'last_undegraded': '2024-05-08T04:41:07.111665+0000', 'last_fullsized': '2024-05-08T04:41:07.111665+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104932+0000', 'last_change': '2024-05-08T04:41:07.104932+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104932+0000', 'last_undegraded': '2024-05-08T04:41:07.104932+0000', 'last_fullsized': '2024-05-08T04:41:07.104932+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-05-08T04:41:12.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106039+0000', 'last_change': '2024-05-08T04:41:07.106039+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106039+0000', 'last_undegraded': '2024-05-08T04:41:07.106039+0000', 'last_fullsized': '2024-05-08T04:41:07.106039+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107095+0000', 'last_change': '2024-05-08T04:41:07.107095+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107095+0000', 'last_undegraded': '2024-05-08T04:41:07.107095+0000', 'last_fullsized': '2024-05-08T04:41:07.107095+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106245+0000', 'last_change': '2024-05-08T04:41:07.106245+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106245+0000', 'last_undegraded': '2024-05-08T04:41:07.106245+0000', 'last_fullsized': '2024-05-08T04:41:07.106245+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106478+0000', 'last_change': '2024-05-08T04:41:07.106478+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106478+0000', 'last_undegraded': '2024-05-08T04:41:07.106478+0000', 'last_fullsized': '2024-05-08T04:41:07.106478+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110595+0000', 'last_change': '2024-05-08T04:41:07.110595+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110595+0000', 'last_undegraded': '2024-05-08T04:41:07.110595+0000', 'last_fullsized': '2024-05-08T04:41:07.110595+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104441+0000', 'last_change': '2024-05-08T04:41:07.104441+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104441+0000', 'last_undegraded': '2024-05-08T04:41:07.104441+0000', 'last_fullsized': '2024-05-08T04:41:07.104441+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:12.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.114231+0000', 'last_change': '2024-05-08T04:41:07.114231+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.114231+0000', 'last_undegraded': '2024-05-08T04:41:07.114231+0000', 'last_fullsized': '2024-05-08T04:41:07.114231+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105925+0000', 'last_change': '2024-05-08T04:41:07.105925+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105925+0000', 'last_undegraded': '2024-05-08T04:41:07.105925+0000', 'last_fullsized': '2024-05-08T04:41:07.105925+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105897+0000', 'last_change': '2024-05-08T04:41:07.105897+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105897+0000', 'last_undegraded': '2024-05-08T04:41:07.105897+0000', 'last_fullsized': '2024-05-08T04:41:07.105897+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106157+0000', 'last_change': '2024-05-08T04:41:07.106157+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106157+0000', 'last_undegraded': '2024-05-08T04:41:07.106157+0000', 'last_fullsized': '2024-05-08T04:41:07.106157+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.112836+0000', 'last_change': '2024-05-08T04:41:07.112836+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.112836+0000', 'last_undegraded': '2024-05-08T04:41:07.112836+0000', 'last_fullsized': '2024-05-08T04:41:07.112836+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-05-08T04:41:12.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107394+0000', 'last_change': '2024-05-08T04:41:07.107394+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107394+0000', 'last_undegraded': '2024-05-08T04:41:07.107394+0000', 'last_fullsized': '2024-05-08T04:41:07.107394+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106637+0000', 'last_change': '2024-05-08T04:41:07.106637+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106637+0000', 'last_undegraded': '2024-05-08T04:41:07.106637+0000', 'last_fullsized': '2024-05-08T04:41:07.106637+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104164+0000', 'last_change': '2024-05-08T04:41:07.104164+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104164+0000', 'last_undegraded': '2024-05-08T04:41:07.104164+0000', 'last_fullsized': '2024-05-08T04:41:07.104164+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108566+0000', 'last_change': '2024-05-08T04:41:07.108566+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108566+0000', 'last_undegraded': '2024-05-08T04:41:07.108566+0000', 'last_fullsized': '2024-05-08T04:41:07.108566+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108233+0000', 'last_change': '2024-05-08T04:41:07.108233+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108233+0000', 'last_undegraded': '2024-05-08T04:41:07.108233+0000', 'last_fullsized': '2024-05-08T04:41:07.108233+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.113288+0000', 'last_change': '2024-05-08T04:41:07.113288+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.113288+0000', 'last_undegraded': '2024-05-08T04:41:07.113288+0000', 'last_fullsized': '2024-05-08T04:41:07.113288+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-05-08T04:41:12.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104159+0000', 'last_change': '2024-05-08T04:41:07.104159+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104159+0000', 'last_undegraded': '2024-05-08T04:41:07.104159+0000', 'last_fullsized': '2024-05-08T04:41:07.104159+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.103467+0000', 'last_change': '2024-05-08T04:41:07.103467+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.103467+0000', 'last_undegraded': '2024-05-08T04:41:07.103467+0000', 'last_fullsized': '2024-05-08T04:41:07.103467+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.114887+0000', 'last_change': '2024-05-08T04:41:07.114887+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.114887+0000', 'last_undegraded': '2024-05-08T04:41:07.114887+0000', 'last_fullsized': '2024-05-08T04:41:07.114887+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.109849+0000', 'last_change': '2024-05-08T04:41:07.109849+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.109849+0000', 'last_undegraded': '2024-05-08T04:41:07.109849+0000', 'last_fullsized': '2024-05-08T04:41:07.109849+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110185+0000', 'last_change': '2024-05-08T04:41:07.110185+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110185+0000', 'last_undegraded': '2024-05-08T04:41:07.110185+0000', 'last_fullsized': '2024-05-08T04:41:07.110185+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110634+0000', 'last_change': '2024-05-08T04:41:07.110634+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110634+0000', 'last_undegraded': '2024-05-08T04:41:07.110634+0000', 'last_fullsized': '2024-05-08T04:41:07.110634+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.111227+0000', 'last_change': '2024-05-08T04:41:07.111227+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.111227+0000', 'last_undegraded': '2024-05-08T04:41:07.111227+0000', 'last_fullsized': '2024-05-08T04:41:07.111227+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.103412+0000', 'last_change': '2024-05-08T04:41:07.103412+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.103412+0000', 'last_undegraded': '2024-05-08T04:41:07.103412+0000', 'last_fullsized': '2024-05-08T04:41:07.103412+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105256+0000', 'last_change': '2024-05-08T04:41:07.105256+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105256+0000', 'last_undegraded': '2024-05-08T04:41:07.105256+0000', 'last_fullsized': '2024-05-08T04:41:07.105256+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110394+0000', 'last_change': '2024-05-08T04:41:07.110394+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110394+0000', 'last_undegraded': '2024-05-08T04:41:07.110394+0000', 'last_fullsized': '2024-05-08T04:41:07.110394+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105796+0000', 'last_change': '2024-05-08T04:41:07.105796+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105796+0000', 'last_undegraded': '2024-05-08T04:41:07.105796+0000', 'last_fullsized': '2024-05-08T04:41:07.105796+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-05-08T04:41:12.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.103521+0000', 'last_change': '2024-05-08T04:41:07.103521+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.103521+0000', 'last_undegraded': '2024-05-08T04:41:07.103521+0000', 'last_fullsized': '2024-05-08T04:41:07.103521+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110219+0000', 'last_change': '2024-05-08T04:41:07.110219+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110219+0000', 'last_undegraded': '2024-05-08T04:41:07.110219+0000', 'last_fullsized': '2024-05-08T04:41:07.110219+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110012+0000', 'last_change': '2024-05-08T04:41:07.110012+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110012+0000', 'last_undegraded': '2024-05-08T04:41:07.110012+0000', 'last_fullsized': '2024-05-08T04:41:07.110012+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.109383+0000', 'last_change': '2024-05-08T04:41:07.109383+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.109383+0000', 'last_undegraded': '2024-05-08T04:41:07.109383+0000', 'last_fullsized': '2024-05-08T04:41:07.109383+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.112472+0000', 'last_change': '2024-05-08T04:41:07.112472+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.112472+0000', 'last_undegraded': '2024-05-08T04:41:07.112472+0000', 'last_fullsized': '2024-05-08T04:41:07.112472+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104512+0000', 'last_change': '2024-05-08T04:41:07.104512+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104512+0000', 'last_undegraded': '2024-05-08T04:41:07.104512+0000', 'last_fullsized': '2024-05-08T04:41:07.104512+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105487+0000', 'last_change': '2024-05-08T04:41:07.105487+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105487+0000', 'last_undegraded': '2024-05-08T04:41:07.105487+0000', 'last_fullsized': '2024-05-08T04:41:07.105487+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107314+0000', 'last_change': '2024-05-08T04:41:07.107314+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107314+0000', 'last_undegraded': '2024-05-08T04:41:07.107314+0000', 'last_fullsized': '2024-05-08T04:41:07.107314+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107179+0000', 'last_change': '2024-05-08T04:41:07.107179+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107179+0000', 'last_undegraded': '2024-05-08T04:41:07.107179+0000', 'last_fullsized': '2024-05-08T04:41:07.107179+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.109605+0000', 'last_change': '2024-05-08T04:41:07.109605+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.109605+0000', 'last_undegraded': '2024-05-08T04:41:07.109605+0000', 'last_fullsized': '2024-05-08T04:41:07.109605+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104470+0000', 'last_change': '2024-05-08T04:41:07.104470+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104470+0000', 'last_undegraded': '2024-05-08T04:41:07.104470+0000', 'last_fullsized': '2024-05-08T04:41:07.104470+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108987+0000', 'last_change': '2024-05-08T04:41:07.108987+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108987+0000', 'last_undegraded': '2024-05-08T04:41:07.108987+0000', 'last_fullsized': '2024-05-08T04:41:07.108987+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.111401+0000', 'last_change': '2024-05-08T04:41:07.111401+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.111401+0000', 'last_undegraded': '2024-05-08T04:41:07.111401+0000', 'last_fullsized': '2024-05-08T04:41:07.111401+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.107497+0000', 'last_change': '2024-05-08T04:41:07.107497+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.107497+0000', 'last_undegraded': '2024-05-08T04:41:07.107497+0000', 'last_fullsized': '2024-05-08T04:41:07.107497+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108207+0000', 'last_change': '2024-05-08T04:41:07.108207+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108207+0000', 'last_undegraded': '2024-05-08T04:41:07.108207+0000', 'last_fullsized': '2024-05-08T04:41:07.108207+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.106673+0000', 'last_change': '2024-05-08T04:41:07.106673+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.106673+0000', 'last_undegraded': '2024-05-08T04:41:07.106673+0000', 'last_fullsized': '2024-05-08T04:41:07.106673+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.104255+0000', 'last_change': '2024-05-08T04:41:07.104255+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.104255+0000', 'last_undegraded': '2024-05-08T04:41:07.104255+0000', 'last_fullsized': '2024-05-08T04:41:07.104255+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-05-08T04:41:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.108488+0000', 'last_change': '2024-05-08T04:41:07.108488+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.108488+0000', 'last_undegraded': '2024-05-08T04:41:07.108488+0000', 'last_fullsized': '2024-05-08T04:41:07.108488+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.109534+0000', 'last_change': '2024-05-08T04:41:07.109534+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.109534+0000', 'last_undegraded': '2024-05-08T04:41:07.109534+0000', 'last_fullsized': '2024-05-08T04:41:07.109534+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105368+0000', 'last_change': '2024-05-08T04:41:07.105368+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105368+0000', 'last_undegraded': '2024-05-08T04:41:07.105368+0000', 'last_fullsized': '2024-05-08T04:41:07.105368+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110538+0000', 'last_change': '2024-05-08T04:41:07.110538+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110538+0000', 'last_undegraded': '2024-05-08T04:41:07.110538+0000', 'last_fullsized': '2024-05-08T04:41:07.110538+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.105714+0000', 'last_change': '2024-05-08T04:41:07.105714+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.105714+0000', 'last_undegraded': '2024-05-08T04:41:07.105714+0000', 'last_fullsized': '2024-05-08T04:41:07.105714+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.110425+0000', 'last_change': '2024-05-08T04:41:07.110425+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.110425+0000', 'last_undegraded': '2024-05-08T04:41:07.110425+0000', 'last_fullsized': '2024-05-08T04:41:07.110425+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:12.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-05-08T04:41:12.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:07.109871+0000', 'last_change': '2024-05-08T04:41:07.109871+0000', 'last_active': '2024-05-08T04:41:07.092845+0000', 'last_peered': '2024-05-08T04:41:07.092845+0000', 'last_clean': '2024-05-08T04:41:07.092845+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:07.109871+0000', 'last_undegraded': '2024-05-08T04:41:07.109871+0000', 'last_fullsized': '2024-05-08T04:41:07.109871+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:07.092845+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:12.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T04:41:13.081 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-05-08T04:41:13.076+0000 7f2b45400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:41:13.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 Processor -- start 2024-05-08T04:41:13.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 -- start start 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5ac15ab60 0x7fd5ac072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac072640 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac06c770 con 0x7fd5ac0b26c0 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.124+0000 7fd5b1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ac06ca50 con 0x7fd5ac15ab60 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5ab400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd5ac0cfda0 0x7fd5ac0cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59832/0 (socket says 172.21.15.164:59832) 2024-05-08T04:41:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5ab400640 1 -- 172.21.15.164:0/583450601 learned_addr learned my addr 172.21.15.164:0/583450601 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:13.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aaa00640 1 --2- 172.21.15.164:0/583450601 >> v2:172.21.15.172:3300/0 conn(0x7fd5ac15ab60 0x7fd5ac072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086798787 0 0) 0x7fd5ac072640 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd590003530 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050663525 0 0) 0x7fd5ac06c770 con 0x7fd5ac0b26c0 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ac072640 con 0x7fd5ac0b26c0 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aaa00640 1 -- 172.21.15.164:0/583450601 >> v1:172.21.15.172:6789/0 conn(0x7fd5ac0b26c0 legacy=0x7fd5ac0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2107238427 0 0) 0x7fd590003530 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aaa00640 1 -- 172.21.15.164:0/583450601 >> v1:172.21.15.164:6789/0 conn(0x7fd5ac0cfda0 legacy=0x7fd5ac0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aaa00640 1 -- 172.21.15.164:0/583450601 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac06cd30 con 0x7fd5ac15ab60 2024-05-08T04:41:13.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.128+0000 7fd5aaa00640 1 --2- 172.21.15.164:0/583450601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac072100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0097c0 tx=0x7fd59c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704bcda74027e45a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0b0070 con 0x7fd5ac15ab60 2024-05-08T04:41:13.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c0ab400 con 0x7fd5ac15ab60 2024-05-08T04:41:13.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5aa000640 1 -- 172.21.15.164:0/583450601 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4001a00 con 0x7fd5ac15ab60 2024-05-08T04:41:13.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- 172.21.15.164:0/583450601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 msgr2=0x7fd5ac072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 --2- 172.21.15.164:0/583450601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac072100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0097c0 tx=0x7fd59c01c3c0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- 172.21.15.164:0/583450601 shutdown_connections 2024-05-08T04:41:13.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 --2- 172.21.15.164:0/583450601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac072100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- 172.21.15.164:0/583450601 >> 172.21.15.164:0/583450601 conn(0x7fd5ac06ac90 msgr2=0x7fd5ac06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- 172.21.15.164:0/583450601 shutdown_connections 2024-05-08T04:41:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- 172.21.15.164:0/583450601 wait complete. 2024-05-08T04:41:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 Processor -- start 2024-05-08T04:41:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- start start 2024-05-08T04:41:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0b26c0 0x7fd5ac151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0aeeb0 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0aed30 con 0x7fd5ac15ab60 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5b1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac168980 con 0x7fd5ac0b26c0 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41834/0 (socket says 172.21.15.164:41834) 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5a2000640 1 -- 172.21.15.164:0/3335134194 learned_addr learned my addr 172.21.15.164:0/3335134194 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5aaa00640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0b26c0 0x7fd5ac151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.132+0000 7fd5ab400640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 -- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0b26c0 msgr2=0x7fd5ac151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0b26c0 0x7fd5ac151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 -- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 msgr2=0x7fd5ac14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 -- 172.21.15.164:0/3335134194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac156570 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a2000640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c490 tx=0x7fd59400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940052d0 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594005470 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac152220 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594012870 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac156ad0 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac069d60 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fd594012a10 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 0x7fd588069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd5940852c0 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.136+0000 7fd5aaa00640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 0x7fd588069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.140+0000 7fd5aaa00640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 0x7fd588069810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0ad000 tx=0x7fd59c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.140+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5ac156570 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd5ac14ee50 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5a3400640 1 -- 172.21.15.164:0/3335134194 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1453 (secure 0 0 0) 0x7fd594055640 con 0x7fd5ac0cfda0 2024-05-08T04:41:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:13.357 INFO:teuthology.orchestra.run.smithi164.stdout:{"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":188,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1715143096,"num_in_osds":8,"osd_in_since":1715143086,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":250851328,"bytes_avail":772843261952,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6465}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-08T04:40:55.460197+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 msgr2=0x7fd588069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 0x7fd588069810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0ad000 tx=0x7fd59c0a6000 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 msgr2=0x7fd5ac14dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c490 tx=0x7fd59400c950 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 shutdown_connections 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd588067350 0x7fd588069810 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac15ab60 0x7fd5ac14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5ac0cfda0 0x7fd5ac14dfb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 --2- 172.21.15.164:0/3335134194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0b26c0 0x7fd5ac151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 >> 172.21.15.164:0/3335134194 conn(0x7fd5ac06ac90 msgr2=0x7fd5ac070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:13.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 shutdown_connections 2024-05-08T04:41:13.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.352+0000 7fd5b1600640 1 -- 172.21.15.164:0/3335134194 wait complete. 2024-05-08T04:41:13.378 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T04:41:13.378 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:13.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 Processor -- start 2024-05-08T04:41:13.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 -- start start 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7b0c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c072640 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c06c770 con 0x7f7b0c0cfda0 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b10a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06ca50 con 0x7f7b0c15ab60 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b01400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7b0c0b26c0 0x7f7b0c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59838/0 (socket says 172.21.15.164:59838) 2024-05-08T04:41:13.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b01400640 1 -- 172.21.15.164:0/1013368383 learned_addr learned my addr 172.21.15.164:0/1013368383 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.508+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7b0c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780980285 0 0) 0x7f7b0c072640 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af0005740 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967969590 0 0) 0x7f7b0c06c770 con 0x7f7b0c0cfda0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b0c072640 con 0x7f7b0c0cfda0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 894713774 0 0) 0x7f7af0005740 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 msgr2=0x7f7b0c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7b0c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 >> v1:172.21.15.172:6789/0 conn(0x7f7b0c0cfda0 legacy=0x7f7b0c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c06cd30 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f7af4001680 con 0x7f7b0c0b26c0 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 >> v1:172.21.15.164:6789/0 conn(0x7f7b0c0b26c0 legacy=0x7f7b0c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0005d90 0x7f7af0006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0006a20 0x7f7af0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7af00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06ca50 con 0x7f7af0005d90 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af000dc90 con 0x7f7af0006a20 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af000de60 con 0x7f7b0c15ab60 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0005d90 0x7f7af0006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b01400640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0006a20 0x7f7af0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b0a800640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7af00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 -- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 msgr2=0x7f7af00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c15ab60 0x7f7af00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 -- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0006a20 msgr2=0x7f7af0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0006a20 0x7f7af0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 -- 172.21.15.164:0/1013368383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0005740 con 0x7f7af0005d90 2024-05-08T04:41:13.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.512+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0005d90 0x7f7af0006190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009fc0 tx=0x7f7afc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9391a73730adddcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0b2070 con 0x7f7af0005d90 2024-05-08T04:41:13.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc0a5a10 con 0x7f7af0005d90 2024-05-08T04:41:13.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b09400640 1 -- 172.21.15.164:0/1013368383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc009c10 con 0x7f7af0005d90 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- 172.21.15.164:0/1013368383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0005d90 msgr2=0x7f7af0006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1013368383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af0005d90 0x7f7af0006190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009fc0 tx=0x7f7afc01cc20 comp rx=0 tx=0).stop 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b0a800640 1 -- 172.21.15.164:0/1013368383 reap_dead start 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- 172.21.15.164:0/1013368383 shutdown_connections 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- 172.21.15.164:0/1013368383 >> 172.21.15.164:0/1013368383 conn(0x7f7b0c06ac90 msgr2=0x7f7b0c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- 172.21.15.164:0/1013368383 shutdown_connections 2024-05-08T04:41:13.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- 172.21.15.164:0/1013368383 wait complete. 2024-05-08T04:41:13.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 Processor -- start 2024-05-08T04:41:13.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- start start 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0005d90 0x7f7b0c1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c144770 0x7f7b0c1f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0aeeb0 con 0x7f7b0c0cc570 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0aed30 con 0x7f7af0005d90 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b10a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0cc1b0 con 0x7f7b0c144770 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b0a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b0a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41840/0 (socket says 172.21.15.164:41840) 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.516+0000 7f7b0a800640 1 -- 172.21.15.164:0/1933247388 learned_addr learned my addr 172.21.15.164:0/1933247388 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0005d90 0x7f7b0c1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b01400640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c144770 0x7f7b0c1f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 -- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c144770 msgr2=0x7f7b0c1f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c144770 0x7f7b0c1f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 -- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0005d90 msgr2=0x7f7b0c1416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0005d90 0x7f7b0c1416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 -- 172.21.15.164:0/1933247388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c141c00 con 0x7f7b0c0cc570 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b0a800640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0ceb40 tx=0x7f7b0400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040050d0 con 0x7f7b0c0cc570 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04005290 con 0x7f7b0c0cc570 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04006c30 con 0x7f7b0c0cc570 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c1f6e70 con 0x7f7b0c0cc570 2024-05-08T04:41:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c15d870 con 0x7f7b0c0cc570 2024-05-08T04:41:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c069d60 con 0x7f7b0c0cc570 2024-05-08T04:41:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f7b04007030 con 0x7f7b0c0cc570 2024-05-08T04:41:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 0x7f7ae8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.520+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7b04084b30 con 0x7f7b0c0cc570 2024-05-08T04:41:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.524+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 0x7f7ae8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.524+0000 7f7b09e00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 0x7f7ae8069810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0af000 tx=0x7f7afc004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:13.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.528+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b0c141c00 con 0x7f7b0c0cc570 2024-05-08T04:41:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.676+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b0c0ccf30 con 0x7f7ae8067350 2024-05-08T04:41:13.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.684+0000 7f7b02a00640 1 -- 172.21.15.164:0/1933247388 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7f7afc003aa0 con 0x7f7ae8067350 2024-05-08T04:41:13.691 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:13.692 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:13.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 msgr2=0x7f7ae8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 0x7f7ae8069810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0af000 tx=0x7f7afc004ac0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 msgr2=0x7f7b0c0cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:13.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0ceb40 tx=0x7f7b0400e820 comp rx=0 tx=0).stop 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 shutdown_connections 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ae8067350 0x7f7ae8069810 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b0c144770 0x7f7b0c1f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c0cc570 0x7f7b0c0cc9f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 --2- 172.21.15.164:0/1933247388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af0005d90 0x7f7b0c1416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 >> 172.21.15.164:0/1933247388 conn(0x7f7b0c06ac90 msgr2=0x7f7b0c15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 shutdown_connections 2024-05-08T04:41:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:13.688+0000 7f7b10a00640 1 -- 172.21.15.164:0/1933247388 wait complete. 2024-05-08T04:41:13.711 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-05-08T04:41:13.463251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245232,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729488,"statfs":{"total":773094113280,"available":772842995712,"internally_reserved":0,"allocated":8462336,"data_stored":4256068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.362586"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398898+0000","last_change":"2024-05-08T04:41:08.127072+0000","last_active":"2024-05-08T04:41:11.398898+0000","last_peered":"2024-05-08T04:41:11.398898+0000","last_clean":"2024-05-08T04:41:11.398898+0000","last_became_active":"2024-05-08T04:41:08.126826+0000","last_became_peered":"2024-05-08T04:41:08.126826+0000","last_unstale":"2024-05-08T04:41:11.398898+0000","last_undegraded":"2024-05-08T04:41:11.398898+0000","last_fullsized":"2024-05-08T04:41:11.398898+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:07.240152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398306+0000","last_change":"2024-05-08T04:41:08.128226+0000","last_active":"2024-05-08T04:41:11.398306+0000","last_peered":"2024-05-08T04:41:11.398306+0000","last_clean":"2024-05-08T04:41:11.398306+0000","last_became_active":"2024-05-08T04:41:08.127832+0000","last_became_peered":"2024-05-08T04:41:08.127832+0000","last_unstale":"2024-05-08T04:41:11.398306+0000","last_undegraded":"2024-05-08T04:41:11.398306+0000","last_fullsized":"2024-05-08T04:41:11.398306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.882813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403536+0000","last_change":"2024-05-08T04:41:08.134248+0000","last_active":"2024-05-08T04:41:10.403536+0000","last_peered":"2024-05-08T04:41:10.403536+0000","last_clean":"2024-05-08T04:41:10.403536+0000","last_became_active":"2024-05-08T04:41:08.132591+0000","last_became_peered":"2024-05-08T04:41:08.132591+0000","last_unstale":"2024-05-08T04:41:10.403536+0000","last_undegraded":"2024-05-08T04:41:10.403536+0000","last_fullsized":"2024-05-08T04:41:10.403536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:42:23.370715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397021+0000","last_change":"2024-05-08T04:41:08.122766+0000","last_active":"2024-05-08T04:41:11.397021+0000","last_peered":"2024-05-08T04:41:11.397021+0000","last_clean":"2024-05-08T04:41:11.397021+0000","last_became_active":"2024-05-08T04:41:08.122519+0000","last_became_peered":"2024-05-08T04:41:08.122519+0000","last_unstale":"2024-05-08T04:41:11.397021+0000","last_undegraded":"2024-05-08T04:41:11.397021+0000","last_fullsized":"2024-05-08T04:41:11.397021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:45.594443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349941+0000","last_change":"2024-05-08T04:41:08.127668+0000","last_active":"2024-05-08T04:41:10.349941+0000","last_peered":"2024-05-08T04:41:10.349941+0000","last_clean":"2024-05-08T04:41:10.349941+0000","last_became_active":"2024-05-08T04:41:08.126586+0000","last_became_peered":"2024-05-08T04:41:08.126586+0000","last_unstale":"2024-05-08T04:41:10.349941+0000","last_undegraded":"2024-05-08T04:41:10.349941+0000","last_fullsized":"2024-05-08T04:41:10.349941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:10.023708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397984+0000","last_change":"2024-05-08T04:41:08.128568+0000","last_active":"2024-05-08T04:41:11.397984+0000","last_peered":"2024-05-08T04:41:11.397984+0000","last_clean":"2024-05-08T04:41:11.397984+0000","last_became_active":"2024-05-08T04:41:08.128337+0000","last_became_peered":"2024-05-08T04:41:08.128337+0000","last_unstale":"2024-05-08T04:41:11.397984+0000","last_undegraded":"2024-05-08T04:41:11.397984+0000","last_fullsized":"2024-05-08T04:41:11.397984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:15.637405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107253+0000","last_change":"2024-05-08T04:41:08.132376+0000","last_active":"2024-05-08T04:41:09.107253+0000","last_peered":"2024-05-08T04:41:09.107253+0000","last_clean":"2024-05-08T04:41:09.107253+0000","last_became_active":"2024-05-08T04:41:08.132069+0000","last_became_peered":"2024-05-08T04:41:08.132069+0000","last_unstale":"2024-05-08T04:41:09.107253+0000","last_undegraded":"2024-05-08T04:41:09.107253+0000","last_fullsized":"2024-05-08T04:41:09.107253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:17.664203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397939+0000","last_change":"2024-05-08T04:41:08.113444+0000","last_active":"2024-05-08T04:41:11.397939+0000","last_peered":"2024-05-08T04:41:11.397939+0000","last_clean":"2024-05-08T04:41:11.397939+0000","last_became_active":"2024-05-08T04:41:08.113163+0000","last_became_peered":"2024-05-08T04:41:08.113163+0000","last_unstale":"2024-05-08T04:41:11.397939+0000","last_undegraded":"2024-05-08T04:41:11.397939+0000","last_fullsized":"2024-05-08T04:41:11.397939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:56.557764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350506+0000","last_change":"2024-05-08T04:41:08.131823+0000","last_active":"2024-05-08T04:41:10.350506+0000","last_peered":"2024-05-08T04:41:10.350506+0000","last_clean":"2024-05-08T04:41:10.350506+0000","last_became_active":"2024-05-08T04:41:08.131333+0000","last_became_peered":"2024-05-08T04:41:08.131333+0000","last_unstale":"2024-05-08T04:41:10.350506+0000","last_undegraded":"2024-05-08T04:41:10.350506+0000","last_fullsized":"2024-05-08T04:41:10.350506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:15.224219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107755+0000","last_change":"2024-05-08T04:41:08.142366+0000","last_active":"2024-05-08T04:41:09.107755+0000","last_peered":"2024-05-08T04:41:09.107755+0000","last_clean":"2024-05-08T04:41:09.107755+0000","last_became_active":"2024-05-08T04:41:08.142048+0000","last_became_peered":"2024-05-08T04:41:08.142048+0000","last_unstale":"2024-05-08T04:41:09.107755+0000","last_undegraded":"2024-05-08T04:41:09.107755+0000","last_fullsized":"2024-05-08T04:41:09.107755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:57.264085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398738+0000","last_change":"2024-05-08T04:41:08.132008+0000","last_active":"2024-05-08T04:41:11.398738+0000","last_peered":"2024-05-08T04:41:11.398738+0000","last_clean":"2024-05-08T04:41:11.398738+0000","last_became_active":"2024-05-08T04:41:08.131176+0000","last_became_peered":"2024-05-08T04:41:08.131176+0000","last_unstale":"2024-05-08T04:41:11.398738+0000","last_undegraded":"2024-05-08T04:41:11.398738+0000","last_fullsized":"2024-05-08T04:41:11.398738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:10.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397076+0000","last_change":"2024-05-08T04:41:08.136053+0000","last_active":"2024-05-08T04:41:11.397076+0000","last_peered":"2024-05-08T04:41:11.397076+0000","last_clean":"2024-05-08T04:41:11.397076+0000","last_became_active":"2024-05-08T04:41:08.135577+0000","last_became_peered":"2024-05-08T04:41:08.135577+0000","last_unstale":"2024-05-08T04:41:11.397076+0000","last_undegraded":"2024-05-08T04:41:11.397076+0000","last_fullsized":"2024-05-08T04:41:11.397076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:31:46.547484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106637+0000","last_change":"2024-05-08T04:41:08.137581+0000","last_active":"2024-05-08T04:41:09.106637+0000","last_peered":"2024-05-08T04:41:09.106637+0000","last_clean":"2024-05-08T04:41:09.106637+0000","last_became_active":"2024-05-08T04:41:08.137323+0000","last_became_peered":"2024-05-08T04:41:08.137323+0000","last_unstale":"2024-05-08T04:41:09.106637+0000","last_undegraded":"2024-05-08T04:41:09.106637+0000","last_fullsized":"2024-05-08T04:41:09.106637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:44.499342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350057+0000","last_change":"2024-05-08T04:41:08.138410+0000","last_active":"2024-05-08T04:41:10.350057+0000","last_peered":"2024-05-08T04:41:10.350057+0000","last_clean":"2024-05-08T04:41:10.350057+0000","last_became_active":"2024-05-08T04:41:08.138151+0000","last_became_peered":"2024-05-08T04:41:08.138151+0000","last_unstale":"2024-05-08T04:41:10.350057+0000","last_undegraded":"2024-05-08T04:41:10.350057+0000","last_fullsized":"2024-05-08T04:41:10.350057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:28.065788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106197+0000","last_change":"2024-05-08T04:41:08.135434+0000","last_active":"2024-05-08T04:41:09.106197+0000","last_peered":"2024-05-08T04:41:09.106197+0000","last_clean":"2024-05-08T04:41:09.106197+0000","last_became_active":"2024-05-08T04:41:08.134807+0000","last_became_peered":"2024-05-08T04:41:08.134807+0000","last_unstale":"2024-05-08T04:41:09.106197+0000","last_undegraded":"2024-05-08T04:41:09.106197+0000","last_fullsized":"2024-05-08T04:41:09.106197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:35.735387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107340+0000","last_change":"2024-05-08T04:41:08.129549+0000","last_active":"2024-05-08T04:41:09.107340+0000","last_peered":"2024-05-08T04:41:09.107340+0000","last_clean":"2024-05-08T04:41:09.107340+0000","last_became_active":"2024-05-08T04:41:08.129329+0000","last_became_peered":"2024-05-08T04:41:08.129329+0000","last_unstale":"2024-05-08T04:41:09.107340+0000","last_undegraded":"2024-05-08T04:41:09.107340+0000","last_fullsized":"2024-05-08T04:41:09.107340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:40.632007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350518+0000","last_change":"2024-05-08T04:41:08.131597+0000","last_active":"2024-05-08T04:41:10.350518+0000","last_peered":"2024-05-08T04:41:10.350518+0000","last_clean":"2024-05-08T04:41:10.350518+0000","last_became_active":"2024-05-08T04:41:08.130988+0000","last_became_peered":"2024-05-08T04:41:08.130988+0000","last_unstale":"2024-05-08T04:41:10.350518+0000","last_undegraded":"2024-05-08T04:41:10.350518+0000","last_fullsized":"2024-05-08T04:41:10.350518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.550232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397213+0000","last_change":"2024-05-08T04:41:08.113183+0000","last_active":"2024-05-08T04:41:11.397213+0000","last_peered":"2024-05-08T04:41:11.397213+0000","last_clean":"2024-05-08T04:41:11.397213+0000","last_became_active":"2024-05-08T04:41:08.112872+0000","last_became_peered":"2024-05-08T04:41:08.112872+0000","last_unstale":"2024-05-08T04:41:11.397213+0000","last_undegraded":"2024-05-08T04:41:11.397213+0000","last_fullsized":"2024-05-08T04:41:11.397213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398674+0000","last_change":"2024-05-08T04:41:08.115306+0000","last_active":"2024-05-08T04:41:11.398674+0000","last_peered":"2024-05-08T04:41:11.398674+0000","last_clean":"2024-05-08T04:41:11.398674+0000","last_became_active":"2024-05-08T04:41:08.115034+0000","last_became_peered":"2024-05-08T04:41:08.115034+0000","last_unstale":"2024-05-08T04:41:11.398674+0000","last_undegraded":"2024-05-08T04:41:11.398674+0000","last_fullsized":"2024-05-08T04:41:11.398674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:07.340180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106655+0000","last_change":"2024-05-08T04:41:08.138896+0000","last_active":"2024-05-08T04:41:09.106655+0000","last_peered":"2024-05-08T04:41:09.106655+0000","last_clean":"2024-05-08T04:41:09.106655+0000","last_became_active":"2024-05-08T04:41:08.138640+0000","last_became_peered":"2024-05-08T04:41:08.138640+0000","last_unstale":"2024-05-08T04:41:09.106655+0000","last_undegraded":"2024-05-08T04:41:09.106655+0000","last_fullsized":"2024-05-08T04:41:09.106655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:25.200286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108325+0000","last_change":"2024-05-08T04:41:08.128257+0000","last_active":"2024-05-08T04:41:09.108325+0000","last_peered":"2024-05-08T04:41:09.108325+0000","last_clean":"2024-05-08T04:41:09.108325+0000","last_became_active":"2024-05-08T04:41:08.127977+0000","last_became_peered":"2024-05-08T04:41:08.127977+0000","last_unstale":"2024-05-08T04:41:09.108325+0000","last_undegraded":"2024-05-08T04:41:09.108325+0000","last_fullsized":"2024-05-08T04:41:09.108325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:16.067442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108286+0000","last_change":"2024-05-08T04:41:08.142408+0000","last_active":"2024-05-08T04:41:09.108286+0000","last_peered":"2024-05-08T04:41:09.108286+0000","last_clean":"2024-05-08T04:41:09.108286+0000","last_became_active":"2024-05-08T04:41:08.142126+0000","last_became_peered":"2024-05-08T04:41:08.142126+0000","last_unstale":"2024-05-08T04:41:09.108286+0000","last_undegraded":"2024-05-08T04:41:09.108286+0000","last_fullsized":"2024-05-08T04:41:09.108286+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:50.631832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397756+0000","last_change":"2024-05-08T04:41:08.120012+0000","last_active":"2024-05-08T04:41:11.397756+0000","last_peered":"2024-05-08T04:41:11.397756+0000","last_clean":"2024-05-08T04:41:11.397756+0000","last_became_active":"2024-05-08T04:41:08.119719+0000","last_became_peered":"2024-05-08T04:41:08.119719+0000","last_unstale":"2024-05-08T04:41:11.397756+0000","last_undegraded":"2024-05-08T04:41:11.397756+0000","last_fullsized":"2024-05-08T04:41:11.397756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:01.724943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397596+0000","last_change":"2024-05-08T04:41:08.121568+0000","last_active":"2024-05-08T04:41:11.397596+0000","last_peered":"2024-05-08T04:41:11.397596+0000","last_clean":"2024-05-08T04:41:11.397596+0000","last_became_active":"2024-05-08T04:41:08.121297+0000","last_became_peered":"2024-05-08T04:41:08.121297+0000","last_unstale":"2024-05-08T04:41:11.397596+0000","last_undegraded":"2024-05-08T04:41:11.397596+0000","last_fullsized":"2024-05-08T04:41:11.397596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:41:52.288134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108018+0000","last_change":"2024-05-08T04:41:08.129042+0000","last_active":"2024-05-08T04:41:09.108018+0000","last_peered":"2024-05-08T04:41:09.108018+0000","last_clean":"2024-05-08T04:41:09.108018+0000","last_became_active":"2024-05-08T04:41:08.125396+0000","last_became_peered":"2024-05-08T04:41:08.125396+0000","last_unstale":"2024-05-08T04:41:09.108018+0000","last_undegraded":"2024-05-08T04:41:09.108018+0000","last_fullsized":"2024-05-08T04:41:09.108018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:58.986860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108322+0000","last_change":"2024-05-08T04:41:08.135386+0000","last_active":"2024-05-08T04:41:09.108322+0000","last_peered":"2024-05-08T04:41:09.108322+0000","last_clean":"2024-05-08T04:41:09.108322+0000","last_became_active":"2024-05-08T04:41:08.135092+0000","last_became_peered":"2024-05-08T04:41:08.135092+0000","last_unstale":"2024-05-08T04:41:09.108322+0000","last_undegraded":"2024-05-08T04:41:09.108322+0000","last_fullsized":"2024-05-08T04:41:09.108322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:25.328081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403586+0000","last_change":"2024-05-08T04:41:08.130633+0000","last_active":"2024-05-08T04:41:10.403586+0000","last_peered":"2024-05-08T04:41:10.403586+0000","last_clean":"2024-05-08T04:41:10.403586+0000","last_became_active":"2024-05-08T04:41:08.130304+0000","last_became_peered":"2024-05-08T04:41:08.130304+0000","last_unstale":"2024-05-08T04:41:10.403586+0000","last_undegraded":"2024-05-08T04:41:10.403586+0000","last_fullsized":"2024-05-08T04:41:10.403586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:47.871026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108426+0000","last_change":"2024-05-08T04:41:08.142669+0000","last_active":"2024-05-08T04:41:09.108426+0000","last_peered":"2024-05-08T04:41:09.108426+0000","last_clean":"2024-05-08T04:41:09.108426+0000","last_became_active":"2024-05-08T04:41:08.142339+0000","last_became_peered":"2024-05-08T04:41:08.142339+0000","last_unstale":"2024-05-08T04:41:09.108426+0000","last_undegraded":"2024-05-08T04:41:09.108426+0000","last_fullsized":"2024-05-08T04:41:09.108426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:54.892363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403434+0000","last_change":"2024-05-08T04:41:08.140417+0000","last_active":"2024-05-08T04:41:10.403434+0000","last_peered":"2024-05-08T04:41:10.403434+0000","last_clean":"2024-05-08T04:41:10.403434+0000","last_became_active":"2024-05-08T04:41:08.140178+0000","last_became_peered":"2024-05-08T04:41:08.140178+0000","last_unstale":"2024-05-08T04:41:10.403434+0000","last_undegraded":"2024-05-08T04:41:10.403434+0000","last_fullsized":"2024-05-08T04:41:10.403434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:56:54.293327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107502+0000","last_change":"2024-05-08T04:41:08.123508+0000","last_active":"2024-05-08T04:41:09.107502+0000","last_peered":"2024-05-08T04:41:09.107502+0000","last_clean":"2024-05-08T04:41:09.107502+0000","last_became_active":"2024-05-08T04:41:08.123201+0000","last_became_peered":"2024-05-08T04:41:08.123201+0000","last_unstale":"2024-05-08T04:41:09.107502+0000","last_undegraded":"2024-05-08T04:41:09.107502+0000","last_fullsized":"2024-05-08T04:41:09.107502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:58.063316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108042+0000","last_change":"2024-05-08T04:41:08.137226+0000","last_active":"2024-05-08T04:41:09.108042+0000","last_peered":"2024-05-08T04:41:09.108042+0000","last_clean":"2024-05-08T04:41:09.108042+0000","last_became_active":"2024-05-08T04:41:08.136952+0000","last_became_peered":"2024-05-08T04:41:08.136952+0000","last_unstale":"2024-05-08T04:41:09.108042+0000","last_undegraded":"2024-05-08T04:41:09.108042+0000","last_fullsized":"2024-05-08T04:41:09.108042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:19.622961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397997+0000","last_change":"2024-05-08T04:41:08.135567+0000","last_active":"2024-05-08T04:41:11.397997+0000","last_peered":"2024-05-08T04:41:11.397997+0000","last_clean":"2024-05-08T04:41:11.397997+0000","last_became_active":"2024-05-08T04:41:08.135134+0000","last_became_peered":"2024-05-08T04:41:08.135134+0000","last_unstale":"2024-05-08T04:41:11.397997+0000","last_undegraded":"2024-05-08T04:41:11.397997+0000","last_fullsized":"2024-05-08T04:41:11.397997+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:20.277016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398610+0000","last_change":"2024-05-08T04:41:08.114608+0000","last_active":"2024-05-08T04:41:11.398610+0000","last_peered":"2024-05-08T04:41:11.398610+0000","last_clean":"2024-05-08T04:41:11.398610+0000","last_became_active":"2024-05-08T04:41:08.114224+0000","last_became_peered":"2024-05-08T04:41:08.114224+0000","last_unstale":"2024-05-08T04:41:11.398610+0000","last_undegraded":"2024-05-08T04:41:11.398610+0000","last_fullsized":"2024-05-08T04:41:11.398610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:25.200286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.399076+0000","last_change":"2024-05-08T04:41:07.123950+0000","last_active":"2024-05-08T04:41:11.399076+0000","last_peered":"2024-05-08T04:41:11.399076+0000","last_clean":"2024-05-08T04:41:11.399076+0000","last_became_active":"2024-05-08T04:41:07.123577+0000","last_became_peered":"2024-05-08T04:41:07.123577+0000","last_unstale":"2024-05-08T04:41:11.399076+0000","last_undegraded":"2024-05-08T04:41:11.399076+0000","last_fullsized":"2024-05-08T04:41:11.399076+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:16.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350586+0000","last_change":"2024-05-08T04:41:08.132345+0000","last_active":"2024-05-08T04:41:10.350586+0000","last_peered":"2024-05-08T04:41:10.350586+0000","last_clean":"2024-05-08T04:41:10.350586+0000","last_became_active":"2024-05-08T04:41:08.132056+0000","last_became_peered":"2024-05-08T04:41:08.132056+0000","last_unstale":"2024-05-08T04:41:10.350586+0000","last_undegraded":"2024-05-08T04:41:10.350586+0000","last_fullsized":"2024-05-08T04:41:10.350586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:25.447697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106753+0000","last_change":"2024-05-08T04:41:07.122634+0000","last_active":"2024-05-08T04:41:09.106753+0000","last_peered":"2024-05-08T04:41:09.106753+0000","last_clean":"2024-05-08T04:41:09.106753+0000","last_became_active":"2024-05-08T04:41:07.122307+0000","last_became_peered":"2024-05-08T04:41:07.122307+0000","last_unstale":"2024-05-08T04:41:09.106753+0000","last_undegraded":"2024-05-08T04:41:09.106753+0000","last_fullsized":"2024-05-08T04:41:09.106753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:33.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398222+0000","last_change":"2024-05-08T04:41:08.121693+0000","last_active":"2024-05-08T04:41:11.398222+0000","last_peered":"2024-05-08T04:41:11.398222+0000","last_clean":"2024-05-08T04:41:11.398222+0000","last_became_active":"2024-05-08T04:41:08.120911+0000","last_became_peered":"2024-05-08T04:41:08.120911+0000","last_unstale":"2024-05-08T04:41:11.398222+0000","last_undegraded":"2024-05-08T04:41:11.398222+0000","last_fullsized":"2024-05-08T04:41:11.398222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:49.132730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107858+0000","last_change":"2024-05-08T04:41:07.124366+0000","last_active":"2024-05-08T04:41:09.107858+0000","last_peered":"2024-05-08T04:41:09.107858+0000","last_clean":"2024-05-08T04:41:09.107858+0000","last_became_active":"2024-05-08T04:41:07.124039+0000","last_became_peered":"2024-05-08T04:41:07.124039+0000","last_unstale":"2024-05-08T04:41:09.107858+0000","last_undegraded":"2024-05-08T04:41:09.107858+0000","last_fullsized":"2024-05-08T04:41:09.107858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:37.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398889+0000","last_change":"2024-05-08T04:41:08.135532+0000","last_active":"2024-05-08T04:41:11.398889+0000","last_peered":"2024-05-08T04:41:11.398889+0000","last_clean":"2024-05-08T04:41:11.398889+0000","last_became_active":"2024-05-08T04:41:08.135263+0000","last_became_peered":"2024-05-08T04:41:08.135263+0000","last_unstale":"2024-05-08T04:41:11.398889+0000","last_undegraded":"2024-05-08T04:41:11.398889+0000","last_fullsized":"2024-05-08T04:41:11.398889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:25:57.683492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398672+0000","last_change":"2024-05-08T04:41:07.126964+0000","last_active":"2024-05-08T04:41:11.398672+0000","last_peered":"2024-05-08T04:41:11.398672+0000","last_clean":"2024-05-08T04:41:11.398672+0000","last_became_active":"2024-05-08T04:41:07.126433+0000","last_became_peered":"2024-05-08T04:41:07.126433+0000","last_unstale":"2024-05-08T04:41:11.398672+0000","last_undegraded":"2024-05-08T04:41:11.398672+0000","last_fullsized":"2024-05-08T04:41:11.398672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:53.603895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108380+0000","last_change":"2024-05-08T04:41:08.140083+0000","last_active":"2024-05-08T04:41:09.108380+0000","last_peered":"2024-05-08T04:41:09.108380+0000","last_clean":"2024-05-08T04:41:09.108380+0000","last_became_active":"2024-05-08T04:41:08.137922+0000","last_became_peered":"2024-05-08T04:41:08.137922+0000","last_unstale":"2024-05-08T04:41:09.108380+0000","last_undegraded":"2024-05-08T04:41:09.108380+0000","last_fullsized":"2024-05-08T04:41:09.108380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:44.006515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403542+0000","last_change":"2024-05-08T04:41:07.118893+0000","last_active":"2024-05-08T04:41:10.403542+0000","last_peered":"2024-05-08T04:41:10.403542+0000","last_clean":"2024-05-08T04:41:10.403542+0000","last_became_active":"2024-05-08T04:41:07.117180+0000","last_became_peered":"2024-05-08T04:41:07.117180+0000","last_unstale":"2024-05-08T04:41:10.403542+0000","last_undegraded":"2024-05-08T04:41:10.403542+0000","last_fullsized":"2024-05-08T04:41:10.403542+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:46:54.270506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108349+0000","last_change":"2024-05-08T04:41:08.126629+0000","last_active":"2024-05-08T04:41:09.108349+0000","last_peered":"2024-05-08T04:41:09.108349+0000","last_clean":"2024-05-08T04:41:09.108349+0000","last_became_active":"2024-05-08T04:41:08.126149+0000","last_became_peered":"2024-05-08T04:41:08.126149+0000","last_unstale":"2024-05-08T04:41:09.108349+0000","last_undegraded":"2024-05-08T04:41:09.108349+0000","last_fullsized":"2024-05-08T04:41:09.108349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:29.429030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398176+0000","last_change":"2024-05-08T04:41:07.122025+0000","last_active":"2024-05-08T04:41:11.398176+0000","last_peered":"2024-05-08T04:41:11.398176+0000","last_clean":"2024-05-08T04:41:11.398176+0000","last_became_active":"2024-05-08T04:41:07.120512+0000","last_became_peered":"2024-05-08T04:41:07.120512+0000","last_unstale":"2024-05-08T04:41:11.398176+0000","last_undegraded":"2024-05-08T04:41:11.398176+0000","last_fullsized":"2024-05-08T04:41:11.398176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:34.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349880+0000","last_change":"2024-05-08T04:41:08.121064+0000","last_active":"2024-05-08T04:41:10.349880+0000","last_peered":"2024-05-08T04:41:10.349880+0000","last_clean":"2024-05-08T04:41:10.349880+0000","last_became_active":"2024-05-08T04:41:08.120796+0000","last_became_peered":"2024-05-08T04:41:08.120796+0000","last_unstale":"2024-05-08T04:41:10.349880+0000","last_undegraded":"2024-05-08T04:41:10.349880+0000","last_fullsized":"2024-05-08T04:41:10.349880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:06.329830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396793+0000","last_change":"2024-05-08T04:41:07.115029+0000","last_active":"2024-05-08T04:41:11.396793+0000","last_peered":"2024-05-08T04:41:11.396793+0000","last_clean":"2024-05-08T04:41:11.396793+0000","last_became_active":"2024-05-08T04:41:07.114753+0000","last_became_peered":"2024-05-08T04:41:07.114753+0000","last_unstale":"2024-05-08T04:41:11.396793+0000","last_undegraded":"2024-05-08T04:41:11.396793+0000","last_fullsized":"2024-05-08T04:41:11.396793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:36.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106339+0000","last_change":"2024-05-08T04:41:08.136986+0000","last_active":"2024-05-08T04:41:09.106339+0000","last_peered":"2024-05-08T04:41:09.106339+0000","last_clean":"2024-05-08T04:41:09.106339+0000","last_became_active":"2024-05-08T04:41:08.136620+0000","last_became_peered":"2024-05-08T04:41:08.136620+0000","last_unstale":"2024-05-08T04:41:09.106339+0000","last_undegraded":"2024-05-08T04:41:09.106339+0000","last_fullsized":"2024-05-08T04:41:09.106339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:01.724943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349845+0000","last_change":"2024-05-08T04:41:07.113389+0000","last_active":"2024-05-08T04:41:10.349845+0000","last_peered":"2024-05-08T04:41:10.349845+0000","last_clean":"2024-05-08T04:41:10.349845+0000","last_became_active":"2024-05-08T04:41:07.113100+0000","last_became_peered":"2024-05-08T04:41:07.113100+0000","last_unstale":"2024-05-08T04:41:10.349845+0000","last_undegraded":"2024-05-08T04:41:10.349845+0000","last_fullsized":"2024-05-08T04:41:10.349845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:04.686860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108512+0000","last_change":"2024-05-08T04:41:08.133105+0000","last_active":"2024-05-08T04:41:09.108512+0000","last_peered":"2024-05-08T04:41:09.108512+0000","last_clean":"2024-05-08T04:41:09.108512+0000","last_became_active":"2024-05-08T04:41:08.132842+0000","last_became_peered":"2024-05-08T04:41:08.132842+0000","last_unstale":"2024-05-08T04:41:09.108512+0000","last_undegraded":"2024-05-08T04:41:09.108512+0000","last_fullsized":"2024-05-08T04:41:09.108512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:03.829699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108104+0000","last_change":"2024-05-08T04:41:07.113877+0000","last_active":"2024-05-08T04:41:09.108104+0000","last_peered":"2024-05-08T04:41:09.108104+0000","last_clean":"2024-05-08T04:41:09.108104+0000","last_became_active":"2024-05-08T04:41:07.112924+0000","last_became_peered":"2024-05-08T04:41:07.112924+0000","last_unstale":"2024-05-08T04:41:09.108104+0000","last_undegraded":"2024-05-08T04:41:09.108104+0000","last_fullsized":"2024-05-08T04:41:09.108104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:20:56.309860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398965+0000","last_change":"2024-05-08T04:41:08.113830+0000","last_active":"2024-05-08T04:41:11.398965+0000","last_peered":"2024-05-08T04:41:11.398965+0000","last_clean":"2024-05-08T04:41:11.398965+0000","last_became_active":"2024-05-08T04:41:08.112170+0000","last_became_peered":"2024-05-08T04:41:08.112170+0000","last_unstale":"2024-05-08T04:41:11.398965+0000","last_undegraded":"2024-05-08T04:41:11.398965+0000","last_fullsized":"2024-05-08T04:41:11.398965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:46.059471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398489+0000","last_change":"2024-05-08T04:41:07.127217+0000","last_active":"2024-05-08T04:41:11.398489+0000","last_peered":"2024-05-08T04:41:11.398489+0000","last_clean":"2024-05-08T04:41:11.398489+0000","last_became_active":"2024-05-08T04:41:07.126634+0000","last_became_peered":"2024-05-08T04:41:07.126634+0000","last_unstale":"2024-05-08T04:41:11.398489+0000","last_undegraded":"2024-05-08T04:41:11.398489+0000","last_fullsized":"2024-05-08T04:41:11.398489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:20.842180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398562+0000","last_change":"2024-05-08T04:41:08.132666+0000","last_active":"2024-05-08T04:41:11.398562+0000","last_peered":"2024-05-08T04:41:11.398562+0000","last_clean":"2024-05-08T04:41:11.398562+0000","last_became_active":"2024-05-08T04:41:08.132376+0000","last_became_peered":"2024-05-08T04:41:08.132376+0000","last_unstale":"2024-05-08T04:41:11.398562+0000","last_undegraded":"2024-05-08T04:41:11.398562+0000","last_fullsized":"2024-05-08T04:41:11.398562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:57.264085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403637+0000","last_change":"2024-05-08T04:41:07.120447+0000","last_active":"2024-05-08T04:41:10.403637+0000","last_peered":"2024-05-08T04:41:10.403637+0000","last_clean":"2024-05-08T04:41:10.403637+0000","last_became_active":"2024-05-08T04:41:07.120185+0000","last_became_peered":"2024-05-08T04:41:07.120185+0000","last_unstale":"2024-05-08T04:41:10.403637+0000","last_undegraded":"2024-05-08T04:41:10.403637+0000","last_fullsized":"2024-05-08T04:41:10.403637+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:44.441957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350141+0000","last_change":"2024-05-08T04:41:08.136327+0000","last_active":"2024-05-08T04:41:10.350141+0000","last_peered":"2024-05-08T04:41:10.350141+0000","last_clean":"2024-05-08T04:41:10.350141+0000","last_became_active":"2024-05-08T04:41:08.135591+0000","last_became_peered":"2024-05-08T04:41:08.135591+0000","last_unstale":"2024-05-08T04:41:10.350141+0000","last_undegraded":"2024-05-08T04:41:10.350141+0000","last_fullsized":"2024-05-08T04:41:10.350141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:34.338272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397173+0000","last_change":"2024-05-08T04:41:07.114558+0000","last_active":"2024-05-08T04:41:11.397173+0000","last_peered":"2024-05-08T04:41:11.397173+0000","last_clean":"2024-05-08T04:41:11.397173+0000","last_became_active":"2024-05-08T04:41:07.114273+0000","last_became_peered":"2024-05-08T04:41:07.114273+0000","last_unstale":"2024-05-08T04:41:11.397173+0000","last_undegraded":"2024-05-08T04:41:11.397173+0000","last_fullsized":"2024-05-08T04:41:11.397173+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:34.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397617+0000","last_change":"2024-05-08T04:41:08.123195+0000","last_active":"2024-05-08T04:41:11.397617+0000","last_peered":"2024-05-08T04:41:11.397617+0000","last_clean":"2024-05-08T04:41:11.397617+0000","last_became_active":"2024-05-08T04:41:08.122905+0000","last_became_peered":"2024-05-08T04:41:08.122905+0000","last_unstale":"2024-05-08T04:41:11.397617+0000","last_undegraded":"2024-05-08T04:41:11.397617+0000","last_fullsized":"2024-05-08T04:41:11.397617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:35.735387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396893+0000","last_change":"2024-05-08T04:41:07.119807+0000","last_active":"2024-05-08T04:41:11.396893+0000","last_peered":"2024-05-08T04:41:11.396893+0000","last_clean":"2024-05-08T04:41:11.396893+0000","last_became_active":"2024-05-08T04:41:07.119549+0000","last_became_peered":"2024-05-08T04:41:07.119549+0000","last_unstale":"2024-05-08T04:41:11.396893+0000","last_undegraded":"2024-05-08T04:41:11.396893+0000","last_fullsized":"2024-05-08T04:41:11.396893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:22.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108138+0000","last_change":"2024-05-08T04:41:08.129404+0000","last_active":"2024-05-08T04:41:09.108138+0000","last_peered":"2024-05-08T04:41:09.108138+0000","last_clean":"2024-05-08T04:41:09.108138+0000","last_became_active":"2024-05-08T04:41:08.129114+0000","last_became_peered":"2024-05-08T04:41:08.129114+0000","last_unstale":"2024-05-08T04:41:09.108138+0000","last_undegraded":"2024-05-08T04:41:09.108138+0000","last_fullsized":"2024-05-08T04:41:09.108138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:25.302556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398261+0000","last_change":"2024-05-08T04:41:07.118440+0000","last_active":"2024-05-08T04:41:11.398261+0000","last_peered":"2024-05-08T04:41:11.398261+0000","last_clean":"2024-05-08T04:41:11.398261+0000","last_became_active":"2024-05-08T04:41:07.118074+0000","last_became_peered":"2024-05-08T04:41:07.118074+0000","last_unstale":"2024-05-08T04:41:11.398261+0000","last_undegraded":"2024-05-08T04:41:11.398261+0000","last_fullsized":"2024-05-08T04:41:11.398261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:36.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106163+0000","last_change":"2024-05-08T04:41:08.140655+0000","last_active":"2024-05-08T04:41:09.106163+0000","last_peered":"2024-05-08T04:41:09.106163+0000","last_clean":"2024-05-08T04:41:09.106163+0000","last_became_active":"2024-05-08T04:41:08.140408+0000","last_became_peered":"2024-05-08T04:41:08.140408+0000","last_unstale":"2024-05-08T04:41:09.106163+0000","last_undegraded":"2024-05-08T04:41:09.106163+0000","last_fullsized":"2024-05-08T04:41:09.106163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:41:52.288134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107251+0000","last_change":"2024-05-08T04:41:07.111672+0000","last_active":"2024-05-08T04:41:09.107251+0000","last_peered":"2024-05-08T04:41:09.107251+0000","last_clean":"2024-05-08T04:41:09.107251+0000","last_became_active":"2024-05-08T04:41:07.110877+0000","last_became_peered":"2024-05-08T04:41:07.110877+0000","last_unstale":"2024-05-08T04:41:09.107251+0000","last_undegraded":"2024-05-08T04:41:09.107251+0000","last_fullsized":"2024-05-08T04:41:09.107251+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:48.121832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349859+0000","last_change":"2024-05-08T04:41:08.139824+0000","last_active":"2024-05-08T04:41:10.349859+0000","last_peered":"2024-05-08T04:41:10.349859+0000","last_clean":"2024-05-08T04:41:10.349859+0000","last_became_active":"2024-05-08T04:41:08.139559+0000","last_became_peered":"2024-05-08T04:41:08.139559+0000","last_unstale":"2024-05-08T04:41:10.349859+0000","last_undegraded":"2024-05-08T04:41:10.349859+0000","last_fullsized":"2024-05-08T04:41:10.349859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:07.686483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107456+0000","last_change":"2024-05-08T04:41:07.120313+0000","last_active":"2024-05-08T04:41:09.107456+0000","last_peered":"2024-05-08T04:41:09.107456+0000","last_clean":"2024-05-08T04:41:09.107456+0000","last_became_active":"2024-05-08T04:41:07.120022+0000","last_became_peered":"2024-05-08T04:41:07.120022+0000","last_unstale":"2024-05-08T04:41:09.107456+0000","last_undegraded":"2024-05-08T04:41:09.107456+0000","last_fullsized":"2024-05-08T04:41:09.107456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:02.922831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107858+0000","last_change":"2024-05-08T04:41:08.130066+0000","last_active":"2024-05-08T04:41:09.107858+0000","last_peered":"2024-05-08T04:41:09.107858+0000","last_clean":"2024-05-08T04:41:09.107858+0000","last_became_active":"2024-05-08T04:41:08.129799+0000","last_became_peered":"2024-05-08T04:41:08.129799+0000","last_unstale":"2024-05-08T04:41:09.107858+0000","last_undegraded":"2024-05-08T04:41:09.107858+0000","last_fullsized":"2024-05-08T04:41:09.107858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:07.340180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398040+0000","last_change":"2024-05-08T04:41:07.128722+0000","last_active":"2024-05-08T04:41:11.398040+0000","last_peered":"2024-05-08T04:41:11.398040+0000","last_clean":"2024-05-08T04:41:11.398040+0000","last_became_active":"2024-05-08T04:41:07.128152+0000","last_became_peered":"2024-05-08T04:41:07.128152+0000","last_unstale":"2024-05-08T04:41:11.398040+0000","last_undegraded":"2024-05-08T04:41:11.398040+0000","last_fullsized":"2024-05-08T04:41:11.398040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:13.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.399102+0000","last_change":"2024-05-08T04:41:08.132342+0000","last_active":"2024-05-08T04:41:11.399102+0000","last_peered":"2024-05-08T04:41:11.399102+0000","last_clean":"2024-05-08T04:41:11.399102+0000","last_became_active":"2024-05-08T04:41:08.132038+0000","last_became_peered":"2024-05-08T04:41:08.132038+0000","last_unstale":"2024-05-08T04:41:11.399102+0000","last_undegraded":"2024-05-08T04:41:11.399102+0000","last_fullsized":"2024-05-08T04:41:11.399102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:57.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107950+0000","last_change":"2024-05-08T04:41:07.124431+0000","last_active":"2024-05-08T04:41:09.107950+0000","last_peered":"2024-05-08T04:41:09.107950+0000","last_clean":"2024-05-08T04:41:09.107950+0000","last_became_active":"2024-05-08T04:41:07.124047+0000","last_became_peered":"2024-05-08T04:41:07.124047+0000","last_unstale":"2024-05-08T04:41:09.107950+0000","last_undegraded":"2024-05-08T04:41:09.107950+0000","last_fullsized":"2024-05-08T04:41:09.107950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:03.276360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397180+0000","last_change":"2024-05-08T04:41:08.135819+0000","last_active":"2024-05-08T04:41:11.397180+0000","last_peered":"2024-05-08T04:41:11.397180+0000","last_clean":"2024-05-08T04:41:11.397180+0000","last_became_active":"2024-05-08T04:41:08.135204+0000","last_became_peered":"2024-05-08T04:41:08.135204+0000","last_unstale":"2024-05-08T04:41:11.397180+0000","last_undegraded":"2024-05-08T04:41:11.397180+0000","last_fullsized":"2024-05-08T04:41:11.397180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:22.003182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398671+0000","last_change":"2024-05-08T04:41:07.126275+0000","last_active":"2024-05-08T04:41:11.398671+0000","last_peered":"2024-05-08T04:41:11.398671+0000","last_clean":"2024-05-08T04:41:11.398671+0000","last_became_active":"2024-05-08T04:41:07.125938+0000","last_became_peered":"2024-05-08T04:41:07.125938+0000","last_unstale":"2024-05-08T04:41:11.398671+0000","last_undegraded":"2024-05-08T04:41:11.398671+0000","last_fullsized":"2024-05-08T04:41:11.398671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:33.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350233+0000","last_change":"2024-05-08T04:41:08.133530+0000","last_active":"2024-05-08T04:41:10.350233+0000","last_peered":"2024-05-08T04:41:10.350233+0000","last_clean":"2024-05-08T04:41:10.350233+0000","last_became_active":"2024-05-08T04:41:08.133246+0000","last_became_peered":"2024-05-08T04:41:08.133246+0000","last_unstale":"2024-05-08T04:41:10.350233+0000","last_undegraded":"2024-05-08T04:41:10.350233+0000","last_fullsized":"2024-05-08T04:41:10.350233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:27.447856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398603+0000","last_change":"2024-05-08T04:41:07.129166+0000","last_active":"2024-05-08T04:41:11.398603+0000","last_peered":"2024-05-08T04:41:11.398603+0000","last_clean":"2024-05-08T04:41:11.398603+0000","last_became_active":"2024-05-08T04:41:07.128785+0000","last_became_peered":"2024-05-08T04:41:07.128785+0000","last_unstale":"2024-05-08T04:41:11.398603+0000","last_undegraded":"2024-05-08T04:41:11.398603+0000","last_fullsized":"2024-05-08T04:41:11.398603+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:57.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403575+0000","last_change":"2024-05-08T04:41:08.132349+0000","last_active":"2024-05-08T04:41:10.403575+0000","last_peered":"2024-05-08T04:41:10.403575+0000","last_clean":"2024-05-08T04:41:10.403575+0000","last_became_active":"2024-05-08T04:41:08.132012+0000","last_became_peered":"2024-05-08T04:41:08.132012+0000","last_unstale":"2024-05-08T04:41:10.403575+0000","last_undegraded":"2024-05-08T04:41:10.403575+0000","last_fullsized":"2024-05-08T04:41:10.403575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:00:45.269537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398366+0000","last_change":"2024-05-08T04:41:07.122473+0000","last_active":"2024-05-08T04:41:11.398366+0000","last_peered":"2024-05-08T04:41:11.398366+0000","last_clean":"2024-05-08T04:41:11.398366+0000","last_became_active":"2024-05-08T04:41:07.122179+0000","last_became_peered":"2024-05-08T04:41:07.122179+0000","last_unstale":"2024-05-08T04:41:11.398366+0000","last_undegraded":"2024-05-08T04:41:11.398366+0000","last_fullsized":"2024-05-08T04:41:11.398366+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:22.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403438+0000","last_change":"2024-05-08T04:41:08.130647+0000","last_active":"2024-05-08T04:41:10.403438+0000","last_peered":"2024-05-08T04:41:10.403438+0000","last_clean":"2024-05-08T04:41:10.403438+0000","last_became_active":"2024-05-08T04:41:08.130316+0000","last_became_peered":"2024-05-08T04:41:08.130316+0000","last_unstale":"2024-05-08T04:41:10.403438+0000","last_undegraded":"2024-05-08T04:41:10.403438+0000","last_fullsized":"2024-05-08T04:41:10.403438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:06.854948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107889+0000","last_change":"2024-05-08T04:41:07.119592+0000","last_active":"2024-05-08T04:41:09.107889+0000","last_peered":"2024-05-08T04:41:09.107889+0000","last_clean":"2024-05-08T04:41:09.107889+0000","last_became_active":"2024-05-08T04:41:07.119253+0000","last_became_peered":"2024-05-08T04:41:07.119253+0000","last_unstale":"2024-05-08T04:41:09.107889+0000","last_undegraded":"2024-05-08T04:41:09.107889+0000","last_fullsized":"2024-05-08T04:41:09.107889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:19.266117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403410+0000","last_change":"2024-05-08T04:41:08.128660+0000","last_active":"2024-05-08T04:41:10.403410+0000","last_peered":"2024-05-08T04:41:10.403410+0000","last_clean":"2024-05-08T04:41:10.403410+0000","last_became_active":"2024-05-08T04:41:08.128340+0000","last_became_peered":"2024-05-08T04:41:08.128340+0000","last_unstale":"2024-05-08T04:41:10.403410+0000","last_undegraded":"2024-05-08T04:41:10.403410+0000","last_fullsized":"2024-05-08T04:41:10.403410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:12.389136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106070+0000","last_change":"2024-05-08T04:41:07.118353+0000","last_active":"2024-05-08T04:41:09.106070+0000","last_peered":"2024-05-08T04:41:09.106070+0000","last_clean":"2024-05-08T04:41:09.106070+0000","last_became_active":"2024-05-08T04:41:07.118022+0000","last_became_peered":"2024-05-08T04:41:07.118022+0000","last_unstale":"2024-05-08T04:41:09.106070+0000","last_undegraded":"2024-05-08T04:41:09.106070+0000","last_fullsized":"2024-05-08T04:41:09.106070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:24.291658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106778+0000","last_change":"2024-05-08T04:41:08.133842+0000","last_active":"2024-05-08T04:41:09.106778+0000","last_peered":"2024-05-08T04:41:09.106778+0000","last_clean":"2024-05-08T04:41:09.106778+0000","last_became_active":"2024-05-08T04:41:08.133594+0000","last_became_peered":"2024-05-08T04:41:08.133594+0000","last_unstale":"2024-05-08T04:41:09.106778+0000","last_undegraded":"2024-05-08T04:41:09.106778+0000","last_fullsized":"2024-05-08T04:41:09.106778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:10.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398042+0000","last_change":"2024-05-08T04:41:07.125972+0000","last_active":"2024-05-08T04:41:11.398042+0000","last_peered":"2024-05-08T04:41:11.398042+0000","last_clean":"2024-05-08T04:41:11.398042+0000","last_became_active":"2024-05-08T04:41:07.125631+0000","last_became_peered":"2024-05-08T04:41:07.125631+0000","last_unstale":"2024-05-08T04:41:11.398042+0000","last_undegraded":"2024-05-08T04:41:11.398042+0000","last_fullsized":"2024-05-08T04:41:11.398042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:24.785925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108367+0000","last_change":"2024-05-08T04:41:08.133318+0000","last_active":"2024-05-08T04:41:09.108367+0000","last_peered":"2024-05-08T04:41:09.108367+0000","last_clean":"2024-05-08T04:41:09.108367+0000","last_became_active":"2024-05-08T04:41:08.133028+0000","last_became_peered":"2024-05-08T04:41:08.133028+0000","last_unstale":"2024-05-08T04:41:09.108367+0000","last_undegraded":"2024-05-08T04:41:09.108367+0000","last_fullsized":"2024-05-08T04:41:09.108367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:34.383175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108519+0000","last_change":"2024-05-08T04:41:07.115163+0000","last_active":"2024-05-08T04:41:09.108519+0000","last_peered":"2024-05-08T04:41:09.108519+0000","last_clean":"2024-05-08T04:41:09.108519+0000","last_became_active":"2024-05-08T04:41:07.114516+0000","last_became_peered":"2024-05-08T04:41:07.114516+0000","last_unstale":"2024-05-08T04:41:09.108519+0000","last_undegraded":"2024-05-08T04:41:09.108519+0000","last_fullsized":"2024-05-08T04:41:09.108519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:41.871914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398518+0000","last_change":"2024-05-08T04:41:08.114585+0000","last_active":"2024-05-08T04:41:11.398518+0000","last_peered":"2024-05-08T04:41:11.398518+0000","last_clean":"2024-05-08T04:41:11.398518+0000","last_became_active":"2024-05-08T04:41:08.114294+0000","last_became_peered":"2024-05-08T04:41:08.114294+0000","last_unstale":"2024-05-08T04:41:11.398518+0000","last_undegraded":"2024-05-08T04:41:11.398518+0000","last_fullsized":"2024-05-08T04:41:11.398518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:44.499342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350239+0000","last_change":"2024-05-08T04:41:07.123483+0000","last_active":"2024-05-08T04:41:10.350239+0000","last_peered":"2024-05-08T04:41:10.350239+0000","last_clean":"2024-05-08T04:41:10.350239+0000","last_became_active":"2024-05-08T04:41:07.121115+0000","last_became_peered":"2024-05-08T04:41:07.121115+0000","last_unstale":"2024-05-08T04:41:10.350239+0000","last_undegraded":"2024-05-08T04:41:10.350239+0000","last_fullsized":"2024-05-08T04:41:10.350239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:50.974023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398999+0000","last_change":"2024-05-08T04:41:08.134611+0000","last_active":"2024-05-08T04:41:11.398999+0000","last_peered":"2024-05-08T04:41:11.398999+0000","last_clean":"2024-05-08T04:41:11.398999+0000","last_became_active":"2024-05-08T04:41:08.133850+0000","last_became_peered":"2024-05-08T04:41:08.133850+0000","last_unstale":"2024-05-08T04:41:11.398999+0000","last_undegraded":"2024-05-08T04:41:11.398999+0000","last_fullsized":"2024-05-08T04:41:11.398999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:56:31.222655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106605+0000","last_change":"2024-05-08T04:41:07.120042+0000","last_active":"2024-05-08T04:41:09.106605+0000","last_peered":"2024-05-08T04:41:09.106605+0000","last_clean":"2024-05-08T04:41:09.106605+0000","last_became_active":"2024-05-08T04:41:07.119795+0000","last_became_peered":"2024-05-08T04:41:07.119795+0000","last_unstale":"2024-05-08T04:41:09.106605+0000","last_undegraded":"2024-05-08T04:41:09.106605+0000","last_fullsized":"2024-05-08T04:41:09.106605+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:24.317182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350109+0000","last_change":"2024-05-08T04:41:08.139854+0000","last_active":"2024-05-08T04:41:10.350109+0000","last_peered":"2024-05-08T04:41:10.350109+0000","last_clean":"2024-05-08T04:41:10.350109+0000","last_became_active":"2024-05-08T04:41:08.139568+0000","last_became_peered":"2024-05-08T04:41:08.139568+0000","last_unstale":"2024-05-08T04:41:10.350109+0000","last_undegraded":"2024-05-08T04:41:10.350109+0000","last_fullsized":"2024-05-08T04:41:10.350109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:59.667811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350009+0000","last_change":"2024-05-08T04:41:07.122490+0000","last_active":"2024-05-08T04:41:10.350009+0000","last_peered":"2024-05-08T04:41:10.350009+0000","last_clean":"2024-05-08T04:41:10.350009+0000","last_became_active":"2024-05-08T04:41:07.122220+0000","last_became_peered":"2024-05-08T04:41:07.122220+0000","last_unstale":"2024-05-08T04:41:10.350009+0000","last_undegraded":"2024-05-08T04:41:10.350009+0000","last_fullsized":"2024-05-08T04:41:10.350009+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:37.570345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398544+0000","last_change":"2024-05-08T04:41:08.113335+0000","last_active":"2024-05-08T04:41:11.398544+0000","last_peered":"2024-05-08T04:41:11.398544+0000","last_clean":"2024-05-08T04:41:11.398544+0000","last_became_active":"2024-05-08T04:41:08.113016+0000","last_became_peered":"2024-05-08T04:41:08.113016+0000","last_unstale":"2024-05-08T04:41:11.398544+0000","last_undegraded":"2024-05-08T04:41:11.398544+0000","last_fullsized":"2024-05-08T04:41:11.398544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:40.632007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396788+0000","last_change":"2024-05-08T04:41:07.114940+0000","last_active":"2024-05-08T04:41:11.396788+0000","last_peered":"2024-05-08T04:41:11.396788+0000","last_clean":"2024-05-08T04:41:11.396788+0000","last_became_active":"2024-05-08T04:41:07.114663+0000","last_became_peered":"2024-05-08T04:41:07.114663+0000","last_unstale":"2024-05-08T04:41:11.396788+0000","last_undegraded":"2024-05-08T04:41:11.396788+0000","last_fullsized":"2024-05-08T04:41:11.396788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:10.342497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396973+0000","last_change":"2024-05-08T04:41:08.131304+0000","last_active":"2024-05-08T04:41:11.396973+0000","last_peered":"2024-05-08T04:41:11.396973+0000","last_clean":"2024-05-08T04:41:11.396973+0000","last_became_active":"2024-05-08T04:41:08.131007+0000","last_became_peered":"2024-05-08T04:41:08.131007+0000","last_unstale":"2024-05-08T04:41:11.396973+0000","last_undegraded":"2024-05-08T04:41:11.396973+0000","last_fullsized":"2024-05-08T04:41:11.396973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:25.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107490+0000","last_change":"2024-05-08T04:41:07.121852+0000","last_active":"2024-05-08T04:41:09.107490+0000","last_peered":"2024-05-08T04:41:09.107490+0000","last_clean":"2024-05-08T04:41:09.107490+0000","last_became_active":"2024-05-08T04:41:07.121549+0000","last_became_peered":"2024-05-08T04:41:07.121549+0000","last_unstale":"2024-05-08T04:41:09.107490+0000","last_undegraded":"2024-05-08T04:41:09.107490+0000","last_fullsized":"2024-05-08T04:41:09.107490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:20.992283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397753+0000","last_change":"2024-05-08T04:41:08.133463+0000","last_active":"2024-05-08T04:41:11.397753+0000","last_peered":"2024-05-08T04:41:11.397753+0000","last_clean":"2024-05-08T04:41:11.397753+0000","last_became_active":"2024-05-08T04:41:08.133119+0000","last_became_peered":"2024-05-08T04:41:08.133119+0000","last_unstale":"2024-05-08T04:41:11.397753+0000","last_undegraded":"2024-05-08T04:41:11.397753+0000","last_fullsized":"2024-05-08T04:41:11.397753+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:59.304053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398088+0000","last_change":"2024-05-08T04:41:07.125169+0000","last_active":"2024-05-08T04:41:11.398088+0000","last_peered":"2024-05-08T04:41:11.398088+0000","last_clean":"2024-05-08T04:41:11.398088+0000","last_became_active":"2024-05-08T04:41:07.124870+0000","last_became_peered":"2024-05-08T04:41:07.124870+0000","last_unstale":"2024-05-08T04:41:11.398088+0000","last_undegraded":"2024-05-08T04:41:11.398088+0000","last_fullsized":"2024-05-08T04:41:11.398088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:10.342497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403434+0000","last_change":"2024-05-08T04:41:08.135971+0000","last_active":"2024-05-08T04:41:10.403434+0000","last_peered":"2024-05-08T04:41:10.403434+0000","last_clean":"2024-05-08T04:41:10.403434+0000","last_became_active":"2024-05-08T04:41:08.131865+0000","last_became_peered":"2024-05-08T04:41:08.131865+0000","last_unstale":"2024-05-08T04:41:10.403434+0000","last_undegraded":"2024-05-08T04:41:10.403434+0000","last_fullsized":"2024-05-08T04:41:10.403434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:31.546708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398238+0000","last_change":"2024-05-08T04:41:07.121887+0000","last_active":"2024-05-08T04:41:11.398238+0000","last_peered":"2024-05-08T04:41:11.398238+0000","last_clean":"2024-05-08T04:41:11.398238+0000","last_became_active":"2024-05-08T04:41:07.121407+0000","last_became_peered":"2024-05-08T04:41:07.121407+0000","last_unstale":"2024-05-08T04:41:11.398238+0000","last_undegraded":"2024-05-08T04:41:11.398238+0000","last_fullsized":"2024-05-08T04:41:11.398238+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:28.853767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397599+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:11.397599+0000","last_peered":"2024-05-08T04:41:11.397599+0000","last_clean":"2024-05-08T04:41:11.397599+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:11.397599+0000","last_undegraded":"2024-05-08T04:41:11.397599+0000","last_fullsized":"2024-05-08T04:41:11.397599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":974798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":383906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":383904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":383884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":383926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":385426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":386042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1495040,"data_stored":974182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:16.715 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:16.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.856+0000 7fe976000640 1 Processor -- start 2024-05-08T04:41:16.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.856+0000 7fe976000640 1 -- start start 2024-05-08T04:41:16.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe976000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015ba70 0x7fe970160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe976000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe97015be50 0x7fe9700cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe976000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9700cd9f0 con 0x7fe9700aeda0 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe976000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97015fe70 con 0x7fe97015be50 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe976000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700cdcd0 con 0x7fe97015ba70 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe97015be50 0x7fe9700cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe97015be50 0x7fe9700cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41848/0 (socket says 172.21.15.164:41848) 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 -- 172.21.15.164:0/1235289404 learned_addr learned my addr 172.21.15.164:0/1235289404 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96fe00640 1 --1- 172.21.15.164:0/1235289404 >> v1:172.21.15.164:6789/0 conn(0x7fe9700aeda0 0x7fe9700b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59846/0 (socket says 172.21.15.164:59846) 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96f400640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015ba70 0x7fe970160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 -- 172.21.15.164:0/1235289404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015ba70 msgr2=0x7fe970160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:16.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015ba70 0x7fe970160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 -- 172.21.15.164:0/1235289404 >> v1:172.21.15.164:6789/0 conn(0x7fe9700aeda0 legacy=0x7fe9700b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.860+0000 7fe96ea00640 1 -- 172.21.15.164:0/1235289404 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97015f820 con 0x7fe97015be50 2024-05-08T04:41:16.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe96ea00640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015be50 0x7fe9700cf690 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009870 tx=0x7fe95c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ae1ea01c26f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:16.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe96e000640 1 -- 172.21.15.164:0/1235289404 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0ab640 con 0x7fe97015be50 2024-05-08T04:41:16.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe96e000640 1 -- 172.21.15.164:0/1235289404 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0b0070 con 0x7fe97015be50 2024-05-08T04:41:16.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe96e000640 1 -- 172.21.15.164:0/1235289404 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0aba00 con 0x7fe97015be50 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- 172.21.15.164:0/1235289404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015be50 msgr2=0x7fe9700cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015be50 0x7fe9700cf690 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009870 tx=0x7fe95c01c530 comp rx=0 tx=0).stop 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- 172.21.15.164:0/1235289404 shutdown_connections 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015be50 0x7fe9700cf690 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- 172.21.15.164:0/1235289404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015ba70 0x7fe970160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- 172.21.15.164:0/1235289404 >> 172.21.15.164:0/1235289404 conn(0x7fe97006ac90 msgr2=0x7fe97006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- 172.21.15.164:0/1235289404 shutdown_connections 2024-05-08T04:41:16.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- 172.21.15.164:0/1235289404 wait complete. 2024-05-08T04:41:16.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 Processor -- start 2024-05-08T04:41:16.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- start start 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 0x7fe97014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015be50 0x7fe970154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970168d40 con 0x7fe97015ba70 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970168bc0 con 0x7fe9700aeda0 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.864+0000 7fe976000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe970168ec0 con 0x7fe97015be50 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41860/0 (socket says 172.21.15.164:41860) 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 -- 172.21.15.164:0/2916122271 learned_addr learned my addr 172.21.15.164:0/2916122271 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96f400640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 0x7fe97014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96fe00640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015be50 0x7fe970154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 -- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015be50 msgr2=0x7fe970154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:16.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015be50 0x7fe970154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 -- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 msgr2=0x7fe97014d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 0x7fe97014d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 -- 172.21.15.164:0/2916122271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97014d860 con 0x7fe97015ba70 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96f400640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 0x7fe97014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe96ea00640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ad040 tx=0x7fe95c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0be3f0 con 0x7fe97015ba70 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0be590 con 0x7fe97015ba70 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9700b0100 con 0x7fe97015ba70 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0be980 con 0x7fe97015ba70 2024-05-08T04:41:16.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9700b06c0 con 0x7fe97015ba70 2024-05-08T04:41:16.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970069d60 con 0x7fe97015ba70 2024-05-08T04:41:16.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.868+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fe95c0c63f0 con 0x7fe97015ba70 2024-05-08T04:41:16.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.872+0000 7fe967400640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 0x7fe950069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:16.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.872+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe95c11fd20 con 0x7fe97015ba70 2024-05-08T04:41:16.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.872+0000 7fe96f400640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 0x7fe950069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:16.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.872+0000 7fe96f400640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 0x7fe950069810 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9600074c0 tx=0x7fe96000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:16.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:16.876+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe95c016150 con 0x7fe97015ba70 2024-05-08T04:41:17.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.024+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe97006e210 con 0x7fe950067350 2024-05-08T04:41:17.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.032+0000 7fe967400640 1 -- 172.21.15.164:0/2916122271 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7fe97006e210 con 0x7fe950067350 2024-05-08T04:41:17.039 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:17.040 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:17.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 msgr2=0x7fe950069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 0x7fe950069810 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9600074c0 tx=0x7fe96000b040 comp rx=0 tx=0).stop 2024-05-08T04:41:17.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 msgr2=0x7fe970153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0ad040 tx=0x7fe95c01ad70 comp rx=0 tx=0).stop 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 shutdown_connections 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe950067350 0x7fe950069810 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97015be50 0x7fe970154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe97015ba70 0x7fe970153c20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 --2- 172.21.15.164:0/2916122271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9700aeda0 0x7fe97014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 >> 172.21.15.164:0/2916122271 conn(0x7fe97006ac90 msgr2=0x7fe9701587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 shutdown_connections 2024-05-08T04:41:17.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.036+0000 7fe976000640 1 -- 172.21.15.164:0/2916122271 wait complete. 2024-05-08T04:41:17.059 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-05-08T04:41:15.463884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245232,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729488,"statfs":{"total":773094113280,"available":772842995712,"internally_reserved":0,"allocated":8462336,"data_stored":4256068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001714"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398898+0000","last_change":"2024-05-08T04:41:08.127072+0000","last_active":"2024-05-08T04:41:11.398898+0000","last_peered":"2024-05-08T04:41:11.398898+0000","last_clean":"2024-05-08T04:41:11.398898+0000","last_became_active":"2024-05-08T04:41:08.126826+0000","last_became_peered":"2024-05-08T04:41:08.126826+0000","last_unstale":"2024-05-08T04:41:11.398898+0000","last_undegraded":"2024-05-08T04:41:11.398898+0000","last_fullsized":"2024-05-08T04:41:11.398898+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:07.240152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398306+0000","last_change":"2024-05-08T04:41:08.128226+0000","last_active":"2024-05-08T04:41:11.398306+0000","last_peered":"2024-05-08T04:41:11.398306+0000","last_clean":"2024-05-08T04:41:11.398306+0000","last_became_active":"2024-05-08T04:41:08.127832+0000","last_became_peered":"2024-05-08T04:41:08.127832+0000","last_unstale":"2024-05-08T04:41:11.398306+0000","last_undegraded":"2024-05-08T04:41:11.398306+0000","last_fullsized":"2024-05-08T04:41:11.398306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.882813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403536+0000","last_change":"2024-05-08T04:41:08.134248+0000","last_active":"2024-05-08T04:41:10.403536+0000","last_peered":"2024-05-08T04:41:10.403536+0000","last_clean":"2024-05-08T04:41:10.403536+0000","last_became_active":"2024-05-08T04:41:08.132591+0000","last_became_peered":"2024-05-08T04:41:08.132591+0000","last_unstale":"2024-05-08T04:41:10.403536+0000","last_undegraded":"2024-05-08T04:41:10.403536+0000","last_fullsized":"2024-05-08T04:41:10.403536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:42:23.370715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397021+0000","last_change":"2024-05-08T04:41:08.122766+0000","last_active":"2024-05-08T04:41:11.397021+0000","last_peered":"2024-05-08T04:41:11.397021+0000","last_clean":"2024-05-08T04:41:11.397021+0000","last_became_active":"2024-05-08T04:41:08.122519+0000","last_became_peered":"2024-05-08T04:41:08.122519+0000","last_unstale":"2024-05-08T04:41:11.397021+0000","last_undegraded":"2024-05-08T04:41:11.397021+0000","last_fullsized":"2024-05-08T04:41:11.397021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:45.594443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349941+0000","last_change":"2024-05-08T04:41:08.127668+0000","last_active":"2024-05-08T04:41:10.349941+0000","last_peered":"2024-05-08T04:41:10.349941+0000","last_clean":"2024-05-08T04:41:10.349941+0000","last_became_active":"2024-05-08T04:41:08.126586+0000","last_became_peered":"2024-05-08T04:41:08.126586+0000","last_unstale":"2024-05-08T04:41:10.349941+0000","last_undegraded":"2024-05-08T04:41:10.349941+0000","last_fullsized":"2024-05-08T04:41:10.349941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:10.023708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397984+0000","last_change":"2024-05-08T04:41:08.128568+0000","last_active":"2024-05-08T04:41:11.397984+0000","last_peered":"2024-05-08T04:41:11.397984+0000","last_clean":"2024-05-08T04:41:11.397984+0000","last_became_active":"2024-05-08T04:41:08.128337+0000","last_became_peered":"2024-05-08T04:41:08.128337+0000","last_unstale":"2024-05-08T04:41:11.397984+0000","last_undegraded":"2024-05-08T04:41:11.397984+0000","last_fullsized":"2024-05-08T04:41:11.397984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:15.637405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107253+0000","last_change":"2024-05-08T04:41:08.132376+0000","last_active":"2024-05-08T04:41:09.107253+0000","last_peered":"2024-05-08T04:41:09.107253+0000","last_clean":"2024-05-08T04:41:09.107253+0000","last_became_active":"2024-05-08T04:41:08.132069+0000","last_became_peered":"2024-05-08T04:41:08.132069+0000","last_unstale":"2024-05-08T04:41:09.107253+0000","last_undegraded":"2024-05-08T04:41:09.107253+0000","last_fullsized":"2024-05-08T04:41:09.107253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:17.664203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397939+0000","last_change":"2024-05-08T04:41:08.113444+0000","last_active":"2024-05-08T04:41:11.397939+0000","last_peered":"2024-05-08T04:41:11.397939+0000","last_clean":"2024-05-08T04:41:11.397939+0000","last_became_active":"2024-05-08T04:41:08.113163+0000","last_became_peered":"2024-05-08T04:41:08.113163+0000","last_unstale":"2024-05-08T04:41:11.397939+0000","last_undegraded":"2024-05-08T04:41:11.397939+0000","last_fullsized":"2024-05-08T04:41:11.397939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:56.557764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350506+0000","last_change":"2024-05-08T04:41:08.131823+0000","last_active":"2024-05-08T04:41:10.350506+0000","last_peered":"2024-05-08T04:41:10.350506+0000","last_clean":"2024-05-08T04:41:10.350506+0000","last_became_active":"2024-05-08T04:41:08.131333+0000","last_became_peered":"2024-05-08T04:41:08.131333+0000","last_unstale":"2024-05-08T04:41:10.350506+0000","last_undegraded":"2024-05-08T04:41:10.350506+0000","last_fullsized":"2024-05-08T04:41:10.350506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:18:15.224219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107755+0000","last_change":"2024-05-08T04:41:08.142366+0000","last_active":"2024-05-08T04:41:09.107755+0000","last_peered":"2024-05-08T04:41:09.107755+0000","last_clean":"2024-05-08T04:41:09.107755+0000","last_became_active":"2024-05-08T04:41:08.142048+0000","last_became_peered":"2024-05-08T04:41:08.142048+0000","last_unstale":"2024-05-08T04:41:09.107755+0000","last_undegraded":"2024-05-08T04:41:09.107755+0000","last_fullsized":"2024-05-08T04:41:09.107755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:57.264085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398738+0000","last_change":"2024-05-08T04:41:08.132008+0000","last_active":"2024-05-08T04:41:11.398738+0000","last_peered":"2024-05-08T04:41:11.398738+0000","last_clean":"2024-05-08T04:41:11.398738+0000","last_became_active":"2024-05-08T04:41:08.131176+0000","last_became_peered":"2024-05-08T04:41:08.131176+0000","last_unstale":"2024-05-08T04:41:11.398738+0000","last_undegraded":"2024-05-08T04:41:11.398738+0000","last_fullsized":"2024-05-08T04:41:11.398738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:10.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397076+0000","last_change":"2024-05-08T04:41:08.136053+0000","last_active":"2024-05-08T04:41:11.397076+0000","last_peered":"2024-05-08T04:41:11.397076+0000","last_clean":"2024-05-08T04:41:11.397076+0000","last_became_active":"2024-05-08T04:41:08.135577+0000","last_became_peered":"2024-05-08T04:41:08.135577+0000","last_unstale":"2024-05-08T04:41:11.397076+0000","last_undegraded":"2024-05-08T04:41:11.397076+0000","last_fullsized":"2024-05-08T04:41:11.397076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:31:46.547484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106637+0000","last_change":"2024-05-08T04:41:08.137581+0000","last_active":"2024-05-08T04:41:09.106637+0000","last_peered":"2024-05-08T04:41:09.106637+0000","last_clean":"2024-05-08T04:41:09.106637+0000","last_became_active":"2024-05-08T04:41:08.137323+0000","last_became_peered":"2024-05-08T04:41:08.137323+0000","last_unstale":"2024-05-08T04:41:09.106637+0000","last_undegraded":"2024-05-08T04:41:09.106637+0000","last_fullsized":"2024-05-08T04:41:09.106637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:44.499342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350057+0000","last_change":"2024-05-08T04:41:08.138410+0000","last_active":"2024-05-08T04:41:10.350057+0000","last_peered":"2024-05-08T04:41:10.350057+0000","last_clean":"2024-05-08T04:41:10.350057+0000","last_became_active":"2024-05-08T04:41:08.138151+0000","last_became_peered":"2024-05-08T04:41:08.138151+0000","last_unstale":"2024-05-08T04:41:10.350057+0000","last_undegraded":"2024-05-08T04:41:10.350057+0000","last_fullsized":"2024-05-08T04:41:10.350057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:28.065788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106197+0000","last_change":"2024-05-08T04:41:08.135434+0000","last_active":"2024-05-08T04:41:09.106197+0000","last_peered":"2024-05-08T04:41:09.106197+0000","last_clean":"2024-05-08T04:41:09.106197+0000","last_became_active":"2024-05-08T04:41:08.134807+0000","last_became_peered":"2024-05-08T04:41:08.134807+0000","last_unstale":"2024-05-08T04:41:09.106197+0000","last_undegraded":"2024-05-08T04:41:09.106197+0000","last_fullsized":"2024-05-08T04:41:09.106197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:35.735387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107340+0000","last_change":"2024-05-08T04:41:08.129549+0000","last_active":"2024-05-08T04:41:09.107340+0000","last_peered":"2024-05-08T04:41:09.107340+0000","last_clean":"2024-05-08T04:41:09.107340+0000","last_became_active":"2024-05-08T04:41:08.129329+0000","last_became_peered":"2024-05-08T04:41:08.129329+0000","last_unstale":"2024-05-08T04:41:09.107340+0000","last_undegraded":"2024-05-08T04:41:09.107340+0000","last_fullsized":"2024-05-08T04:41:09.107340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:40.632007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350518+0000","last_change":"2024-05-08T04:41:08.131597+0000","last_active":"2024-05-08T04:41:10.350518+0000","last_peered":"2024-05-08T04:41:10.350518+0000","last_clean":"2024-05-08T04:41:10.350518+0000","last_became_active":"2024-05-08T04:41:08.130988+0000","last_became_peered":"2024-05-08T04:41:08.130988+0000","last_unstale":"2024-05-08T04:41:10.350518+0000","last_undegraded":"2024-05-08T04:41:10.350518+0000","last_fullsized":"2024-05-08T04:41:10.350518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.550232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397213+0000","last_change":"2024-05-08T04:41:08.113183+0000","last_active":"2024-05-08T04:41:11.397213+0000","last_peered":"2024-05-08T04:41:11.397213+0000","last_clean":"2024-05-08T04:41:11.397213+0000","last_became_active":"2024-05-08T04:41:08.112872+0000","last_became_peered":"2024-05-08T04:41:08.112872+0000","last_unstale":"2024-05-08T04:41:11.397213+0000","last_undegraded":"2024-05-08T04:41:11.397213+0000","last_fullsized":"2024-05-08T04:41:11.397213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398674+0000","last_change":"2024-05-08T04:41:08.115306+0000","last_active":"2024-05-08T04:41:11.398674+0000","last_peered":"2024-05-08T04:41:11.398674+0000","last_clean":"2024-05-08T04:41:11.398674+0000","last_became_active":"2024-05-08T04:41:08.115034+0000","last_became_peered":"2024-05-08T04:41:08.115034+0000","last_unstale":"2024-05-08T04:41:11.398674+0000","last_undegraded":"2024-05-08T04:41:11.398674+0000","last_fullsized":"2024-05-08T04:41:11.398674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:07.340180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106655+0000","last_change":"2024-05-08T04:41:08.138896+0000","last_active":"2024-05-08T04:41:09.106655+0000","last_peered":"2024-05-08T04:41:09.106655+0000","last_clean":"2024-05-08T04:41:09.106655+0000","last_became_active":"2024-05-08T04:41:08.138640+0000","last_became_peered":"2024-05-08T04:41:08.138640+0000","last_unstale":"2024-05-08T04:41:09.106655+0000","last_undegraded":"2024-05-08T04:41:09.106655+0000","last_fullsized":"2024-05-08T04:41:09.106655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:25.200286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108325+0000","last_change":"2024-05-08T04:41:08.128257+0000","last_active":"2024-05-08T04:41:09.108325+0000","last_peered":"2024-05-08T04:41:09.108325+0000","last_clean":"2024-05-08T04:41:09.108325+0000","last_became_active":"2024-05-08T04:41:08.127977+0000","last_became_peered":"2024-05-08T04:41:08.127977+0000","last_unstale":"2024-05-08T04:41:09.108325+0000","last_undegraded":"2024-05-08T04:41:09.108325+0000","last_fullsized":"2024-05-08T04:41:09.108325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:16.067442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108286+0000","last_change":"2024-05-08T04:41:08.142408+0000","last_active":"2024-05-08T04:41:09.108286+0000","last_peered":"2024-05-08T04:41:09.108286+0000","last_clean":"2024-05-08T04:41:09.108286+0000","last_became_active":"2024-05-08T04:41:08.142126+0000","last_became_peered":"2024-05-08T04:41:08.142126+0000","last_unstale":"2024-05-08T04:41:09.108286+0000","last_undegraded":"2024-05-08T04:41:09.108286+0000","last_fullsized":"2024-05-08T04:41:09.108286+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:50.631832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397756+0000","last_change":"2024-05-08T04:41:08.120012+0000","last_active":"2024-05-08T04:41:11.397756+0000","last_peered":"2024-05-08T04:41:11.397756+0000","last_clean":"2024-05-08T04:41:11.397756+0000","last_became_active":"2024-05-08T04:41:08.119719+0000","last_became_peered":"2024-05-08T04:41:08.119719+0000","last_unstale":"2024-05-08T04:41:11.397756+0000","last_undegraded":"2024-05-08T04:41:11.397756+0000","last_fullsized":"2024-05-08T04:41:11.397756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:01.724943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397596+0000","last_change":"2024-05-08T04:41:08.121568+0000","last_active":"2024-05-08T04:41:11.397596+0000","last_peered":"2024-05-08T04:41:11.397596+0000","last_clean":"2024-05-08T04:41:11.397596+0000","last_became_active":"2024-05-08T04:41:08.121297+0000","last_became_peered":"2024-05-08T04:41:08.121297+0000","last_unstale":"2024-05-08T04:41:11.397596+0000","last_undegraded":"2024-05-08T04:41:11.397596+0000","last_fullsized":"2024-05-08T04:41:11.397596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:41:52.288134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108018+0000","last_change":"2024-05-08T04:41:08.129042+0000","last_active":"2024-05-08T04:41:09.108018+0000","last_peered":"2024-05-08T04:41:09.108018+0000","last_clean":"2024-05-08T04:41:09.108018+0000","last_became_active":"2024-05-08T04:41:08.125396+0000","last_became_peered":"2024-05-08T04:41:08.125396+0000","last_unstale":"2024-05-08T04:41:09.108018+0000","last_undegraded":"2024-05-08T04:41:09.108018+0000","last_fullsized":"2024-05-08T04:41:09.108018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:58.986860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108322+0000","last_change":"2024-05-08T04:41:08.135386+0000","last_active":"2024-05-08T04:41:09.108322+0000","last_peered":"2024-05-08T04:41:09.108322+0000","last_clean":"2024-05-08T04:41:09.108322+0000","last_became_active":"2024-05-08T04:41:08.135092+0000","last_became_peered":"2024-05-08T04:41:08.135092+0000","last_unstale":"2024-05-08T04:41:09.108322+0000","last_undegraded":"2024-05-08T04:41:09.108322+0000","last_fullsized":"2024-05-08T04:41:09.108322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:25.328081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403586+0000","last_change":"2024-05-08T04:41:08.130633+0000","last_active":"2024-05-08T04:41:10.403586+0000","last_peered":"2024-05-08T04:41:10.403586+0000","last_clean":"2024-05-08T04:41:10.403586+0000","last_became_active":"2024-05-08T04:41:08.130304+0000","last_became_peered":"2024-05-08T04:41:08.130304+0000","last_unstale":"2024-05-08T04:41:10.403586+0000","last_undegraded":"2024-05-08T04:41:10.403586+0000","last_fullsized":"2024-05-08T04:41:10.403586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:47.871026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108426+0000","last_change":"2024-05-08T04:41:08.142669+0000","last_active":"2024-05-08T04:41:09.108426+0000","last_peered":"2024-05-08T04:41:09.108426+0000","last_clean":"2024-05-08T04:41:09.108426+0000","last_became_active":"2024-05-08T04:41:08.142339+0000","last_became_peered":"2024-05-08T04:41:08.142339+0000","last_unstale":"2024-05-08T04:41:09.108426+0000","last_undegraded":"2024-05-08T04:41:09.108426+0000","last_fullsized":"2024-05-08T04:41:09.108426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:54.892363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403434+0000","last_change":"2024-05-08T04:41:08.140417+0000","last_active":"2024-05-08T04:41:10.403434+0000","last_peered":"2024-05-08T04:41:10.403434+0000","last_clean":"2024-05-08T04:41:10.403434+0000","last_became_active":"2024-05-08T04:41:08.140178+0000","last_became_peered":"2024-05-08T04:41:08.140178+0000","last_unstale":"2024-05-08T04:41:10.403434+0000","last_undegraded":"2024-05-08T04:41:10.403434+0000","last_fullsized":"2024-05-08T04:41:10.403434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:56:54.293327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107502+0000","last_change":"2024-05-08T04:41:08.123508+0000","last_active":"2024-05-08T04:41:09.107502+0000","last_peered":"2024-05-08T04:41:09.107502+0000","last_clean":"2024-05-08T04:41:09.107502+0000","last_became_active":"2024-05-08T04:41:08.123201+0000","last_became_peered":"2024-05-08T04:41:08.123201+0000","last_unstale":"2024-05-08T04:41:09.107502+0000","last_undegraded":"2024-05-08T04:41:09.107502+0000","last_fullsized":"2024-05-08T04:41:09.107502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:58.063316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108042+0000","last_change":"2024-05-08T04:41:08.137226+0000","last_active":"2024-05-08T04:41:09.108042+0000","last_peered":"2024-05-08T04:41:09.108042+0000","last_clean":"2024-05-08T04:41:09.108042+0000","last_became_active":"2024-05-08T04:41:08.136952+0000","last_became_peered":"2024-05-08T04:41:08.136952+0000","last_unstale":"2024-05-08T04:41:09.108042+0000","last_undegraded":"2024-05-08T04:41:09.108042+0000","last_fullsized":"2024-05-08T04:41:09.108042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:19.622961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397997+0000","last_change":"2024-05-08T04:41:08.135567+0000","last_active":"2024-05-08T04:41:11.397997+0000","last_peered":"2024-05-08T04:41:11.397997+0000","last_clean":"2024-05-08T04:41:11.397997+0000","last_became_active":"2024-05-08T04:41:08.135134+0000","last_became_peered":"2024-05-08T04:41:08.135134+0000","last_unstale":"2024-05-08T04:41:11.397997+0000","last_undegraded":"2024-05-08T04:41:11.397997+0000","last_fullsized":"2024-05-08T04:41:11.397997+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:20.277016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398610+0000","last_change":"2024-05-08T04:41:08.114608+0000","last_active":"2024-05-08T04:41:11.398610+0000","last_peered":"2024-05-08T04:41:11.398610+0000","last_clean":"2024-05-08T04:41:11.398610+0000","last_became_active":"2024-05-08T04:41:08.114224+0000","last_became_peered":"2024-05-08T04:41:08.114224+0000","last_unstale":"2024-05-08T04:41:11.398610+0000","last_undegraded":"2024-05-08T04:41:11.398610+0000","last_fullsized":"2024-05-08T04:41:11.398610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:25.200286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.399076+0000","last_change":"2024-05-08T04:41:07.123950+0000","last_active":"2024-05-08T04:41:11.399076+0000","last_peered":"2024-05-08T04:41:11.399076+0000","last_clean":"2024-05-08T04:41:11.399076+0000","last_became_active":"2024-05-08T04:41:07.123577+0000","last_became_peered":"2024-05-08T04:41:07.123577+0000","last_unstale":"2024-05-08T04:41:11.399076+0000","last_undegraded":"2024-05-08T04:41:11.399076+0000","last_fullsized":"2024-05-08T04:41:11.399076+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:16.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350586+0000","last_change":"2024-05-08T04:41:08.132345+0000","last_active":"2024-05-08T04:41:10.350586+0000","last_peered":"2024-05-08T04:41:10.350586+0000","last_clean":"2024-05-08T04:41:10.350586+0000","last_became_active":"2024-05-08T04:41:08.132056+0000","last_became_peered":"2024-05-08T04:41:08.132056+0000","last_unstale":"2024-05-08T04:41:10.350586+0000","last_undegraded":"2024-05-08T04:41:10.350586+0000","last_fullsized":"2024-05-08T04:41:10.350586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:25.447697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106753+0000","last_change":"2024-05-08T04:41:07.122634+0000","last_active":"2024-05-08T04:41:09.106753+0000","last_peered":"2024-05-08T04:41:09.106753+0000","last_clean":"2024-05-08T04:41:09.106753+0000","last_became_active":"2024-05-08T04:41:07.122307+0000","last_became_peered":"2024-05-08T04:41:07.122307+0000","last_unstale":"2024-05-08T04:41:09.106753+0000","last_undegraded":"2024-05-08T04:41:09.106753+0000","last_fullsized":"2024-05-08T04:41:09.106753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:33.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398222+0000","last_change":"2024-05-08T04:41:08.121693+0000","last_active":"2024-05-08T04:41:11.398222+0000","last_peered":"2024-05-08T04:41:11.398222+0000","last_clean":"2024-05-08T04:41:11.398222+0000","last_became_active":"2024-05-08T04:41:08.120911+0000","last_became_peered":"2024-05-08T04:41:08.120911+0000","last_unstale":"2024-05-08T04:41:11.398222+0000","last_undegraded":"2024-05-08T04:41:11.398222+0000","last_fullsized":"2024-05-08T04:41:11.398222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:49.132730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107858+0000","last_change":"2024-05-08T04:41:07.124366+0000","last_active":"2024-05-08T04:41:09.107858+0000","last_peered":"2024-05-08T04:41:09.107858+0000","last_clean":"2024-05-08T04:41:09.107858+0000","last_became_active":"2024-05-08T04:41:07.124039+0000","last_became_peered":"2024-05-08T04:41:07.124039+0000","last_unstale":"2024-05-08T04:41:09.107858+0000","last_undegraded":"2024-05-08T04:41:09.107858+0000","last_fullsized":"2024-05-08T04:41:09.107858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:37.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398889+0000","last_change":"2024-05-08T04:41:08.135532+0000","last_active":"2024-05-08T04:41:11.398889+0000","last_peered":"2024-05-08T04:41:11.398889+0000","last_clean":"2024-05-08T04:41:11.398889+0000","last_became_active":"2024-05-08T04:41:08.135263+0000","last_became_peered":"2024-05-08T04:41:08.135263+0000","last_unstale":"2024-05-08T04:41:11.398889+0000","last_undegraded":"2024-05-08T04:41:11.398889+0000","last_fullsized":"2024-05-08T04:41:11.398889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:25:57.683492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398672+0000","last_change":"2024-05-08T04:41:07.126964+0000","last_active":"2024-05-08T04:41:11.398672+0000","last_peered":"2024-05-08T04:41:11.398672+0000","last_clean":"2024-05-08T04:41:11.398672+0000","last_became_active":"2024-05-08T04:41:07.126433+0000","last_became_peered":"2024-05-08T04:41:07.126433+0000","last_unstale":"2024-05-08T04:41:11.398672+0000","last_undegraded":"2024-05-08T04:41:11.398672+0000","last_fullsized":"2024-05-08T04:41:11.398672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:18:53.603895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108380+0000","last_change":"2024-05-08T04:41:08.140083+0000","last_active":"2024-05-08T04:41:09.108380+0000","last_peered":"2024-05-08T04:41:09.108380+0000","last_clean":"2024-05-08T04:41:09.108380+0000","last_became_active":"2024-05-08T04:41:08.137922+0000","last_became_peered":"2024-05-08T04:41:08.137922+0000","last_unstale":"2024-05-08T04:41:09.108380+0000","last_undegraded":"2024-05-08T04:41:09.108380+0000","last_fullsized":"2024-05-08T04:41:09.108380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:44.006515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403542+0000","last_change":"2024-05-08T04:41:07.118893+0000","last_active":"2024-05-08T04:41:10.403542+0000","last_peered":"2024-05-08T04:41:10.403542+0000","last_clean":"2024-05-08T04:41:10.403542+0000","last_became_active":"2024-05-08T04:41:07.117180+0000","last_became_peered":"2024-05-08T04:41:07.117180+0000","last_unstale":"2024-05-08T04:41:10.403542+0000","last_undegraded":"2024-05-08T04:41:10.403542+0000","last_fullsized":"2024-05-08T04:41:10.403542+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:46:54.270506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108349+0000","last_change":"2024-05-08T04:41:08.126629+0000","last_active":"2024-05-08T04:41:09.108349+0000","last_peered":"2024-05-08T04:41:09.108349+0000","last_clean":"2024-05-08T04:41:09.108349+0000","last_became_active":"2024-05-08T04:41:08.126149+0000","last_became_peered":"2024-05-08T04:41:08.126149+0000","last_unstale":"2024-05-08T04:41:09.108349+0000","last_undegraded":"2024-05-08T04:41:09.108349+0000","last_fullsized":"2024-05-08T04:41:09.108349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:29.429030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398176+0000","last_change":"2024-05-08T04:41:07.122025+0000","last_active":"2024-05-08T04:41:11.398176+0000","last_peered":"2024-05-08T04:41:11.398176+0000","last_clean":"2024-05-08T04:41:11.398176+0000","last_became_active":"2024-05-08T04:41:07.120512+0000","last_became_peered":"2024-05-08T04:41:07.120512+0000","last_unstale":"2024-05-08T04:41:11.398176+0000","last_undegraded":"2024-05-08T04:41:11.398176+0000","last_fullsized":"2024-05-08T04:41:11.398176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:34.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349880+0000","last_change":"2024-05-08T04:41:08.121064+0000","last_active":"2024-05-08T04:41:10.349880+0000","last_peered":"2024-05-08T04:41:10.349880+0000","last_clean":"2024-05-08T04:41:10.349880+0000","last_became_active":"2024-05-08T04:41:08.120796+0000","last_became_peered":"2024-05-08T04:41:08.120796+0000","last_unstale":"2024-05-08T04:41:10.349880+0000","last_undegraded":"2024-05-08T04:41:10.349880+0000","last_fullsized":"2024-05-08T04:41:10.349880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:06.329830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396793+0000","last_change":"2024-05-08T04:41:07.115029+0000","last_active":"2024-05-08T04:41:11.396793+0000","last_peered":"2024-05-08T04:41:11.396793+0000","last_clean":"2024-05-08T04:41:11.396793+0000","last_became_active":"2024-05-08T04:41:07.114753+0000","last_became_peered":"2024-05-08T04:41:07.114753+0000","last_unstale":"2024-05-08T04:41:11.396793+0000","last_undegraded":"2024-05-08T04:41:11.396793+0000","last_fullsized":"2024-05-08T04:41:11.396793+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:36.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106339+0000","last_change":"2024-05-08T04:41:08.136986+0000","last_active":"2024-05-08T04:41:09.106339+0000","last_peered":"2024-05-08T04:41:09.106339+0000","last_clean":"2024-05-08T04:41:09.106339+0000","last_became_active":"2024-05-08T04:41:08.136620+0000","last_became_peered":"2024-05-08T04:41:08.136620+0000","last_unstale":"2024-05-08T04:41:09.106339+0000","last_undegraded":"2024-05-08T04:41:09.106339+0000","last_fullsized":"2024-05-08T04:41:09.106339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:01.724943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349845+0000","last_change":"2024-05-08T04:41:07.113389+0000","last_active":"2024-05-08T04:41:10.349845+0000","last_peered":"2024-05-08T04:41:10.349845+0000","last_clean":"2024-05-08T04:41:10.349845+0000","last_became_active":"2024-05-08T04:41:07.113100+0000","last_became_peered":"2024-05-08T04:41:07.113100+0000","last_unstale":"2024-05-08T04:41:10.349845+0000","last_undegraded":"2024-05-08T04:41:10.349845+0000","last_fullsized":"2024-05-08T04:41:10.349845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:04.686860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108512+0000","last_change":"2024-05-08T04:41:08.133105+0000","last_active":"2024-05-08T04:41:09.108512+0000","last_peered":"2024-05-08T04:41:09.108512+0000","last_clean":"2024-05-08T04:41:09.108512+0000","last_became_active":"2024-05-08T04:41:08.132842+0000","last_became_peered":"2024-05-08T04:41:08.132842+0000","last_unstale":"2024-05-08T04:41:09.108512+0000","last_undegraded":"2024-05-08T04:41:09.108512+0000","last_fullsized":"2024-05-08T04:41:09.108512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:03.829699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108104+0000","last_change":"2024-05-08T04:41:07.113877+0000","last_active":"2024-05-08T04:41:09.108104+0000","last_peered":"2024-05-08T04:41:09.108104+0000","last_clean":"2024-05-08T04:41:09.108104+0000","last_became_active":"2024-05-08T04:41:07.112924+0000","last_became_peered":"2024-05-08T04:41:07.112924+0000","last_unstale":"2024-05-08T04:41:09.108104+0000","last_undegraded":"2024-05-08T04:41:09.108104+0000","last_fullsized":"2024-05-08T04:41:09.108104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:20:56.309860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398965+0000","last_change":"2024-05-08T04:41:08.113830+0000","last_active":"2024-05-08T04:41:11.398965+0000","last_peered":"2024-05-08T04:41:11.398965+0000","last_clean":"2024-05-08T04:41:11.398965+0000","last_became_active":"2024-05-08T04:41:08.112170+0000","last_became_peered":"2024-05-08T04:41:08.112170+0000","last_unstale":"2024-05-08T04:41:11.398965+0000","last_undegraded":"2024-05-08T04:41:11.398965+0000","last_fullsized":"2024-05-08T04:41:11.398965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:46.059471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398489+0000","last_change":"2024-05-08T04:41:07.127217+0000","last_active":"2024-05-08T04:41:11.398489+0000","last_peered":"2024-05-08T04:41:11.398489+0000","last_clean":"2024-05-08T04:41:11.398489+0000","last_became_active":"2024-05-08T04:41:07.126634+0000","last_became_peered":"2024-05-08T04:41:07.126634+0000","last_unstale":"2024-05-08T04:41:11.398489+0000","last_undegraded":"2024-05-08T04:41:11.398489+0000","last_fullsized":"2024-05-08T04:41:11.398489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:20.842180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398562+0000","last_change":"2024-05-08T04:41:08.132666+0000","last_active":"2024-05-08T04:41:11.398562+0000","last_peered":"2024-05-08T04:41:11.398562+0000","last_clean":"2024-05-08T04:41:11.398562+0000","last_became_active":"2024-05-08T04:41:08.132376+0000","last_became_peered":"2024-05-08T04:41:08.132376+0000","last_unstale":"2024-05-08T04:41:11.398562+0000","last_undegraded":"2024-05-08T04:41:11.398562+0000","last_fullsized":"2024-05-08T04:41:11.398562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:57.264085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403637+0000","last_change":"2024-05-08T04:41:07.120447+0000","last_active":"2024-05-08T04:41:10.403637+0000","last_peered":"2024-05-08T04:41:10.403637+0000","last_clean":"2024-05-08T04:41:10.403637+0000","last_became_active":"2024-05-08T04:41:07.120185+0000","last_became_peered":"2024-05-08T04:41:07.120185+0000","last_unstale":"2024-05-08T04:41:10.403637+0000","last_undegraded":"2024-05-08T04:41:10.403637+0000","last_fullsized":"2024-05-08T04:41:10.403637+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:44.441957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350141+0000","last_change":"2024-05-08T04:41:08.136327+0000","last_active":"2024-05-08T04:41:10.350141+0000","last_peered":"2024-05-08T04:41:10.350141+0000","last_clean":"2024-05-08T04:41:10.350141+0000","last_became_active":"2024-05-08T04:41:08.135591+0000","last_became_peered":"2024-05-08T04:41:08.135591+0000","last_unstale":"2024-05-08T04:41:10.350141+0000","last_undegraded":"2024-05-08T04:41:10.350141+0000","last_fullsized":"2024-05-08T04:41:10.350141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:34.338272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397173+0000","last_change":"2024-05-08T04:41:07.114558+0000","last_active":"2024-05-08T04:41:11.397173+0000","last_peered":"2024-05-08T04:41:11.397173+0000","last_clean":"2024-05-08T04:41:11.397173+0000","last_became_active":"2024-05-08T04:41:07.114273+0000","last_became_peered":"2024-05-08T04:41:07.114273+0000","last_unstale":"2024-05-08T04:41:11.397173+0000","last_undegraded":"2024-05-08T04:41:11.397173+0000","last_fullsized":"2024-05-08T04:41:11.397173+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:34.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397617+0000","last_change":"2024-05-08T04:41:08.123195+0000","last_active":"2024-05-08T04:41:11.397617+0000","last_peered":"2024-05-08T04:41:11.397617+0000","last_clean":"2024-05-08T04:41:11.397617+0000","last_became_active":"2024-05-08T04:41:08.122905+0000","last_became_peered":"2024-05-08T04:41:08.122905+0000","last_unstale":"2024-05-08T04:41:11.397617+0000","last_undegraded":"2024-05-08T04:41:11.397617+0000","last_fullsized":"2024-05-08T04:41:11.397617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:35.735387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396893+0000","last_change":"2024-05-08T04:41:07.119807+0000","last_active":"2024-05-08T04:41:11.396893+0000","last_peered":"2024-05-08T04:41:11.396893+0000","last_clean":"2024-05-08T04:41:11.396893+0000","last_became_active":"2024-05-08T04:41:07.119549+0000","last_became_peered":"2024-05-08T04:41:07.119549+0000","last_unstale":"2024-05-08T04:41:11.396893+0000","last_undegraded":"2024-05-08T04:41:11.396893+0000","last_fullsized":"2024-05-08T04:41:11.396893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:22.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108138+0000","last_change":"2024-05-08T04:41:08.129404+0000","last_active":"2024-05-08T04:41:09.108138+0000","last_peered":"2024-05-08T04:41:09.108138+0000","last_clean":"2024-05-08T04:41:09.108138+0000","last_became_active":"2024-05-08T04:41:08.129114+0000","last_became_peered":"2024-05-08T04:41:08.129114+0000","last_unstale":"2024-05-08T04:41:09.108138+0000","last_undegraded":"2024-05-08T04:41:09.108138+0000","last_fullsized":"2024-05-08T04:41:09.108138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:25.302556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398261+0000","last_change":"2024-05-08T04:41:07.118440+0000","last_active":"2024-05-08T04:41:11.398261+0000","last_peered":"2024-05-08T04:41:11.398261+0000","last_clean":"2024-05-08T04:41:11.398261+0000","last_became_active":"2024-05-08T04:41:07.118074+0000","last_became_peered":"2024-05-08T04:41:07.118074+0000","last_unstale":"2024-05-08T04:41:11.398261+0000","last_undegraded":"2024-05-08T04:41:11.398261+0000","last_fullsized":"2024-05-08T04:41:11.398261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:36.524117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106163+0000","last_change":"2024-05-08T04:41:08.140655+0000","last_active":"2024-05-08T04:41:09.106163+0000","last_peered":"2024-05-08T04:41:09.106163+0000","last_clean":"2024-05-08T04:41:09.106163+0000","last_became_active":"2024-05-08T04:41:08.140408+0000","last_became_peered":"2024-05-08T04:41:08.140408+0000","last_unstale":"2024-05-08T04:41:09.106163+0000","last_undegraded":"2024-05-08T04:41:09.106163+0000","last_fullsized":"2024-05-08T04:41:09.106163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:41:52.288134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107251+0000","last_change":"2024-05-08T04:41:07.111672+0000","last_active":"2024-05-08T04:41:09.107251+0000","last_peered":"2024-05-08T04:41:09.107251+0000","last_clean":"2024-05-08T04:41:09.107251+0000","last_became_active":"2024-05-08T04:41:07.110877+0000","last_became_peered":"2024-05-08T04:41:07.110877+0000","last_unstale":"2024-05-08T04:41:09.107251+0000","last_undegraded":"2024-05-08T04:41:09.107251+0000","last_fullsized":"2024-05-08T04:41:09.107251+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:48.121832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.349859+0000","last_change":"2024-05-08T04:41:08.139824+0000","last_active":"2024-05-08T04:41:10.349859+0000","last_peered":"2024-05-08T04:41:10.349859+0000","last_clean":"2024-05-08T04:41:10.349859+0000","last_became_active":"2024-05-08T04:41:08.139559+0000","last_became_peered":"2024-05-08T04:41:08.139559+0000","last_unstale":"2024-05-08T04:41:10.349859+0000","last_undegraded":"2024-05-08T04:41:10.349859+0000","last_fullsized":"2024-05-08T04:41:10.349859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:07.686483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107456+0000","last_change":"2024-05-08T04:41:07.120313+0000","last_active":"2024-05-08T04:41:09.107456+0000","last_peered":"2024-05-08T04:41:09.107456+0000","last_clean":"2024-05-08T04:41:09.107456+0000","last_became_active":"2024-05-08T04:41:07.120022+0000","last_became_peered":"2024-05-08T04:41:07.120022+0000","last_unstale":"2024-05-08T04:41:09.107456+0000","last_undegraded":"2024-05-08T04:41:09.107456+0000","last_fullsized":"2024-05-08T04:41:09.107456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:02.922831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107858+0000","last_change":"2024-05-08T04:41:08.130066+0000","last_active":"2024-05-08T04:41:09.107858+0000","last_peered":"2024-05-08T04:41:09.107858+0000","last_clean":"2024-05-08T04:41:09.107858+0000","last_became_active":"2024-05-08T04:41:08.129799+0000","last_became_peered":"2024-05-08T04:41:08.129799+0000","last_unstale":"2024-05-08T04:41:09.107858+0000","last_undegraded":"2024-05-08T04:41:09.107858+0000","last_fullsized":"2024-05-08T04:41:09.107858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:07.340180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398040+0000","last_change":"2024-05-08T04:41:07.128722+0000","last_active":"2024-05-08T04:41:11.398040+0000","last_peered":"2024-05-08T04:41:11.398040+0000","last_clean":"2024-05-08T04:41:11.398040+0000","last_became_active":"2024-05-08T04:41:07.128152+0000","last_became_peered":"2024-05-08T04:41:07.128152+0000","last_unstale":"2024-05-08T04:41:11.398040+0000","last_undegraded":"2024-05-08T04:41:11.398040+0000","last_fullsized":"2024-05-08T04:41:11.398040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:13.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.399102+0000","last_change":"2024-05-08T04:41:08.132342+0000","last_active":"2024-05-08T04:41:11.399102+0000","last_peered":"2024-05-08T04:41:11.399102+0000","last_clean":"2024-05-08T04:41:11.399102+0000","last_became_active":"2024-05-08T04:41:08.132038+0000","last_became_peered":"2024-05-08T04:41:08.132038+0000","last_unstale":"2024-05-08T04:41:11.399102+0000","last_undegraded":"2024-05-08T04:41:11.399102+0000","last_fullsized":"2024-05-08T04:41:11.399102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:57.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107950+0000","last_change":"2024-05-08T04:41:07.124431+0000","last_active":"2024-05-08T04:41:09.107950+0000","last_peered":"2024-05-08T04:41:09.107950+0000","last_clean":"2024-05-08T04:41:09.107950+0000","last_became_active":"2024-05-08T04:41:07.124047+0000","last_became_peered":"2024-05-08T04:41:07.124047+0000","last_unstale":"2024-05-08T04:41:09.107950+0000","last_undegraded":"2024-05-08T04:41:09.107950+0000","last_fullsized":"2024-05-08T04:41:09.107950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:03.276360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397180+0000","last_change":"2024-05-08T04:41:08.135819+0000","last_active":"2024-05-08T04:41:11.397180+0000","last_peered":"2024-05-08T04:41:11.397180+0000","last_clean":"2024-05-08T04:41:11.397180+0000","last_became_active":"2024-05-08T04:41:08.135204+0000","last_became_peered":"2024-05-08T04:41:08.135204+0000","last_unstale":"2024-05-08T04:41:11.397180+0000","last_undegraded":"2024-05-08T04:41:11.397180+0000","last_fullsized":"2024-05-08T04:41:11.397180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:22.003182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398671+0000","last_change":"2024-05-08T04:41:07.126275+0000","last_active":"2024-05-08T04:41:11.398671+0000","last_peered":"2024-05-08T04:41:11.398671+0000","last_clean":"2024-05-08T04:41:11.398671+0000","last_became_active":"2024-05-08T04:41:07.125938+0000","last_became_peered":"2024-05-08T04:41:07.125938+0000","last_unstale":"2024-05-08T04:41:11.398671+0000","last_undegraded":"2024-05-08T04:41:11.398671+0000","last_fullsized":"2024-05-08T04:41:11.398671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:33.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350233+0000","last_change":"2024-05-08T04:41:08.133530+0000","last_active":"2024-05-08T04:41:10.350233+0000","last_peered":"2024-05-08T04:41:10.350233+0000","last_clean":"2024-05-08T04:41:10.350233+0000","last_became_active":"2024-05-08T04:41:08.133246+0000","last_became_peered":"2024-05-08T04:41:08.133246+0000","last_unstale":"2024-05-08T04:41:10.350233+0000","last_undegraded":"2024-05-08T04:41:10.350233+0000","last_fullsized":"2024-05-08T04:41:10.350233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:27.447856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398603+0000","last_change":"2024-05-08T04:41:07.129166+0000","last_active":"2024-05-08T04:41:11.398603+0000","last_peered":"2024-05-08T04:41:11.398603+0000","last_clean":"2024-05-08T04:41:11.398603+0000","last_became_active":"2024-05-08T04:41:07.128785+0000","last_became_peered":"2024-05-08T04:41:07.128785+0000","last_unstale":"2024-05-08T04:41:11.398603+0000","last_undegraded":"2024-05-08T04:41:11.398603+0000","last_fullsized":"2024-05-08T04:41:11.398603+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:10:57.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403575+0000","last_change":"2024-05-08T04:41:08.132349+0000","last_active":"2024-05-08T04:41:10.403575+0000","last_peered":"2024-05-08T04:41:10.403575+0000","last_clean":"2024-05-08T04:41:10.403575+0000","last_became_active":"2024-05-08T04:41:08.132012+0000","last_became_peered":"2024-05-08T04:41:08.132012+0000","last_unstale":"2024-05-08T04:41:10.403575+0000","last_undegraded":"2024-05-08T04:41:10.403575+0000","last_fullsized":"2024-05-08T04:41:10.403575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:00:45.269537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398366+0000","last_change":"2024-05-08T04:41:07.122473+0000","last_active":"2024-05-08T04:41:11.398366+0000","last_peered":"2024-05-08T04:41:11.398366+0000","last_clean":"2024-05-08T04:41:11.398366+0000","last_became_active":"2024-05-08T04:41:07.122179+0000","last_became_peered":"2024-05-08T04:41:07.122179+0000","last_unstale":"2024-05-08T04:41:11.398366+0000","last_undegraded":"2024-05-08T04:41:11.398366+0000","last_fullsized":"2024-05-08T04:41:11.398366+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:22.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403438+0000","last_change":"2024-05-08T04:41:08.130647+0000","last_active":"2024-05-08T04:41:10.403438+0000","last_peered":"2024-05-08T04:41:10.403438+0000","last_clean":"2024-05-08T04:41:10.403438+0000","last_became_active":"2024-05-08T04:41:08.130316+0000","last_became_peered":"2024-05-08T04:41:08.130316+0000","last_unstale":"2024-05-08T04:41:10.403438+0000","last_undegraded":"2024-05-08T04:41:10.403438+0000","last_fullsized":"2024-05-08T04:41:10.403438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:06.854948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107889+0000","last_change":"2024-05-08T04:41:07.119592+0000","last_active":"2024-05-08T04:41:09.107889+0000","last_peered":"2024-05-08T04:41:09.107889+0000","last_clean":"2024-05-08T04:41:09.107889+0000","last_became_active":"2024-05-08T04:41:07.119253+0000","last_became_peered":"2024-05-08T04:41:07.119253+0000","last_unstale":"2024-05-08T04:41:09.107889+0000","last_undegraded":"2024-05-08T04:41:09.107889+0000","last_fullsized":"2024-05-08T04:41:09.107889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:19.266117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403410+0000","last_change":"2024-05-08T04:41:08.128660+0000","last_active":"2024-05-08T04:41:10.403410+0000","last_peered":"2024-05-08T04:41:10.403410+0000","last_clean":"2024-05-08T04:41:10.403410+0000","last_became_active":"2024-05-08T04:41:08.128340+0000","last_became_peered":"2024-05-08T04:41:08.128340+0000","last_unstale":"2024-05-08T04:41:10.403410+0000","last_undegraded":"2024-05-08T04:41:10.403410+0000","last_fullsized":"2024-05-08T04:41:10.403410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:12.389136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106070+0000","last_change":"2024-05-08T04:41:07.118353+0000","last_active":"2024-05-08T04:41:09.106070+0000","last_peered":"2024-05-08T04:41:09.106070+0000","last_clean":"2024-05-08T04:41:09.106070+0000","last_became_active":"2024-05-08T04:41:07.118022+0000","last_became_peered":"2024-05-08T04:41:07.118022+0000","last_unstale":"2024-05-08T04:41:09.106070+0000","last_undegraded":"2024-05-08T04:41:09.106070+0000","last_fullsized":"2024-05-08T04:41:09.106070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:24.291658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106778+0000","last_change":"2024-05-08T04:41:08.133842+0000","last_active":"2024-05-08T04:41:09.106778+0000","last_peered":"2024-05-08T04:41:09.106778+0000","last_clean":"2024-05-08T04:41:09.106778+0000","last_became_active":"2024-05-08T04:41:08.133594+0000","last_became_peered":"2024-05-08T04:41:08.133594+0000","last_unstale":"2024-05-08T04:41:09.106778+0000","last_undegraded":"2024-05-08T04:41:09.106778+0000","last_fullsized":"2024-05-08T04:41:09.106778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:10.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398042+0000","last_change":"2024-05-08T04:41:07.125972+0000","last_active":"2024-05-08T04:41:11.398042+0000","last_peered":"2024-05-08T04:41:11.398042+0000","last_clean":"2024-05-08T04:41:11.398042+0000","last_became_active":"2024-05-08T04:41:07.125631+0000","last_became_peered":"2024-05-08T04:41:07.125631+0000","last_unstale":"2024-05-08T04:41:11.398042+0000","last_undegraded":"2024-05-08T04:41:11.398042+0000","last_fullsized":"2024-05-08T04:41:11.398042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:24.785925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108367+0000","last_change":"2024-05-08T04:41:08.133318+0000","last_active":"2024-05-08T04:41:09.108367+0000","last_peered":"2024-05-08T04:41:09.108367+0000","last_clean":"2024-05-08T04:41:09.108367+0000","last_became_active":"2024-05-08T04:41:08.133028+0000","last_became_peered":"2024-05-08T04:41:08.133028+0000","last_unstale":"2024-05-08T04:41:09.108367+0000","last_undegraded":"2024-05-08T04:41:09.108367+0000","last_fullsized":"2024-05-08T04:41:09.108367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:34.383175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.108519+0000","last_change":"2024-05-08T04:41:07.115163+0000","last_active":"2024-05-08T04:41:09.108519+0000","last_peered":"2024-05-08T04:41:09.108519+0000","last_clean":"2024-05-08T04:41:09.108519+0000","last_became_active":"2024-05-08T04:41:07.114516+0000","last_became_peered":"2024-05-08T04:41:07.114516+0000","last_unstale":"2024-05-08T04:41:09.108519+0000","last_undegraded":"2024-05-08T04:41:09.108519+0000","last_fullsized":"2024-05-08T04:41:09.108519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:41.871914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398518+0000","last_change":"2024-05-08T04:41:08.114585+0000","last_active":"2024-05-08T04:41:11.398518+0000","last_peered":"2024-05-08T04:41:11.398518+0000","last_clean":"2024-05-08T04:41:11.398518+0000","last_became_active":"2024-05-08T04:41:08.114294+0000","last_became_peered":"2024-05-08T04:41:08.114294+0000","last_unstale":"2024-05-08T04:41:11.398518+0000","last_undegraded":"2024-05-08T04:41:11.398518+0000","last_fullsized":"2024-05-08T04:41:11.398518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:44.499342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350239+0000","last_change":"2024-05-08T04:41:07.123483+0000","last_active":"2024-05-08T04:41:10.350239+0000","last_peered":"2024-05-08T04:41:10.350239+0000","last_clean":"2024-05-08T04:41:10.350239+0000","last_became_active":"2024-05-08T04:41:07.121115+0000","last_became_peered":"2024-05-08T04:41:07.121115+0000","last_unstale":"2024-05-08T04:41:10.350239+0000","last_undegraded":"2024-05-08T04:41:10.350239+0000","last_fullsized":"2024-05-08T04:41:10.350239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:50.974023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398999+0000","last_change":"2024-05-08T04:41:08.134611+0000","last_active":"2024-05-08T04:41:11.398999+0000","last_peered":"2024-05-08T04:41:11.398999+0000","last_clean":"2024-05-08T04:41:11.398999+0000","last_became_active":"2024-05-08T04:41:08.133850+0000","last_became_peered":"2024-05-08T04:41:08.133850+0000","last_unstale":"2024-05-08T04:41:11.398999+0000","last_undegraded":"2024-05-08T04:41:11.398999+0000","last_fullsized":"2024-05-08T04:41:11.398999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:56:31.222655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.106605+0000","last_change":"2024-05-08T04:41:07.120042+0000","last_active":"2024-05-08T04:41:09.106605+0000","last_peered":"2024-05-08T04:41:09.106605+0000","last_clean":"2024-05-08T04:41:09.106605+0000","last_became_active":"2024-05-08T04:41:07.119795+0000","last_became_peered":"2024-05-08T04:41:07.119795+0000","last_unstale":"2024-05-08T04:41:09.106605+0000","last_undegraded":"2024-05-08T04:41:09.106605+0000","last_fullsized":"2024-05-08T04:41:09.106605+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:24.317182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350109+0000","last_change":"2024-05-08T04:41:08.139854+0000","last_active":"2024-05-08T04:41:10.350109+0000","last_peered":"2024-05-08T04:41:10.350109+0000","last_clean":"2024-05-08T04:41:10.350109+0000","last_became_active":"2024-05-08T04:41:08.139568+0000","last_became_peered":"2024-05-08T04:41:08.139568+0000","last_unstale":"2024-05-08T04:41:10.350109+0000","last_undegraded":"2024-05-08T04:41:10.350109+0000","last_fullsized":"2024-05-08T04:41:10.350109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:22:59.667811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.350009+0000","last_change":"2024-05-08T04:41:07.122490+0000","last_active":"2024-05-08T04:41:10.350009+0000","last_peered":"2024-05-08T04:41:10.350009+0000","last_clean":"2024-05-08T04:41:10.350009+0000","last_became_active":"2024-05-08T04:41:07.122220+0000","last_became_peered":"2024-05-08T04:41:07.122220+0000","last_unstale":"2024-05-08T04:41:10.350009+0000","last_undegraded":"2024-05-08T04:41:10.350009+0000","last_fullsized":"2024-05-08T04:41:10.350009+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:37.570345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398544+0000","last_change":"2024-05-08T04:41:08.113335+0000","last_active":"2024-05-08T04:41:11.398544+0000","last_peered":"2024-05-08T04:41:11.398544+0000","last_clean":"2024-05-08T04:41:11.398544+0000","last_became_active":"2024-05-08T04:41:08.113016+0000","last_became_peered":"2024-05-08T04:41:08.113016+0000","last_unstale":"2024-05-08T04:41:11.398544+0000","last_undegraded":"2024-05-08T04:41:11.398544+0000","last_fullsized":"2024-05-08T04:41:11.398544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:11:40.632007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396788+0000","last_change":"2024-05-08T04:41:07.114940+0000","last_active":"2024-05-08T04:41:11.396788+0000","last_peered":"2024-05-08T04:41:11.396788+0000","last_clean":"2024-05-08T04:41:11.396788+0000","last_became_active":"2024-05-08T04:41:07.114663+0000","last_became_peered":"2024-05-08T04:41:07.114663+0000","last_unstale":"2024-05-08T04:41:11.396788+0000","last_undegraded":"2024-05-08T04:41:11.396788+0000","last_fullsized":"2024-05-08T04:41:11.396788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:10.342497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.396973+0000","last_change":"2024-05-08T04:41:08.131304+0000","last_active":"2024-05-08T04:41:11.396973+0000","last_peered":"2024-05-08T04:41:11.396973+0000","last_clean":"2024-05-08T04:41:11.396973+0000","last_became_active":"2024-05-08T04:41:08.131007+0000","last_became_peered":"2024-05-08T04:41:08.131007+0000","last_unstale":"2024-05-08T04:41:11.396973+0000","last_undegraded":"2024-05-08T04:41:11.396973+0000","last_fullsized":"2024-05-08T04:41:11.396973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:25.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:09.107490+0000","last_change":"2024-05-08T04:41:07.121852+0000","last_active":"2024-05-08T04:41:09.107490+0000","last_peered":"2024-05-08T04:41:09.107490+0000","last_clean":"2024-05-08T04:41:09.107490+0000","last_became_active":"2024-05-08T04:41:07.121549+0000","last_became_peered":"2024-05-08T04:41:07.121549+0000","last_unstale":"2024-05-08T04:41:09.107490+0000","last_undegraded":"2024-05-08T04:41:09.107490+0000","last_fullsized":"2024-05-08T04:41:09.107490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:20.992283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397753+0000","last_change":"2024-05-08T04:41:08.133463+0000","last_active":"2024-05-08T04:41:11.397753+0000","last_peered":"2024-05-08T04:41:11.397753+0000","last_clean":"2024-05-08T04:41:11.397753+0000","last_became_active":"2024-05-08T04:41:08.133119+0000","last_became_peered":"2024-05-08T04:41:08.133119+0000","last_unstale":"2024-05-08T04:41:11.397753+0000","last_undegraded":"2024-05-08T04:41:11.397753+0000","last_fullsized":"2024-05-08T04:41:11.397753+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:59.304053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398088+0000","last_change":"2024-05-08T04:41:07.125169+0000","last_active":"2024-05-08T04:41:11.398088+0000","last_peered":"2024-05-08T04:41:11.398088+0000","last_clean":"2024-05-08T04:41:11.398088+0000","last_became_active":"2024-05-08T04:41:07.124870+0000","last_became_peered":"2024-05-08T04:41:07.124870+0000","last_unstale":"2024-05-08T04:41:11.398088+0000","last_undegraded":"2024-05-08T04:41:11.398088+0000","last_fullsized":"2024-05-08T04:41:11.398088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:10.342497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:10.403434+0000","last_change":"2024-05-08T04:41:08.135971+0000","last_active":"2024-05-08T04:41:10.403434+0000","last_peered":"2024-05-08T04:41:10.403434+0000","last_clean":"2024-05-08T04:41:10.403434+0000","last_became_active":"2024-05-08T04:41:08.131865+0000","last_became_peered":"2024-05-08T04:41:08.131865+0000","last_unstale":"2024-05-08T04:41:10.403434+0000","last_undegraded":"2024-05-08T04:41:10.403434+0000","last_fullsized":"2024-05-08T04:41:10.403434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:07.092845+0000","last_clean_scrub_stamp":"2024-05-08T04:41:07.092845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:31.546708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.398238+0000","last_change":"2024-05-08T04:41:07.121887+0000","last_active":"2024-05-08T04:41:11.398238+0000","last_peered":"2024-05-08T04:41:11.398238+0000","last_clean":"2024-05-08T04:41:11.398238+0000","last_became_active":"2024-05-08T04:41:07.121407+0000","last_became_peered":"2024-05-08T04:41:07.121407+0000","last_unstale":"2024-05-08T04:41:11.398238+0000","last_undegraded":"2024-05-08T04:41:11.398238+0000","last_fullsized":"2024-05-08T04:41:11.398238+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:06.081946+0000","last_clean_scrub_stamp":"2024-05-08T04:41:06.081946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:28.853767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T04:41:11.397599+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:11.397599+0000","last_peered":"2024-05-08T04:41:11.397599+0000","last_clean":"2024-05-08T04:41:11.397599+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:11.397599+0000","last_undegraded":"2024-05-08T04:41:11.397599+0000","last_fullsized":"2024-05-08T04:41:11.397599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":974798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":383906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":383904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":383884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":383926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":385426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":386042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1495040,"data_stored":974182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:17.063 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:41:17.063 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:41:17.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 Processor -- start 2024-05-08T04:41:17.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 -- start start 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f576406cce0 0x7f57640ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 0x7f57640ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576415e870 con 0x7f576415ab10 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640cf130 con 0x7f576406cce0 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.244+0000 7f5769e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576406bb80 con 0x7f57640cecc0 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f576406cce0 0x7f57640ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f576406cce0 0x7f57640ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41874/0 (socket says 172.21.15.164:41874) 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 -- 172.21.15.164:0/2833853383 learned_addr learned my addr 172.21.15.164:0/2833853383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:17.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f5763400640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 0x7f57640ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 -- 172.21.15.164:0/2833853383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 msgr2=0x7f57640ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:17.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 0x7f57640ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 -- 172.21.15.164:0/2833853383 >> v1:172.21.15.172:6789/0 conn(0x7f576415ab10 legacy=0x7f57640ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 -- 172.21.15.164:0/2833853383 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576415eb50 con 0x7f576406cce0 2024-05-08T04:41:17.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f5763400640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 0x7f57640ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:17.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f575a000640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f576406cce0 0x7f57640ce780 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f574c009870 tx=0x7f574c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ca090b8709e0e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f5762000640 1 -- 172.21.15.164:0/2833853383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0aa600 con 0x7f576406cce0 2024-05-08T04:41:17.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f5762000640 1 -- 172.21.15.164:0/2833853383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c0af070 con 0x7f576406cce0 2024-05-08T04:41:17.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.248+0000 7f5762000640 1 -- 172.21.15.164:0/2833853383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0aa9c0 con 0x7f576406cce0 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- 172.21.15.164:0/2833853383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f576406cce0 msgr2=0x7f57640ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f576406cce0 0x7f57640ce780 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f574c009870 tx=0x7f574c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- 172.21.15.164:0/2833853383 shutdown_connections 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57640cecc0 0x7f57640ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- 172.21.15.164:0/2833853383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f576406cce0 0x7f57640ce780 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- 172.21.15.164:0/2833853383 >> 172.21.15.164:0/2833853383 conn(0x7f57640cf770 msgr2=0x7f57640cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- 172.21.15.164:0/2833853383 shutdown_connections 2024-05-08T04:41:17.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- 172.21.15.164:0/2833853383 wait complete. 2024-05-08T04:41:17.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 Processor -- start 2024-05-08T04:41:17.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- start start 2024-05-08T04:41:17.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f576406cce0 0x7f5764156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f576415ab10 0x7f5764153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640aedf0 con 0x7f57640cecc0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576415aef0 con 0x7f576406cce0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5769e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640aef70 con 0x7f576415ab10 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41884/0 (socket says 172.21.15.164:41884) 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 -- 172.21.15.164:0/2059751802 learned_addr learned my addr 172.21.15.164:0/2059751802 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5762a00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f576406cce0 0x7f5764156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f5763400640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f576415ab10 0x7f5764153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 -- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f576415ab10 msgr2=0x7f5764153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f576415ab10 0x7f5764153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 -- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f576406cce0 msgr2=0x7f5764156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f576406cce0 0x7f5764156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.252+0000 7f575a000640 1 -- 172.21.15.164:0/2059751802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57641fa140 con 0x7f57640cecc0 2024-05-08T04:41:17.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575a000640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f574c01cd40 tx=0x7f574c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0bd5f0 con 0x7f57640cecc0 2024-05-08T04:41:17.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c0bd790 con 0x7f57640cecc0 2024-05-08T04:41:17.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57641fa420 con 0x7f57640cecc0 2024-05-08T04:41:17.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0bdb80 con 0x7f57640cecc0 2024-05-08T04:41:17.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57641fa9e0 con 0x7f57640cecc0 2024-05-08T04:41:17.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5764072d30 con 0x7f57640cecc0 2024-05-08T04:41:17.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f574c0a9070 con 0x7f57640cecc0 2024-05-08T04:41:17.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 0x7f5740069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f574c1200e0 con 0x7f57640cecc0 2024-05-08T04:41:17.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.256+0000 7f5762a00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 0x7f5740069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.260+0000 7f5762a00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 0x7f5740069840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f57540015b0 tx=0x7f5754009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.260+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f574c016150 con 0x7f57640cecc0 2024-05-08T04:41:17.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.472+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f576404f010 con 0x7f57640cecc0 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.472+0000 7f575be00640 1 -- 172.21.15.164:0/2059751802 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f574c0f0a30 con 0x7f57640cecc0 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:41:17.478 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:41:17.479 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:41:17.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 msgr2=0x7f5740069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 0x7f5740069840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f57540015b0 tx=0x7f5754009040 comp rx=0 tx=0).stop 2024-05-08T04:41:17.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 msgr2=0x7f57640ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f574c01cd40 tx=0x7f574c01ace0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 shutdown_connections 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5740067380 0x7f5740069840 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f576415ab10 0x7f5764153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57640cecc0 0x7f57640ae360 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 --2- 172.21.15.164:0/2059751802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f576406cce0 0x7f5764156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 >> 172.21.15.164:0/2059751802 conn(0x7f57640cf770 msgr2=0x7f57640cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 shutdown_connections 2024-05-08T04:41:17.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.476+0000 7f5769e00640 1 -- 172.21.15.164:0/2059751802 wait complete. 2024-05-08T04:41:17.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:17.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.628+0000 7f5f02800640 1 Processor -- start 2024-05-08T04:41:17.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5f02800640 1 -- start start 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5f02800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5efc0ccfe0 0x7f5efc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5f02800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b42a0 con 0x7f5efc15bc50 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5f02800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0cd4e0 con 0x7f5efc0d05b0 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5f02800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc0cd7c0 con 0x7f5efc0ccfe0 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5efc15bc50 0x7f5efc0b4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59852/0 (socket says 172.21.15.164:59852) 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 -- 172.21.15.164:0/295932734 learned_addr learned my addr 172.21.15.164:0/295932734 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efaa00640 1 --2- 172.21.15.164:0/295932734 >> v2:172.21.15.172:3300/0 conn(0x7f5efc0ccfe0 0x7f5efc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275840229 0 0) 0x7f5efc0b42a0 con 0x7f5efc15bc50 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee0003530 con 0x7f5efc15bc50 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525007539 0 0) 0x7f5ee0003530 con 0x7f5efc15bc50 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 >> v2:172.21.15.172:3300/0 conn(0x7f5efc0ccfe0 msgr2=0x7f5efc0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:17.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 --2- 172.21.15.164:0/295932734 >> v2:172.21.15.172:3300/0 conn(0x7f5efc0ccfe0 0x7f5efc0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 >> v1:172.21.15.172:6789/0 conn(0x7f5efc0d05b0 legacy=0x7f5efc15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0cf590 con 0x7f5efc15bc50 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5eec001680 con 0x7f5efc15bc50 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 >> v1:172.21.15.164:6789/0 conn(0x7f5efc15bc50 legacy=0x7f5efc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 0x7f5ee0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ee0004810 0x7f5ee0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00051d0 0x7f5ee000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0cd7c0 con 0x7f5ee00039c0 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000c170 con 0x7f5ee00051d0 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000c370 con 0x7f5ee0004810 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 0x7f5ee0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efaa00640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ee0004810 0x7f5ee0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efbe00640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00051d0 0x7f5ee000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 -- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ee0004810 msgr2=0x7f5ee0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ee0004810 0x7f5ee0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 -- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00051d0 msgr2=0x7f5ee000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00051d0 0x7f5ee000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efb400640 1 -- 172.21.15.164:0/295932734 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0003530 con 0x7f5ee00039c0 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.632+0000 7f5efbe00640 1 -- 172.21.15.164:0/295932734 reap_dead start 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5efb400640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 0x7f5ee0003de0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c210 tx=0x7f5eec01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72147a3774b2f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec007d40 con 0x7f5ee00039c0 2024-05-08T04:41:17.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0a9070 con 0x7f5ee00039c0 2024-05-08T04:41:17.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5efa000640 1 -- 172.21.15.164:0/295932734 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec005d80 con 0x7f5ee00039c0 2024-05-08T04:41:17.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 -- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 msgr2=0x7f5ee0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 0x7f5ee0003de0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c210 tx=0x7f5eec01cca0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 -- 172.21.15.164:0/295932734 shutdown_connections 2024-05-08T04:41:17.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 --2- 172.21.15.164:0/295932734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ee00039c0 0x7f5ee0003de0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 -- 172.21.15.164:0/295932734 >> 172.21.15.164:0/295932734 conn(0x7f5efc0b34f0 msgr2=0x7f5efc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:17.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 -- 172.21.15.164:0/295932734 shutdown_connections 2024-05-08T04:41:17.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.636+0000 7f5f02800640 1 -- 172.21.15.164:0/295932734 wait complete. 2024-05-08T04:41:17.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 Processor -- start 2024-05-08T04:41:17.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- start start 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00039c0 0x7f5efc154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc15cc50 0x7f5efc158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ce280 con 0x7f5efc159240 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ce100 con 0x7f5ee00039c0 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ce400 con 0x7f5efc15cc50 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efb400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00039c0 0x7f5efc154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41902/0 (socket says 172.21.15.164:41902) 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc15cc50 0x7f5efc158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 -- 172.21.15.164:0/142854362 learned_addr learned my addr 172.21.15.164:0/142854362 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 -- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc15cc50 msgr2=0x7f5efc158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc15cc50 0x7f5efc158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 -- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00039c0 msgr2=0x7f5efc154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00039c0 0x7f5efc154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 -- 172.21.15.164:0/142854362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc15db70 con 0x7f5efc159240 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5efbe00640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5efc072270 tx=0x7f5ef400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef401a070 con 0x7f5efc159240 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4006d70 con 0x7f5efc159240 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4003bc0 con 0x7f5efc159240 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc15de50 con 0x7f5efc159240 2024-05-08T04:41:17.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc15edc0 con 0x7f5efc159240 2024-05-08T04:41:17.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.640+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc069d60 con 0x7f5efc159240 2024-05-08T04:41:17.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.644+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f5ef4005110 con 0x7f5efc159240 2024-05-08T04:41:17.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.644+0000 7f5ef3400640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 0x7f5edc069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:17.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.644+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5ef4084b50 con 0x7f5efc159240 2024-05-08T04:41:17.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.644+0000 7f5efb400640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 0x7f5edc069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:17.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.644+0000 7f5efb400640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 0x7f5edc069840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0ab000 tx=0x7f5eec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:17.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.648+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ef4054f60 con 0x7f5efc159240 2024-05-08T04:41:17.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.832+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5efc0b7940 con 0x7f5efc159240 2024-05-08T04:41:17.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.832+0000 7f5ef3400640 1 -- 172.21.15.164:0/142854362 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5ef4059000 con 0x7f5efc159240 2024-05-08T04:41:17.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:17.839 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:17.840 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 43 2024-05-08T04:41:17.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 msgr2=0x7f5edc069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 0x7f5edc069840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0ab000 tx=0x7f5eec0a7070 comp rx=0 tx=0).stop 2024-05-08T04:41:17.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 msgr2=0x7f5efc0c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:17.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5efc072270 tx=0x7f5ef400fbb0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 shutdown_connections 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5edc067380 0x7f5edc069840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5efc15cc50 0x7f5efc158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc159240 0x7f5efc0c5e20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 --2- 172.21.15.164:0/142854362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee00039c0 0x7f5efc154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 >> 172.21.15.164:0/142854362 conn(0x7f5efc0b34f0 msgr2=0x7f5efc0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 shutdown_connections 2024-05-08T04:41:17.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:17.836+0000 7f5f02800640 1 -- 172.21.15.164:0/142854362 wait complete. 2024-05-08T04:41:18.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:41:07.483520+0000', 'modified': '2024-05-08T04:41:10.507554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6465}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6465': {'gid': 6465, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/239509813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 239509813}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 239509813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:41:18.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:41:18.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T04:41:18.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 Processor -- start 2024-05-08T04:41:18.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 -- start start 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff16c15ab60 0x7ff16c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 0x7ff16c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c072640 con 0x7ff16c0cfbd0 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c0d0060 con 0x7ff16c15ab60 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff171800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c06c770 con 0x7ff16c0b26c0 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff16c15ab60 0x7ff16c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff16c15ab60 0x7ff16c072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41912/0 (socket says 172.21.15.164:41912) 2024-05-08T04:41:18.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 -- 172.21.15.164:0/2710815604 learned_addr learned my addr 172.21.15.164:0/2710815604 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16a000640 1 --2- 172.21.15.164:0/2710815604 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 0x7ff16c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 -- 172.21.15.164:0/2710815604 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 msgr2=0x7ff16c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2710815604 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 0x7ff16c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 -- 172.21.15.164:0/2710815604 >> v1:172.21.15.172:6789/0 conn(0x7ff16c0cfbd0 legacy=0x7ff16c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16aa00640 1 -- 172.21.15.164:0/2710815604 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c06ca50 con 0x7ff16c15ab60 2024-05-08T04:41:18.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.396+0000 7ff16a000640 1 --2- 172.21.15.164:0/2710815604 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 0x7ff16c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2710815604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c15ab60 0x7ff16c072100 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06e880 tx=0x7ff15c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a97348f73514a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff169600640 1 -- 172.21.15.164:0/2710815604 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0a5670 con 0x7ff16c15ab60 2024-05-08T04:41:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff169600640 1 -- 172.21.15.164:0/2710815604 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c0a5810 con 0x7ff16c15ab60 2024-05-08T04:41:18.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff169600640 1 -- 172.21.15.164:0/2710815604 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0a5be0 con 0x7ff16c15ab60 2024-05-08T04:41:18.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 -- 172.21.15.164:0/2710815604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c15ab60 msgr2=0x7ff16c072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 --2- 172.21.15.164:0/2710815604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c15ab60 0x7ff16c072100 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06e880 tx=0x7ff15c01c9b0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 -- 172.21.15.164:0/2710815604 shutdown_connections 2024-05-08T04:41:18.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 --2- 172.21.15.164:0/2710815604 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0b26c0 0x7ff16c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 --2- 172.21.15.164:0/2710815604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c15ab60 0x7ff16c072100 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 -- 172.21.15.164:0/2710815604 >> 172.21.15.164:0/2710815604 conn(0x7ff16c06ac90 msgr2=0x7ff16c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 -- 172.21.15.164:0/2710815604 shutdown_connections 2024-05-08T04:41:18.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.400+0000 7ff171800640 1 -- 172.21.15.164:0/2710815604 wait complete. 2024-05-08T04:41:18.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 Processor -- start 2024-05-08T04:41:18.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- start start 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c0cfbd0 0x7ff16c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 0x7ff16c14ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c06c550 con 0x7ff16c0b26c0 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c06c3d0 con 0x7ff16c15ab60 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0aed30 con 0x7ff16c0cfbd0 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41918/0 (socket says 172.21.15.164:41918) 2024-05-08T04:41:18.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 -- 172.21.15.164:0/1299072222 learned_addr learned my addr 172.21.15.164:0/1299072222 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16a000640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c0cfbd0 0x7ff16c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16b400640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 0x7ff16c14ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 -- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c0cfbd0 msgr2=0x7ff16c14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c0cfbd0 0x7ff16c14e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 -- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 msgr2=0x7ff16c14ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 0x7ff16c14ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 -- 172.21.15.164:0/1299072222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c15b630 con 0x7ff16c0b26c0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16b400640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 0x7ff16c14ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff16aa00640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0af040 tx=0x7ff15c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff162a00640 1 -- 172.21.15.164:0/1299072222 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0bc020 con 0x7ff16c0b26c0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff162a00640 1 -- 172.21.15.164:0/1299072222 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c0b8670 con 0x7ff16c0b26c0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff162a00640 1 -- 172.21.15.164:0/1299072222 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0b8a60 con 0x7ff16c0b26c0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c15b910 con 0x7ff16c0b26c0 2024-05-08T04:41:18.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.404+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c15c1b0 con 0x7ff16c0b26c0 2024-05-08T04:41:18.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff162a00640 1 -- 172.21.15.164:0/1299072222 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7ff15c0bf020 con 0x7ff16c0b26c0 2024-05-08T04:41:18.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 Processor -- start 2024-05-08T04:41:18.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff162a00640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 0x7ff14c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 -- start start 2024-05-08T04:41:18.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff162a00640 1 -- 172.21.15.164:0/1299072222 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff15c1209f0 con 0x7ff16c0b26c0 2024-05-08T04:41:18.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 0x7ff14c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 0x7ff14c069810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff16c1f68d0 tx=0x7ff164009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b5262d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b51005f0 con 0x5644b5218f30 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b5067da0 con 0x5644b5263240 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644b4ff0150 con 0x5644b5262920 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41928/0 (socket says 172.21.15.164:41928) 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 -- 172.21.15.164:0/1548412187 learned_addr learned my addr 172.21.15.164:0/1548412187 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16aa00640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b5262d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16a000640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.408+0000 7ff16b400640 1 -- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 msgr2=0x5644b5262d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b5262d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 -- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 msgr2=0x5644b52189f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52189f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 -- 172.21.15.164:0/1548412187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b5009ca0 con 0x5644b5218f30 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b5262d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff164007130 tx=0x7ff164011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f13da0a445220b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff157400640 1 -- 172.21.15.164:0/1548412187 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1640069f0 con 0x5644b5218f30 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff157400640 1 -- 172.21.15.164:0/1548412187 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164006b90 con 0x5644b5218f30 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff157400640 1 -- 172.21.15.164:0/1548412187 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164006d40 con 0x5644b5218f30 2024-05-08T04:41:18.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- 172.21.15.164:0/1548412187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 msgr2=0x5644b5219310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff164007130 tx=0x7ff164011000 comp rx=0 tx=0).stop 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- 172.21.15.164:0/1548412187 shutdown_connections 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5219310 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52189f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- 172.21.15.164:0/1548412187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b5262d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- 172.21.15.164:0/1548412187 >> 172.21.15.164:0/1548412187 conn(0x5644b521f2a0 msgr2=0x5644b52172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- 172.21.15.164:0/1548412187 shutdown_connections 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- 172.21.15.164:0/1548412187 wait complete. 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- start start 2024-05-08T04:41:18.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 Processor -- start 2024-05-08T04:41:18.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 0x5644b526a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b521caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b5022990 con 0x5644b5218f30 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b5063e40 con 0x5644b5262920 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644b4ff3900 con 0x5644b5263240 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff174c18000 10 client.0 _ll_register_callbacks cb 0x5644b511b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41940/0 (socket says 172.21.15.164:41940) 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 -- 172.21.15.164:0/246469292 learned_addr learned my addr 172.21.15.164:0/246469292 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16a000640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 0x5644b526a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16b400640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b521caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 -- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 msgr2=0x5644b521caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b521caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 -- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 msgr2=0x5644b526a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 0x5644b526a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 -- 172.21.15.164:0/246469292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644b5020630 con 0x5644b5218f30 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16a000640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 0x5644b526a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.412+0000 7ff16aa00640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0f0e40 tx=0x7ff15c0ef3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622d0083e8471d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:41:18.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0f72b0 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff15c0f1110 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff164006b90 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff15c0f0eb0 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 10 client.6627 fetch_fsmap learned FSMap version 43 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5644b4ff03a0 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164006b90 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff15c0f9130 con 0x5644b5218f30 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 10 client.6627 fetch_fsmap finished waiting for FSMap version 43 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 10 client.6627 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:41:18.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 20 client.6627 populate_metadata read hostname 'smithi164' 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b521e160 0x5644b5273d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 4 client.6627 mds_command: new command op to 6465 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 --> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644b5003a50 con 0x5644b521e160 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff16a000640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b521e160 0x5644b5273d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 10 client.6627 ms_handle_connect on v2:172.21.15.164:6834/239509813 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff16a000640 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b521e160 0x5644b5273d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c0dc5dfc85183e server_cookie=89202fc39bc13136 in_seq=0 out_seq=0 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 1 -- 172.21.15.164:0/246469292 <== mds.0 v2:172.21.15.164:6834/239509813 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644b5003a50 con 0x5644b521e160 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 10 client.6627 handle_command_reply: tid=0 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 client.6627 shutdown 2024-05-08T04:41:18.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 msgr2=0x5644b5269cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0f0e40 tx=0x7ff15c0ef3c0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 shutdown_connections 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b521caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b521e160 0x5644b5273d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5262920 0x5644b526a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 --2- 172.21.15.164:0/246469292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b5269cf0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff14be00640 0 client.6627 ms_handle_reset on v2:172.21.15.164:6834/239509813 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 >> 172.21.15.164:0/246469292 conn(0x5644b521f2a0 msgr2=0x5644b52179e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 shutdown_connections 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 1 -- 172.21.15.164:0/246469292 wait complete. 2024-05-08T04:41:18.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.416+0000 7ff174c18000 20 client.6627 trim_cache size 0 max 16384 2024-05-08T04:41:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 Processor -- start 2024-05-08T04:41:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 -- start start 2024-05-08T04:41:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5218f30 0x5644b52be200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b52bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c202050 con 0x5644b5262920 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1549f0 con 0x5644b5218f30 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c154ff0 con 0x5644b5263240 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41944/0 (socket says 172.21.15.164:41944) 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 -- 172.21.15.164:0/2752281791 learned_addr learned my addr 172.21.15.164:0/2752281791 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5218f30 0x5644b52be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16b400640 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b52bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 -- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 msgr2=0x5644b52bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b52bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 -- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5218f30 msgr2=0x5644b52be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5218f30 0x5644b52be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 -- 172.21.15.164:0/2752281791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b5263070 con 0x5644b5262920 2024-05-08T04:41:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff16a000640 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x5644b52628f0 tx=0x7ff158008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b33ae554f27895d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff156a00640 1 -- 172.21.15.164:0/2752281791 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15800fd90 con 0x5644b5262920 2024-05-08T04:41:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff156a00640 1 -- 172.21.15.164:0/2752281791 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158020030 con 0x5644b5262920 2024-05-08T04:41:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.432+0000 7ff156a00640 1 -- 172.21.15.164:0/2752281791 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158004260 con 0x5644b5262920 2024-05-08T04:41:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2752281791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 msgr2=0x5644b52be740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x5644b52628f0 tx=0x7ff158008040 comp rx=0 tx=0).stop 2024-05-08T04:41:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2752281791 shutdown_connections 2024-05-08T04:41:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5263240 0x5644b52bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5262920 0x5644b52be740 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- 172.21.15.164:0/2752281791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5218f30 0x5644b52be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2752281791 >> 172.21.15.164:0/2752281791 conn(0x5644b521f2a0 msgr2=0x5644b521c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2752281791 shutdown_connections 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2752281791 wait complete. 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- start start 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 Processor -- start 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b52aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b506c3d0 con 0x5644b5218f30 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b5123d50 con 0x5644b5263240 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644b5002980 con 0x5644b5262920 2024-05-08T04:41:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 10 client.0 _ll_register_callbacks cb 0x5644b5008330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41950/0 (socket says 172.21.15.164:41950) 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 -- 172.21.15.164:0/2562510933 learned_addr learned my addr 172.21.15.164:0/2562510933 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16b400640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16a000640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b52aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 -- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 msgr2=0x5644b52aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b52aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 -- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 msgr2=0x5644b52aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 -- 172.21.15.164:0/2562510933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644b50e9600 con 0x5644b5218f30 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff16aa00640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0ef160 tx=0x7ff15c0c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507ce1bbab9d8bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff14be00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:41:18.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0f09f0 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff15c0f0300 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff158020030 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff15c0f0500 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 10 client.6633 fetch_fsmap learned FSMap version 43 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5644b4ffc9b0 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.436+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158020030 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff15c100040 con 0x5644b5218f30 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 10 client.6633 fetch_fsmap finished waiting for FSMap version 43 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 10 client.6633 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 20 client.6633 populate_metadata read hostname 'smithi164' 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b52ab250 0x5644b52ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 4 client.6633 mds_command: new command op to 6465 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:41:18.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 --> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644b50752e0 con 0x5644b52ab250 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff16a000640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b52ab250 0x5644b52ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff16a000640 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b52ab250 0x5644b52ab630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edbe7ca332f6a68 server_cookie=4b401cb28f38f6e2 in_seq=0 out_seq=0 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff14be00640 10 client.6633 ms_handle_connect on v2:172.21.15.164:6834/239509813 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff14be00640 1 -- 172.21.15.164:0/2562510933 <== mds.0 v2:172.21.15.164:6834/239509813 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5644b50752e0 con 0x5644b52ab250 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff14be00640 10 client.6633 handle_command_reply: tid=0 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 client.6633 shutdown 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 msgr2=0x5644b52a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0ef160 tx=0x7ff15c0c2000 comp rx=0 tx=0).stop 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 shutdown_connections 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644b5262920 0x5644b52aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:6834/239509813,v1:172.21.15.164:6837/239509813] conn(0x5644b52ab250 0x5644b52ab630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644b5263240 0x5644b52aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 --2- 172.21.15.164:0/2562510933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644b5218f30 0x5644b52a9e10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 >> 172.21.15.164:0/2562510933 conn(0x5644b521f2a0 msgr2=0x5644b526a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff14be00640 0 client.6633 ms_handle_reset on v2:172.21.15.164:6834/239509813 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 shutdown_connections 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 1 -- 172.21.15.164:0/2562510933 wait complete. 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff174c18000 20 client.6633 trim_cache size 0 max 16384 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 6465, 2024-05-08T04:41:18.450 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 10.342273728, 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 43, 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 74, 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 74, 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 16.232032485000001 2024-05-08T04:41:18.451 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:41:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 msgr2=0x7ff14c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 0x7ff14c069810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff16c1f68d0 tx=0x7ff164009040 comp rx=0 tx=0).stop 2024-05-08T04:41:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 msgr2=0x7ff16c15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0af040 tx=0x7ff15c019750 comp rx=0 tx=0).stop 2024-05-08T04:41:18.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 shutdown_connections 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff14c067350 0x7ff14c069810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c0cfbd0 0x7ff16c14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c15ab60 0x7ff16c14ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 --2- 172.21.15.164:0/1299072222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff16c0b26c0 0x7ff16c15b050 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 >> 172.21.15.164:0/1299072222 conn(0x7ff16c06ac90 msgr2=0x7ff16c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 shutdown_connections 2024-05-08T04:41:18.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.440+0000 7ff171800640 1 -- 172.21.15.164:0/1299072222 wait complete. 2024-05-08T04:41:18.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:41:18.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 Processor -- start 2024-05-08T04:41:18.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 -- start start 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f480815da10 0x7f4808071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 0x7f48080b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4808072400 con 0x7f480806ac90 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080d1410 con 0x7f480815da10 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f480d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080d04d0 con 0x7f48080d0fb0 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f480815da10 0x7f4808071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f480815da10 0x7f4808071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41956/0 (socket says 172.21.15.164:41956) 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 -- 172.21.15.164:0/1707793683 learned_addr learned my addr 172.21.15.164:0/1707793683 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f47ffe00640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 0x7f48080b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 -- 172.21.15.164:0/1707793683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 msgr2=0x7f48080b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 0x7f48080b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 -- 172.21.15.164:0/1707793683 >> v1:172.21.15.172:6789/0 conn(0x7f480806ac90 legacy=0x7f48080ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:18.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f4807e00640 1 -- 172.21.15.164:0/1707793683 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080d07b0 con 0x7f480815da10 2024-05-08T04:41:18.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.596+0000 7f47ffe00640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 0x7f48080b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:18.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f4807e00640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808071e20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009870 tx=0x7f47f801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b3d8b5d9e3337a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f4807400640 1 -- 172.21.15.164:0/1707793683 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80ab660 con 0x7f480815da10 2024-05-08T04:41:18.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f4807400640 1 -- 172.21.15.164:0/1707793683 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f80b0070 con 0x7f480815da10 2024-05-08T04:41:18.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f4807400640 1 -- 172.21.15.164:0/1707793683 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80aba20 con 0x7f480815da10 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 -- 172.21.15.164:0/1707793683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 msgr2=0x7f4808071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808071e20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009870 tx=0x7f47f801c530 comp rx=0 tx=0).stop 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 -- 172.21.15.164:0/1707793683 shutdown_connections 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48080d0fb0 0x7f48080b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 --2- 172.21.15.164:0/1707793683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808071e20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 -- 172.21.15.164:0/1707793683 >> 172.21.15.164:0/1707793683 conn(0x7f4808066fa0 msgr2=0x7f480806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 -- 172.21.15.164:0/1707793683 shutdown_connections 2024-05-08T04:41:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.600+0000 7f480d800640 1 -- 172.21.15.164:0/1707793683 wait complete. 2024-05-08T04:41:18.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 Processor -- start 2024-05-08T04:41:18.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- start start 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f480806ac90 0x7f480813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48080d0fb0 0x7f480814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080b3d90 con 0x7f480815da10 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080b3c10 con 0x7f48080d0fb0 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080ae3b0 con 0x7f480806ac90 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41970/0 (socket says 172.21.15.164:41970) 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 -- 172.21.15.164:0/3935249185 learned_addr learned my addr 172.21.15.164:0/3935249185 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f4807e00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f480806ac90 0x7f480813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f47ffe00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48080d0fb0 0x7f480814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 -- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f480806ac90 msgr2=0x7f480813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f480806ac90 0x7f480813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 -- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48080d0fb0 msgr2=0x7f480814cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48080d0fb0 0x7f480814cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 -- 172.21.15.164:0/3935249185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480814ae00 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480ce00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f480000c7a0 tx=0x7f480000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800003b80 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800003d20 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000aa30 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480814b0e0 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.604+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4808155310 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480804ef30 con 0x7f480815da10 2024-05-08T04:41:18.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f480000abd0 con 0x7f480815da10 2024-05-08T04:41:18.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f4804c00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 0x7f47e4069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:18.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f48000854a0 con 0x7f480815da10 2024-05-08T04:41:18.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f4807e00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 0x7f47e4069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:18.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.608+0000 7f4807e00640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 0x7f47e4069810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f80af000 tx=0x7f47f80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:18.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.612+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48000558b0 con 0x7f480815da10 2024-05-08T04:41:18.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.796+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4808152fd0 con 0x7f480815da10 2024-05-08T04:41:18.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.796+0000 7f4804c00640 1 -- 172.21.15.164:0/3935249185 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4800059950 con 0x7f480815da10 2024-05-08T04:41:18.804 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:41:18.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 msgr2=0x7f47e4069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 0x7f47e4069810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f80af000 tx=0x7f47f80a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:18.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 msgr2=0x7f4808149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:18.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f480000c7a0 tx=0x7f480000cc60 comp rx=0 tx=0).stop 2024-05-08T04:41:18.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 shutdown_connections 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f47e4067350 0x7f47e4069810 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f480815da10 0x7f4808149e00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48080d0fb0 0x7f480814cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 --2- 172.21.15.164:0/3935249185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f480806ac90 0x7f480813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 >> 172.21.15.164:0/3935249185 conn(0x7f4808066fa0 msgr2=0x7f48080b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 shutdown_connections 2024-05-08T04:41:18.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:18.800+0000 7f480d800640 1 -- 172.21.15.164:0/3935249185 wait complete. 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:41:18.824 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:41:18.825 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:18.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:18.825 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:41:18.834 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:41:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.836 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.837 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:18.838 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:41:18.838 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:41:18.838 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:18.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:18.838 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:41:18.896 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:41:18.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:18.899 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:41:18.954 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:41:18.954 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:41:18.954 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:41:18.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:18.955 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:41:18.955 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:41:18.956 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:41:18.956 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:41:18.956 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:41:18.956 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:41:19.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.047 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:41:19.103 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:41:19.103 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:41:19.104 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:19.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:41:19.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:19.167 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:19.219 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:19.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:19.220 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:19.270 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:41:19.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:19.359 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:19.359 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:19.359 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:41:19.382 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:41:19.386 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:41:19.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:19.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.406 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:41:19.462 INFO:teuthology.orchestra.run.smithi164.stdout:client6642 2024-05-08T04:41:19.463 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:19.463 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:19.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:41:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 Processor -- start 2024-05-08T04:41:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 -- start start 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff86c15ab60 0x7ff86c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff86c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c072640 con 0x7ff86c0b26c0 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86c06c770 con 0x7ff86c15ab60 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff871600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c06ca50 con 0x7ff86c0cfda0 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff86a000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff86c0b26c0 0x7ff86c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53496/0 (socket says 172.21.15.164:53496) 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff86a000640 1 -- 172.21.15.164:0/3328882248 learned_addr learned my addr 172.21.15.164:0/3328882248 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> v2:172.21.15.172:3300/0 conn(0x7ff86c15ab60 0x7ff86c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff86b400640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff86c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384085098 0 0) 0x7ff86c072640 con 0x7ff86c0b26c0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff850003530 con 0x7ff86c0b26c0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862648717 0 0) 0x7ff850003530 con 0x7ff86c0b26c0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 msgr2=0x7ff86c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff86c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 >> v2:172.21.15.172:3300/0 conn(0x7ff86c15ab60 msgr2=0x7ff86c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.684+0000 7ff869600640 1 --2- 172.21.15.164:0/3328882248 >> v2:172.21.15.172:3300/0 conn(0x7ff86c15ab60 0x7ff86c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c06cd30 con 0x7ff86c0b26c0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7ff858001680 con 0x7ff86c0b26c0 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> v2:172.21.15.172:3300/0 conn(0x7ff86c15ab60 0x7ff86c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 >> v1:172.21.15.164:6789/0 conn(0x7ff86c0b26c0 legacy=0x7ff86c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff850003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff850004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850004d50 0x7ff8500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c06c770 con 0x7ff8500039c0 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c06ca50 con 0x7ff850004d50 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000b9d0 con 0x7ff86c0cfda0 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff850003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86b400640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850004d50 0x7ff8500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86a000640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff850004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 -- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 msgr2=0x7ff850004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c0cfda0 0x7ff850004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 -- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850004d50 msgr2=0x7ff8500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff850004d50 0x7ff8500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 -- 172.21.15.164:0/3328882248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850003530 con 0x7ff8500039c0 2024-05-08T04:41:19.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff86aa00640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff850003de0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e350 tx=0x7ff85c01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f0b30119384d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:19.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0b2070 con 0x7ff8500039c0 2024-05-08T04:41:19.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c0a5c70 con 0x7ff8500039c0 2024-05-08T04:41:19.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.688+0000 7ff869600640 1 -- 172.21.15.164:0/3328882248 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c009de0 con 0x7ff8500039c0 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- 172.21.15.164:0/3328882248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 msgr2=0x7ff850003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 --2- 172.21.15.164:0/3328882248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff850003de0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e350 tx=0x7ff85c01ce80 comp rx=0 tx=0).stop 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86b400640 1 -- 172.21.15.164:0/3328882248 reap_dead start 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- 172.21.15.164:0/3328882248 shutdown_connections 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- 172.21.15.164:0/3328882248 >> 172.21.15.164:0/3328882248 conn(0x7ff86c06ac90 msgr2=0x7ff86c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:19.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- 172.21.15.164:0/3328882248 shutdown_connections 2024-05-08T04:41:19.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- 172.21.15.164:0/3328882248 wait complete. 2024-05-08T04:41:19.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 Processor -- start 2024-05-08T04:41:19.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- start start 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c14d3e0 0x7ff86c1f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff86c0af2b0 0x7ff86c1f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0aeeb0 con 0x7ff8500039c0 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0aed30 con 0x7ff86c0af2b0 2024-05-08T04:41:19.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff871600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0ce120 con 0x7ff86c14d3e0 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53380/0 (socket says 172.21.15.164:53380) 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 -- 172.21.15.164:0/2706106956 learned_addr learned my addr 172.21.15.164:0/2706106956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86a000640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c14d3e0 0x7ff86c1f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86b400640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff86c0af2b0 0x7ff86c1f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 -- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c14d3e0 msgr2=0x7ff86c1f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c14d3e0 0x7ff86c1f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 -- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff86c0af2b0 msgr2=0x7ff86c1f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff86c0af2b0 0x7ff86c1f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.692+0000 7ff86aa00640 1 -- 172.21.15.164:0/2706106956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c1f8100 con 0x7ff8500039c0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff86aa00640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b0040 tx=0x7ff85c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0ba020 con 0x7ff8500039c0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c0adb40 con 0x7ff8500039c0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0b2040 con 0x7ff8500039c0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c1f83e0 con 0x7ff8500039c0 2024-05-08T04:41:19.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c1f89a0 con 0x7ff8500039c0 2024-05-08T04:41:19.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c069d60 con 0x7ff8500039c0 2024-05-08T04:41:19.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7ff85c0bd020 con 0x7ff8500039c0 2024-05-08T04:41:19.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 0x7ff84c069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:19.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff85c1200a0 con 0x7ff8500039c0 2024-05-08T04:41:19.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.696+0000 7ff86a000640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 0x7ff84c069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:19.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.700+0000 7ff86a000640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 0x7ff84c069810 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff86c15a580 tx=0x7ff858009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:19.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.700+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff85c0c4030 con 0x7ff8500039c0 2024-05-08T04:41:19.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.888+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff86c06d9a0 con 0x7ff8500039c0 2024-05-08T04:41:19.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.888+0000 7ff862a00640 1 -- 172.21.15.164:0/2706106956 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff85c0a5e20 con 0x7ff8500039c0 2024-05-08T04:41:19.895 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:41:19.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 msgr2=0x7ff84c069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 0x7ff84c069810 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff86c15a580 tx=0x7ff858009040 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 msgr2=0x7ff86c1f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b0040 tx=0x7ff85c009750 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 shutdown_connections 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff84c067350 0x7ff84c069810 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff86c14d3e0 0x7ff86c1f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff86c0af2b0 0x7ff86c1f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 --2- 172.21.15.164:0/2706106956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500039c0 0x7ff86c1f7a70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 >> 172.21.15.164:0/2706106956 conn(0x7ff86c06ac90 msgr2=0x7ff86c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 shutdown_connections 2024-05-08T04:41:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:19.892+0000 7ff871600640 1 -- 172.21.15.164:0/2706106956 wait complete. 2024-05-08T04:41:19.915 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:41:19.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:41:19.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:41:19.915 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:41:19.915 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:19.916 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:41:19.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:41:19.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:41:19.916 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:19.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.916 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:41:19.926 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:41:19.926 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.927 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42191sec preferred_lft 42191sec 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:41:19.928 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:41:19.929 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:19.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:19.930 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:41:19.998 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:41:20.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.001 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:20.059 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:41:20.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:41:20.060 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:41:20.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.155 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:41:20.165 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:41:20.165 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:41:20.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:20.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:41:20.230 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:20.230 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:20.286 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:20.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:20.286 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:20.341 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:41:20.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:20.435 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:20.435 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:20.435 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:20.450 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:20.450 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:20.450 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:20.450 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:20.450 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:20.458 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:41:20.459 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:41:20.462 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:41:20.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:20.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:20.482 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:41:20.537 INFO:teuthology.orchestra.run.smithi172.stdout:client6418 2024-05-08T04:41:20.537 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:20.537 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:20.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:41:20.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.724+0000 7fb49b400640 1 Processor -- start 2024-05-08T04:41:20.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.724+0000 7fb49b400640 1 -- start start 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb49b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4940b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb49b400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49415e300 con 0x7fb4940b4dc0 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb49b400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49415e570 con 0x7fb4940af6d0 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb49b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415e850 con 0x7fb4940adfa0 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb499400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb4940b4dc0 0x7fb49406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53498/0 (socket says 172.21.15.164:53498) 2024-05-08T04:41:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb499400640 1 -- 172.21.15.164:0/1335665171 learned_addr learned my addr 172.21.15.164:0/1335665171 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb48be00640 1 --2- 172.21.15.164:0/1335665171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4940b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786646975 0 0) 0x7fb49415e300 con 0x7fb4940b4dc0 2024-05-08T04:41:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb478003530 con 0x7fb4940b4dc0 2024-05-08T04:41:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415940016 0 0) 0x7fb49415e570 con 0x7fb4940af6d0 2024-05-08T04:41:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb49415e300 con 0x7fb4940af6d0 2024-05-08T04:41:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb48be00640 1 -- 172.21.15.164:0/1335665171 >> v1:172.21.15.172:6789/0 conn(0x7fb4940af6d0 legacy=0x7fb4940ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb48be00640 1 -- 172.21.15.164:0/1335665171 >> v1:172.21.15.164:6789/0 conn(0x7fb4940b4dc0 legacy=0x7fb49406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.728+0000 7fb48be00640 1 -- 172.21.15.164:0/1335665171 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940b5250 con 0x7fb4940adfa0 2024-05-08T04:41:20.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb48be00640 1 --2- 172.21.15.164:0/1335665171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4940b2ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097f0 tx=0x7fb47c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684bee8981050379 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:20.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0b2070 con 0x7fb4940adfa0 2024-05-08T04:41:20.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c0a9070 con 0x7fb4940adfa0 2024-05-08T04:41:20.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb493e00640 1 -- 172.21.15.164:0/1335665171 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0ad860 con 0x7fb4940adfa0 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- 172.21.15.164:0/1335665171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 msgr2=0x7fb4940b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 --2- 172.21.15.164:0/1335665171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4940b2ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097f0 tx=0x7fb47c01c450 comp rx=0 tx=0).stop 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- 172.21.15.164:0/1335665171 shutdown_connections 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 --2- 172.21.15.164:0/1335665171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4940b2ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- 172.21.15.164:0/1335665171 >> 172.21.15.164:0/1335665171 conn(0x7fb49406cca0 msgr2=0x7fb49406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- 172.21.15.164:0/1335665171 shutdown_connections 2024-05-08T04:41:20.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- 172.21.15.164:0/1335665171 wait complete. 2024-05-08T04:41:20.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 Processor -- start 2024-05-08T04:41:20.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.732+0000 7fb49b400640 1 -- start start 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4941465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 0x7fb494160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49416bcf0 con 0x7fb4940af6d0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49416bb70 con 0x7fb4940b4dc0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49416be70 con 0x7fb4940adfa0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53396/0 (socket says 172.21.15.164:53396) 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 -- 172.21.15.164:0/787972981 learned_addr learned my addr 172.21.15.164:0/787972981 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb498a00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4941465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb499400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 0x7fb494160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:20.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 -- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 msgr2=0x7fb4941465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4941465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 -- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 msgr2=0x7fb494160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 0x7fb494160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 -- 172.21.15.164:0/787972981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494146e10 con 0x7fb4940af6d0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb498a00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4941465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb499400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 0x7fb494160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb48be00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb47c000c10 tx=0x7fb47c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0bc070 con 0x7fb4940af6d0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c0adc90 con 0x7fb4940af6d0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0b2040 con 0x7fb4940af6d0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb494158890 con 0x7fb4940af6d0 2024-05-08T04:41:20.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb494158e50 con 0x7fb4940af6d0 2024-05-08T04:41:20.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49406ddf0 con 0x7fb4940af6d0 2024-05-08T04:41:20.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.736+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fb47c0ab070 con 0x7fb4940af6d0 2024-05-08T04:41:20.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.740+0000 7fb491600640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 0x7fb470069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:20.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.740+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb47c11e660 con 0x7fb4940af6d0 2024-05-08T04:41:20.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.740+0000 7fb498a00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 0x7fb470069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:20.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.740+0000 7fb498a00640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 0x7fb470069810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb49415d7a0 tx=0x7fb484009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:20.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.744+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb47c0160d0 con 0x7fb4940af6d0 2024-05-08T04:41:20.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:20.928+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb49404f010 con 0x7fb4940af6d0 2024-05-08T04:41:21.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.176+0000 7fb491600640 1 -- 172.21.15.164:0/787972981 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb47c0eef20 con 0x7fb4940af6d0 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 msgr2=0x7fb470069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 0x7fb470069810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb49415d7a0 tx=0x7fb484009040 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 msgr2=0x7fb494143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb47c000c10 tx=0x7fb47c0191f0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 shutdown_connections 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb470067350 0x7fb470069810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4940b4dc0 0x7fb494160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4940af6d0 0x7fb494143700 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 --2- 172.21.15.164:0/787972981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4940adfa0 0x7fb4941465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 >> 172.21.15.164:0/787972981 conn(0x7fb49406cca0 msgr2=0x7fb49406bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 shutdown_connections 2024-05-08T04:41:21.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.180+0000 7fb49b400640 1 -- 172.21.15.164:0/787972981 wait complete. 2024-05-08T04:41:21.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:41:21.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.336+0000 7efd06200640 1 Processor -- start 2024-05-08T04:41:21.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.336+0000 7efd06200640 1 -- start start 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efd06200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd000cfda0 0x7efd00160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efd06200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efd06200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd000cd9f0 con 0x7efd0015ba70 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efd06200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0015fe70 con 0x7efd000cfda0 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efd06200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd000cdcd0 con 0x7efd0015be50 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcff400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7efd0015ba70 0x7efd000cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53514/0 (socket says 172.21.15.164:53514) 2024-05-08T04:41:21.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcff400640 1 -- 172.21.15.164:0/1092028028 learned_addr learned my addr 172.21.15.164:0/1092028028 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:21.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfea00640 1 --2- 172.21.15.164:0/1092028028 >> v2:172.21.15.172:3300/0 conn(0x7efd000cfda0 0x7efd00160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcffe00640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757385361 0 0) 0x7efd000cd9f0 con 0x7efd0015ba70 2024-05-08T04:41:21.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce4003530 con 0x7efd0015ba70 2024-05-08T04:41:21.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfea00640 1 -- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 msgr2=0x7efd000b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:21.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfea00640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd000b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2422339092 0 0) 0x7efce4003530 con 0x7efd0015ba70 2024-05-08T04:41:21.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfea00640 1 -- 172.21.15.164:0/1092028028 >> v1:172.21.15.164:6789/0 conn(0x7efd0015ba70 legacy=0x7efd000cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcfea00640 1 -- 172.21.15.164:0/1092028028 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0015f820 con 0x7efd000cfda0 2024-05-08T04:41:21.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.340+0000 7efcffe00640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:21.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcfea00640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd000cfda0 0x7efd00160300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097c0 tx=0x7efcec01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2bfd38d74b7a602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b0070 con 0x7efd000cfda0 2024-05-08T04:41:21.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec0ab400 con 0x7efd000cfda0 2024-05-08T04:41:21.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcfe000640 1 -- 172.21.15.164:0/1092028028 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0001a00 con 0x7efd000cfda0 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd000cfda0 msgr2=0x7efd00160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd000cfda0 0x7efd00160300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097c0 tx=0x7efcec01c390 comp rx=0 tx=0).stop 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- 172.21.15.164:0/1092028028 shutdown_connections 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- 172.21.15.164:0/1092028028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd000cfda0 0x7efd00160300 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- 172.21.15.164:0/1092028028 >> 172.21.15.164:0/1092028028 conn(0x7efd0006ac90 msgr2=0x7efd0006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- 172.21.15.164:0/1092028028 shutdown_connections 2024-05-08T04:41:21.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- 172.21.15.164:0/1092028028 wait complete. 2024-05-08T04:41:21.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 Processor -- start 2024-05-08T04:41:21.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- start start 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd0015ba70 0x7efd000b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd001391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd00168d40 con 0x7efd000cfda0 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd00168bc0 con 0x7efd0015ba70 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efd06200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd00168ec0 con 0x7efd0015be50 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcff400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53402/0 (socket says 172.21.15.164:53402) 2024-05-08T04:41:21.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.344+0000 7efcff400640 1 -- 172.21.15.164:0/927146276 learned_addr learned my addr 172.21.15.164:0/927146276 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcfea00640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd0015ba70 0x7efd000b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcffe00640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd001391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 -- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 msgr2=0x7efd001391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd001391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 -- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd0015ba70 msgr2=0x7efd000b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd0015ba70 0x7efd000b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 -- 172.21.15.164:0/927146276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd001389f0 con 0x7efd000cfda0 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcff400640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efcf0001950 tx=0x7efcf000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0012400 con 0x7efd000cfda0 2024-05-08T04:41:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf00125c0 con 0x7efd000cfda0 2024-05-08T04:41:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00129b0 con 0x7efd000cfda0 2024-05-08T04:41:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd000b1120 con 0x7efd000cfda0 2024-05-08T04:41:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0014f470 con 0x7efd000cfda0 2024-05-08T04:41:21.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd00069d60 con 0x7efd000cfda0 2024-05-08T04:41:21.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7efcf0012b50 con 0x7efd000cfda0 2024-05-08T04:41:21.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 0x7efce0069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.348+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efcf0085fd0 con 0x7efd000cfda0 2024-05-08T04:41:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.352+0000 7efcfea00640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 0x7efce0069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.352+0000 7efcfea00640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 0x7efce0069810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ad000 tx=0x7efcec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.352+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcf00569e0 con 0x7efd000cfda0 2024-05-08T04:41:21.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.532+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efd000b09c0 con 0x7efce0067350 2024-05-08T04:41:21.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.532+0000 7efcf7400640 1 -- 172.21.15.164:0/927146276 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7efd000b09c0 con 0x7efce0067350 2024-05-08T04:41:21.538 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:41:21.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:21.538 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:41:21.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:21.539 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:41:21.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 msgr2=0x7efce0069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 0x7efce0069810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ad000 tx=0x7efcec0a6000 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 msgr2=0x7efd00138260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efcf0001950 tx=0x7efcf000a430 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 shutdown_connections 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce0067350 0x7efce0069810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd0015be50 0x7efd001391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd0015ba70 0x7efd000b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 --2- 172.21.15.164:0/927146276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd000cfda0 0x7efd00138260 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 >> 172.21.15.164:0/927146276 conn(0x7efd0006ac90 msgr2=0x7efd0014aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 shutdown_connections 2024-05-08T04:41:21.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.536+0000 7efd06200640 1 -- 172.21.15.164:0/927146276 wait complete. 2024-05-08T04:41:21.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:41:21.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 Processor -- start 2024-05-08T04:41:21.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 -- start start 2024-05-08T04:41:21.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75f80aee90 0x7f75f80d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f80af9a0 con 0x7f75f80d0a90 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f80af750 con 0x7f75f80aee90 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75ff800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80afc80 con 0x7f75f815e900 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75efe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f75f80d0a90 0x7f75f80b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53530/0 (socket says 172.21.15.164:53530) 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75efe00640 1 -- 172.21.15.164:0/1183437578 learned_addr learned my addr 172.21.15.164:0/1183437578 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75fd800640 1 --2- 172.21.15.164:0/1183437578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.692+0000 7f75fce00640 1 --2- 172.21.15.164:0/1183437578 >> v2:172.21.15.172:3300/0 conn(0x7f75f80aee90 0x7f75f80d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75fd800640 1 -- 172.21.15.164:0/1183437578 >> v2:172.21.15.172:3300/0 conn(0x7f75f80aee90 msgr2=0x7f75f80d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75fd800640 1 --2- 172.21.15.164:0/1183437578 >> v2:172.21.15.172:3300/0 conn(0x7f75f80aee90 0x7f75f80d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75fd800640 1 -- 172.21.15.164:0/1183437578 >> v1:172.21.15.164:6789/0 conn(0x7f75f80d0a90 legacy=0x7f75f80b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75fd800640 1 -- 172.21.15.164:0/1183437578 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80aff60 con 0x7f75f815e900 2024-05-08T04:41:21.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75f7e00640 1 -- 172.21.15.164:0/1183437578 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489705988 0 0) 0x7f75f80af9a0 con 0x7f75f80d0a90 2024-05-08T04:41:21.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75fd800640 1 --2- 172.21.15.164:0/1183437578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f815ed90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009d90 tx=0x7f75f001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c49a75eddf26a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75f7e00640 1 -- 172.21.15.164:0/1183437578 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00b2070 con 0x7f75f815e900 2024-05-08T04:41:21.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75f7e00640 1 -- 172.21.15.164:0/1183437578 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f00a5920 con 0x7f75f815e900 2024-05-08T04:41:21.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75f7e00640 1 -- 172.21.15.164:0/1183437578 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00a5cf0 con 0x7f75f815e900 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 -- 172.21.15.164:0/1183437578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 msgr2=0x7f75f815ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 --2- 172.21.15.164:0/1183437578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f815ed90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009d90 tx=0x7f75f001c970 comp rx=0 tx=0).stop 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 -- 172.21.15.164:0/1183437578 shutdown_connections 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 --2- 172.21.15.164:0/1183437578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f815ed90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 --2- 172.21.15.164:0/1183437578 >> v2:172.21.15.172:3300/0 conn(0x7f75f80aee90 0x7f75f80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.696+0000 7f75ff800640 1 -- 172.21.15.164:0/1183437578 >> 172.21.15.164:0/1183437578 conn(0x7f75f8067050 msgr2=0x7f75f806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- 172.21.15.164:0/1183437578 shutdown_connections 2024-05-08T04:41:21.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- 172.21.15.164:0/1183437578 wait complete. 2024-05-08T04:41:21.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 Processor -- start 2024-05-08T04:41:21.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- start start 2024-05-08T04:41:21.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f80aee90 0x7f75f813e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f8154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80720f0 con 0x7f75f80d0a90 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80af270 con 0x7f75f80aee90 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75ff800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8072270 con 0x7f75f815e900 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53414/0 (socket says 172.21.15.164:53414) 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 -- 172.21.15.164:0/1335631469 learned_addr learned my addr 172.21.15.164:0/1335631469 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75fd800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f8154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75fce00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f80aee90 0x7f75f813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 -- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 msgr2=0x7f75f8154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f8154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 -- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f80aee90 msgr2=0x7f75f813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f80aee90 0x7f75f813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 -- 172.21.15.164:0/1335631469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f815b3a0 con 0x7f75f80d0a90 2024-05-08T04:41:21.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75efe00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c4b0 tx=0x7f75e000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.700+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0018070 con 0x7f75f80d0a90 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0007500 con 0x7f75f80d0a90 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004a30 con 0x7f75f80d0a90 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f815b680 con 0x7f75f80d0a90 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f813f2d0 con 0x7f75f80d0a90 2024-05-08T04:41:21.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f8072e80 con 0x7f75f80d0a90 2024-05-08T04:41:21.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f75e0004bd0 con 0x7f75f80d0a90 2024-05-08T04:41:21.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75f5600640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 0x7f75d4069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:21.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f75e00849e0 con 0x7f75f80d0a90 2024-05-08T04:41:21.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.704+0000 7f75fce00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 0x7f75d4069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:21.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.708+0000 7f75fce00640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 0x7f75d4069810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009ac0 tx=0x7f75e8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:21.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.708+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75e0054e20 con 0x7f75f80d0a90 2024-05-08T04:41:21.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75f80d0ff0 con 0x7f75f80d0a90 2024-05-08T04:41:21.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75f5600640 1 -- 172.21.15.164:0/1335631469 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75e0058ec0 con 0x7f75f80d0a90 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 msgr2=0x7f75d4069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 0x7f75d4069810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009ac0 tx=0x7f75e8009190 comp rx=0 tx=0).stop 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 msgr2=0x7f75f815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c4b0 tx=0x7f75e000c970 comp rx=0 tx=0).stop 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 shutdown_connections 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f75d4067350 0x7f75d4069810 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75f815e900 0x7f75f8154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f80d0a90 0x7f75f815aa50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 --2- 172.21.15.164:0/1335631469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f80aee90 0x7f75f813e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:21.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 >> 172.21.15.164:0/1335631469 conn(0x7f75f8067050 msgr2=0x7f75f80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:21.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 shutdown_connections 2024-05-08T04:41:21.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:21.864+0000 7f75ff800640 1 -- 172.21.15.164:0/1335631469 wait complete. 2024-05-08T04:41:21.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp8110 2024-05-08T04:41:22.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 Processor -- start 2024-05-08T04:41:22.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 -- start start 2024-05-08T04:41:22.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3815ba20 0x7fbe38160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbe3806de50 0x7fbe3806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380c6600 con 0x7fbe380ae6e0 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3815fbe0 con 0x7fbe3806de50 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe40000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380c6040 con 0x7fbe3815ba20 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbe380ae6e0 0x7fbe380cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53540/0 (socket says 172.21.15.164:53540) 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbe3806de50 0x7fbe3806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3e000640 1 -- 172.21.15.164:0/883736994 learned_addr learned my addr 172.21.15.164:0/883736994 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3d600640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3815ba20 0x7fbe38160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe37e00640 1 -- 172.21.15.164:0/883736994 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477836858 0 0) 0x7fbe380c6600 con 0x7fbe380ae6e0 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3cc00640 1 -- 172.21.15.164:0/883736994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3815ba20 msgr2=0x7fbe38160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3cc00640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3815ba20 0x7fbe38160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3cc00640 1 -- 172.21.15.164:0/883736994 >> v1:172.21.15.164:6789/0 conn(0x7fbe380ae6e0 legacy=0x7fbe380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.024+0000 7fbe3cc00640 1 -- 172.21.15.164:0/883736994 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380c6320 con 0x7fbe3806de50 2024-05-08T04:41:22.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe3cc00640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe3806e290 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fbe3815fd60 tx=0x7fbe2401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9de797072766774 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe37e00640 1 -- 172.21.15.164:0/883736994 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240ab610 con 0x7fbe3806de50 2024-05-08T04:41:22.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe37e00640 1 -- 172.21.15.164:0/883736994 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe240b0070 con 0x7fbe3806de50 2024-05-08T04:41:22.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe37e00640 1 -- 172.21.15.164:0/883736994 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240ab9d0 con 0x7fbe3806de50 2024-05-08T04:41:22.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 -- 172.21.15.164:0/883736994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 msgr2=0x7fbe3806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe3806e290 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fbe3815fd60 tx=0x7fbe2401c470 comp rx=0 tx=0).stop 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 -- 172.21.15.164:0/883736994 shutdown_connections 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe3806e290 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 --2- 172.21.15.164:0/883736994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3815ba20 0x7fbe38160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 -- 172.21.15.164:0/883736994 >> 172.21.15.164:0/883736994 conn(0x7fbe380cf770 msgr2=0x7fbe380cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 -- 172.21.15.164:0/883736994 shutdown_connections 2024-05-08T04:41:22.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.028+0000 7fbe40000640 1 -- 172.21.15.164:0/883736994 wait complete. 2024-05-08T04:41:22.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 Processor -- start 2024-05-08T04:41:22.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- start start 2024-05-08T04:41:22.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe380ae6e0 0x7fbe381467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 0x7fbe380b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380aec60 con 0x7fbe3806de50 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380aeae0 con 0x7fbe3815ba20 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380aede0 con 0x7fbe380ae6e0 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53424/0 (socket says 172.21.15.164:53424) 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 -- 172.21.15.164:0/3812602973 learned_addr learned my addr 172.21.15.164:0/3812602973 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3cc00640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe380ae6e0 0x7fbe381467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3e000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 0x7fbe380b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 -- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe380ae6e0 msgr2=0x7fbe381467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe380ae6e0 0x7fbe381467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 -- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 msgr2=0x7fbe380b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 0x7fbe380b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 -- 172.21.15.164:0/3812602973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe381475f0 con 0x7fbe3806de50 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3e000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 0x7fbe380b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe3d600640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00e870 tx=0x7fbe2c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017070 con 0x7fbe3806de50 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00c8e0 con 0x7fbe3806de50 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00cc90 con 0x7fbe3806de50 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe380b5010 con 0x7fbe3806de50 2024-05-08T04:41:22.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38071000 con 0x7fbe3806de50 2024-05-08T04:41:22.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.032+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe380af210 con 0x7fbe3806de50 2024-05-08T04:41:22.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.036+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7fbe2c01a070 con 0x7fbe3806de50 2024-05-08T04:41:22.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.036+0000 7fbe35600640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 0x7fbe18069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.036+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbe2c0858a0 con 0x7fbe3806de50 2024-05-08T04:41:22.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.036+0000 7fbe3cc00640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 0x7fbe18069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.036+0000 7fbe3cc00640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 0x7fbe18069810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe240af000 tx=0x7fbe240a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.040+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe381475f0 con 0x7fbe3806de50 2024-05-08T04:41:22.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.228+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp8110", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe38072d30 con 0x7fbe18067350 2024-05-08T04:41:22.241 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:22.236+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:22.241 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:22.236+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:22.241 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:22.236+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:22.241 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:22.236+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:22.241 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:22.236+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:22.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.248+0000 7fbe35600640 1 -- 172.21.15.164:0/3812602973 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbe38072d30 con 0x7fbe18067350 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 msgr2=0x7fbe18069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 0x7fbe18069810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe240af000 tx=0x7fbe240a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 msgr2=0x7fbe38146d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00e870 tx=0x7fbe2c00ec70 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 shutdown_connections 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbe18067350 0x7fbe18069810 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe380ae6e0 0x7fbe381467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3815ba20 0x7fbe380b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 --2- 172.21.15.164:0/3812602973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe3806de50 0x7fbe38146d70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 >> 172.21.15.164:0/3812602973 conn(0x7fbe380cf770 msgr2=0x7fbe3815e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:22.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 shutdown_connections 2024-05-08T04:41:22.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.252+0000 7fbe40000640 1 -- 172.21.15.164:0/3812602973 wait complete. 2024-05-08T04:41:22.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol71870 --group_name subvol_grp8110 2024-05-08T04:41:22.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.412+0000 7f0aff600640 1 Processor -- start 2024-05-08T04:41:22.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.412+0000 7f0aff600640 1 -- start start 2024-05-08T04:41:22.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.412+0000 7f0aff600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0af815ce10 0x7f0af806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.412+0000 7f0aff600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80c6600 con 0x7f0af815ba20 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0aff600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80c6040 con 0x7f0af806e390 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0aff600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af815fbe0 con 0x7f0af815ce10 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0afcc00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0af815ba20 0x7f0af8160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53546/0 (socket says 172.21.15.164:53546) 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0afcc00640 1 -- 172.21.15.164:0/148471546 learned_addr learned my addr 172.21.15.164:0/148471546 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7e00640 1 --2- 172.21.15.164:0/148471546 >> v2:172.21.15.164:3300/0 conn(0x7f0af815ce10 0x7f0af806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887446454 0 0) 0x7f0af80c6600 con 0x7f0af815ba20 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc003530 con 0x7f0af815ba20 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653142814 0 0) 0x7f0af80c6040 con 0x7f0af806e390 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7e00640 1 -- 172.21.15.164:0/148471546 >> v1:172.21.15.172:6789/0 conn(0x7f0af806e390 legacy=0x7f0af80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7e00640 1 -- 172.21.15.164:0/148471546 >> v1:172.21.15.164:6789/0 conn(0x7f0af815ba20 legacy=0x7f0af8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7e00640 1 -- 172.21.15.164:0/148471546 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80c6320 con 0x7f0af815ce10 2024-05-08T04:41:22.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7e00640 1 --2- 172.21.15.164:0/148471546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af815ce10 0x7f0af806de50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40097c0 tx=0x7f0ae401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a4216acc8dc9d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40b1070 con 0x7f0af815ce10 2024-05-08T04:41:22.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae40ac410 con 0x7f0af815ce10 2024-05-08T04:41:22.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.416+0000 7f0af7400640 1 -- 172.21.15.164:0/148471546 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40ac7e0 con 0x7f0af815ce10 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- 172.21.15.164:0/148471546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af815ce10 msgr2=0x7f0af806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 --2- 172.21.15.164:0/148471546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af815ce10 0x7f0af806de50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40097c0 tx=0x7f0ae401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- 172.21.15.164:0/148471546 shutdown_connections 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 --2- 172.21.15.164:0/148471546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af815ce10 0x7f0af806de50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- 172.21.15.164:0/148471546 >> 172.21.15.164:0/148471546 conn(0x7f0af80cf770 msgr2=0x7f0af80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- 172.21.15.164:0/148471546 shutdown_connections 2024-05-08T04:41:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- 172.21.15.164:0/148471546 wait complete. 2024-05-08T04:41:22.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 Processor -- start 2024-05-08T04:41:22.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- start start 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 0x7f0af8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0af815ce10 0x7f0af8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80ce000 con 0x7f0af806e390 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cde80 con 0x7f0af815ce10 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0aff600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8169b80 con 0x7f0af815ba20 2024-05-08T04:41:22.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0afcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0afcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53474/0 (socket says 172.21.15.164:53474) 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0afcc00640 1 -- 172.21.15.164:0/1757699831 learned_addr learned my addr 172.21.15.164:0/1757699831 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0af7e00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 0x7f0af8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.420+0000 7f0afd600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0af815ce10 0x7f0af8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 -- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 msgr2=0x7f0af8154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 0x7f0af8154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 -- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0af815ce10 msgr2=0x7f0af8150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0af815ce10 0x7f0af8150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 -- 172.21.15.164:0/1757699831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af8070db0 con 0x7f0af806e390 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af7e00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 0x7f0af8154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0afcc00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800cb70 tx=0x7f0ae800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8017070 con 0x7f0af806e390 2024-05-08T04:41:22.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800ad40 con 0x7f0af806e390 2024-05-08T04:41:22.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af8071090 con 0x7f0af806e390 2024-05-08T04:41:22.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8012620 con 0x7f0af806e390 2024-05-08T04:41:22.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af8071b70 con 0x7f0af806e390 2024-05-08T04:41:22.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8072d30 con 0x7f0af806e390 2024-05-08T04:41:22.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f0ae801a070 con 0x7f0af806e390 2024-05-08T04:41:22.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 0x7f0ad8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0ae80854b0 con 0x7f0af806e390 2024-05-08T04:41:22.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.424+0000 7f0af7e00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 0x7f0ad8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.428+0000 7f0af7e00640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 0x7f0ad8069810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40ae000 tx=0x7f0ae40a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.428+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ae80558c0 con 0x7f0af806e390 2024-05-08T04:41:22.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.612+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol71870", "group_name": "subvol_grp8110", "target": ["mon-mgr", ""]}) v1 -- 0x7f0af80b49e0 con 0x7f0ad8067350 2024-05-08T04:41:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.632+0000 7f0af4c00640 1 -- 172.21.15.164:0/1757699831 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ae40ac410 con 0x7f0ad8067350 2024-05-08T04:41:22.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 msgr2=0x7f0ad8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 0x7f0ad8069810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40ae000 tx=0x7f0ae40a6040 comp rx=0 tx=0).stop 2024-05-08T04:41:22.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 msgr2=0x7f0af815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800cb70 tx=0x7f0ae800a430 comp rx=0 tx=0).stop 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 shutdown_connections 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ad8067350 0x7f0ad8069810 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0af815ba20 0x7f0af8154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0af815ce10 0x7f0af8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 --2- 172.21.15.164:0/1757699831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af806e390 0x7f0af815add0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 >> 172.21.15.164:0/1757699831 conn(0x7f0af80cf770 msgr2=0x7f0af8155380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 shutdown_connections 2024-05-08T04:41:22.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.636+0000 7f0aff600640 1 -- 172.21.15.164:0/1757699831 wait complete. 2024-05-08T04:41:22.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:22.660 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp8110/.snap/ancestral_snap_1) 2024-05-08T04:41:22.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:22.690 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp8110) 2024-05-08T04:41:22.738 INFO:teuthology.orchestra.run.smithi164.stdout:1099511627777 2024-05-08T04:41:22.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:22.740 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp8110/subvol71870/.snap/_ancestral_snap_1_1099511627777) 2024-05-08T04:41:22.794 INFO:teuthology.orchestra.run.smithi164.stdout:9a450570-eafc-4eac-a436-c0e69a154d62 2024-05-08T04:41:22.795 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol71870 _ancestral_snap_1_1099511627777 subvol_grp8110 2024-05-08T04:41:22.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 Processor -- start 2024-05-08T04:41:22.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 -- start start 2024-05-08T04:41:22.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 0x7f7ef00d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ef00cd560 0x7f7ef015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef0070280 con 0x7f7ef015f2b0 2024-05-08T04:41:22.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef015f690 con 0x7f7ef00cd560 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7ef5400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef00cff80 con 0x7f7ef00b3fe0 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eef200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ef00cd560 0x7f7ef015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eee800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7ef015f2b0 0x7f7ef00cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53554/0 (socket says 172.21.15.164:53554) 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eee800640 1 -- 172.21.15.164:0/2230383188 learned_addr learned my addr 172.21.15.164:0/2230383188 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eede00640 1 --2- 172.21.15.164:0/2230383188 >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 0x7f7ef00d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eef200640 1 -- 172.21.15.164:0/2230383188 >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 msgr2=0x7f7ef00d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:22.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eef200640 1 --2- 172.21.15.164:0/2230383188 >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 0x7f7ef00d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eef200640 1 -- 172.21.15.164:0/2230383188 >> v1:172.21.15.164:6789/0 conn(0x7f7ef015f2b0 legacy=0x7f7ef00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eef200640 1 -- 172.21.15.164:0/2230383188 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0070560 con 0x7f7ef00cd560 2024-05-08T04:41:22.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.980+0000 7f7eede00640 1 --2- 172.21.15.164:0/2230383188 >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 0x7f7ef00d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:22.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7eef200640 1 --2- 172.21.15.164:0/2230383188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef00cd560 0x7f7ef015f8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0071be0 tx=0x7f7ee801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89285d1b4f07e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7eed400640 1 -- 172.21.15.164:0/2230383188 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80b2070 con 0x7f7ef00cd560 2024-05-08T04:41:22.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7eed400640 1 -- 172.21.15.164:0/2230383188 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee80a59c0 con 0x7f7ef00cd560 2024-05-08T04:41:22.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7eed400640 1 -- 172.21.15.164:0/2230383188 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80a5d90 con 0x7f7ef00cd560 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 -- 172.21.15.164:0/2230383188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef00cd560 msgr2=0x7f7ef015f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 --2- 172.21.15.164:0/2230383188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef00cd560 0x7f7ef015f8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0071be0 tx=0x7f7ee801c900 comp rx=0 tx=0).stop 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 -- 172.21.15.164:0/2230383188 shutdown_connections 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 --2- 172.21.15.164:0/2230383188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef00cd560 0x7f7ef015f8b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 --2- 172.21.15.164:0/2230383188 >> v2:172.21.15.172:3300/0 conn(0x7f7ef00b3fe0 0x7f7ef00d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 -- 172.21.15.164:0/2230383188 >> 172.21.15.164:0/2230383188 conn(0x7f7ef00b34f0 msgr2=0x7f7ef00b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 -- 172.21.15.164:0/2230383188 shutdown_connections 2024-05-08T04:41:22.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.984+0000 7f7ef5400640 1 -- 172.21.15.164:0/2230383188 wait complete. 2024-05-08T04:41:22.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 Processor -- start 2024-05-08T04:41:22.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- start start 2024-05-08T04:41:22.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef00b3fe0 0x7f7ef00b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef00cd560 0x7f7ef015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef015f2b0 0x7f7ef0150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0168d70 con 0x7f7ef015f2b0 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0168bf0 con 0x7f7ef00b3fe0 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0168ef0 con 0x7f7ef00cd560 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef015f2b0 0x7f7ef0150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef015f2b0 0x7f7ef0150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53492/0 (socket says 172.21.15.164:53492) 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 -- 172.21.15.164:0/1736564500 learned_addr learned my addr 172.21.15.164:0/1736564500 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eee800640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef00b3fe0 0x7f7ef00b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 -- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef00cd560 msgr2=0x7f7ef015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eede00640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef00cd560 0x7f7ef015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef00cd560 0x7f7ef015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 -- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef00b3fe0 msgr2=0x7f7ef00b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef00b3fe0 0x7f7ef00b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 -- 172.21.15.164:0/1736564500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef01552f0 con 0x7f7ef015f2b0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eee800640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef00b3fe0 0x7f7ef00b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eede00640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef00cd560 0x7f7ef015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7eef200640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ef015f2b0 0x7f7ef0150cc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80b0040 tx=0x7f7ee80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80b2050 con 0x7f7ef015f2b0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee80be070 con 0x7f7ef015f2b0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- 172.21.15.164:0/1736564500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef01555d0 con 0x7f7ef015f2b0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80adb70 con 0x7f7ef015f2b0 2024-05-08T04:41:22.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.988+0000 7f7ef5400640 1 -- 172.21.15.164:0/1736564500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef0155b90 con 0x7f7ef015f2b0 2024-05-08T04:41:22.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7ef5400640 1 -- 172.21.15.164:0/1736564500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef0155d60 con 0x7f7ef015f2b0 2024-05-08T04:41:22.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f7ee80add10 con 0x7f7ef015f2b0 2024-05-08T04:41:22.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7ee6a00640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ecc067350 0x7f7ecc069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:22.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7ee8120c10 con 0x7f7ef015f2b0 2024-05-08T04:41:22.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7eee800640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ecc067350 0x7f7ecc069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:22.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.992+0000 7f7eee800640 1 --2- 172.21.15.164:0/1736564500 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ecc067350 0x7f7ecc069810 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8007fd0 tx=0x7f7ed8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:22.996+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ef0155d60 con 0x7f7ef015f2b0 2024-05-08T04:41:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.184+0000 7f7ef5400640 1 -- 172.21.15.164:0/1736564500 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol71870", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp8110", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ef004f010 con 0x7f7ecc067350 2024-05-08T04:41:23.195 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.192+0000 7ffa56200640 -1 mgr.server reply reply (22) Invalid argument failed to open clone index 2024-05-08T04:41:23.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.192+0000 7f7ee6a00640 1 -- 172.21.15.164:0/1736564500 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -22 failed to open clone index) v1 ==== 34+0+0 (secure 0 0 0) 0x7f7ef004f010 con 0x7f7ecc067350 2024-05-08T04:41:23.196 INFO:teuthology.orchestra.run.smithi164.stderr:Error EINVAL: failed to open clone index 2024-05-08T04:41:23.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T04:41:23.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:23.206 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp8110/.snap/ancestral_snap_1) 2024-05-08T04:41:23.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol71870 --group_name subvol_grp8110 2024-05-08T04:41:23.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.408+0000 7f8a13200640 1 Processor -- start 2024-05-08T04:41:23.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.408+0000 7f8a13200640 1 -- start start 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a13200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a13200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a0c06dfa0 0x7f8a0c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a13200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c06d150 con 0x7f8a0c15a200 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a13200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c06bca0 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a13200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0af150 con 0x7f8a0c0ca850 2024-05-08T04:41:23.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a0c06dfa0 0x7f8a0c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a0be00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8a0c15a200 0x7f8a0c0ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53564/0 (socket says 172.21.15.164:53564) 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a0be00640 1 -- 172.21.15.164:0/39161580 learned_addr learned my addr 172.21.15.164:0/39161580 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a03e00640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 -- 172.21.15.164:0/39161580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 msgr2=0x7f8a0c0aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c0aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 -- 172.21.15.164:0/39161580 >> v1:172.21.15.164:6789/0 conn(0x7f8a0c15a200 legacy=0x7f8a0c0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 -- 172.21.15.164:0/39161580 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c06d430 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a0b400640 1 -- 172.21.15.164:0/39161580 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629919347 0 0) 0x7f8a0c06d150 con 0x7f8a0c15a200 2024-05-08T04:41:23.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.412+0000 7f8a11200640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c0cd670 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400aea0 tx=0x7f8a0401d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656cf88a94b067d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0b400640 1 -- 172.21.15.164:0/39161580 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a6760 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0b400640 1 -- 172.21.15.164:0/39161580 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040a6900 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0b400640 1 -- 172.21.15.164:0/39161580 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a6cd0 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- 172.21.15.164:0/39161580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 msgr2=0x7f8a0c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c0cd670 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400aea0 tx=0x7f8a0401d920 comp rx=0 tx=0).stop 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- 172.21.15.164:0/39161580 shutdown_connections 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c0cd670 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- 172.21.15.164:0/39161580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- 172.21.15.164:0/39161580 >> 172.21.15.164:0/39161580 conn(0x7f8a0c0cf770 msgr2=0x7f8a0c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:23.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- 172.21.15.164:0/39161580 shutdown_connections 2024-05-08T04:41:23.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- 172.21.15.164:0/39161580 wait complete. 2024-05-08T04:41:23.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 Processor -- start 2024-05-08T04:41:23.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- start start 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0c15a200 0x7f8a0c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cdd30 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cdbb0 con 0x7f8a0c15a200 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a13200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cdeb0 con 0x7f8a0c0ca850 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53510/0 (socket says 172.21.15.164:53510) 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 -- 172.21.15.164:0/1540848182 learned_addr learned my addr 172.21.15.164:0/1540848182 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:23.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a11200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0c15a200 0x7f8a0c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a03e00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 -- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 msgr2=0x7f8a0c153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 -- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0c15a200 msgr2=0x7f8a0c15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0c15a200 0x7f8a0c15ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.416+0000 7f8a0be00640 1 -- 172.21.15.164:0/1540848182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c1531c0 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a0be00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c480 tx=0x7f89fc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc018070 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc007500 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004a30 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c1534a0 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c153a60 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c072d30 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f89fc01b070 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 0x7f89e8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f89fc084a40 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.420+0000 7f8a03e00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 0x7f89e8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.424+0000 7f8a03e00640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 0x7f89e8069810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0c4da0 tx=0x7f89f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.424+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89fc0554a0 con 0x7f8a0c06dfa0 2024-05-08T04:41:23.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.604+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol71870", "group_name": "subvol_grp8110", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a0c153c30 con 0x7f89e8067350 2024-05-08T04:41:23.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a08c00640 1 -- 172.21.15.164:0/1540848182 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8a0c153c30 con 0x7f89e8067350 2024-05-08T04:41:23.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 msgr2=0x7f89e8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 0x7f89e8069810 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0c4da0 tx=0x7f89f4008040 comp rx=0 tx=0).stop 2024-05-08T04:41:23.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 msgr2=0x7f8a0c14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c480 tx=0x7f89fc00c940 comp rx=0 tx=0).stop 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 shutdown_connections 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067350 0x7f89e8069810 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0c0ca850 0x7f8a0c153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0c15a200 0x7f8a0c15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 --2- 172.21.15.164:0/1540848182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0c06dfa0 0x7f8a0c14f210 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 >> 172.21.15.164:0/1540848182 conn(0x7f8a0c0cf770 msgr2=0x7f8a0c15e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 shutdown_connections 2024-05-08T04:41:23.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.620+0000 7f8a13200640 1 -- 172.21.15.164:0/1540848182 wait complete. 2024-05-08T04:41:23.630 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.624+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.630 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.624+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.630 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.624+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.630 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.624+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.630 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.624+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.648 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.644+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.649 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.644+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.649 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.644+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.649 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.644+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.649 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:23.644+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:23.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:23.651 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:41:23.666 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:41:23.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp8110 2024-05-08T04:41:23.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 Processor -- start 2024-05-08T04:41:23.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 -- start start 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92a815ab10 0x7f92a806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92a80cee90 0x7f92a80aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a815e870 con 0x7f92a80ae6e0 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a806bb80 con 0x7f92a80cee90 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92ae400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a815eb50 con 0x7f92a815ab10 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f929ea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f92a80ae6e0 0x7f92a80ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53580/0 (socket says 172.21.15.164:53580) 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f929ea00640 1 -- 172.21.15.164:0/585265369 learned_addr learned my addr 172.21.15.164:0/585265369 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a7e00640 1 --2- 172.21.15.164:0/585265369 >> v2:172.21.15.172:3300/0 conn(0x7f92a80cee90 0x7f92a80aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a7400640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92a815ab10 0x7f92a806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158198727 0 0) 0x7f92a815e870 con 0x7f92a80ae6e0 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f928c003530 con 0x7f92a80ae6e0 2024-05-08T04:41:23.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703589347 0 0) 0x7f928c003530 con 0x7f92a80ae6e0 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92a815ab10 msgr2=0x7f92a806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92a815ab10 0x7f92a806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 >> v2:172.21.15.172:3300/0 conn(0x7f92a80cee90 msgr2=0x7f92a80aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 --2- 172.21.15.164:0/585265369 >> v2:172.21.15.172:3300/0 conn(0x7f92a80cee90 0x7f92a80aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a815ee30 con 0x7f92a80ae6e0 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f9290001680 con 0x7f92a80ae6e0 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 >> v1:172.21.15.164:6789/0 conn(0x7f92a80ae6e0 legacy=0x7f92a80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f928c0039c0 0x7f928c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c004810 0x7f928c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c0051d0 0x7f928c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a806bb80 con 0x7f928c0051d0 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a815eb50 con 0x7f928c0039c0 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c00bf60 con 0x7f928c004810 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a7400640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92a815ab10 0x7f92a806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:23.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.848+0000 7f92a7e00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c0051d0 0x7f928c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f929ea00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c004810 0x7f928c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7400640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f928c0039c0 0x7f928c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 -- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c004810 msgr2=0x7f928c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c004810 0x7f928c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 -- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f928c0039c0 msgr2=0x7f928c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f928c0039c0 0x7f928c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 -- 172.21.15.164:0/585265369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c003530 con 0x7f928c0051d0 2024-05-08T04:41:23.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c0051d0 0x7f928c00b9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009f90 tx=0x7f92a001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5875023f7279fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a00b0070 con 0x7f928c0051d0 2024-05-08T04:41:23.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a00ab600 con 0x7f928c0051d0 2024-05-08T04:41:23.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a6a00640 1 -- 172.21.15.164:0/585265369 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0009c10 con 0x7f928c0051d0 2024-05-08T04:41:23.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92ae400640 1 -- 172.21.15.164:0/585265369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c0051d0 msgr2=0x7f928c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92ae400640 1 --2- 172.21.15.164:0/585265369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c0051d0 0x7f928c00b9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009f90 tx=0x7f92a001c8f0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.852+0000 7f92a7e00640 1 -- 172.21.15.164:0/585265369 reap_dead start 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- 172.21.15.164:0/585265369 shutdown_connections 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- 172.21.15.164:0/585265369 >> 172.21.15.164:0/585265369 conn(0x7f92a80cf770 msgr2=0x7f92a80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- 172.21.15.164:0/585265369 shutdown_connections 2024-05-08T04:41:23.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- 172.21.15.164:0/585265369 wait complete. 2024-05-08T04:41:23.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 Processor -- start 2024-05-08T04:41:23.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- start start 2024-05-08T04:41:23.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c0051d0 0x7f92a8155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a8071880 0x7f92a81f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80cac70 con 0x7f92a8070270 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80caaf0 con 0x7f92a8071880 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92ae400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80cadf0 con 0x7f928c0051d0 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53584/0 (socket says 172.21.15.164:53584) 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 -- 172.21.15.164:0/2532330080 learned_addr learned my addr 172.21.15.164:0/2532330080 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92a7400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c0051d0 0x7f92a8155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f92a7e00640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a8071880 0x7f92a81f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 -- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c0051d0 msgr2=0x7f92a8155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c0051d0 0x7f92a8155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 -- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a8071880 msgr2=0x7f92a81f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:23.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a8071880 0x7f92a81f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 -- 172.21.15.164:0/2532330080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80707e0 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.856+0000 7f929ea00640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9290012510 tx=0x7f92900129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929001c070 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290014770 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290014b60 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a8070ac0 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a8071080 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a8072d30 con 0x7f92a8070270 2024-05-08T04:41:23.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) v1 ==== 87084+0+0 (secure 0 0 0) 0x7f9290014d00 con 0x7f92a8070270 2024-05-08T04:41:23.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 0x7f9284069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:23.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9290087e60 con 0x7f92a8070270 2024-05-08T04:41:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f92a7400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 0x7f9284069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:23.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.860+0000 7f92a7400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 0x7f9284069810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f92a80c4920 tx=0x7f9298009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:23.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:23.864+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9290058380 con 0x7f92a8070270 2024-05-08T04:41:24.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.040+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp8110", "target": ["mon-mgr", ""]}) v1 -- 0x7f92a80aef30 con 0x7f9284067350 2024-05-08T04:41:24.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.040+0000 7f929fe00640 1 -- 172.21.15.164:0/2532330080 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f92a80aef30 con 0x7f9284067350 2024-05-08T04:41:24.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 msgr2=0x7f9284069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 0x7f9284069810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f92a80c4920 tx=0x7f9298009190 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 msgr2=0x7f92a81f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9290012510 tx=0x7f92900129d0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 shutdown_connections 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9284067350 0x7f9284069810 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a8071880 0x7f92a81f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a8070270 0x7f92a81f6040 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 --2- 172.21.15.164:0/2532330080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f928c0051d0 0x7f92a8155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 >> 172.21.15.164:0/2532330080 conn(0x7f92a80cf770 msgr2=0x7f92a80cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 shutdown_connections 2024-05-08T04:41:24.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.044+0000 7f92ae400640 1 -- 172.21.15.164:0/2532330080 wait complete. 2024-05-08T04:41:24.068 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:41:24.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:24.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:24.233 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:41:24.233 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:24.233 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:24.242 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:24.242 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:24.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:24.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.468+0000 7f5fbb400640 1 Processor -- start 2024-05-08T04:41:24.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.468+0000 7f5fbb400640 1 -- start start 2024-05-08T04:41:24.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb4161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fbb400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb4161fe0 con 0x7f5fb415ea20 2024-05-08T04:41:24.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fbb400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb41622c0 con 0x7f5fb40cfd60 2024-05-08T04:41:24.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fbb400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb4162880 con 0x7f5fb4144200 2024-05-08T04:41:24.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb9400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5fb415ea20 0x7f5fb40ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53586/0 (socket says 172.21.15.164:53586) 2024-05-08T04:41:24.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb9400640 1 -- 172.21.15.164:0/1514267706 learned_addr learned my addr 172.21.15.164:0/1514267706 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:24.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb8a00640 1 --2- 172.21.15.164:0/1514267706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb4161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956993297 0 0) 0x7f5fb4161fe0 con 0x7f5fb415ea20 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f98003530 con 0x7f5fb415ea20 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923448394 0 0) 0x7f5fb41622c0 con 0x7f5fb40cfd60 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb4161fe0 con 0x7f5fb40cfd60 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb8a00640 1 -- 172.21.15.164:0/1514267706 >> v1:172.21.15.172:6789/0 conn(0x7f5fb40cfd60 legacy=0x7f5fb40cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb8a00640 1 -- 172.21.15.164:0/1514267706 >> v1:172.21.15.164:6789/0 conn(0x7f5fb415ea20 legacy=0x7f5fb40ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.472+0000 7f5fb8a00640 1 -- 172.21.15.164:0/1514267706 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb41625a0 con 0x7f5fb4144200 2024-05-08T04:41:24.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fb8a00640 1 --2- 172.21.15.164:0/1514267706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb4161950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097f0 tx=0x7f5fa401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e8f7c1e21fe91da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b0070 con 0x7f5fb4144200 2024-05-08T04:41:24.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa40ab600 con 0x7f5fb4144200 2024-05-08T04:41:24.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fb3e00640 1 -- 172.21.15.164:0/1514267706 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40ab9b0 con 0x7f5fb4144200 2024-05-08T04:41:24.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- 172.21.15.164:0/1514267706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 msgr2=0x7f5fb4161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 --2- 172.21.15.164:0/1514267706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb4161950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097f0 tx=0x7f5fa401c450 comp rx=0 tx=0).stop 2024-05-08T04:41:24.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- 172.21.15.164:0/1514267706 shutdown_connections 2024-05-08T04:41:24.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 --2- 172.21.15.164:0/1514267706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb4161950 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- 172.21.15.164:0/1514267706 >> 172.21.15.164:0/1514267706 conn(0x7f5fb406ac90 msgr2=0x7f5fb406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:24.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- 172.21.15.164:0/1514267706 shutdown_connections 2024-05-08T04:41:24.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- 172.21.15.164:0/1514267706 wait complete. 2024-05-08T04:41:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 Processor -- start 2024-05-08T04:41:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.476+0000 7f5fbb400640 1 -- start start 2024-05-08T04:41:24.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fb40cfd60 0x7f5fb414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb413cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40cc7c0 con 0x7f5fb415ea20 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40d0160 con 0x7f5fb40cfd60 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40cc940 con 0x7f5fb4144200 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53592/0 (socket says 172.21.15.164:53592) 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 -- 172.21.15.164:0/747392862 learned_addr learned my addr 172.21.15.164:0/747392862 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb8a00640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fb40cfd60 0x7f5fb414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fabe00640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb413cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 -- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 msgr2=0x7f5fb413cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb413cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 -- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fb40cfd60 msgr2=0x7f5fb414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fb40cfd60 0x7f5fb414a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 -- 172.21.15.164:0/747392862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40b2f30 con 0x7f5fb415ea20 2024-05-08T04:41:24.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb9400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00cbd0 tx=0x7f5fac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac017070 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00ace0 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac012600 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb40b3f90 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb40b34b0 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb4069d60 con 0x7f5fb415ea20 2024-05-08T04:41:24.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.480+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 10) v1 ==== 87135+0+0 (secure 0 0 0) 0x7f5fac01a070 con 0x7f5fb415ea20 2024-05-08T04:41:24.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.484+0000 7f5fb1600640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 0x7f5f900698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.484+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5fac085850 con 0x7f5fb415ea20 2024-05-08T04:41:24.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.484+0000 7f5fb8a00640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 0x7f5f900698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.484+0000 7f5fb8a00640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 0x7f5f900698a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ad000 tx=0x7f5fa40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.488+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fac055a60 con 0x7f5fb415ea20 2024-05-08T04:41:24.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.644+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fb40ceab0 con 0x7f5fb415ea20 2024-05-08T04:41:24.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.644+0000 7f5fb1600640 1 -- 172.21.15.164:0/747392862 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f5fac059b00 con 0x7f5fb415ea20 2024-05-08T04:41:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:24.650 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":74,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:09.101885+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:24.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 msgr2=0x7f5f900698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 0x7f5f900698a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ad000 tx=0x7f5fa40a6000 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 msgr2=0x7f5fb413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00cbd0 tx=0x7f5fac00a430 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 shutdown_connections 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5f900673e0 0x7f5f900698a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fb4144200 0x7f5fb413cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fb415ea20 0x7f5fb413d1d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 --2- 172.21.15.164:0/747392862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fb40cfd60 0x7f5fb414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 >> 172.21.15.164:0/747392862 conn(0x7f5fb406ac90 msgr2=0x7f5fb40b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 shutdown_connections 2024-05-08T04:41:24.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.648+0000 7f5fbb400640 1 -- 172.21.15.164:0/747392862 wait complete. 2024-05-08T04:41:24.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:05.195201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:06.483763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:24.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:24.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 Processor -- start 2024-05-08T04:41:24.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 -- start start 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42a015ba20 0x7f42a0160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42a006de50 0x7f42a006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a00c6600 con 0x7f42a00ae6e0 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a015fbe0 con 0x7f42a006de50 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f42a5a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a00c6040 con 0x7f42a015ba20 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f4296000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42a006de50 0x7f42a006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f429f400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f42a00ae6e0 0x7f42a00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53588/0 (socket says 172.21.15.164:53588) 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f429f400640 1 -- 172.21.15.164:0/3977000571 learned_addr learned my addr 172.21.15.164:0/3977000571 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:24.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f429ea00640 1 --2- 172.21.15.164:0/3977000571 >> v2:172.21.15.172:3300/0 conn(0x7f42a015ba20 0x7f42a0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f4296000640 1 -- 172.21.15.164:0/3977000571 >> v2:172.21.15.172:3300/0 conn(0x7f42a015ba20 msgr2=0x7f42a0160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:24.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f4296000640 1 --2- 172.21.15.164:0/3977000571 >> v2:172.21.15.172:3300/0 conn(0x7f42a015ba20 0x7f42a0160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f4296000640 1 -- 172.21.15.164:0/3977000571 >> v1:172.21.15.164:6789/0 conn(0x7f42a00ae6e0 legacy=0x7f42a00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f4296000640 1 -- 172.21.15.164:0/3977000571 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00c6320 con 0x7f42a006de50 2024-05-08T04:41:24.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.804+0000 7f429e000640 1 -- 172.21.15.164:0/3977000571 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671822141 0 0) 0x7f42a00c6600 con 0x7f42a00ae6e0 2024-05-08T04:41:24.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f4296000640 1 --2- 172.21.15.164:0/3977000571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a006de50 0x7f42a006e2c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f42a00cc0e0 tx=0x7f428801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327431660ade3089 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f429e000640 1 -- 172.21.15.164:0/3977000571 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880ab5f0 con 0x7f42a006de50 2024-05-08T04:41:24.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f429e000640 1 -- 172.21.15.164:0/3977000571 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880b0070 con 0x7f42a006de50 2024-05-08T04:41:24.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f429e000640 1 -- 172.21.15.164:0/3977000571 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880ab9b0 con 0x7f42a006de50 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 -- 172.21.15.164:0/3977000571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a006de50 msgr2=0x7f42a006e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 --2- 172.21.15.164:0/3977000571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a006de50 0x7f42a006e2c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f42a00cc0e0 tx=0x7f428801c450 comp rx=0 tx=0).stop 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 -- 172.21.15.164:0/3977000571 shutdown_connections 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 --2- 172.21.15.164:0/3977000571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a006de50 0x7f42a006e2c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 --2- 172.21.15.164:0/3977000571 >> v2:172.21.15.172:3300/0 conn(0x7f42a015ba20 0x7f42a0160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 -- 172.21.15.164:0/3977000571 >> 172.21.15.164:0/3977000571 conn(0x7f42a00cf770 msgr2=0x7f42a00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 -- 172.21.15.164:0/3977000571 shutdown_connections 2024-05-08T04:41:24.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.808+0000 7f42a5a00640 1 -- 172.21.15.164:0/3977000571 wait complete. 2024-05-08T04:41:24.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 Processor -- start 2024-05-08T04:41:24.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- start start 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a006de50 0x7f42a014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a00ae6e0 0x7f42a00cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00aec60 con 0x7f42a015ba20 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00aeae0 con 0x7f42a006de50 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00aede0 con 0x7f42a00ae6e0 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53624/0 (socket says 172.21.15.164:53624) 2024-05-08T04:41:24.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 -- 172.21.15.164:0/1807790269 learned_addr learned my addr 172.21.15.164:0/1807790269 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f4296000640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a00ae6e0 0x7f42a00cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429ea00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a006de50 0x7f42a014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 -- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a00ae6e0 msgr2=0x7f42a00cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a00ae6e0 0x7f42a00cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 -- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a006de50 msgr2=0x7f42a014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a006de50 0x7f42a014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 -- 172.21.15.164:0/1807790269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a014fd30 con 0x7f42a015ba20 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f429f400640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f429800c7a0 tx=0x7f429800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298018070 con 0x7f42a015ba20 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800a640 con 0x7f42a015ba20 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800aab0 con 0x7f42a015ba20 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a014ffd0 con 0x7f42a015ba20 2024-05-08T04:41:24.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.812+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a00cade0 con 0x7f42a015ba20 2024-05-08T04:41:24.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0072d30 con 0x7f42a015ba20 2024-05-08T04:41:24.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 10) v1 ==== 87135+0+0 (secure 0 0 0) 0x7f429800ac50 con 0x7f42a015ba20 2024-05-08T04:41:24.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f4297400640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 0x7f427c0698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:24.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4298084f40 con 0x7f42a015ba20 2024-05-08T04:41:24.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f429ea00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 0x7f427c0698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:24.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.816+0000 7f429ea00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 0x7f427c0698a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4290009a50 tx=0x7f4290008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:24.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.820+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4298055830 con 0x7f42a015ba20 2024-05-08T04:41:24.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.972+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42a0147d60 con 0x7f42a015ba20 2024-05-08T04:41:24.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f4297400640 1 -- 172.21.15.164:0/1807790269 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4298055130 con 0x7f42a015ba20 2024-05-08T04:41:24.982 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:24.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 msgr2=0x7f427c0698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 0x7f427c0698a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4290009a50 tx=0x7f4290008040 comp rx=0 tx=0).stop 2024-05-08T04:41:24.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 msgr2=0x7f42a014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f429800c7a0 tx=0x7f429800cc60 comp rx=0 tx=0).stop 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 shutdown_connections 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f427c0673e0 0x7f427c0698a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a00ae6e0 0x7f42a00cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42a015ba20 0x7f42a014f7f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 --2- 172.21.15.164:0/1807790269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a006de50 0x7f42a014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 >> 172.21.15.164:0/1807790269 conn(0x7f42a00cf770 msgr2=0x7f42a015d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 shutdown_connections 2024-05-08T04:41:24.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:24.976+0000 7f42a5a00640 1 -- 172.21.15.164:0/1807790269 wait complete. 2024-05-08T04:41:25.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:41:25.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.132+0000 7ff0a3200640 1 Processor -- start 2024-05-08T04:41:25.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 -- start start 2024-05-08T04:41:25.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 0x7ff09c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 0x7ff09c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c15e870 con 0x7ff09c15ab10 2024-05-08T04:41:25.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0cf130 con 0x7ff09c0cecc0 2024-05-08T04:41:25.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c06bb80 con 0x7ff09c06cce0 2024-05-08T04:41:25.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a1200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 0x7ff09c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff093400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 0x7ff09c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a1200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 0x7ff09c0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:37482/0 (socket says 172.21.15.164:37482) 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff0a1200640 1 -- 172.21.15.164:0/2901156065 learned_addr learned my addr 172.21.15.164:0/2901156065 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff093400640 1 -- 172.21.15.164:0/2901156065 >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 msgr2=0x7ff09c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff093400640 1 --2- 172.21.15.164:0/2901156065 >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 0x7ff09c0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff093400640 1 -- 172.21.15.164:0/2901156065 >> v1:172.21.15.172:6789/0 conn(0x7ff09c15ab10 legacy=0x7ff09c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.136+0000 7ff093400640 1 -- 172.21.15.164:0/2901156065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c15eb50 con 0x7ff09c06cce0 2024-05-08T04:41:25.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff093400640 1 --2- 172.21.15.164:0/2901156065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 0x7ff09c0ce780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4cfaaec688b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff09b400640 1 -- 172.21.15.164:0/2901156065 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b1070 con 0x7ff09c06cce0 2024-05-08T04:41:25.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff09b400640 1 -- 172.21.15.164:0/2901156065 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840a8050 con 0x7ff09c06cce0 2024-05-08T04:41:25.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff09b400640 1 -- 172.21.15.164:0/2901156065 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840ac860 con 0x7ff09c06cce0 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- 172.21.15.164:0/2901156065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 msgr2=0x7ff09c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 --2- 172.21.15.164:0/2901156065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 0x7ff09c0ce780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c3e0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- 172.21.15.164:0/2901156065 shutdown_connections 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 --2- 172.21.15.164:0/2901156065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c06cce0 0x7ff09c0ce780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 --2- 172.21.15.164:0/2901156065 >> v2:172.21.15.172:3300/0 conn(0x7ff09c0cecc0 0x7ff09c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- 172.21.15.164:0/2901156065 >> 172.21.15.164:0/2901156065 conn(0x7ff09c0cf770 msgr2=0x7ff09c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- 172.21.15.164:0/2901156065 shutdown_connections 2024-05-08T04:41:25.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- 172.21.15.164:0/2901156065 wait complete. 2024-05-08T04:41:25.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 Processor -- start 2024-05-08T04:41:25.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.140+0000 7ff0a3200640 1 -- start start 2024-05-08T04:41:25.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 0x7ff09c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 0x7ff09c1ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0aedf0 con 0x7ff09c06cce0 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c15aef0 con 0x7ff09c15ab10 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0aef70 con 0x7ff09c0cecc0 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53634/0 (socket says 172.21.15.164:53634) 2024-05-08T04:41:25.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 -- 172.21.15.164:0/1695377653 learned_addr learned my addr 172.21.15.164:0/1695377653 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a1200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 0x7ff09c1ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff093400640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 0x7ff09c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 -- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 msgr2=0x7ff09c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 0x7ff09c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 -- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 msgr2=0x7ff09c1ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 0x7ff09c1ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 -- 172.21.15.164:0/1695377653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c150a90 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a1200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 0x7ff09c1ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff093400640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 0x7ff09c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff09be00640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c730 tx=0x7ff08c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c016020 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c006d60 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003bc0 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c14f9f0 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c14ffb0 con 0x7ff09c06cce0 2024-05-08T04:41:25.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.144+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c072d30 con 0x7ff09c06cce0 2024-05-08T04:41:25.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.148+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 10) v1 ==== 87135+0+0 (secure 0 0 0) 0x7ff08c00a850 con 0x7ff09c06cce0 2024-05-08T04:41:25.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.148+0000 7ff098c00640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 0x7ff0780698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.148+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff08c085500 con 0x7ff09c06cce0 2024-05-08T04:41:25.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.148+0000 7ff093400640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 0x7ff0780698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.148+0000 7ff093400640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 0x7ff0780698a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff0840ae000 tx=0x7ff084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.152+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff08c020020 con 0x7ff09c06cce0 2024-05-08T04:41:25.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff09c0c4bf0 con 0x7ff09c06cce0 2024-05-08T04:41:25.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff098c00640 1 -- 172.21.15.164:0/1695377653 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff08c055e80 con 0x7ff09c06cce0 2024-05-08T04:41:25.313 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 msgr2=0x7ff0780698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 0x7ff0780698a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff0840ae000 tx=0x7ff084008040 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 msgr2=0x7ff09c14f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c730 tx=0x7ff08c00cbf0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 shutdown_connections 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0780673e0 0x7ff0780698a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09c0cecc0 0x7ff09c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff09c15ab10 0x7ff09c1ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 --2- 172.21.15.164:0/1695377653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c06cce0 0x7ff09c14f1a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 >> 172.21.15.164:0/1695377653 conn(0x7ff09c0cf770 msgr2=0x7ff09c0ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 shutdown_connections 2024-05-08T04:41:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.308+0000 7ff0a3200640 1 -- 172.21.15.164:0/1695377653 wait complete. 2024-05-08T04:41:25.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:41:25.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 Processor -- start 2024-05-08T04:41:25.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 -- start start 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 0x7f73ec0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73ec0cf760 0x7f73ec0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec0cfc60 con 0x7f73ec160300 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ec0d2120 con 0x7f73ec0cf760 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15f820 con 0x7f73ec15b9f0 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f0e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f73ec160300 0x7f73ec06fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53590/0 (socket says 172.21.15.164:53590) 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73f0e00640 1 -- 172.21.15.164:0/239083100 learned_addr learned my addr 172.21.15.164:0/239083100 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73ebe00640 1 --2- 172.21.15.164:0/239083100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 0x7f73ec0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73eb400640 1 --2- 172.21.15.164:0/239083100 >> v2:172.21.15.172:3300/0 conn(0x7f73ec0cf760 0x7f73ec0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174302389 0 0) 0x7f73ec0cfc60 con 0x7f73ec160300 2024-05-08T04:41:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d0003530 con 0x7f73ec160300 2024-05-08T04:41:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73ebe00640 1 -- 172.21.15.164:0/239083100 >> v2:172.21.15.172:3300/0 conn(0x7f73ec0cf760 msgr2=0x7f73ec0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73ebe00640 1 --2- 172.21.15.164:0/239083100 >> v2:172.21.15.172:3300/0 conn(0x7f73ec0cf760 0x7f73ec0d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73ebe00640 1 -- 172.21.15.164:0/239083100 >> v1:172.21.15.164:6789/0 conn(0x7f73ec160300 legacy=0x7f73ec06fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362617346 0 0) 0x7f73d0003530 con 0x7f73ec160300 2024-05-08T04:41:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.468+0000 7f73ebe00640 1 -- 172.21.15.164:0/239083100 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec15fb00 con 0x7f73ec15b9f0 2024-05-08T04:41:25.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73ebe00640 1 --2- 172.21.15.164:0/239083100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 0x7f73ec0d16a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0097c0 tx=0x7f73dc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59cdeef556d0162e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b0070 con 0x7f73ec15b9f0 2024-05-08T04:41:25.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0ab400 con 0x7f73ec15b9f0 2024-05-08T04:41:25.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73eaa00640 1 -- 172.21.15.164:0/239083100 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4001a00 con 0x7f73ec15b9f0 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 -- 172.21.15.164:0/239083100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 msgr2=0x7f73ec0d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 --2- 172.21.15.164:0/239083100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 0x7f73ec0d16a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0097c0 tx=0x7f73dc01c3c0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 -- 172.21.15.164:0/239083100 shutdown_connections 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 --2- 172.21.15.164:0/239083100 >> v2:172.21.15.172:3300/0 conn(0x7f73ec0cf760 0x7f73ec0d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 --2- 172.21.15.164:0/239083100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec15b9f0 0x7f73ec0d16a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 -- 172.21.15.164:0/239083100 >> 172.21.15.164:0/239083100 conn(0x7f73ec0723e0 msgr2=0x7f73ec0727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 -- 172.21.15.164:0/239083100 shutdown_connections 2024-05-08T04:41:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.472+0000 7f73f1800640 1 -- 172.21.15.164:0/239083100 wait complete. 2024-05-08T04:41:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 Processor -- start 2024-05-08T04:41:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- start start 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec0cf760 0x7f73ec06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec160300 0x7f73ec06ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec168b30 con 0x7f73ec15b9f0 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec1689b0 con 0x7f73ec160300 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec168cb0 con 0x7f73ec0cf760 2024-05-08T04:41:25.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53638/0 (socket says 172.21.15.164:53638) 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 -- 172.21.15.164:0/2734295190 learned_addr learned my addr 172.21.15.164:0/2734295190 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 -- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec0cf760 msgr2=0x7f73ec06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec0cf760 0x7f73ec06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 -- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec160300 msgr2=0x7f73ec06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec160300 0x7f73ec06ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 -- 172.21.15.164:0/2734295190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec06cc80 con 0x7f73ec15b9f0 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73eb400640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f73d800e870 tx=0x7f73d800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8017070 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800c940 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec06d890 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800ccf0 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec06bf40 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.476+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec06ddb0 con 0x7f73ec15b9f0 2024-05-08T04:41:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.480+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 10) v1 ==== 87135+0+0 (secure 0 0 0) 0x7f73d801a070 con 0x7f73ec15b9f0 2024-05-08T04:41:25.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.480+0000 7f73e3e00640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 0x7f73cc0698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.480+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f73d8085710 con 0x7f73ec15b9f0 2024-05-08T04:41:25.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.480+0000 7f73ebe00640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 0x7f73cc0698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.480+0000 7f73ebe00640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 0x7f73cc0698d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ad000 tx=0x7f73dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.484+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73d8056060 con 0x7f73ec15b9f0 2024-05-08T04:41:25.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73ec04ef30 con 0x7f73ec15b9f0 2024-05-08T04:41:25.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73e3e00640 1 -- 172.21.15.164:0/2734295190 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73d8055960 con 0x7f73ec15b9f0 2024-05-08T04:41:25.641 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 msgr2=0x7f73cc0698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 0x7f73cc0698d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ad000 tx=0x7f73dc008040 comp rx=0 tx=0).stop 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 msgr2=0x7f73ec14f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f73d800e870 tx=0x7f73d800ec70 comp rx=0 tx=0).stop 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 shutdown_connections 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f73cc067410 0x7f73cc0698d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec160300 0x7f73ec06ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73ec15b9f0 0x7f73ec14f770 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 --2- 172.21.15.164:0/2734295190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73ec0cf760 0x7f73ec06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 >> 172.21.15.164:0/2734295190 conn(0x7f73ec0723e0 msgr2=0x7f73ec15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 shutdown_connections 2024-05-08T04:41:25.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.636+0000 7f73f1800640 1 -- 172.21.15.164:0/2734295190 wait complete. 2024-05-08T04:41:25.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:41:25.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 Processor -- start 2024-05-08T04:41:25.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 -- start start 2024-05-08T04:41:25.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f96240b26c0 0x7f96240cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f96240cfbd0 0x7f96240cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962406c770 con 0x7f962415ab60 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96240d0060 con 0x7f96240cfbd0 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.792+0000 7f9629200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962406ca50 con 0x7f96240b26c0 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f96240cfbd0 0x7f96240cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f96240cfbd0 0x7f96240cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53644/0 (socket says 172.21.15.164:53644) 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 -- 172.21.15.164:0/4118288706 learned_addr learned my addr 172.21.15.164:0/4118288706 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9619400640 1 --2- 172.21.15.164:0/4118288706 >> v2:172.21.15.172:3300/0 conn(0x7f96240b26c0 0x7f96240cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 -- 172.21.15.164:0/4118288706 >> v2:172.21.15.172:3300/0 conn(0x7f96240b26c0 msgr2=0x7f96240cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:25.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 --2- 172.21.15.164:0/4118288706 >> v2:172.21.15.172:3300/0 conn(0x7f96240b26c0 0x7f96240cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 -- 172.21.15.164:0/4118288706 >> v1:172.21.15.172:6789/0 conn(0x7f962415ab60 legacy=0x7f9624072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 -- 172.21.15.164:0/4118288706 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962406cd30 con 0x7f96240cfbd0 2024-05-08T04:41:25.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9622800640 1 --2- 172.21.15.164:0/4118288706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240cfbd0 0x7f96240cd910 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f962406e880 tx=0x7f961c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff986c1ce165efb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9621400640 1 -- 172.21.15.164:0/4118288706 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c0a56c0 con 0x7f96240cfbd0 2024-05-08T04:41:25.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9621400640 1 -- 172.21.15.164:0/4118288706 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c0a5860 con 0x7f96240cfbd0 2024-05-08T04:41:25.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.796+0000 7f9621400640 1 -- 172.21.15.164:0/4118288706 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c0a5c30 con 0x7f96240cfbd0 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- 172.21.15.164:0/4118288706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240cfbd0 msgr2=0x7f96240cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- 172.21.15.164:0/4118288706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240cfbd0 0x7f96240cd910 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f962406e880 tx=0x7f961c01c9f0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- 172.21.15.164:0/4118288706 shutdown_connections 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- 172.21.15.164:0/4118288706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240cfbd0 0x7f96240cd910 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- 172.21.15.164:0/4118288706 >> v2:172.21.15.172:3300/0 conn(0x7f96240b26c0 0x7f96240cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- 172.21.15.164:0/4118288706 >> 172.21.15.164:0/4118288706 conn(0x7f962406ac90 msgr2=0x7f962406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- 172.21.15.164:0/4118288706 shutdown_connections 2024-05-08T04:41:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- 172.21.15.164:0/4118288706 wait complete. 2024-05-08T04:41:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 Processor -- start 2024-05-08T04:41:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- start start 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 0x7f9624157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f962415ab60 0x7f962414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240aeeb0 con 0x7f96240b26c0 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240aed30 con 0x7f96240cfbd0 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9629200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9624167970 con 0x7f962415ab60 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53656/0 (socket says 172.21.15.164:53656) 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 -- 172.21.15.164:0/692521615 learned_addr learned my addr 172.21.15.164:0/692521615 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9619400640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 0x7f9624157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9622800640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f962415ab60 0x7f962414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 -- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f962415ab60 msgr2=0x7f962414fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f962415ab60 0x7f962414fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 -- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 msgr2=0x7f9624157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 0x7f9624157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9621e00640 1 -- 172.21.15.164:0/692521615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9624154650 con 0x7f96240b26c0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.800+0000 7f9619400640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 0x7f9624157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f9621e00640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f961400c370 tx=0x7f961400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140052d0 con 0x7f96240b26c0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9614005470 con 0x7f96240b26c0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624150ad0 con 0x7f96240b26c0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614007260 con 0x7f96240b26c0 2024-05-08T04:41:25.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962415b0f0 con 0x7f96240b26c0 2024-05-08T04:41:25.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9624069d60 con 0x7f96240b26c0 2024-05-08T04:41:25.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 10) v1 ==== 87135+0+0 (secure 0 0 0) 0x7f9614007400 con 0x7f96240b26c0 2024-05-08T04:41:25.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 0x7f96000698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:25.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9614084d40 con 0x7f96240b26c0 2024-05-08T04:41:25.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.804+0000 7f9619400640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 0x7f96000698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:25.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.808+0000 7f9619400640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 0x7f96000698a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9624165930 tx=0x7f960c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:25.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.808+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f961408b080 con 0x7f96240b26c0 2024-05-08T04:41:25.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.972+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f962406d9a0 con 0x7f96240b26c0 2024-05-08T04:41:25.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.972+0000 7f961aa00640 1 -- 172.21.15.164:0/692521615 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9614055630 con 0x7f96240b26c0 2024-05-08T04:41:25.979 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:41:25.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 msgr2=0x7f96000698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 0x7f96000698a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9624165930 tx=0x7f960c008040 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 msgr2=0x7f96241503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f961400c370 tx=0x7f961400c830 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 shutdown_connections 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f96000673e0 0x7f96000698a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f962415ab60 0x7f962414fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96240cfbd0 0x7f9624157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 --2- 172.21.15.164:0/692521615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96240b26c0 0x7f96241503d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 >> 172.21.15.164:0/692521615 conn(0x7f962406ac90 msgr2=0x7f962415eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 shutdown_connections 2024-05-08T04:41:25.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:25.976+0000 7f9629200640 1 -- 172.21.15.164:0/692521615 wait complete. 2024-05-08T04:41:26.000 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:41:26.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:26.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:26.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:41:26.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:26.027 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:41:26.075 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:26.075 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:26.085 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:26.085 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:26.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:26.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 Processor -- start 2024-05-08T04:41:26.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 -- start start 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde6415ba70 0x7fde64160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 0x7fde640b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde640cd9f0 con 0x7fde6415be50 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6415fe70 con 0x7fde6415ba70 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6c800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640cdcd0 con 0x7fde640cfda0 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde6415ba70 0x7fde64160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde6415ba70 0x7fde64160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53660/0 (socket says 172.21.15.164:53660) 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 -- 172.21.15.164:0/2542898908 learned_addr learned my addr 172.21.15.164:0/2542898908 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:26.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6a800640 1 --2- 172.21.15.164:0/2542898908 >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 0x7fde640b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 -- 172.21.15.164:0/2542898908 >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 msgr2=0x7fde640b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:26.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 --2- 172.21.15.164:0/2542898908 >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 0x7fde640b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 -- 172.21.15.164:0/2542898908 >> v1:172.21.15.172:6789/0 conn(0x7fde6415be50 legacy=0x7fde640cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:26.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde69e00640 1 -- 172.21.15.164:0/2542898908 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6415f820 con 0x7fde6415ba70 2024-05-08T04:41:26.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.268+0000 7fde6a800640 1 --2- 172.21.15.164:0/2542898908 >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 0x7fde640b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:26.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde69e00640 1 --2- 172.21.15.164:0/2542898908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415ba70 0x7fde64160300 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fde6415fde0 tx=0x7fde5801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23ca613ce87a5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde68a00640 1 -- 172.21.15.164:0/2542898908 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580b1070 con 0x7fde6415ba70 2024-05-08T04:41:26.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde68a00640 1 -- 172.21.15.164:0/2542898908 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde580ac620 con 0x7fde6415ba70 2024-05-08T04:41:26.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde68a00640 1 -- 172.21.15.164:0/2542898908 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580ac9f0 con 0x7fde6415ba70 2024-05-08T04:41:26.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 -- 172.21.15.164:0/2542898908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415ba70 msgr2=0x7fde64160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 --2- 172.21.15.164:0/2542898908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415ba70 0x7fde64160300 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fde6415fde0 tx=0x7fde5801c4b0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 -- 172.21.15.164:0/2542898908 shutdown_connections 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 --2- 172.21.15.164:0/2542898908 >> v2:172.21.15.172:3300/0 conn(0x7fde640cfda0 0x7fde640b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 --2- 172.21.15.164:0/2542898908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415ba70 0x7fde64160300 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 -- 172.21.15.164:0/2542898908 >> 172.21.15.164:0/2542898908 conn(0x7fde6406ac90 msgr2=0x7fde6406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 -- 172.21.15.164:0/2542898908 shutdown_connections 2024-05-08T04:41:26.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.272+0000 7fde6c800640 1 -- 172.21.15.164:0/2542898908 wait complete. 2024-05-08T04:41:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 Processor -- start 2024-05-08T04:41:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- start start 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde640cfda0 0x7fde6414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde6415ba70 0x7fde6415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64168d40 con 0x7fde6415be50 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64168bc0 con 0x7fde6415ba70 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde64168ec0 con 0x7fde640cfda0 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53662/0 (socket says 172.21.15.164:53662) 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 -- 172.21.15.164:0/2957255954 learned_addr learned my addr 172.21.15.164:0/2957255954 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde69400640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde6415ba70 0x7fde6415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde69e00640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde640cfda0 0x7fde6414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 -- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde640cfda0 msgr2=0x7fde6414e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde640cfda0 0x7fde6414e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 -- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde6415ba70 msgr2=0x7fde6415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde6415ba70 0x7fde6415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 -- 172.21.15.164:0/2957255954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64153300 con 0x7fde6415be50 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6a800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fde6000cef0 tx=0x7fde6000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6001a070 con 0x7fde6415be50 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000d860 con 0x7fde6415be50 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000dc50 con 0x7fde6415be50 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde640b0470 con 0x7fde6415be50 2024-05-08T04:41:26.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.276+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde640b0a30 con 0x7fde6415be50 2024-05-08T04:41:26.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde64069d60 con 0x7fde6415be50 2024-05-08T04:41:26.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fde6001d070 con 0x7fde6415be50 2024-05-08T04:41:26.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde5e000640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 0x7fde480699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fde60086270 con 0x7fde6415be50 2024-05-08T04:41:26.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde69e00640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 0x7fde480699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.280+0000 7fde69e00640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 0x7fde480699c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fde580ae000 tx=0x7fde580a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.284+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde600564b0 con 0x7fde6415be50 2024-05-08T04:41:26.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.440+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde6415ca90 con 0x7fde6415be50 2024-05-08T04:41:26.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.440+0000 7fde5e000640 1 -- 172.21.15.164:0/2957255954 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16771 (secure 0 0 0) 0x7fde6005a550 con 0x7fde6415be50 2024-05-08T04:41:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:26.447 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":74,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:09.101885+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:26.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 msgr2=0x7fde480699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 0x7fde480699c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fde580ae000 tx=0x7fde580a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:26.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 msgr2=0x7fde641486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fde6000cef0 tx=0x7fde6000a710 comp rx=0 tx=0).stop 2024-05-08T04:41:26.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 shutdown_connections 2024-05-08T04:41:26.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fde48067500 0x7fde480699c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde6415be50 0x7fde641486b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde6415ba70 0x7fde6415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 --2- 172.21.15.164:0/2957255954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde640cfda0 0x7fde6414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 >> 172.21.15.164:0/2957255954 conn(0x7fde6406ac90 msgr2=0x7fde6406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:26.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 shutdown_connections 2024-05-08T04:41:26.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.444+0000 7fde6c800640 1 -- 172.21.15.164:0/2957255954 wait complete. 2024-05-08T04:41:26.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:05.195201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:06.483763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:26.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:26.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 Processor -- start 2024-05-08T04:41:26.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 -- start start 2024-05-08T04:41:26.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb53c15ab10 0x7fb53c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 0x7fb53c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53c15e870 con 0x7fb53c0ae6e0 2024-05-08T04:41:26.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c06bb80 con 0x7fb53c15ab10 2024-05-08T04:41:26.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb544000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c15eb50 con 0x7fb53c0cee90 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb542000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 0x7fb53c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb541600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb53c15ab10 0x7fb53c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb540c00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb53c0ae6e0 0x7fb53c0ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:54564/0 (socket says 172.21.15.164:54564) 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb540c00640 1 -- 172.21.15.164:0/4111764805 learned_addr learned my addr 172.21.15.164:0/4111764805 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb542000640 1 -- 172.21.15.164:0/4111764805 >> v2:172.21.15.172:3300/0 conn(0x7fb53c15ab10 msgr2=0x7fb53c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:26.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb542000640 1 --2- 172.21.15.164:0/4111764805 >> v2:172.21.15.172:3300/0 conn(0x7fb53c15ab10 0x7fb53c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb53be00640 1 -- 172.21.15.164:0/4111764805 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687088689 0 0) 0x7fb53c15e870 con 0x7fb53c0ae6e0 2024-05-08T04:41:26.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb542000640 1 -- 172.21.15.164:0/4111764805 >> v1:172.21.15.172:6789/0 conn(0x7fb53c0ae6e0 legacy=0x7fb53c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.604+0000 7fb542000640 1 -- 172.21.15.164:0/4111764805 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c15ee30 con 0x7fb53c0cee90 2024-05-08T04:41:26.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb542000640 1 --2- 172.21.15.164:0/4111764805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 0x7fb53c0aeb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5340097c0 tx=0x7fb53401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774474200137b3d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb53be00640 1 -- 172.21.15.164:0/4111764805 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340b1070 con 0x7fb53c0cee90 2024-05-08T04:41:26.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb53be00640 1 -- 172.21.15.164:0/4111764805 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340ac600 con 0x7fb53c0cee90 2024-05-08T04:41:26.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb53be00640 1 -- 172.21.15.164:0/4111764805 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340ac9d0 con 0x7fb53c0cee90 2024-05-08T04:41:26.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 -- 172.21.15.164:0/4111764805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 msgr2=0x7fb53c0aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 --2- 172.21.15.164:0/4111764805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 0x7fb53c0aeb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5340097c0 tx=0x7fb53401c3e0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 -- 172.21.15.164:0/4111764805 shutdown_connections 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 --2- 172.21.15.164:0/4111764805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0cee90 0x7fb53c0aeb70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 --2- 172.21.15.164:0/4111764805 >> v2:172.21.15.172:3300/0 conn(0x7fb53c15ab10 0x7fb53c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 -- 172.21.15.164:0/4111764805 >> 172.21.15.164:0/4111764805 conn(0x7fb53c0cf770 msgr2=0x7fb53c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 -- 172.21.15.164:0/4111764805 shutdown_connections 2024-05-08T04:41:26.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.608+0000 7fb544000640 1 -- 172.21.15.164:0/4111764805 wait complete. 2024-05-08T04:41:26.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 Processor -- start 2024-05-08T04:41:26.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- start start 2024-05-08T04:41:26.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0ae6e0 0x7fb53c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb53c15ab10 0x7fb53c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0cb070 con 0x7fb53c0cee90 2024-05-08T04:41:26.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0caef0 con 0x7fb53c15ab10 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0cb1f0 con 0x7fb53c0ae6e0 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53668/0 (socket says 172.21.15.164:53668) 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 -- 172.21.15.164:0/4258429397 learned_addr learned my addr 172.21.15.164:0/4258429397 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb542000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb53c15ab10 0x7fb53c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb541600640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0ae6e0 0x7fb53c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 -- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0ae6e0 msgr2=0x7fb53c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0ae6e0 0x7fb53c06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 -- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb53c15ab10 msgr2=0x7fb53c14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb53c15ab10 0x7fb53c14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 -- 172.21.15.164:0/4258429397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c159000 con 0x7fb53c0cee90 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb540c00640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb5280011f0 tx=0x7fb52800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528018070 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5280074e0 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528004a10 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1592e0 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1598a0 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.612+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c072d30 con 0x7fb53c0cee90 2024-05-08T04:41:26.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.616+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fb528004bb0 con 0x7fb53c0cee90 2024-05-08T04:41:26.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.616+0000 7fb539600640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 0x7fb51c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.616+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb528084e90 con 0x7fb53c0cee90 2024-05-08T04:41:26.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.616+0000 7fb541600640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 0x7fb51c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.616+0000 7fb541600640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 0x7fb51c0699c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5300073f0 tx=0x7fb53000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.620+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb528055610 con 0x7fb53c0cee90 2024-05-08T04:41:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.776+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb53c0c4bf0 con 0x7fb53c0cee90 2024-05-08T04:41:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.776+0000 7fb539600640 1 -- 172.21.15.164:0/4258429397 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb528054f10 con 0x7fb53c0cee90 2024-05-08T04:41:26.783 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 msgr2=0x7fb51c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 0x7fb51c0699c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5300073f0 tx=0x7fb53000b040 comp rx=0 tx=0).stop 2024-05-08T04:41:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 msgr2=0x7fb53c0adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb5280011f0 tx=0x7fb52800c9b0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 shutdown_connections 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb51c067500 0x7fb51c0699c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb53c15ab10 0x7fb53c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb53c0cee90 0x7fb53c0adaf0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 --2- 172.21.15.164:0/4258429397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb53c0ae6e0 0x7fb53c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 >> 172.21.15.164:0/4258429397 conn(0x7fb53c0cf770 msgr2=0x7fb53c06e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 shutdown_connections 2024-05-08T04:41:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.780+0000 7fb544000640 1 -- 172.21.15.164:0/4258429397 wait complete. 2024-05-08T04:41:26.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:41:26.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 Processor -- start 2024-05-08T04:41:26.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 -- start start 2024-05-08T04:41:26.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db815ab10 0x7f5db806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db815e870 con 0x7f5db80ae6e0 2024-05-08T04:41:26.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db815eb50 con 0x7f5db80cee90 2024-05-08T04:41:26.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5dbce00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db806bb80 con 0x7f5db815ab10 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.936+0000 7f5db5e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db815ab10 0x7f5db806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db815ab10 0x7f5db806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53682/0 (socket says 172.21.15.164:53682) 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 -- 172.21.15.164:0/2292941719 learned_addr learned my addr 172.21.15.164:0/2292941719 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5400640 1 --1- 172.21.15.164:0/2292941719 >> v1:172.21.15.164:6789/0 conn(0x7f5db80ae6e0 0x7f5db80ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53598/0 (socket says 172.21.15.164:53598) 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 -- 172.21.15.164:0/2292941719 >> v1:172.21.15.172:6789/0 conn(0x7f5db80cee90 legacy=0x7f5db80ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 -- 172.21.15.164:0/2292941719 >> v1:172.21.15.164:6789/0 conn(0x7f5db80ae6e0 legacy=0x7f5db80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 -- 172.21.15.164:0/2292941719 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db815ee30 con 0x7f5db815ab10 2024-05-08T04:41:26.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db5e00640 1 --2- 172.21.15.164:0/2292941719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db815ab10 0x7f5db806cce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009e50 tx=0x7f5da801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5837daa61509872 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db4a00640 1 -- 172.21.15.164:0/2292941719 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80a5830 con 0x7f5db815ab10 2024-05-08T04:41:26.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db4a00640 1 -- 172.21.15.164:0/2292941719 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da80a59d0 con 0x7f5db815ab10 2024-05-08T04:41:26.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.940+0000 7f5db4a00640 1 -- 172.21.15.164:0/2292941719 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80a5da0 con 0x7f5db815ab10 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- 172.21.15.164:0/2292941719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db815ab10 msgr2=0x7f5db806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2292941719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db815ab10 0x7f5db806cce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009e50 tx=0x7f5da801c970 comp rx=0 tx=0).stop 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- 172.21.15.164:0/2292941719 shutdown_connections 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2292941719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db815ab10 0x7f5db806cce0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- 172.21.15.164:0/2292941719 >> 172.21.15.164:0/2292941719 conn(0x7f5db80cf770 msgr2=0x7f5db80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- 172.21.15.164:0/2292941719 shutdown_connections 2024-05-08T04:41:26.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- 172.21.15.164:0/2292941719 wait complete. 2024-05-08T04:41:26.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 Processor -- start 2024-05-08T04:41:26.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- start start 2024-05-08T04:41:26.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 0x7f5db81f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db815ab10 0x7f5db815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80cd670 con 0x7f5db80cee90 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db815aef0 con 0x7f5db80ae6e0 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5dbce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80cd7f0 con 0x7f5db815ab10 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53690/0 (socket says 172.21.15.164:53690) 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 -- 172.21.15.164:0/2109123110 learned_addr learned my addr 172.21.15.164:0/2109123110 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db6800640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db815ab10 0x7f5db815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5e00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 0x7f5db81f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 -- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db815ab10 msgr2=0x7f5db815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db815ab10 0x7f5db815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 -- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 msgr2=0x7f5db81f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 0x7f5db81f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 -- 172.21.15.164:0/2109123110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80c9ff0 con 0x7f5db80cee90 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5e00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 0x7f5db81f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.944+0000 7f5db5400640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c490 tx=0x7f5da400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4017070 con 0x7f5db80cee90 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4006480 con 0x7f5db80cee90 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4004a10 con 0x7f5db80cee90 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db81f7de0 con 0x7f5db80cee90 2024-05-08T04:41:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db81f83a0 con 0x7f5db80cee90 2024-05-08T04:41:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db80c71d0 con 0x7f5db80cee90 2024-05-08T04:41:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f5da401a070 con 0x7f5db80cee90 2024-05-08T04:41:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 0x7f5d980699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5da4084c60 con 0x7f5db80cee90 2024-05-08T04:41:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5db5e00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 0x7f5d980699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:26.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.948+0000 7f5db5e00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 0x7f5d980699c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5da80ae000 tx=0x7f5da80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:26.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:26.952+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5da4054e20 con 0x7f5db80cee90 2024-05-08T04:41:27.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.108+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5db8072d30 con 0x7f5db80cee90 2024-05-08T04:41:27.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.108+0000 7f5dae000640 1 -- 172.21.15.164:0/2109123110 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5da4058ec0 con 0x7f5db80cee90 2024-05-08T04:41:27.115 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 msgr2=0x7f5d980699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 0x7f5d980699c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5da80ae000 tx=0x7f5da80a7070 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 msgr2=0x7f5db8153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c490 tx=0x7f5da400c950 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 shutdown_connections 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d98067500 0x7f5d980699c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db815ab10 0x7f5db815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db80cee90 0x7f5db8153f10 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 --2- 172.21.15.164:0/2109123110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db80ae6e0 0x7f5db81f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 >> 172.21.15.164:0/2109123110 conn(0x7f5db80cf770 msgr2=0x7f5db80cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 shutdown_connections 2024-05-08T04:41:27.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.112+0000 7f5dbce00640 1 -- 172.21.15.164:0/2109123110 wait complete. 2024-05-08T04:41:27.136 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:41:27.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 Processor -- start 2024-05-08T04:41:27.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 -- start start 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f46b40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46b40cecc0 0x7f46b40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b415e870 con 0x7f46b415ab10 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40cf130 con 0x7f46b40cecc0 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46bc800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b406bb80 con 0x7f46b406cce0 2024-05-08T04:41:27.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f46b415ab10 0x7f46b40ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53612/0 (socket says 172.21.15.164:53612) 2024-05-08T04:41:27.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 -- 172.21.15.164:0/3721485353 learned_addr learned my addr 172.21.15.164:0/3721485353 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:27.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46ba800640 1 --2- 172.21.15.164:0/3721485353 >> v2:172.21.15.172:3300/0 conn(0x7f46b40cecc0 0x7f46b40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096979306 0 0) 0x7f46b415e870 con 0x7f46b415ab10 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9400640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f46b40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f469c003530 con 0x7f46b415ab10 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242457876 0 0) 0x7f469c003530 con 0x7f46b415ab10 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 msgr2=0x7f46b40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f46b40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 >> v2:172.21.15.172:3300/0 conn(0x7f46b40cecc0 msgr2=0x7f46b40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 --2- 172.21.15.164:0/3721485353 >> v2:172.21.15.172:3300/0 conn(0x7f46b40cecc0 0x7f46b40ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b415eb50 con 0x7f46b415ab10 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f46a8001680 con 0x7f46b415ab10 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 >> v1:172.21.15.164:6789/0 conn(0x7f46b415ab10 legacy=0x7f46b40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f469c0039c0 0x7f469c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f469c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469c004d50 0x7f469c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40cf130 con 0x7f469c0039c0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b406bb80 con 0x7f469c004d50 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c00b9d0 con 0x7f46b406cce0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f469c0039c0 0x7f469c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9400640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f469c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46ba800640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469c004d50 0x7f469c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 -- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 msgr2=0x7f469c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46b406cce0 0x7f469c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 -- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469c004d50 msgr2=0x7f469c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469c004d50 0x7f469c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46b9e00640 1 -- 172.21.15.164:0/3721485353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c003530 con 0x7f469c0039c0 2024-05-08T04:41:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.272+0000 7f46ba800640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469c004d50 0x7f469c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:27.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46b9e00640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f469c0039c0 0x7f469c003de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c240 tx=0x7f46a801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3717a3d585ac5806 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8007d60 con 0x7f469c0039c0 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80a9020 con 0x7f469c0039c0 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46b8a00640 1 -- 172.21.15.164:0/3721485353 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8005ae0 con 0x7f469c0039c0 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 -- 172.21.15.164:0/3721485353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f469c0039c0 msgr2=0x7f469c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 --2- 172.21.15.164:0/3721485353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f469c0039c0 0x7f469c003de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c240 tx=0x7f46a801cc90 comp rx=0 tx=0).stop 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46ba800640 1 -- 172.21.15.164:0/3721485353 reap_dead start 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 -- 172.21.15.164:0/3721485353 shutdown_connections 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 -- 172.21.15.164:0/3721485353 >> 172.21.15.164:0/3721485353 conn(0x7f46b40cf770 msgr2=0x7f46b40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 -- 172.21.15.164:0/3721485353 shutdown_connections 2024-05-08T04:41:27.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.276+0000 7f46bc800640 1 -- 172.21.15.164:0/3721485353 wait complete. 2024-05-08T04:41:27.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 Processor -- start 2024-05-08T04:41:27.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- start start 2024-05-08T04:41:27.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f469c0039c0 0x7f46b40b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46b40b4090 0x7f46b40b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40aedf0 con 0x7f46b40b29c0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415aef0 con 0x7f46b40b4090 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40aef70 con 0x7f469c0039c0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53718/0 (socket says 172.21.15.164:53718) 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 -- 172.21.15.164:0/1950827208 learned_addr learned my addr 172.21.15.164:0/1950827208 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46b9e00640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f469c0039c0 0x7f46b40b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46b9400640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46b40b4090 0x7f46b40b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 -- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f469c0039c0 msgr2=0x7f46b40b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f469c0039c0 0x7f46b40b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 -- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46b40b4090 msgr2=0x7f46b40b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46b40b4090 0x7f46b40b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 -- 172.21.15.164:0/1950827208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40b2ff0 con 0x7f46b40b29c0 2024-05-08T04:41:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ba800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f46b40cf6d0 tx=0x7f46b0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0007ea0 con 0x7f46b40b29c0 2024-05-08T04:41:27.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0010d80 con 0x7f46b40b29c0 2024-05-08T04:41:27.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b40b32d0 con 0x7f46b40b29c0 2024-05-08T04:41:27.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0003be0 con 0x7f46b40b29c0 2024-05-08T04:41:27.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.280+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b40b3890 con 0x7f46b40b29c0 2024-05-08T04:41:27.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4072d30 con 0x7f46b40b29c0 2024-05-08T04:41:27.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f46b001e020 con 0x7f46b40b29c0 2024-05-08T04:41:27.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46ae000640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 0x7f46980699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f46b0086000 con 0x7f46b40b29c0 2024-05-08T04:41:27.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46b9e00640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 0x7f46980699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.284+0000 7f46b9e00640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 0x7f46980699c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f46a80aa000 tx=0x7f46a8001ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.288+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46b0025020 con 0x7f46b40b29c0 2024-05-08T04:41:27.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46b40c4bf0 con 0x7f46b40b29c0 2024-05-08T04:41:27.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46ae000640 1 -- 172.21.15.164:0/1950827208 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46b0056790 con 0x7f46b40b29c0 2024-05-08T04:41:27.448 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 msgr2=0x7f46980699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 0x7f46980699c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f46a80aa000 tx=0x7f46a8001ac0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 msgr2=0x7f46b4160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f46b40cf6d0 tx=0x7f46b0010590 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 shutdown_connections 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4698067500 0x7f46980699c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46b40b29c0 0x7f46b4160a20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46b40b4090 0x7f46b40b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 --2- 172.21.15.164:0/1950827208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f469c0039c0 0x7f46b40b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 >> 172.21.15.164:0/1950827208 conn(0x7f46b40cf770 msgr2=0x7f46b406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 shutdown_connections 2024-05-08T04:41:27.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.444+0000 7f46bc800640 1 -- 172.21.15.164:0/1950827208 wait complete. 2024-05-08T04:41:27.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:41:27.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 Processor -- start 2024-05-08T04:41:27.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 -- start start 2024-05-08T04:41:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015ba70 0x7f9350160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f935015be50 0x7f93500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93500aeda0 0x7f93500cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935015fe70 con 0x7f935015be50 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500cd9f0 con 0x7f93500aeda0 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9354e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cdcd0 con 0x7f935015ba70 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f935015be50 0x7f93500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f935015be50 0x7f93500cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53722/0 (socket says 172.21.15.164:53722) 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 -- 172.21.15.164:0/3223317727 learned_addr learned my addr 172.21.15.164:0/3223317727 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f934de00640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015ba70 0x7f9350160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f934e800640 1 --2- 172.21.15.164:0/3223317727 >> v2:172.21.15.172:3300/0 conn(0x7f93500aeda0 0x7f93500cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 -- 172.21.15.164:0/3223317727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015ba70 msgr2=0x7f9350160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015ba70 0x7f9350160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 -- 172.21.15.164:0/3223317727 >> v2:172.21.15.172:3300/0 conn(0x7f93500aeda0 msgr2=0x7f93500cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 --2- 172.21.15.164:0/3223317727 >> v2:172.21.15.172:3300/0 conn(0x7f93500aeda0 0x7f93500cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.604+0000 7f9345400640 1 -- 172.21.15.164:0/3223317727 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935015f820 con 0x7f935015be50 2024-05-08T04:41:27.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9345400640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935015be50 0x7f93500cf690 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f93500b4830 tx=0x7f933801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a50121a746a1dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f934d400640 1 -- 172.21.15.164:0/3223317727 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380b1070 con 0x7f935015be50 2024-05-08T04:41:27.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f934d400640 1 -- 172.21.15.164:0/3223317727 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93380ac620 con 0x7f935015be50 2024-05-08T04:41:27.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f934d400640 1 -- 172.21.15.164:0/3223317727 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380ac9f0 con 0x7f935015be50 2024-05-08T04:41:27.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 -- 172.21.15.164:0/3223317727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935015be50 msgr2=0x7f93500cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935015be50 0x7f93500cf690 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f93500b4830 tx=0x7f933801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 -- 172.21.15.164:0/3223317727 shutdown_connections 2024-05-08T04:41:27.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 --2- 172.21.15.164:0/3223317727 >> v2:172.21.15.172:3300/0 conn(0x7f93500aeda0 0x7f93500cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935015be50 0x7f93500cf690 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 --2- 172.21.15.164:0/3223317727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015ba70 0x7f9350160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 -- 172.21.15.164:0/3223317727 >> 172.21.15.164:0/3223317727 conn(0x7f935006ac90 msgr2=0x7f935006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 -- 172.21.15.164:0/3223317727 shutdown_connections 2024-05-08T04:41:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.608+0000 7f9354e00640 1 -- 172.21.15.164:0/3223317727 wait complete. 2024-05-08T04:41:27.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 Processor -- start 2024-05-08T04:41:27.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- start start 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 0x7f9350150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015be50 0x7f9350153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b3d40 con 0x7f93500aeda0 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b3bc0 con 0x7f935015ba70 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b3ec0 con 0x7f935015be50 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53726/0 (socket says 172.21.15.164:53726) 2024-05-08T04:41:27.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 -- 172.21.15.164:0/2867188339 learned_addr learned my addr 172.21.15.164:0/2867188339 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9345400640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 0x7f9350150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934e800640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015be50 0x7f9350153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 -- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015be50 msgr2=0x7f9350153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015be50 0x7f9350153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 -- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 msgr2=0x7f9350150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 0x7f9350150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 -- 172.21.15.164:0/2867188339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935014f590 con 0x7f93500aeda0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9345400640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 0x7f9350150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f934de00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f934000d830 tx=0x7f934000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000b6c0 con 0x7f93500aeda0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340019070 con 0x7f93500aeda0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000ba60 con 0x7f93500aeda0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935014f870 con 0x7f93500aeda0 2024-05-08T04:41:27.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.612+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935014fe30 con 0x7f93500aeda0 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9350069d60 con 0x7f93500aeda0 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f934001e460 con 0x7f93500aeda0 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9346a00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 0x7f932c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9340086100 con 0x7f93500aeda0 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9345400640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 0x7f932c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:27.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.616+0000 7f9345400640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 0x7f932c0699f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93380ae000 tx=0x7f9338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:27.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.620+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93400568d0 con 0x7f93500aeda0 2024-05-08T04:41:27.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f935006e210 con 0x7f93500aeda0 2024-05-08T04:41:27.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9346a00640 1 -- 172.21.15.164:0/2867188339 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f93400561d0 con 0x7f93500aeda0 2024-05-08T04:41:27.793 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 msgr2=0x7f932c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 0x7f932c0699f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93380ae000 tx=0x7f9338008040 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 msgr2=0x7f9350147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f934000d830 tx=0x7f934000dcf0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 shutdown_connections 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f932c067530 0x7f932c0699f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935015be50 0x7f9350153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935015ba70 0x7f9350150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 --2- 172.21.15.164:0/2867188339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93500aeda0 0x7f9350147ea0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 >> 172.21.15.164:0/2867188339 conn(0x7f935006ac90 msgr2=0x7f93501583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 shutdown_connections 2024-05-08T04:41:27.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:27.788+0000 7f9354e00640 1 -- 172.21.15.164:0/2867188339 wait complete. 2024-05-08T04:41:27.815 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:41:27.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:27.815 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:27.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:41:27.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:27.845 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:41:27.899 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:27.899 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:27.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:41:27.908 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:27.908 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:27.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:41:27.950 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 Processor -- start 2024-05-08T04:41:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 -- start start 2024-05-08T04:41:28.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5080168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508006cc70 con 0x7f508015dd70 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508006cf50 con 0x7f508006c7b0 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.080+0000 7f5086c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50800b1570 con 0x7f50800b2d80 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f508015dd70 0x7f50801619a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53614/0 (socket says 172.21.15.164:53614) 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 -- 172.21.15.164:0/2251768921 learned_addr learned my addr 172.21.15.164:0/2251768921 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649221492 0 0) 0x7f508006cc70 con 0x7f508015dd70 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5064003530 con 0x7f508015dd70 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f5084c00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5080168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528812092 0 0) 0x7f5064003530 con 0x7f508015dd70 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 msgr2=0x7f5080168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5080168cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 >> v1:172.21.15.172:6789/0 conn(0x7f508006c7b0 legacy=0x7f50801620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800b1850 con 0x7f508015dd70 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5070001680 con 0x7f508015dd70 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 >> v1:172.21.15.164:6789/0 conn(0x7f508015dd70 legacy=0x7f50801619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640039f0 0x7f5064003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5064004810 0x7f5064004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5064005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50800b1570 con 0x7f50640039f0 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506400bb90 con 0x7f5064004810 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506400bd90 con 0x7f50800b2d80 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640039f0 0x7f5064003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f5077400640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5064004810 0x7f5064004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f5084c00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5064005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 -- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 msgr2=0x7f5064005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5064005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 -- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5064004810 msgr2=0x7f5064004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5064004810 0x7f5064004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f507fe00640 1 -- 172.21.15.164:0/2251768921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064003530 con 0x7f50640039f0 2024-05-08T04:41:28.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.084+0000 7f5084c00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50800b2d80 0x7f5064005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f507fe00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640039f0 0x7f5064003e10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f507000c210 tx=0x7f507001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d0bea14bedb30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070007d40 con 0x7f50640039f0 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070005b00 con 0x7f50640039f0 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f507f400640 1 -- 172.21.15.164:0/2251768921 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070007070 con 0x7f50640039f0 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- 172.21.15.164:0/2251768921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640039f0 msgr2=0x7f5064003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 --2- 172.21.15.164:0/2251768921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640039f0 0x7f5064003e10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f507000c210 tx=0x7f507001f3f0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5084c00640 1 -- 172.21.15.164:0/2251768921 reap_dead start 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- 172.21.15.164:0/2251768921 shutdown_connections 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- 172.21.15.164:0/2251768921 >> 172.21.15.164:0/2251768921 conn(0x7f50800b34f0 msgr2=0x7f50800b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- 172.21.15.164:0/2251768921 shutdown_connections 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- 172.21.15.164:0/2251768921 wait complete. 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 Processor -- start 2024-05-08T04:41:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 -- start start 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.088+0000 7f5086c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640039f0 0x7f5080149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50800c7b10 0x7f50800c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508015e150 con 0x7f50800c5500 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50800b4230 con 0x7f50800c7b10 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508015e2d0 con 0x7f50640039f0 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53742/0 (socket says 172.21.15.164:53742) 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 -- 172.21.15.164:0/4265940382 learned_addr learned my addr 172.21.15.164:0/4265940382 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f507fe00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640039f0 0x7f5080149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5084c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50800c7b10 0x7f50800c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 -- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640039f0 msgr2=0x7f5080149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640039f0 0x7f5080149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 -- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50800c7b10 msgr2=0x7f50800c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50800c7b10 0x7f50800c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 -- 172.21.15.164:0/4265940382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800b5ca0 con 0x7f50800c5500 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5077400640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f506800f850 tx=0x7f506800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068001cf0 con 0x7f50800c5500 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068018070 con 0x7f50800c5500 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800d730 con 0x7f50800c5500 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50800b5f10 con 0x7f50800c5500 2024-05-08T04:41:28.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50800b64d0 con 0x7f50800c5500 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5080069d60 con 0x7f50800c5500 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.092+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f506800d8d0 con 0x7f50800c5500 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.096+0000 7f507cc00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 0x7f505c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.096+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5068085810 con 0x7f50800c5500 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.096+0000 7f507fe00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 0x7f505c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.096+0000 7f507fe00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 0x7f505c0699f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f50700af000 tx=0x7f50700aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.100+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5068055900 con 0x7f50800c5500 2024-05-08T04:41:28.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.288+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50800b4010 con 0x7f50800c5500 2024-05-08T04:41:28.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.288+0000 7f507cc00640 1 -- 172.21.15.164:0/4265940382 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f50680599a0 con 0x7f50800c5500 2024-05-08T04:41:28.295 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:28.295 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:28.295 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 44 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 msgr2=0x7f505c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 0x7f505c0699f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f50700af000 tx=0x7f50700aa000 comp rx=0 tx=0).stop 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 msgr2=0x7f508014a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f506800f850 tx=0x7f506800fd10 comp rx=0 tx=0).stop 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 shutdown_connections 2024-05-08T04:41:28.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f505c067530 0x7f505c0699f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50800c7b10 0x7f50800c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50800c5500 0x7f508014a830 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 --2- 172.21.15.164:0/4265940382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640039f0 0x7f5080149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 >> 172.21.15.164:0/4265940382 conn(0x7f50800b34f0 msgr2=0x7f50800b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 shutdown_connections 2024-05-08T04:41:28.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.292+0000 7f5086c00640 1 -- 172.21.15.164:0/4265940382 wait complete. 2024-05-08T04:41:28.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:28.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.448+0000 7f4441600640 1 Processor -- start 2024-05-08T04:41:28.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.448+0000 7f4441600640 1 -- start start 2024-05-08T04:41:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4441600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f443c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4441600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443c072640 con 0x7f443c0b26c0 2024-05-08T04:41:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4441600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443c06c770 con 0x7f443c0cfda0 2024-05-08T04:41:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4441600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06ca50 con 0x7f443c15ab60 2024-05-08T04:41:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f443c0b26c0 0x7f443c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:53628/0 (socket says 172.21.15.164:53628) 2024-05-08T04:41:28.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 -- 172.21.15.164:0/1840761151 learned_addr learned my addr 172.21.15.164:0/1840761151 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:28.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443aa00640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f443c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790885358 0 0) 0x7f443c072640 con 0x7f443c0b26c0 2024-05-08T04:41:28.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4420005740 con 0x7f443c0b26c0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127716332 0 0) 0x7f443c06c770 con 0x7f443c0cfda0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f443c072640 con 0x7f443c0cfda0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3736851217 0 0) 0x7f4420005740 con 0x7f443c0b26c0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 msgr2=0x7f443c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f443c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 >> v1:172.21.15.172:6789/0 conn(0x7f443c0cfda0 legacy=0x7f443c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c06cd30 con 0x7f443c0b26c0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f4424001680 con 0x7f443c0b26c0 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 >> v1:172.21.15.164:6789/0 conn(0x7f443c0b26c0 legacy=0x7f443c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4420005d90 0x7f4420006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4420006a20 0x7f4420006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f44200073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06ca50 con 0x7f4420006a20 2024-05-08T04:41:28.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442000dc90 con 0x7f4420005d90 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442000de60 con 0x7f443c15ab60 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4420006a20 0x7f4420006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443aa00640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4420005d90 0x7f4420006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f443b400640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f44200073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 -- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 msgr2=0x7f44200073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c15ab60 0x7f44200073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 -- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4420005d90 msgr2=0x7f4420006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4420005d90 0x7f4420006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.452+0000 7f4432000640 1 -- 172.21.15.164:0/1840761151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420005740 con 0x7f4420006a20 2024-05-08T04:41:28.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4432000640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4420006a20 0x7f4420006ea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f442400c210 tx=0x7f442401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c04a7e31b9a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424007cf0 con 0x7f4420006a20 2024-05-08T04:41:28.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44240a9020 con 0x7f4420006a20 2024-05-08T04:41:28.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f443a000640 1 -- 172.21.15.164:0/1840761151 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424005b00 con 0x7f4420006a20 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 -- 172.21.15.164:0/1840761151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4420006a20 msgr2=0x7f4420006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 --2- 172.21.15.164:0/1840761151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4420006a20 0x7f4420006ea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f442400c210 tx=0x7f442401cba0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f443b400640 1 -- 172.21.15.164:0/1840761151 reap_dead start 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 -- 172.21.15.164:0/1840761151 shutdown_connections 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 -- 172.21.15.164:0/1840761151 >> 172.21.15.164:0/1840761151 conn(0x7f443c06ac90 msgr2=0x7f443c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 -- 172.21.15.164:0/1840761151 shutdown_connections 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.456+0000 7f4441600640 1 -- 172.21.15.164:0/1840761151 wait complete. 2024-05-08T04:41:28.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 Processor -- start 2024-05-08T04:41:28.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- start start 2024-05-08T04:41:28.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c144190 0x7f443c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 0x7f443c15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06c3d0 con 0x7f443c147eb0 2024-05-08T04:41:28.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06d2f0 con 0x7f443c148330 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06c550 con 0x7f443c144190 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53768/0 (socket says 172.21.15.164:53768) 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 -- 172.21.15.164:0/3833274018 learned_addr learned my addr 172.21.15.164:0/3833274018 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f443aa00640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c144190 0x7f443c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f443b400640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 0x7f443c15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 -- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c144190 msgr2=0x7f443c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c144190 0x7f443c0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 -- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 msgr2=0x7f443c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 0x7f443c15e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 -- 172.21.15.164:0/3833274018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c144da0 con 0x7f443c147eb0 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f443b400640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 0x7f443c15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:28.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4432000640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f442400c210 tx=0x7f4420007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424007cf0 con 0x7f443c147eb0 2024-05-08T04:41:28.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44240b2070 con 0x7f443c147eb0 2024-05-08T04:41:28.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424001c00 con 0x7f443c147eb0 2024-05-08T04:41:28.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c145080 con 0x7f443c147eb0 2024-05-08T04:41:28.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c145640 con 0x7f443c147eb0 2024-05-08T04:41:28.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.460+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c069d60 con 0x7f443c147eb0 2024-05-08T04:41:28.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.464+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f44240a5cc0 con 0x7f443c147eb0 2024-05-08T04:41:28.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.464+0000 7f4433400640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 0x7f44180699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.464+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f442411efc0 con 0x7f443c147eb0 2024-05-08T04:41:28.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.464+0000 7f443aa00640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 0x7f44180699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.464+0000 7f443aa00640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 0x7f44180699f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f442c014430 tx=0x7f442c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.468+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44240168b0 con 0x7f443c147eb0 2024-05-08T04:41:28.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:41:28.488 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:41:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.632+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f443c06d9a0 con 0x7f443c147eb0 2024-05-08T04:41:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.632+0000 7f4433400640 1 -- 172.21.15.164:0/3833274018 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f44240ef0b0 con 0x7f443c147eb0 2024-05-08T04:41:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:28.638 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:28.209436+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 msgr2=0x7f44180699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 0x7f44180699f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f442c014430 tx=0x7f442c00b040 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 msgr2=0x7f443c15e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f442400c210 tx=0x7f4420007a50 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 shutdown_connections 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4418067530 0x7f44180699f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c148330 0x7f443c15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f443c147eb0 0x7f443c15e290 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 --2- 172.21.15.164:0/3833274018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c144190 0x7f443c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 >> 172.21.15.164:0/3833274018 conn(0x7f443c06ac90 msgr2=0x7f443c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 shutdown_connections 2024-05-08T04:41:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.636+0000 7f4441600640 1 -- 172.21.15.164:0/3833274018 wait complete. 2024-05-08T04:41:28.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:05.195201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:06.483763+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:28.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 Processor -- start 2024-05-08T04:41:28.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 -- start start 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 0x7f77cc0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc15e870 con 0x7f77cc0ae6e0 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc06bb80 con 0x7f77cc0cee90 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.840+0000 7f77d2800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc15eb50 con 0x7f77cc15ab10 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:44158/0 (socket says 172.21.15.164:44158) 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 -- 172.21.15.164:0/2936049798 learned_addr learned my addr 172.21.15.164:0/2936049798 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 0x7f77cc0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77c3e00640 1 --1- 172.21.15.164:0/2936049798 >> v1:172.21.15.172:6789/0 conn(0x7f77cc0ae6e0 0x7f77cc0ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:54596/0 (socket says 172.21.15.164:54596) 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 -- 172.21.15.164:0/2936049798 >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 msgr2=0x7f77cc0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 --2- 172.21.15.164:0/2936049798 >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 0x7f77cc0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 -- 172.21.15.164:0/2936049798 >> v1:172.21.15.172:6789/0 conn(0x7f77cc0ae6e0 legacy=0x7f77cc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 -- 172.21.15.164:0/2936049798 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc15ee30 con 0x7f77cc15ab10 2024-05-08T04:41:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77caa00640 1 -- 172.21.15.164:0/2936049798 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096702114 0 0) 0x7f77cc15e870 con 0x7f77cc0ae6e0 2024-05-08T04:41:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cbe00640 1 --2- 172.21.15.164:0/2936049798 >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 0x7f77cc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.844+0000 7f77cb400640 1 --2- 172.21.15.164:0/2936049798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b5d30bb18f31ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77caa00640 1 -- 172.21.15.164:0/2936049798 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0ab660 con 0x7f77cc15ab10 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77caa00640 1 -- 172.21.15.164:0/2936049798 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0b0070 con 0x7f77cc15ab10 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77caa00640 1 -- 172.21.15.164:0/2936049798 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0aba20 con 0x7f77cc15ab10 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- 172.21.15.164:0/2936049798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 msgr2=0x7f77cc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- 172.21.15.164:0/2936049798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- 172.21.15.164:0/2936049798 shutdown_connections 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- 172.21.15.164:0/2936049798 >> v2:172.21.15.172:3300/0 conn(0x7f77cc0cee90 0x7f77cc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- 172.21.15.164:0/2936049798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc15ab10 0x7f77cc06cce0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- 172.21.15.164:0/2936049798 >> 172.21.15.164:0/2936049798 conn(0x7f77cc0cf770 msgr2=0x7f77cc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- 172.21.15.164:0/2936049798 shutdown_connections 2024-05-08T04:41:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- 172.21.15.164:0/2936049798 wait complete. 2024-05-08T04:41:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 Processor -- start 2024-05-08T04:41:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- start start 2024-05-08T04:41:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc0ae6e0 0x7f77cc14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77cc0cee90 0x7f77cc14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0cb070 con 0x7f77cc15ab10 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0caef0 con 0x7f77cc0cee90 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77d2800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0cb1f0 con 0x7f77cc0ae6e0 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53780/0 (socket says 172.21.15.164:53780) 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77cbe00640 1 -- 172.21.15.164:0/187349874 learned_addr learned my addr 172.21.15.164:0/187349874 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77cb400640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc0ae6e0 0x7f77cc14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.848+0000 7f77c3e00640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77cc0cee90 0x7f77cc14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 -- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc0ae6e0 msgr2=0x7f77cc14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc0ae6e0 0x7f77cc14a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 -- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77cc0cee90 msgr2=0x7f77cc14baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77cc0cee90 0x7f77cc14baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 -- 172.21.15.164:0/187349874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc14aa00 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77cbe00640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009ea0 tx=0x7f77c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4018020 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4014480 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4014870 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc14ace0 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc14b2a0 con 0x7f77cc15ab10 2024-05-08T04:41:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc072d30 con 0x7f77cc15ab10 2024-05-08T04:41:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.852+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f77c401b020 con 0x7f77cc15ab10 2024-05-08T04:41:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.856+0000 7f77c3400640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 0x7f77a80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.856+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f77c4086920 con 0x7f77cc15ab10 2024-05-08T04:41:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.856+0000 7f77cb400640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 0x7f77a80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.856+0000 7f77cb400640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 0x7f77a80699c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0af000 tx=0x7f77bc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:28.860+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77c4022020 con 0x7f77cc15ab10 2024-05-08T04:41:29.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.008+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77cc04f010 con 0x7f77cc15ab10 2024-05-08T04:41:29.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77c3400640 1 -- 172.21.15.164:0/187349874 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77c4010040 con 0x7f77cc15ab10 2024-05-08T04:41:29.016 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:29.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 msgr2=0x7f77a80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 0x7f77a80699c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0af000 tx=0x7f77bc0a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 msgr2=0x7f77cc0b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009ea0 tx=0x7f77c4009f90 comp rx=0 tx=0).stop 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 shutdown_connections 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f77a8067500 0x7f77a80699c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77cc15ab10 0x7f77cc0b3660 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77cc0cee90 0x7f77cc14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 --2- 172.21.15.164:0/187349874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77cc0ae6e0 0x7f77cc14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 >> 172.21.15.164:0/187349874 conn(0x7f77cc0cf770 msgr2=0x7f77cc06df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 shutdown_connections 2024-05-08T04:41:29.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.012+0000 7f77d2800640 1 -- 172.21.15.164:0/187349874 wait complete. 2024-05-08T04:41:29.037 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:41:29.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 Processor -- start 2024-05-08T04:41:29.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 -- start start 2024-05-08T04:41:29.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 0x7f6fe80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe80cfe60 con 0x7f6fe806f520 2024-05-08T04:41:29.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe80d0790 con 0x7f6fe80d0c20 2024-05-08T04:41:29.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fef400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80d0140 con 0x7f6fe80cdb10 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fe7e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 0x7f6fe80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:59966/0 (socket says 172.21.15.164:59966) 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fed400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6fe806f520 0x7f6fe8161b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:35954/0 (socket says 172.21.15.164:35954) 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6fe7e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 0x7f6fe80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:34010/0 (socket says 172.21.15.164:34010) 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 -- 172.21.15.164:0/3685129498 learned_addr learned my addr 172.21.15.164:0/3685129498 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 -- 172.21.15.164:0/3685129498 >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 msgr2=0x7f6fe80ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 --2- 172.21.15.164:0/3685129498 >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 0x7f6fe80ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 -- 172.21.15.164:0/3685129498 >> v1:172.21.15.172:6789/0 conn(0x7f6fe806f520 legacy=0x7f6fe8161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.172+0000 7f6feca00640 1 -- 172.21.15.164:0/3685129498 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe80d0420 con 0x7f6fe80cdb10 2024-05-08T04:41:29.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6feca00640 1 --2- 172.21.15.164:0/3685129498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098a0 tx=0x7f6fdc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f53af3800db0208 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fe7400640 1 -- 172.21.15.164:0/3685129498 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0b1070 con 0x7f6fe80cdb10 2024-05-08T04:41:29.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fe7400640 1 -- 172.21.15.164:0/3685129498 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc0ac620 con 0x7f6fe80cdb10 2024-05-08T04:41:29.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fe7400640 1 -- 172.21.15.164:0/3685129498 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0ac9f0 con 0x7f6fe80cdb10 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 -- 172.21.15.164:0/3685129498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 msgr2=0x7f6fe806efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 --2- 172.21.15.164:0/3685129498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098a0 tx=0x7f6fdc01c4b0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 -- 172.21.15.164:0/3685129498 shutdown_connections 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 --2- 172.21.15.164:0/3685129498 >> v2:172.21.15.172:3300/0 conn(0x7f6fe80d0c20 0x7f6fe80ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 --2- 172.21.15.164:0/3685129498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe806efe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 -- 172.21.15.164:0/3685129498 >> 172.21.15.164:0/3685129498 conn(0x7f6fe80b34f0 msgr2=0x7f6fe80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 -- 172.21.15.164:0/3685129498 shutdown_connections 2024-05-08T04:41:29.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.176+0000 7f6fef400640 1 -- 172.21.15.164:0/3685129498 wait complete. 2024-05-08T04:41:29.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 Processor -- start 2024-05-08T04:41:29.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- start start 2024-05-08T04:41:29.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe806f520 0x7f6fe8155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe81560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe816b260 con 0x7f6fe80d0c20 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe816b0e0 con 0x7f6fe806f520 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe816b3e0 con 0x7f6fe80cdb10 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45494/0 (socket says 172.21.15.164:45494) 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 -- 172.21.15.164:0/3345210427 learned_addr learned my addr 172.21.15.164:0/3345210427 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6feca00640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe806f520 0x7f6fe8155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fe7e00640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe81560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 -- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 msgr2=0x7f6fe81560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe81560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 -- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe806f520 msgr2=0x7f6fe8155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe806f520 0x7f6fe8155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 -- 172.21.15.164:0/3345210427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8155ed0 con 0x7f6fe80d0c20 2024-05-08T04:41:29.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fed400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c6c0 tx=0x7f6fe000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0018070 con 0x7f6fe80d0c20 2024-05-08T04:41:29.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0004d70 con 0x7f6fe80d0c20 2024-05-08T04:41:29.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe80b67c0 con 0x7f6fe80d0c20 2024-05-08T04:41:29.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000a840 con 0x7f6fe80d0c20 2024-05-08T04:41:29.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe80b6d80 con 0x7f6fe80d0c20 2024-05-08T04:41:29.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.180+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe809fae0 con 0x7f6fe80d0c20 2024-05-08T04:41:29.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.184+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f6fe001b070 con 0x7f6fe80d0c20 2024-05-08T04:41:29.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.184+0000 7f6fe4c00640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 0x7f6fc80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.184+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f6fe0084f70 con 0x7f6fe80d0c20 2024-05-08T04:41:29.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.184+0000 7f6feca00640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 0x7f6fc80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.184+0000 7f6feca00640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 0x7f6fc80699c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0ae000 tx=0x7f6fdc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.188+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fe00550b0 con 0x7f6fe80d0c20 2024-05-08T04:41:29.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fe804f010 con 0x7f6fe80d0c20 2024-05-08T04:41:29.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fe4c00640 1 -- 172.21.15.164:0/3345210427 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fe0059150 con 0x7f6fe80d0c20 2024-05-08T04:41:29.349 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:41:29.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 msgr2=0x7f6fc80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 0x7f6fc80699c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0ae000 tx=0x7f6fdc0a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 msgr2=0x7f6fe8156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c6c0 tx=0x7f6fe000cb80 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 shutdown_connections 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fc8067500 0x7f6fc80699c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe80cdb10 0x7f6fe81560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fe80d0c20 0x7f6fe8156600 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 --2- 172.21.15.164:0/3345210427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe806f520 0x7f6fe8155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 >> 172.21.15.164:0/3345210427 conn(0x7f6fe80b34f0 msgr2=0x7f6fe80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 shutdown_connections 2024-05-08T04:41:29.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.344+0000 7f6fef400640 1 -- 172.21.15.164:0/3345210427 wait complete. 2024-05-08T04:41:29.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:41:29.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 Processor -- start 2024-05-08T04:41:29.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 -- start start 2024-05-08T04:41:29.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d3c06cce0 0x7f3d3c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 0x7f3d3c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3c15e870 con 0x7f3d3c15ab10 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c0cf130 con 0x7f3d3c06cce0 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d43800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c06bb80 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d3c06cce0 0x7f3d3c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d3c06cce0 0x7f3d3c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45502/0 (socket says 172.21.15.164:45502) 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 -- 172.21.15.164:0/442634727 learned_addr learned my addr 172.21.15.164:0/442634727 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d41800640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 0x7f3d3c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 -- 172.21.15.164:0/442634727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 msgr2=0x7f3d3c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 0x7f3d3c0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 -- 172.21.15.164:0/442634727 >> v1:172.21.15.172:6789/0 conn(0x7f3d3c15ab10 legacy=0x7f3d3c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d3be00640 1 -- 172.21.15.164:0/442634727 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c15eb50 con 0x7f3d3c06cce0 2024-05-08T04:41:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.504+0000 7f3d41800640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 0x7f3d3c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d3be00640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c0ce780 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009870 tx=0x7f3d2801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd58664f97fdd9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d3b400640 1 -- 172.21.15.164:0/442634727 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280aa600 con 0x7f3d3c06cce0 2024-05-08T04:41:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d3b400640 1 -- 172.21.15.164:0/442634727 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d280af070 con 0x7f3d3c06cce0 2024-05-08T04:41:29.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d3b400640 1 -- 172.21.15.164:0/442634727 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280aa9c0 con 0x7f3d3c06cce0 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 -- 172.21.15.164:0/442634727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c06cce0 msgr2=0x7f3d3c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c0ce780 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009870 tx=0x7f3d2801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 -- 172.21.15.164:0/442634727 shutdown_connections 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c0cecc0 0x7f3d3c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 --2- 172.21.15.164:0/442634727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c0ce780 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 -- 172.21.15.164:0/442634727 >> 172.21.15.164:0/442634727 conn(0x7f3d3c0cf770 msgr2=0x7f3d3c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 -- 172.21.15.164:0/442634727 shutdown_connections 2024-05-08T04:41:29.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.508+0000 7f3d43800640 1 -- 172.21.15.164:0/442634727 wait complete. 2024-05-08T04:41:29.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 Processor -- start 2024-05-08T04:41:29.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- start start 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c15ab10 0x7f3d3c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0aedf0 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c15aef0 con 0x7f3d3c06cce0 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0aef70 con 0x7f3d3c15ab10 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45514/0 (socket says 172.21.15.164:45514) 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 -- 172.21.15.164:0/1999459883 learned_addr learned my addr 172.21.15.164:0/1999459883 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d41800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c15ab10 0x7f3d3c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d40e00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 -- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c15ab10 msgr2=0x7f3d3c0b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c15ab10 0x7f3d3c0b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 -- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d3c06cce0 msgr2=0x7f3d3c1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c1525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 -- 172.21.15.164:0/1999459883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c0b4c30 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d3be00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3d2801a0d0 tx=0x7f3d2801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280bd5f0 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d280bd790 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c0b4f10 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280bdb80 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.512+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c146830 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3c0b5430 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f3d280a9070 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d38c00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 0x7f3d1c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3d28120280 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d40e00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 0x7f3d1c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.516+0000 7f3d40e00640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 0x7f3d1c0699c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c007fd0 tx=0x7f3d2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.520+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d3c0b5430 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.676+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d3c072d30 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.676+0000 7f3d38c00640 1 -- 172.21.15.164:0/1999459883 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d28016150 con 0x7f3d3c0cecc0 2024-05-08T04:41:29.683 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 msgr2=0x7f3d1c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 0x7f3d1c0699c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c007fd0 tx=0x7f3d2c009040 comp rx=0 tx=0).stop 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 msgr2=0x7f3d3c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f3d2801a0d0 tx=0x7f3d2801ace0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 shutdown_connections 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3d1c067500 0x7f3d1c0699c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d3c15ab10 0x7f3d3c0b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d3c0cecc0 0x7f3d3c0b45c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 --2- 172.21.15.164:0/1999459883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d3c06cce0 0x7f3d3c1525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 >> 172.21.15.164:0/1999459883 conn(0x7f3d3c0cf770 msgr2=0x7f3d3c1541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 shutdown_connections 2024-05-08T04:41:29.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.680+0000 7f3d43800640 1 -- 172.21.15.164:0/1999459883 wait complete. 2024-05-08T04:41:29.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:29.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.832+0000 7f4ff2200640 1 Processor -- start 2024-05-08T04:41:29.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.832+0000 7f4ff2200640 1 -- start start 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.832+0000 7f4ff2200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4fec0af480 0x7f4fec0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4ff2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec06c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4ff2200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec071980 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4ff2200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec161810 con 0x7f4fec0af480 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4ff2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec06c650 con 0x7f4fec06bca0 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4fec0af480 0x7f4fec0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4fec0af480 0x7f4fec0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45526/0 (socket says 172.21.15.164:45526) 2024-05-08T04:41:29.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 -- 172.21.15.164:0/2742811626 learned_addr learned my addr 172.21.15.164:0/2742811626 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4febe00640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 -- 172.21.15.164:0/2742811626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 msgr2=0x7f4fec06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec06c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 -- 172.21.15.164:0/2742811626 >> v1:172.21.15.172:6789/0 conn(0x7f4fec0cdfa0 legacy=0x7f4fec06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feb400640 1 -- 172.21.15.164:0/2742811626 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec071c00 con 0x7f4fec0af480 2024-05-08T04:41:29.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.836+0000 7f4feaa00640 1 -- 172.21.15.164:0/2742811626 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667793664 0 0) 0x7f4fd4001120 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4feb400640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0af480 0x7f4fec0cf470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009870 tx=0x7f4fdc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8e28dc76026c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4feaa00640 1 -- 172.21.15.164:0/2742811626 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a7070 con 0x7f4fec0af480 2024-05-08T04:41:29.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4feaa00640 1 -- 172.21.15.164:0/2742811626 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0b0070 con 0x7f4fec0af480 2024-05-08T04:41:29.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4feaa00640 1 -- 172.21.15.164:0/2742811626 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0ab890 con 0x7f4fec0af480 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- 172.21.15.164:0/2742811626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0af480 msgr2=0x7f4fec0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0af480 0x7f4fec0cf470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009870 tx=0x7f4fdc01c560 comp rx=0 tx=0).stop 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- 172.21.15.164:0/2742811626 shutdown_connections 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- 172.21.15.164:0/2742811626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0af480 0x7f4fec0cf470 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- 172.21.15.164:0/2742811626 >> 172.21.15.164:0/2742811626 conn(0x7f4fec0ccab0 msgr2=0x7f4fec0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- 172.21.15.164:0/2742811626 shutdown_connections 2024-05-08T04:41:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- 172.21.15.164:0/2742811626 wait complete. 2024-05-08T04:41:29.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 Processor -- start 2024-05-08T04:41:29.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- start start 2024-05-08T04:41:29.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec0af480 0x7f4fec15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec071f50 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec06eed0 con 0x7f4fec0af480 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.840+0000 7f4ff2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec072bf0 con 0x7f4fec06bca0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45532/0 (socket says 172.21.15.164:45532) 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 -- 172.21.15.164:0/4006260198 learned_addr learned my addr 172.21.15.164:0/4006260198 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4feb400640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe2a00640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec0af480 0x7f4fec15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 -- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 msgr2=0x7f4fec159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 -- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec0af480 msgr2=0x7f4fec15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec0af480 0x7f4fec15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 -- 172.21.15.164:0/4006260198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec09d430 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4febe00640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400efc0 tx=0x7f4fe400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4018020 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4014480 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec09d6d0 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4014870 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec15cd30 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec069d60 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f4fe401b020 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.844+0000 7f4fe3e00640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 0x7f4fc80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:29.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.848+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4fe4086570 con 0x7f4fec0cdfa0 2024-05-08T04:41:29.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.848+0000 7f4feb400640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 0x7f4fc80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:29.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.848+0000 7f4feb400640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 0x7f4fc80699c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0af000 tx=0x7f4fdc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:29.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:29.848+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fe4022070 con 0x7f4fec0cdfa0 2024-05-08T04:41:30.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.036+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fec066df0 con 0x7f4fec0cdfa0 2024-05-08T04:41:30.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4fe3e00640 1 -- 172.21.15.164:0/4006260198 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4fe4010040 con 0x7f4fec0cdfa0 2024-05-08T04:41:30.044 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:30.044 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:30.044 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 44 2024-05-08T04:41:30.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 msgr2=0x7f4fc80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 0x7f4fc80699c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0af000 tx=0x7f4fdc0a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:30.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 msgr2=0x7f4fec15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400efc0 tx=0x7f4fe400c490 comp rx=0 tx=0).stop 2024-05-08T04:41:30.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 shutdown_connections 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fc8067500 0x7f4fc80699c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fec0cdfa0 0x7f4fec15ddf0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec0af480 0x7f4fec15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 --2- 172.21.15.164:0/4006260198 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fec06bca0 0x7f4fec159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 >> 172.21.15.164:0/4006260198 conn(0x7f4fec0ccab0 msgr2=0x7f4fec09e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 shutdown_connections 2024-05-08T04:41:30.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.040+0000 7f4ff2200640 1 -- 172.21.15.164:0/4006260198 wait complete. 2024-05-08T04:41:30.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:30.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 Processor -- start 2024-05-08T04:41:30.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 -- start start 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2c80b26c0 0x7fd2c80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c806c770 con 0x7fd2c815ab60 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c806ca50 con 0x7fd2c80cfbd0 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2cd600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80d0060 con 0x7fd2c80b26c0 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2c6a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd2c815ab60 0x7fd2c8072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39912/0 (socket says 172.21.15.164:39912) 2024-05-08T04:41:30.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.196+0000 7fd2c6a00640 1 -- 172.21.15.164:0/552471008 learned_addr learned my addr 172.21.15.164:0/552471008 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2be000640 1 --2- 172.21.15.164:0/552471008 >> v2:172.21.15.172:3300/0 conn(0x7fd2c80b26c0 0x7fd2c80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411955376 0 0) 0x7fd2c806c770 con 0x7fd2c815ab60 2024-05-08T04:41:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003530 con 0x7fd2c815ab60 2024-05-08T04:41:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000149138 0 0) 0x7fd2ac003530 con 0x7fd2c815ab60 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 >> v2:172.21.15.172:3300/0 conn(0x7fd2c80b26c0 msgr2=0x7fd2c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 --2- 172.21.15.164:0/552471008 >> v2:172.21.15.172:3300/0 conn(0x7fd2c80b26c0 0x7fd2c80cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 >> v1:172.21.15.172:6789/0 conn(0x7fd2c80cfbd0 legacy=0x7fd2c80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c806cd30 con 0x7fd2c815ab60 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd2b8001680 con 0x7fd2c815ab60 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 >> v1:172.21.15.164:6789/0 conn(0x7fd2c815ab60 legacy=0x7fd2c8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2ac0039c0 0x7fd2ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2ac0043c0 0x7fd2ac004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00bfc0 con 0x7fd2ac005090 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00c170 con 0x7fd2ac0043c0 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00c370 con 0x7fd2ac0039c0 2024-05-08T04:41:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2be000640 1 --2- 172.21.15.164:0/552471008 >> v2:172.21.15.172:3300/0 conn(0x7fd2c80b26c0 0x7fd2c80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c6a00640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2ac0039c0 0x7fd2ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2be000640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2ac0043c0 0x7fd2ac004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 -- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2ac0039c0 msgr2=0x7fd2ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2ac0039c0 0x7fd2ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 -- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2ac0043c0 msgr2=0x7fd2ac004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2ac0043c0 0x7fd2ac004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2c7400640 1 -- 172.21.15.164:0/552471008 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac003530 con 0x7fd2ac005090 2024-05-08T04:41:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.200+0000 7fd2be000640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2ac0043c0 0x7fd2ac004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:30.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2c7400640 1 -- 172.21.15.164:0/552471008 reap_dead start 2024-05-08T04:41:30.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2c7400640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2ac00b9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001f60 tx=0x7fd2c001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ee9aac7c1fa385 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0001b60 con 0x7fd2ac005090 2024-05-08T04:41:30.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0003b80 con 0x7fd2ac005090 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2c6000640 1 -- 172.21.15.164:0/552471008 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00050d0 con 0x7fd2ac005090 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 msgr2=0x7fd2ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2ac00b9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001f60 tx=0x7fd2c001c880 comp rx=0 tx=0).stop 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- 172.21.15.164:0/552471008 shutdown_connections 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 --2- 172.21.15.164:0/552471008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2ac00b9d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- 172.21.15.164:0/552471008 >> 172.21.15.164:0/552471008 conn(0x7fd2c806ac90 msgr2=0x7fd2c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- 172.21.15.164:0/552471008 shutdown_connections 2024-05-08T04:41:30.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- 172.21.15.164:0/552471008 wait complete. 2024-05-08T04:41:30.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 Processor -- start 2024-05-08T04:41:30.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- start start 2024-05-08T04:41:30.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 0x7fd2c81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2c8159c10 0x7fd2c80b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80aed30 con 0x7fd2ac005090 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c806d2f0 con 0x7fd2c8148140 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.204+0000 7fd2cd600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80aeeb0 con 0x7fd2c8159c10 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45562/0 (socket says 172.21.15.164:45562) 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 -- 172.21.15.164:0/3680824608 learned_addr learned my addr 172.21.15.164:0/3680824608 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2be000640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 0x7fd2c81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c7400640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2c8159c10 0x7fd2c80b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 -- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2c8159c10 msgr2=0x7fd2c80b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2c8159c10 0x7fd2c80b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 -- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 msgr2=0x7fd2c81596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 0x7fd2c81596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 -- 172.21.15.164:0/3680824608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c815afe0 con 0x7fd2ac005090 2024-05-08T04:41:30.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2be000640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 0x7fd2c81596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2c6a00640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8012340 tx=0x7fd2b8012800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b801c070 con 0x7fd2ac005090 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8014830 con 0x7fd2ac005090 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8014c20 con 0x7fd2ac005090 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c815bfc0 con 0x7fd2ac005090 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c815b280 con 0x7fd2ac005090 2024-05-08T04:41:30.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c8069d60 con 0x7fd2ac005090 2024-05-08T04:41:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fd2b801f070 con 0x7fd2ac005090 2024-05-08T04:41:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.208+0000 7fd2bf400640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 0x7fd2a40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.212+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd2b8088360 con 0x7fd2ac005090 2024-05-08T04:41:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.212+0000 7fd2be000640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 0x7fd2a40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.212+0000 7fd2be000640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 0x7fd2a40699c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2c81f4dc0 tx=0x7fd2b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.212+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2b8058b30 con 0x7fd2ac005090 2024-05-08T04:41:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c80ce580 con 0x7fd2ac005090 2024-05-08T04:41:30.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2bf400640 1 -- 172.21.15.164:0/3680824608 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fd2b8058430 con 0x7fd2ac005090 2024-05-08T04:41:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:30.377 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:28.209436+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 msgr2=0x7fd2a40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 0x7fd2a40699c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2c81f4dc0 tx=0x7fd2b0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 msgr2=0x7fd2c815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8012340 tx=0x7fd2b8012800 comp rx=0 tx=0).stop 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 shutdown_connections 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2a4067500 0x7fd2a40699c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2c8159c10 0x7fd2c80b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c8148140 0x7fd2c81596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 --2- 172.21.15.164:0/3680824608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ac005090 0x7fd2c815ba80 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 >> 172.21.15.164:0/3680824608 conn(0x7fd2c806ac90 msgr2=0x7fd2c806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 shutdown_connections 2024-05-08T04:41:30.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.372+0000 7fd2cd600640 1 -- 172.21.15.164:0/3680824608 wait complete. 2024-05-08T04:41:30.398 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:41:30.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:30.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 Processor -- start 2024-05-08T04:41:30.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 -- start start 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8e00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e0071980 con 0x7ff8e00af480 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e0071c00 con 0x7ff8e006bca0 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8e5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0161810 con 0x7ff8e00cdfa0 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8dea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff8e00af480 0x7ff8e006cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39922/0 (socket says 172.21.15.164:39922) 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.528+0000 7ff8dea00640 1 -- 172.21.15.164:0/2537868414 learned_addr learned my addr 172.21.15.164:0/2537868414 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8d6000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8e00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612439420 0 0) 0x7ff8d0001120 con 0x7ff8e00af480 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c4003530 con 0x7ff8e00af480 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706673788 0 0) 0x7ff8e0071c00 con 0x7ff8e006bca0 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0001120 con 0x7ff8e006bca0 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247707395 0 0) 0x7ff8c4003530 con 0x7ff8e00af480 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 msgr2=0x7ff8e00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8e00cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 >> v1:172.21.15.172:6789/0 conn(0x7ff8e006bca0 legacy=0x7ff8e00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e006f120 con 0x7ff8e00af480 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7ff8d0001850 con 0x7ff8e00af480 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8d6000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8e00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 >> v1:172.21.15.164:6789/0 conn(0x7ff8e00af480 legacy=0x7ff8e006cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8c4003b80 0x7ff8c4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c4004810 0x7ff8c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0161810 con 0x7ff8c4003b80 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c400ba80 con 0x7ff8c4004810 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c400bc50 con 0x7ff8e00cdfa0 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8c4003b80 0x7ff8c4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8d6000640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c4004810 0x7ff8c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8df400640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 -- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 msgr2=0x7ff8c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8e00cdfa0 0x7ff8c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 -- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c4004810 msgr2=0x7ff8c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c4004810 0x7ff8c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.532+0000 7ff8dea00640 1 -- 172.21.15.164:0/2537868414 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c4003530 con 0x7ff8c4003b80 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8dea00640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8c4003b80 0x7ff8c4003f80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c2f0 tx=0x7ff8d001cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdce331b61c5aeed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0007dc0 con 0x7ff8c4003b80 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d00a9070 con 0x7ff8c4003b80 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8de000640 1 -- 172.21.15.164:0/2537868414 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0005d20 con 0x7ff8c4003b80 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- 172.21.15.164:0/2537868414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8c4003b80 msgr2=0x7ff8c4003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 --2- 172.21.15.164:0/2537868414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8c4003b80 0x7ff8c4003f80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c2f0 tx=0x7ff8d001cb00 comp rx=0 tx=0).stop 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8df400640 1 -- 172.21.15.164:0/2537868414 reap_dead start 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- 172.21.15.164:0/2537868414 shutdown_connections 2024-05-08T04:41:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- 172.21.15.164:0/2537868414 >> 172.21.15.164:0/2537868414 conn(0x7ff8e00ccab0 msgr2=0x7ff8e00ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- 172.21.15.164:0/2537868414 shutdown_connections 2024-05-08T04:41:30.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- 172.21.15.164:0/2537868414 wait complete. 2024-05-08T04:41:30.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 Processor -- start 2024-05-08T04:41:30.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- start start 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c4003b80 0x7ff8e015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e015c780 0x7ff8e00b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e006c440 con 0x7ff8e015c250 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e006c2c0 con 0x7ff8e015c780 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8e5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e006c5c0 con 0x7ff8c4003b80 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8d6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8d6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45582/0 (socket says 172.21.15.164:45582) 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.536+0000 7ff8d6000640 1 -- 172.21.15.164:0/4256191810 learned_addr learned my addr 172.21.15.164:0/4256191810 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8dea00640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c4003b80 0x7ff8e015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8df400640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e015c780 0x7ff8e00b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 -- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c4003b80 msgr2=0x7ff8e015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c4003b80 0x7ff8e015bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 -- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e015c780 msgr2=0x7ff8e00b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e015c780 0x7ff8e00b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 -- 172.21.15.164:0/4256191810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e01fadb0 con 0x7ff8e015c250 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d6000640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8011910 tx=0x7ff8c8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800faf0 con 0x7ff8e015c250 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800fc90 con 0x7ff8e015c250 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e01fb090 con 0x7ff8e015c250 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8004030 con 0x7ff8e015c250 2024-05-08T04:41:30.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e01fb650 con 0x7ff8e015c250 2024-05-08T04:41:30.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0069d60 con 0x7ff8e015c250 2024-05-08T04:41:30.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7ff8c8012040 con 0x7ff8e015c250 2024-05-08T04:41:30.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 0x7ff8bc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.540+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7ff8c8085e00 con 0x7ff8e015c250 2024-05-08T04:41:30.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.544+0000 7ff8dea00640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 0x7ff8bc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.544+0000 7ff8dea00640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 0x7ff8bc0699c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00ab000 tx=0x7ff8d00b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.548+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8c8019070 con 0x7ff8e015c250 2024-05-08T04:41:30.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.732+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e00a0950 con 0x7ff8e015c250 2024-05-08T04:41:30.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.732+0000 7ff8d7400640 1 -- 172.21.15.164:0/4256191810 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff8c8055eb0 con 0x7ff8e015c250 2024-05-08T04:41:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:30.740 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:30.740 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 44 2024-05-08T04:41:30.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 msgr2=0x7ff8bc0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 0x7ff8bc0699c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00ab000 tx=0x7ff8d00b1040 comp rx=0 tx=0).stop 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 msgr2=0x7ff8e00afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8011910 tx=0x7ff8c8011dd0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 shutdown_connections 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff8bc067500 0x7ff8bc0699c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e015c780 0x7ff8e00b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8e015c250 0x7ff8e00afd90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 --2- 172.21.15.164:0/4256191810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c4003b80 0x7ff8e015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 >> 172.21.15.164:0/4256191810 conn(0x7ff8e00ccab0 msgr2=0x7ff8e00cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 shutdown_connections 2024-05-08T04:41:30.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.736+0000 7ff8e5600640 1 -- 172.21.15.164:0/4256191810 wait complete. 2024-05-08T04:41:30.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 Processor -- start 2024-05-08T04:41:30.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 -- start start 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87c015bc50 0x7f87c00b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c00b42a0 con 0x7f87c00ccfe0 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c00cd4e0 con 0x7f87c00d0780 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87c6200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c00cd7c0 con 0x7f87c015bc50 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87b6a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f87c00ccfe0 0x7f87c00d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39936/0 (socket says 172.21.15.164:39936) 2024-05-08T04:41:30.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87b6a00640 1 -- 172.21.15.164:0/692009160 learned_addr learned my addr 172.21.15.164:0/692009160 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bf400640 1 --2- 172.21.15.164:0/692009160 >> v2:172.21.15.172:3300/0 conn(0x7f87c015bc50 0x7f87c00b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117552220 0 0) 0x7f87c00b42a0 con 0x7f87c00ccfe0 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a4003530 con 0x7f87c00ccfe0 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773827363 0 0) 0x7f87a4003530 con 0x7f87c00ccfe0 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 >> v2:172.21.15.172:3300/0 conn(0x7f87c015bc50 msgr2=0x7f87c00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 --2- 172.21.15.164:0/692009160 >> v2:172.21.15.172:3300/0 conn(0x7f87c015bc50 0x7f87c00b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 >> v1:172.21.15.172:6789/0 conn(0x7f87c00d0780 legacy=0x7f87c015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00cf590 con 0x7f87c00ccfe0 2024-05-08T04:41:30.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f87a80016a0 con 0x7f87c00ccfe0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 >> v1:172.21.15.164:6789/0 conn(0x7f87c00ccfe0 legacy=0x7f87c00d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 0x7f87a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a40043c0 0x7f87a4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a4005130 0x7f87a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00cd7c0 con 0x7f87a40043c0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a400c070 con 0x7f87a40039c0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a400c270 con 0x7f87a4005130 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87b6a00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a40043c0 0x7f87a4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.892+0000 7f87bf400640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 0x7f87a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bfe00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a4005130 0x7f87a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 -- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a4005130 msgr2=0x7f87a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a4005130 0x7f87a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 -- 172.21.15.164:0/692009160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a40043c0 msgr2=0x7f87a4004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a40043c0 0x7f87a4004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 -- 172.21.15.164:0/692009160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4003530 con 0x7f87a40039c0 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bfe00640 1 -- 172.21.15.164:0/692009160 reap_dead start 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87b6a00640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a40043c0 0x7f87a4004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bf400640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 0x7f87a4003da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009fc0 tx=0x7f87b001cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e369a6a0cd0031 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00b2070 con 0x7f87a40039c0 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00a5cb0 con 0x7f87a40039c0 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.896+0000 7f87bea00640 1 -- 172.21.15.164:0/692009160 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0009c10 con 0x7f87a40039c0 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 msgr2=0x7f87a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 0x7f87a4003da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009fc0 tx=0x7f87b001cca0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/692009160 shutdown_connections 2024-05-08T04:41:30.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 --2- 172.21.15.164:0/692009160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a40039c0 0x7f87a4003da0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/692009160 >> 172.21.15.164:0/692009160 conn(0x7f87c00b34f0 msgr2=0x7f87c00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/692009160 shutdown_connections 2024-05-08T04:41:30.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/692009160 wait complete. 2024-05-08T04:41:30.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 Processor -- start 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- start start 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a40039c0 0x7f87c01f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c00cac20 0x7f87c00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c015c030 con 0x7f87c0144790 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00cfb50 con 0x7f87c00cac20 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00d0070 con 0x7f87a40039c0 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45596/0 (socket says 172.21.15.164:45596) 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 -- 172.21.15.164:0/79238572 learned_addr learned my addr 172.21.15.164:0/79238572 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87bf400640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a40039c0 0x7f87c01f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87bfe00640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c00cac20 0x7f87c00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 -- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a40039c0 msgr2=0x7f87c01f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a40039c0 0x7f87c01f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 -- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c00cac20 msgr2=0x7f87c00cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c00cac20 0x7f87c00cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 -- 172.21.15.164:0/79238572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00c9b80 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b6a00640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000c10 tx=0x7f87a8007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007d70 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8018070 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800fc20 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.900+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c00c9e60 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c00ca420 con 0x7f87c0144790 2024-05-08T04:41:30.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0069d60 con 0x7f87c0144790 2024-05-08T04:41:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f87a801d460 con 0x7f87c0144790 2024-05-08T04:41:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87b7e00640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 0x7f879c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f87a8085a70 con 0x7f87c0144790 2024-05-08T04:41:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87bf400640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 0x7f879c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.904+0000 7f87bf400640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 0x7f879c0699f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f87b00af000 tx=0x7f87b00a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:30.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:30.908+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87a8056170 con 0x7f87c0144790 2024-05-08T04:41:31.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.092+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87c00b7940 con 0x7f87c0144790 2024-05-08T04:41:31.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87b7e00640 1 -- 172.21.15.164:0/79238572 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f87a8055a70 con 0x7f87c0144790 2024-05-08T04:41:31.100 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:31.100 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:07.483520+0000","modified":"2024-05-08T04:41:10.507554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6465},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6465":{"gid":6465,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/239509813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":239509813},{"type":"v1","addr":"172.21.15.164:6837","nonce":239509813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-08T04:41:31.100 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 44 2024-05-08T04:41:31.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 msgr2=0x7f879c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 0x7f879c0699f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f87b00af000 tx=0x7f87b00a7040 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 msgr2=0x7f87c015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000c10 tx=0x7f87a8007720 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 shutdown_connections 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f879c067530 0x7f879c0699f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c00cac20 0x7f87c00cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87c0144790 0x7f87c015c8a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 --2- 172.21.15.164:0/79238572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87a40039c0 0x7f87c01f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 >> 172.21.15.164:0/79238572 conn(0x7f87c00b34f0 msgr2=0x7f87c006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 shutdown_connections 2024-05-08T04:41:31.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.096+0000 7f87c6200640 1 -- 172.21.15.164:0/79238572 wait complete. 2024-05-08T04:41:31.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:31.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.252+0000 7f4782e00640 1 Processor -- start 2024-05-08T04:41:31.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.252+0000 7f4782e00640 1 -- start start 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4782e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f477c15ab60 0x7f477c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4782e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c072640 con 0x7f477c0cfda0 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4782e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c06c770 con 0x7f477c0b26c0 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4782e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c06ca50 con 0x7f477c15ab60 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4780e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f477c0cfda0 0x7f477c0cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39944/0 (socket says 172.21.15.164:39944) 2024-05-08T04:41:31.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f4780e00640 1 -- 172.21.15.164:0/3398295030 learned_addr learned my addr 172.21.15.164:0/3398295030 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477be00640 1 --2- 172.21.15.164:0/3398295030 >> v2:172.21.15.172:3300/0 conn(0x7f477c15ab60 0x7f477c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477b400640 1 -- 172.21.15.164:0/3398295030 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283224683 0 0) 0x7f477c072640 con 0x7f477c0cfda0 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477b400640 1 -- 172.21.15.164:0/3398295030 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4760003530 con 0x7f477c0cfda0 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477be00640 1 -- 172.21.15.164:0/3398295030 >> v1:172.21.15.172:6789/0 conn(0x7f477c0b26c0 legacy=0x7f477c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477be00640 1 -- 172.21.15.164:0/3398295030 >> v1:172.21.15.164:6789/0 conn(0x7f477c0cfda0 legacy=0x7f477c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.256+0000 7f477be00640 1 -- 172.21.15.164:0/3398295030 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c06cd30 con 0x7f477c15ab60 2024-05-08T04:41:31.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f477be00640 1 --2- 172.21.15.164:0/3398295030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c15ab60 0x7f477c072100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f476c0097c0 tx=0x7f476c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fa10c46ee137d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f477b400640 1 -- 172.21.15.164:0/3398295030 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0b0070 con 0x7f477c15ab60 2024-05-08T04:41:31.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f477b400640 1 -- 172.21.15.164:0/3398295030 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c0ab600 con 0x7f477c15ab60 2024-05-08T04:41:31.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f477b400640 1 -- 172.21.15.164:0/3398295030 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0ab9d0 con 0x7f477c15ab60 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- 172.21.15.164:0/3398295030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c15ab60 msgr2=0x7f477c072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 --2- 172.21.15.164:0/3398295030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c15ab60 0x7f477c072100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f476c0097c0 tx=0x7f476c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- 172.21.15.164:0/3398295030 shutdown_connections 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 --2- 172.21.15.164:0/3398295030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c15ab60 0x7f477c072100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- 172.21.15.164:0/3398295030 >> 172.21.15.164:0/3398295030 conn(0x7f477c06ac90 msgr2=0x7f477c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- 172.21.15.164:0/3398295030 shutdown_connections 2024-05-08T04:41:31.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- 172.21.15.164:0/3398295030 wait complete. 2024-05-08T04:41:31.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 Processor -- start 2024-05-08T04:41:31.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- start start 2024-05-08T04:41:31.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477c0b26c0 0x7f477c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 0x7f477c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0aeeb0 con 0x7f477c15ab60 2024-05-08T04:41:31.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0aed30 con 0x7f477c0cfda0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4782e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c168980 con 0x7f477c0b26c0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4780e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4780e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45612/0 (socket says 172.21.15.164:45612) 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.260+0000 7f4780e00640 1 -- 172.21.15.164:0/2221590772 learned_addr learned my addr 172.21.15.164:0/2221590772 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4773400640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 0x7f477c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f477be00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477c0b26c0 0x7f477c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 -- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477c0b26c0 msgr2=0x7f477c150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477c0b26c0 0x7f477c150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 -- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 msgr2=0x7f477c1519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 0x7f477c1519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 -- 172.21.15.164:0/2221590772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c150900 con 0x7f477c15ab60 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4773400640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 0x7f477c1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4780e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f477400cbb0 tx=0x7f477400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774017070 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400ad40 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774012600 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c150be0 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c1511a0 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c069d60 con 0x7f477c15ab60 2024-05-08T04:41:31.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f477401a070 con 0x7f477c15ab60 2024-05-08T04:41:31.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 0x7f47580699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.264+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4774085830 con 0x7f477c15ab60 2024-05-08T04:41:31.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.268+0000 7f477be00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 0x7f47580699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.268+0000 7f477be00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 0x7f47580699c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ad000 tx=0x7f476c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.268+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4774055920 con 0x7f477c15ab60 2024-05-08T04:41:31.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.424+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f477c15c640 con 0x7f477c15ab60 2024-05-08T04:41:31.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.424+0000 7f4778c00640 1 -- 172.21.15.164:0/2221590772 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f47740599c0 con 0x7f477c15ab60 2024-05-08T04:41:31.431 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:31.431 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:28.209436+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:05.195201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:06.483763+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 msgr2=0x7f47580699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 0x7f47580699c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ad000 tx=0x7f476c0a6000 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 msgr2=0x7f477c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f477400cbb0 tx=0x7f477400a430 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 shutdown_connections 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4758067500 0x7f47580699c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477c15ab60 0x7f477c15c100 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f477c0cfda0 0x7f477c1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 --2- 172.21.15.164:0/2221590772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477c0b26c0 0x7f477c150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 >> 172.21.15.164:0/2221590772 conn(0x7f477c06ac90 msgr2=0x7f477c15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 shutdown_connections 2024-05-08T04:41:31.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.428+0000 7f4782e00640 1 -- 172.21.15.164:0/2221590772 wait complete. 2024-05-08T04:41:31.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:41:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 Processor -- start 2024-05-08T04:41:31.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 -- start start 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f510015dd70 0x7f51001619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f510006c7b0 0x7f5100161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51001628b0 con 0x7f5100162420 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510006cc70 con 0x7f510015dd70 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f5106200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510006cf50 con 0x7f510006c7b0 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f510015dd70 0x7f51001619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f510015dd70 0x7f51001619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45622/0 (socket says 172.21.15.164:45622) 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 -- 172.21.15.164:0/748508726 learned_addr learned my addr 172.21.15.164:0/748508726 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50fea00640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f510006c7b0 0x7f5100161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 -- 172.21.15.164:0/748508726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f510006c7b0 msgr2=0x7f5100161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:31.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f510006c7b0 0x7f5100161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 -- 172.21.15.164:0/748508726 >> v1:172.21.15.164:6789/0 conn(0x7f5100162420 legacy=0x7f51000cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.584+0000 7f50ff400640 1 -- 172.21.15.164:0/748508726 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000b1570 con 0x7f510015dd70 2024-05-08T04:41:31.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f50ff400640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51001619a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009e50 tx=0x7f50f001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c034e04ca093a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f50fe000640 1 -- 172.21.15.164:0/748508726 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00b4070 con 0x7f510015dd70 2024-05-08T04:41:31.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f50fe000640 1 -- 172.21.15.164:0/748508726 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f00af400 con 0x7f510015dd70 2024-05-08T04:41:31.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f50fe000640 1 -- 172.21.15.164:0/748508726 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00af7d0 con 0x7f510015dd70 2024-05-08T04:41:31.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 -- 172.21.15.164:0/748508726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 msgr2=0x7f51001619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51001619a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009e50 tx=0x7f50f001f210 comp rx=0 tx=0).stop 2024-05-08T04:41:31.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 -- 172.21.15.164:0/748508726 shutdown_connections 2024-05-08T04:41:31.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f510006c7b0 0x7f5100161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 --2- 172.21.15.164:0/748508726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51001619a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 -- 172.21.15.164:0/748508726 >> 172.21.15.164:0/748508726 conn(0x7f51000b34f0 msgr2=0x7f51000b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 -- 172.21.15.164:0/748508726 shutdown_connections 2024-05-08T04:41:31.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.588+0000 7f5106200640 1 -- 172.21.15.164:0/748508726 wait complete. 2024-05-08T04:41:31.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 Processor -- start 2024-05-08T04:41:31.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- start start 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 0x7f5100159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5100162420 0x7f51000b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510015e150 con 0x7f510015dd70 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000b4230 con 0x7f510006c7b0 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510015e2d0 con 0x7f5100162420 2024-05-08T04:41:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45626/0 (socket says 172.21.15.164:45626) 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 -- 172.21.15.164:0/3310113939 learned_addr learned my addr 172.21.15.164:0/3310113939 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 -- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5100162420 msgr2=0x7f51000b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5100162420 0x7f51000b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 -- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 msgr2=0x7f5100159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50ff400640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 0x7f5100159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 0x7f5100159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 -- 172.21.15.164:0/3310113939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000aee40 con 0x7f510015dd70 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50ff400640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 0x7f5100159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50fea00640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e960 tx=0x7f50ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec017070 con 0x7f510015dd70 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec00ca00 con 0x7f510015dd70 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00cdb0 con 0x7f510015dd70 2024-05-08T04:41:31.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51000af120 con 0x7f510015dd70 2024-05-08T04:41:31.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51000af6e0 con 0x7f510015dd70 2024-05-08T04:41:31.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.592+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5100069d60 con 0x7f510015dd70 2024-05-08T04:41:31.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.596+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f50ec01a070 con 0x7f510015dd70 2024-05-08T04:41:31.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.596+0000 7f50f7400640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 0x7f50e00699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.596+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f50ec085eb0 con 0x7f510015dd70 2024-05-08T04:41:31.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.596+0000 7f50ff400640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 0x7f50e00699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.596+0000 7f50ff400640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 0x7f50e00699f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b1000 tx=0x7f50f00aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.600+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50ec0565b0 con 0x7f510015dd70 2024-05-08T04:41:31.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.784+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f510006e3c0 con 0x7f510015dd70 2024-05-08T04:41:31.789 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:41:31.784+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:41:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f50f7400640 1 -- 172.21.15.164:0/3310113939 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f50ec055eb0 con 0x7f510015dd70 2024-05-08T04:41:31.792 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:41:31.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 msgr2=0x7f50e00699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 0x7f50e00699f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b1000 tx=0x7f50f00aa000 comp rx=0 tx=0).stop 2024-05-08T04:41:31.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 msgr2=0x7f51000afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e960 tx=0x7f50ec00ee20 comp rx=0 tx=0).stop 2024-05-08T04:41:31.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 shutdown_connections 2024-05-08T04:41:31.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f50e0067530 0x7f50e00699f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5100162420 0x7f51000b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f510015dd70 0x7f51000afee0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 --2- 172.21.15.164:0/3310113939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f510006c7b0 0x7f5100159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 >> 172.21.15.164:0/3310113939 conn(0x7f51000b34f0 msgr2=0x7f510006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 shutdown_connections 2024-05-08T04:41:31.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.788+0000 7f5106200640 1 -- 172.21.15.164:0/3310113939 wait complete. 2024-05-08T04:41:31.820 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:41:31.863 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:41:31.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 Processor -- start 2024-05-08T04:41:31.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 -- start start 2024-05-08T04:41:31.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1a9c15ab10 0x7f1a9c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c15e870 con 0x7f1a9c0ae6e0 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c06bb80 con 0x7f1a9c0cee90 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa2e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c15eb50 con 0x7f1a9c15ab10 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45668/0 (socket says 172.21.15.164:45668) 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 -- 172.21.15.164:0/2761596125 learned_addr learned my addr 172.21.15.164:0/2761596125 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1a9be00640 1 --2- 172.21.15.164:0/2761596125 >> v2:172.21.15.172:3300/0 conn(0x7f1a9c15ab10 0x7f1a9c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 -- 172.21.15.164:0/2761596125 >> v2:172.21.15.172:3300/0 conn(0x7f1a9c15ab10 msgr2=0x7f1a9c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 --2- 172.21.15.164:0/2761596125 >> v2:172.21.15.172:3300/0 conn(0x7f1a9c15ab10 0x7f1a9c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 -- 172.21.15.164:0/2761596125 >> v1:172.21.15.172:6789/0 conn(0x7f1a9c0ae6e0 legacy=0x7f1a9c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.960+0000 7f1aa0e00640 1 -- 172.21.15.164:0/2761596125 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c15ee30 con 0x7f1a9c0cee90 2024-05-08T04:41:31.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa0e00640 1 --2- 172.21.15.164:0/2761596125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0c4920 tx=0x7f1a9401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53ed8230cc72a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1a9b400640 1 -- 172.21.15.164:0/2761596125 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940ab660 con 0x7f1a9c0cee90 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1a9b400640 1 -- 172.21.15.164:0/2761596125 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a940b0070 con 0x7f1a9c0cee90 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1a9b400640 1 -- 172.21.15.164:0/2761596125 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940aba20 con 0x7f1a9c0cee90 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- 172.21.15.164:0/2761596125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c0cee90 msgr2=0x7f1a9c0aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/2761596125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0c4920 tx=0x7f1a9401c460 comp rx=0 tx=0).stop 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- 172.21.15.164:0/2761596125 shutdown_connections 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/2761596125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c0cee90 0x7f1a9c0aeb70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/2761596125 >> v2:172.21.15.172:3300/0 conn(0x7f1a9c15ab10 0x7f1a9c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- 172.21.15.164:0/2761596125 >> 172.21.15.164:0/2761596125 conn(0x7f1a9c0cf770 msgr2=0x7f1a9c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:31.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- 172.21.15.164:0/2761596125 shutdown_connections 2024-05-08T04:41:31.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- 172.21.15.164:0/2761596125 wait complete. 2024-05-08T04:41:31.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 Processor -- start 2024-05-08T04:41:31.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.964+0000 7f1aa2e00640 1 -- start start 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 0x7f1a9c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 0x7f1a9c0714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0cb070 con 0x7f1a9c15ab10 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0caef0 con 0x7f1a9c0ae6e0 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0cb1f0 con 0x7f1a9c0cee90 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45682/0 (socket says 172.21.15.164:45682) 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 -- 172.21.15.164:0/3776001419 learned_addr learned my addr 172.21.15.164:0/3776001419 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a93400640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 0x7f1a9c0714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a9be00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 0x7f1a9c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 -- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 msgr2=0x7f1a9c0714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 0x7f1a9c0714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 -- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 msgr2=0x7f1a9c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 0x7f1a9c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 -- 172.21.15.164:0/3776001419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c14ba70 con 0x7f1a9c15ab10 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a93400640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 0x7f1a9c0714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a9be00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 0x7f1a9c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa0e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a94019070 tx=0x7f1a9401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940be3f0 con 0x7f1a9c15ab10 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a940be590 con 0x7f1a9c15ab10 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940be980 con 0x7f1a9c15ab10 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c151200 con 0x7f1a9c15ab10 2024-05-08T04:41:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c150100 con 0x7f1a9c15ab10 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.968+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c072d30 con 0x7f1a9c15ab10 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.972+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f1a940b43f0 con 0x7f1a9c15ab10 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.972+0000 7f1a98c00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 0x7f1a780699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.972+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f1a9411ffa0 con 0x7f1a9c15ab10 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.972+0000 7f1a9be00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 0x7f1a780699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.972+0000 7f1a9be00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 0x7f1a780699c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00bce0 tx=0x7f1a8c00b3a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:31.976+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a9c14ba70 con 0x7f1a9c15ab10 2024-05-08T04:41:32.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.168+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1a9c04f010 con 0x7f1a9c15ab10 2024-05-08T04:41:32.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.788+0000 7f1a98c00640 1 -- 172.21.15.164:0/3776001419 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1a94016110 con 0x7f1a9c15ab10 2024-05-08T04:41:32.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 msgr2=0x7f1a780699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:32.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 0x7f1a780699c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00bce0 tx=0x7f1a8c00b3a0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 msgr2=0x7f1a9c0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a94019070 tx=0x7f1a9401aca0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 shutdown_connections 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a78067500 0x7f1a780699c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a9c0cee90 0x7f1a9c0714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a9c15ab10 0x7f1a9c0719f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 --2- 172.21.15.164:0/3776001419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a9c0ae6e0 0x7f1a9c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 >> 172.21.15.164:0/3776001419 conn(0x7f1a9c0cf770 msgr2=0x7f1a9c0b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 shutdown_connections 2024-05-08T04:41:32.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.792+0000 7f1aa2e00640 1 -- 172.21.15.164:0/3776001419 wait complete. 2024-05-08T04:41:32.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:41:32.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 Processor -- start 2024-05-08T04:41:32.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 -- start start 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f535c0cf900 0x7f535c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0cfce0 con 0x7f535c070bd0 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c071be0 con 0x7f535c072720 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5363200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c071060 con 0x7f535c0cf900 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5353400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f535c070bd0 0x7f535c072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39972/0 (socket says 172.21.15.164:39972) 2024-05-08T04:41:33.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f5353400640 1 -- 172.21.15.164:0/1674535585 learned_addr learned my addr 172.21.15.164:0/1674535585 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.992+0000 7f535be00640 1 --2- 172.21.15.164:0/1674535585 >> v2:172.21.15.172:3300/0 conn(0x7f535c0cf900 0x7f535c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571204379 0 0) 0x7f535c0cfce0 con 0x7f535c070bd0 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5340003530 con 0x7f535c070bd0 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141813121 0 0) 0x7f535c071be0 con 0x7f535c072720 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f535c0cfce0 con 0x7f535c072720 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535be00640 1 -- 172.21.15.164:0/1674535585 >> v1:172.21.15.172:6789/0 conn(0x7f535c072720 legacy=0x7f535c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535be00640 1 -- 172.21.15.164:0/1674535585 >> v1:172.21.15.164:6789/0 conn(0x7f535c070bd0 legacy=0x7f535c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535be00640 1 -- 172.21.15.164:0/1674535585 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c071340 con 0x7f535c0cf900 2024-05-08T04:41:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535be00640 1 --2- 172.21.15.164:0/1674535585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c0cf900 0x7f535c15cb20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f534c0097c0 tx=0x7f534c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d97b328ee8bdcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0b1070 con 0x7f535c0cf900 2024-05-08T04:41:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c0ac5f0 con 0x7f535c0cf900 2024-05-08T04:41:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:32.996+0000 7f535b400640 1 -- 172.21.15.164:0/1674535585 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0ac9c0 con 0x7f535c0cf900 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- 172.21.15.164:0/1674535585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c0cf900 msgr2=0x7f535c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 --2- 172.21.15.164:0/1674535585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c0cf900 0x7f535c15cb20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f534c0097c0 tx=0x7f534c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- 172.21.15.164:0/1674535585 shutdown_connections 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 --2- 172.21.15.164:0/1674535585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c0cf900 0x7f535c15cb20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- 172.21.15.164:0/1674535585 >> 172.21.15.164:0/1674535585 conn(0x7f535c06cca0 msgr2=0x7f535c06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- 172.21.15.164:0/1674535585 shutdown_connections 2024-05-08T04:41:33.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- 172.21.15.164:0/1674535585 wait complete. 2024-05-08T04:41:33.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 Processor -- start 2024-05-08T04:41:33.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- start start 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 0x7f535c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 0x7f535c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c152ee0 con 0x7f535c0cf900 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c152d60 con 0x7f535c1472d0 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5363200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c16a250 con 0x7f535c072720 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f535be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 0x7f535c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45696/0 (socket says 172.21.15.164:45696) 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 -- 172.21.15.164:0/3120179586 learned_addr learned my addr 172.21.15.164:0/3120179586 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:33.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5361200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 0x7f535c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 -- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 msgr2=0x7f535c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 0x7f535c152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 -- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 msgr2=0x7f535c151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 0x7f535c151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 -- 172.21.15.164:0/3120179586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c15ddf0 con 0x7f535c0cf900 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5361200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 0x7f535c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f535be00640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 0x7f535c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.000+0000 7f5353400640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f534400c4d0 tx=0x7f534400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f535c0cf900 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344005cf0 con 0x7f535c0cf900 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c15e0d0 con 0x7f535c0cf900 2024-05-08T04:41:33.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440072b0 con 0x7f535c0cf900 2024-05-08T04:41:33.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c15e690 con 0x7f535c0cf900 2024-05-08T04:41:33.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c15e890 con 0x7f535c0cf900 2024-05-08T04:41:33.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f534401a070 con 0x7f535c0cf900 2024-05-08T04:41:33.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 0x7f53380699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5344084ea0 con 0x7f535c0cf900 2024-05-08T04:41:33.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.004+0000 7f535be00640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 0x7f53380699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.008+0000 7f535be00640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 0x7f53380699f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ae000 tx=0x7f534c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.008+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f535c15e890 con 0x7f535c0cf900 2024-05-08T04:41:33.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.168+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f535c06ddf0 con 0x7f535c0cf900 2024-05-08T04:41:33.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.792+0000 7f5358c00640 1 -- 172.21.15.164:0/3120179586 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5344088030 con 0x7f535c0cf900 2024-05-08T04:41:33.796 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 msgr2=0x7f53380699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 0x7f53380699f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ae000 tx=0x7f534c008040 comp rx=0 tx=0).stop 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 msgr2=0x7f535c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f534400c4d0 tx=0x7f534400c990 comp rx=0 tx=0).stop 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 shutdown_connections 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5338067530 0x7f53380699f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f535c1472d0 0x7f535c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c0cf900 0x7f535c15ebb0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 --2- 172.21.15.164:0/3120179586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535c072720 0x7f535c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 >> 172.21.15.164:0/3120179586 conn(0x7f535c06cca0 msgr2=0x7f535c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 shutdown_connections 2024-05-08T04:41:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.796+0000 7f5363200640 1 -- 172.21.15.164:0/3120179586 wait complete. 2024-05-08T04:41:33.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:41:33.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.960+0000 7f1ba2e00640 1 Processor -- start 2024-05-08T04:41:33.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.960+0000 7f1ba2e00640 1 -- start start 2024-05-08T04:41:33.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba2e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1b9c15ce10 0x7f1b9c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba2e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0c6600 con 0x7f1b9c06e390 2024-05-08T04:41:33.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba2e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0c6040 con 0x7f1b9c15ba20 2024-05-08T04:41:33.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba2e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c15fbe0 con 0x7f1b9c15ce10 2024-05-08T04:41:33.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1b9c06e390 0x7f1b9c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39976/0 (socket says 172.21.15.164:39976) 2024-05-08T04:41:33.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 -- 172.21.15.164:0/634249611 learned_addr learned my addr 172.21.15.164:0/634249611 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731549200 0 0) 0x7f1b9c0c6600 con 0x7f1b9c06e390 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80003530 con 0x7f1b9c06e390 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b93400640 1 --2- 172.21.15.164:0/634249611 >> v2:172.21.15.172:3300/0 conn(0x7f1b9c15ce10 0x7f1b9c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2338071180 0 0) 0x7f1b80003530 con 0x7f1b9c06e390 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 >> v2:172.21.15.172:3300/0 conn(0x7f1b9c15ce10 msgr2=0x7f1b9c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 --2- 172.21.15.164:0/634249611 >> v2:172.21.15.172:3300/0 conn(0x7f1b9c15ce10 0x7f1b9c06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 >> v1:172.21.15.172:6789/0 conn(0x7f1b9c15ba20 legacy=0x7f1b9c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0c6320 con 0x7f1b9c06e390 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f1b94001a00 con 0x7f1b9c06e390 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 >> v1:172.21.15.164:6789/0 conn(0x7f1b9c06e390 legacy=0x7f1b9c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80003a10 0x7f1b80003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b80004810 0x7f1b80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c15fbe0 con 0x7f1b80005220 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000c170 con 0x7f1b80003a10 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000c370 con 0x7f1b80004810 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b9be00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80003a10 0x7f1b80003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1b93400640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b80004810 0x7f1b80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 -- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b80004810 msgr2=0x7f1b80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b80004810 0x7f1b80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 -- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80003a10 msgr2=0x7f1b80003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80003a10 0x7f1b80003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.964+0000 7f1ba0e00640 1 -- 172.21.15.164:0/634249611 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80003530 con 0x7f1b80005220 2024-05-08T04:41:33.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba0e00640 1 -- 172.21.15.164:0/634249611 reap_dead start 2024-05-08T04:41:33.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba0e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b8000b9d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400a770 tx=0x7f1b9401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da3549fa586c37e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400a390 con 0x7f1b80005220 2024-05-08T04:41:33.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b940ac640 con 0x7f1b80005220 2024-05-08T04:41:33.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1b9b400640 1 -- 172.21.15.164:0/634249611 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940aca30 con 0x7f1b80005220 2024-05-08T04:41:33.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 -- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 msgr2=0x7f1b8000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b8000b9d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400a770 tx=0x7f1b9401d2a0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 -- 172.21.15.164:0/634249611 shutdown_connections 2024-05-08T04:41:33.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/634249611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b8000b9d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 -- 172.21.15.164:0/634249611 >> 172.21.15.164:0/634249611 conn(0x7f1b9c0cf770 msgr2=0x7f1b9c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:33.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 -- 172.21.15.164:0/634249611 shutdown_connections 2024-05-08T04:41:33.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.968+0000 7f1ba2e00640 1 -- 172.21.15.164:0/634249611 wait complete. 2024-05-08T04:41:33.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 Processor -- start 2024-05-08T04:41:33.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- start start 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c0b3a90 0x7f1b9c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b3f10 0x7f1b9c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0cf410 con 0x7f1b80005220 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0c5ec0 con 0x7f1b9c0b3a90 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c06de50 con 0x7f1b9c0b3f10 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45714/0 (socket says 172.21.15.164:45714) 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 -- 172.21.15.164:0/1919212324 learned_addr learned my addr 172.21.15.164:0/1919212324 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba0e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b3f10 0x7f1b9c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b93400640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c0b3a90 0x7f1b9c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 -- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b3f10 msgr2=0x7f1b9c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b3f10 0x7f1b9c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 -- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c0b3a90 msgr2=0x7f1b9c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c0b3a90 0x7f1b9c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 -- 172.21.15.164:0/1919212324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0b44e0 con 0x7f1b80005220 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b9be00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00f7f0 tx=0x7f1b8c00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0050d0 con 0x7f1b80005220 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c018070 con 0x7f1b80005220 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c005490 con 0x7f1b80005220 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c0b47c0 con 0x7f1b80005220 2024-05-08T04:41:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c0b4aa0 con 0x7f1b80005220 2024-05-08T04:41:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.972+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c072d30 con 0x7f1b80005220 2024-05-08T04:41:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.976+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f1b8c00d6d0 con 0x7f1b80005220 2024-05-08T04:41:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.976+0000 7f1b98c00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 0x7f1b78069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.976+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5413+0+0 (secure 0 0 0) 0x7f1b8c085580 con 0x7f1b80005220 2024-05-08T04:41:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.976+0000 7f1b93400640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 0x7f1b78069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.976+0000 7f1b93400640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 0x7f1b78069b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0c9b90 tx=0x7f1b84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:33.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:33.980+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b8c055870 con 0x7f1b80005220 2024-05-08T04:41:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.144+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b9c04f010 con 0x7f1b80005220 2024-05-08T04:41:34.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.792+0000 7f1b98c00640 1 -- 172.21.15.164:0/1919212324 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1b8c059910 con 0x7f1b80005220 2024-05-08T04:41:34.798 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:41:34.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 msgr2=0x7f1b78069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 0x7f1b78069b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0c9b90 tx=0x7f1b84008040 comp rx=0 tx=0).stop 2024-05-08T04:41:34.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 msgr2=0x7f1b9c1f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00f7f0 tx=0x7f1b8c00fcb0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 shutdown_connections 2024-05-08T04:41:34.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b780676a0 0x7f1b78069b60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b3f10 0x7f1b9c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c0b3a90 0x7f1b9c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 --2- 172.21.15.164:0/1919212324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b80005220 0x7f1b9c1f1cd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 >> 172.21.15.164:0/1919212324 conn(0x7f1b9c0cf770 msgr2=0x7f1b9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 shutdown_connections 2024-05-08T04:41:34.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.796+0000 7f1ba2e00640 1 -- 172.21.15.164:0/1919212324 wait complete. 2024-05-08T04:41:34.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:41:34.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.956+0000 7f205f800640 1 Processor -- start 2024-05-08T04:41:34.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205f800640 1 -- start start 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f20580c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205f800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20581627f0 con 0x7f205806ac90 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205f800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20580adc50 con 0x7f20580cb290 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20580ae860 con 0x7f2058144360 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f205806ac90 0x7f20581619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39984/0 (socket says 172.21.15.164:39984) 2024-05-08T04:41:34.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 -- 172.21.15.164:0/771119378 learned_addr learned my addr 172.21.15.164:0/771119378 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205ce00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f20580c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406812341 0 0) 0x7f20581627f0 con 0x7f205806ac90 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c003530 con 0x7f205806ac90 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117459498 0 0) 0x7f20580adc50 con 0x7f20580cb290 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20581627f0 con 0x7f20580cb290 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545354454 0 0) 0x7f203c003530 con 0x7f205806ac90 2024-05-08T04:41:34.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 msgr2=0x7f20580c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f20580c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 >> v1:172.21.15.172:6789/0 conn(0x7f20580cb290 legacy=0x7f20581620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580adf30 con 0x7f205806ac90 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2044001680 con 0x7f205806ac90 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 >> v1:172.21.15.164:6789/0 conn(0x7f205806ac90 legacy=0x7f20581619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f203c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f203c0040e0 0x7f203c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c004d50 0x7f203c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580ae860 con 0x7f203c0040e0 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c00bab0 con 0x7f203c004d50 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c00bcb0 con 0x7f2058144360 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f203c0040e0 0x7f203c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205d800640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c004d50 0x7f203c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f205ce00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f203c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 -- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 msgr2=0x7f203c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:34.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2058144360 0x7f203c003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 -- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c004d50 msgr2=0x7f203c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:34.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c004d50 0x7f203c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.960+0000 7f2057e00640 1 -- 172.21.15.164:0/771119378 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c003530 con 0x7f203c0040e0 2024-05-08T04:41:34.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f2057e00640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f203c0040e0 0x7f203c004810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f204400c160 tx=0x7f204401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87c1eb11798e0b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:34.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044007cb0 con 0x7f203c0040e0 2024-05-08T04:41:34.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20440a9020 con 0x7f203c0040e0 2024-05-08T04:41:34.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f2057400640 1 -- 172.21.15.164:0/771119378 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044005ae0 con 0x7f203c0040e0 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 -- 172.21.15.164:0/771119378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f203c0040e0 msgr2=0x7f203c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 --2- 172.21.15.164:0/771119378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f203c0040e0 0x7f203c004810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f204400c160 tx=0x7f204401cbe0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205d800640 1 -- 172.21.15.164:0/771119378 reap_dead start 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 -- 172.21.15.164:0/771119378 shutdown_connections 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 -- 172.21.15.164:0/771119378 >> 172.21.15.164:0/771119378 conn(0x7f20580cf770 msgr2=0x7f20580cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 -- 172.21.15.164:0/771119378 shutdown_connections 2024-05-08T04:41:34.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.964+0000 7f205f800640 1 -- 172.21.15.164:0/771119378 wait complete. 2024-05-08T04:41:34.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 Processor -- start 2024-05-08T04:41:34.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- start start 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c0040e0 0x7f2058131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205806d4f0 0x7f205806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580ae4f0 con 0x7f2058070f10 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058144740 con 0x7f203c0040e0 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20580ad890 con 0x7f205806d4f0 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45740/0 (socket says 172.21.15.164:45740) 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 -- 172.21.15.164:0/1140081224 learned_addr learned my addr 172.21.15.164:0/1140081224 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205ce00640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c0040e0 0x7f2058131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f2057e00640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205806d4f0 0x7f205806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 -- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205806d4f0 msgr2=0x7f205806d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205806d4f0 0x7f205806d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 -- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c0040e0 msgr2=0x7f2058131d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:34.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c0040e0 0x7f2058131d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:34.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 -- 172.21.15.164:0/1140081224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20581625b0 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205d800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2058070320 tx=0x7f2050012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050010910 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205001b070 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050010d10 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2058132260 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.968+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2058071ee0 con 0x7f2058070f10 2024-05-08T04:41:34.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058072d30 con 0x7f2058070f10 2024-05-08T04:41:34.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f2050016480 con 0x7f2058070f10 2024-05-08T04:41:34.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f2054c00640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 0x7f20380699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:34.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2050088a80 con 0x7f2058070f10 2024-05-08T04:41:34.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f205ce00640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 0x7f20380699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:34.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.972+0000 7f205ce00640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 0x7f20380699c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2048014430 tx=0x7f204800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:34.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:34.976+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2058132260 con 0x7f2058070f10 2024-05-08T04:41:35.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.140+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f205804f010 con 0x7f2058070f10 2024-05-08T04:41:35.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f2054c00640 1 -- 172.21.15.164:0/1140081224 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2050058ee0 con 0x7f2058070f10 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 msgr2=0x7f20380699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 0x7f20380699c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2048014430 tx=0x7f204800b040 comp rx=0 tx=0).stop 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 msgr2=0x7f20580713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2058070320 tx=0x7f2050012c90 comp rx=0 tx=0).stop 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 shutdown_connections 2024-05-08T04:41:35.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2038067500 0x7f20380699c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f205806d4f0 0x7f205806d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2058070f10 0x7f20580713c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 --2- 172.21.15.164:0/1140081224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203c0040e0 0x7f2058131d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 >> 172.21.15.164:0/1140081224 conn(0x7f20580cf770 msgr2=0x7f20581605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 shutdown_connections 2024-05-08T04:41:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.144+0000 7f205f800640 1 -- 172.21.15.164:0/1140081224 wait complete. 2024-05-08T04:41:35.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2024-05-08T04:41:35.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 Processor -- start 2024-05-08T04:41:35.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 -- start start 2024-05-08T04:41:35.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f470415ab60 0x7f4704072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 0x7f47040cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4704072640 con 0x7f47040b26c0 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470406c770 con 0x7f470415ab60 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f470b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470406ca50 con 0x7f47040cfda0 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f470415ab60 0x7f4704072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f470415ab60 0x7f4704072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45744/0 (socket says 172.21.15.164:45744) 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 -- 172.21.15.164:0/942078955 learned_addr learned my addr 172.21.15.164:0/942078955 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4709200640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 0x7f47040cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 -- 172.21.15.164:0/942078955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 msgr2=0x7f47040cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 0x7f47040cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 -- 172.21.15.164:0/942078955 >> v1:172.21.15.172:6789/0 conn(0x7f47040b26c0 legacy=0x7f47040cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4703e00640 1 -- 172.21.15.164:0/942078955 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470406cd30 con 0x7f470415ab60 2024-05-08T04:41:35.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.308+0000 7f4709200640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 0x7f47040cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:35.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f4703e00640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f4704072100 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f46f400d150 tx=0x7f46f401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae767b9f8f851ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f4703400640 1 -- 172.21.15.164:0/942078955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400b2d0 con 0x7f470415ab60 2024-05-08T04:41:35.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f4703400640 1 -- 172.21.15.164:0/942078955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f400b470 con 0x7f470415ab60 2024-05-08T04:41:35.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f4703400640 1 -- 172.21.15.164:0/942078955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f40a7b20 con 0x7f470415ab60 2024-05-08T04:41:35.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 -- 172.21.15.164:0/942078955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 msgr2=0x7f4704072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f4704072100 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f46f400d150 tx=0x7f46f401ec30 comp rx=0 tx=0).stop 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 -- 172.21.15.164:0/942078955 shutdown_connections 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040cfda0 0x7f47040cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 --2- 172.21.15.164:0/942078955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f4704072100 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 -- 172.21.15.164:0/942078955 >> 172.21.15.164:0/942078955 conn(0x7f470406ac90 msgr2=0x7f470406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 -- 172.21.15.164:0/942078955 shutdown_connections 2024-05-08T04:41:35.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.312+0000 7f470b200640 1 -- 172.21.15.164:0/942078955 wait complete. 2024-05-08T04:41:35.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 Processor -- start 2024-05-08T04:41:35.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- start start 2024-05-08T04:41:35.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040b26c0 0x7f4704148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47040cfda0 0x7f470415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040aeeb0 con 0x7f470415ab60 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040aed30 con 0x7f47040cfda0 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040cc1b0 con 0x7f47040b26c0 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45748/0 (socket says 172.21.15.164:45748) 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4703e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040b26c0 0x7f4704148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 -- 172.21.15.164:0/1749607694 learned_addr learned my addr 172.21.15.164:0/1749607694 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f46fb400640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47040cfda0 0x7f470415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 -- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040b26c0 msgr2=0x7f4704148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040b26c0 0x7f4704148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 -- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47040cfda0 msgr2=0x7f470415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47040cfda0 0x7f470415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 -- 172.21.15.164:0/1749607694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704148d80 con 0x7f470415ab60 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4709200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e810 tx=0x7f46fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc017070 con 0x7f470415ab60 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc00c6d0 con 0x7f470415ab60 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4704159950 con 0x7f470415ab60 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00f040 con 0x7f470415ab60 2024-05-08T04:41:35.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.316+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4704159f10 con 0x7f470415ab60 2024-05-08T04:41:35.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4704069d60 con 0x7f470415ab60 2024-05-08T04:41:35.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f46fc01a070 con 0x7f470415ab60 2024-05-08T04:41:35.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f4700c00640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 0x7f46e00699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f46fc0853d0 con 0x7f470415ab60 2024-05-08T04:41:35.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f4703e00640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 0x7f46e00699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.320+0000 7f4703e00640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 0x7f46e00699c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46f40b0000 tx=0x7f46f40a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.324+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46fc0556d0 con 0x7f470415ab60 2024-05-08T04:41:35.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.532+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f470406d9a0 con 0x7f470415ab60 2024-05-08T04:41:35.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.804+0000 7f4700c00640 1 -- 172.21.15.164:0/1749607694 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v202) v1 ==== 166+0+0 (secure 0 0 0) 0x7f46fc059770 con 0x7f470415ab60 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 msgr2=0x7f46e00699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 0x7f46e00699c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46f40b0000 tx=0x7f46f40a9070 comp rx=0 tx=0).stop 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 msgr2=0x7f470415daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e810 tx=0x7f46fc00ecd0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 shutdown_connections 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46e0067500 0x7f46e00699c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f470415ab60 0x7f470415daa0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47040cfda0 0x7f470415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 --2- 172.21.15.164:0/1749607694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47040b26c0 0x7f4704148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 >> 172.21.15.164:0/1749607694 conn(0x7f470406ac90 msgr2=0x7f47040afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:35.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 shutdown_connections 2024-05-08T04:41:35.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.808+0000 7f470b200640 1 -- 172.21.15.164:0/1749607694 wait complete. 2024-05-08T04:41:35.831 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2024-05-08T04:41:35.831 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:41:35.831 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:35.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.960+0000 7fb2e3a00640 1 Processor -- start 2024-05-08T04:41:35.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.960+0000 7fb2e3a00640 1 -- start start 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e3a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2dc15ab60 0x7fb2dc072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0b26c0 0x7fb2dc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e3a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc072640 con 0x7fb2dc0cfbd0 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e3a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc0d0060 con 0x7fb2dc15ab60 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc06c770 con 0x7fb2dc0b26c0 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2dc15ab60 0x7fb2dc072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2dc15ab60 0x7fb2dc072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45750/0 (socket says 172.21.15.164:45750) 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 -- 172.21.15.164:0/157187381 learned_addr learned my addr 172.21.15.164:0/157187381 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2dbe00640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0b26c0 0x7fb2dc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 -- 172.21.15.164:0/157187381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0b26c0 msgr2=0x7fb2dc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0b26c0 0x7fb2dc0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 -- 172.21.15.164:0/157187381 >> v1:172.21.15.172:6789/0 conn(0x7fb2dc0cfbd0 legacy=0x7fb2dc0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:35.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.964+0000 7fb2e1000640 1 -- 172.21.15.164:0/157187381 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc06ca50 con 0x7fb2dc15ab60 2024-05-08T04:41:35.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e1000640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc072100 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009870 tx=0x7fb2d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c6bd242654d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2db400640 1 -- 172.21.15.164:0/157187381 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00b2070 con 0x7fb2dc15ab60 2024-05-08T04:41:35.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2db400640 1 -- 172.21.15.164:0/157187381 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d00a9070 con 0x7fb2dc15ab60 2024-05-08T04:41:35.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2db400640 1 -- 172.21.15.164:0/157187381 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00ad880 con 0x7fb2dc15ab60 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- 172.21.15.164:0/157187381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 msgr2=0x7fb2dc072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc072100 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009870 tx=0x7fb2d001c560 comp rx=0 tx=0).stop 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- 172.21.15.164:0/157187381 shutdown_connections 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0b26c0 0x7fb2dc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/157187381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc072100 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- 172.21.15.164:0/157187381 >> 172.21.15.164:0/157187381 conn(0x7fb2dc06ac90 msgr2=0x7fb2dc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- 172.21.15.164:0/157187381 shutdown_connections 2024-05-08T04:41:35.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- 172.21.15.164:0/157187381 wait complete. 2024-05-08T04:41:35.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 Processor -- start 2024-05-08T04:41:35.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.968+0000 7fb2e3a00640 1 -- start start 2024-05-08T04:41:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0b26c0 0x7fb2dc14b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0cfbd0 0x7fb2dc0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc06c550 con 0x7fb2dc15ab60 2024-05-08T04:41:35.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc06c3d0 con 0x7fb2dc0b26c0 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0aed30 con 0x7fb2dc0cfbd0 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2dbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0cfbd0 0x7fb2dc0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0b26c0 0x7fb2dc14b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45762/0 (socket says 172.21.15.164:45762) 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 -- 172.21.15.164:0/4101464255 learned_addr learned my addr 172.21.15.164:0/4101464255 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 -- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0cfbd0 msgr2=0x7fb2dc0afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0cfbd0 0x7fb2dc0afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 -- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0b26c0 msgr2=0x7fb2dc14b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0b26c0 0x7fb2dc14b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 -- 172.21.15.164:0/4101464255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc14b7b0 con 0x7fb2dc15ab60 2024-05-08T04:41:35.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e1a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c7a0 tx=0x7fb2d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4003b80 con 0x7fb2dc15ab60 2024-05-08T04:41:35.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d4003d20 con 0x7fb2dc15ab60 2024-05-08T04:41:35.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc157190 con 0x7fb2dc15ab60 2024-05-08T04:41:35.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400aa90 con 0x7fb2dc15ab60 2024-05-08T04:41:35.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc157750 con 0x7fb2dc15ab60 2024-05-08T04:41:35.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc069d60 con 0x7fb2dc15ab60 2024-05-08T04:41:35.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.972+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fb2d400ac30 con 0x7fb2dc15ab60 2024-05-08T04:41:35.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.976+0000 7fb2d8c00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 0x7fb2bc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:35.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.976+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fb2d4085560 con 0x7fb2dc15ab60 2024-05-08T04:41:35.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.976+0000 7fb2e1000640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 0x7fb2bc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:35.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.976+0000 7fb2e1000640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 0x7fb2bc0699c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00af000 tx=0x7fb2d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:35.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:35.980+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2d40559c0 con 0x7fb2dc15ab60 2024-05-08T04:41:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.132+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2dc0af780 con 0x7fb2dc15ab60 2024-05-08T04:41:36.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2d8c00640 1 -- 172.21.15.164:0/4101464255 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13169 (secure 0 0 0) 0x7fb2d4059a60 con 0x7fb2dc15ab60 2024-05-08T04:41:36.353 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:36.353 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":78,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:34.796093+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/239509813":"2024-05-09T04:41:31.790129+0000","172.21.15.164:6834/239509813":"2024-05-09T04:41:31.790129+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:36.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 msgr2=0x7fb2bc0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 0x7fb2bc0699c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00af000 tx=0x7fb2d0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 msgr2=0x7fb2dc0b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c7a0 tx=0x7fb2d400cc60 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 shutdown_connections 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb2bc067500 0x7fb2bc0699c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0cfbd0 0x7fb2dc0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2dc15ab60 0x7fb2dc0b0280 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 --2- 172.21.15.164:0/4101464255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0b26c0 0x7fb2dc14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 >> 172.21.15.164:0/4101464255 conn(0x7fb2dc06ac90 msgr2=0x7fb2dc0cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 shutdown_connections 2024-05-08T04:41:36.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.136+0000 7fb2e3a00640 1 -- 172.21.15.164:0/4101464255 wait complete. 2024-05-08T04:41:36.356 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:36.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 Processor -- start 2024-05-08T04:41:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 -- start start 2024-05-08T04:41:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea4815ab10 0x7fea4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fea480cee90 0x7fea480aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4815e870 con 0x7fea480ae6e0 2024-05-08T04:41:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4806bb80 con 0x7fea480cee90 2024-05-08T04:41:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4f800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4815eb50 con 0x7fea4815ab10 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fea480cee90 0x7fea480aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fea480cee90 0x7fea480aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45776/0 (socket says 172.21.15.164:45776) 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 -- 172.21.15.164:0/3455620984 learned_addr learned my addr 172.21.15.164:0/3455620984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4ce00640 1 --2- 172.21.15.164:0/3455620984 >> v2:172.21.15.172:3300/0 conn(0x7fea4815ab10 0x7fea4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 -- 172.21.15.164:0/3455620984 >> v2:172.21.15.172:3300/0 conn(0x7fea4815ab10 msgr2=0x7fea4806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 --2- 172.21.15.164:0/3455620984 >> v2:172.21.15.172:3300/0 conn(0x7fea4815ab10 0x7fea4806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 -- 172.21.15.164:0/3455620984 >> v1:172.21.15.164:6789/0 conn(0x7fea480ae6e0 legacy=0x7fea480ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea4d800640 1 -- 172.21.15.164:0/3455620984 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4815ee30 con 0x7fea480cee90 2024-05-08T04:41:36.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.492+0000 7fea47400640 1 -- 172.21.15.164:0/3455620984 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210000176 0 0) 0x7fea4815e870 con 0x7fea480ae6e0 2024-05-08T04:41:36.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4d800640 1 --2- 172.21.15.164:0/3455620984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480cee90 0x7fea480aeb70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fea480c4920 tx=0x7fea4001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac80d7a7cbcf567 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea47400640 1 -- 172.21.15.164:0/3455620984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a57f0 con 0x7fea480cee90 2024-05-08T04:41:36.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea47400640 1 -- 172.21.15.164:0/3455620984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea400a5990 con 0x7fea480cee90 2024-05-08T04:41:36.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea47400640 1 -- 172.21.15.164:0/3455620984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a5d60 con 0x7fea480cee90 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- 172.21.15.164:0/3455620984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480cee90 msgr2=0x7fea480aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 --2- 172.21.15.164:0/3455620984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480cee90 0x7fea480aeb70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fea480c4920 tx=0x7fea4001c930 comp rx=0 tx=0).stop 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- 172.21.15.164:0/3455620984 shutdown_connections 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 --2- 172.21.15.164:0/3455620984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480cee90 0x7fea480aeb70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 --2- 172.21.15.164:0/3455620984 >> v2:172.21.15.172:3300/0 conn(0x7fea4815ab10 0x7fea4806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- 172.21.15.164:0/3455620984 >> 172.21.15.164:0/3455620984 conn(0x7fea480cf770 msgr2=0x7fea480cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- 172.21.15.164:0/3455620984 shutdown_connections 2024-05-08T04:41:36.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- 172.21.15.164:0/3455620984 wait complete. 2024-05-08T04:41:36.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 Processor -- start 2024-05-08T04:41:36.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.496+0000 7fea4f800640 1 -- start start 2024-05-08T04:41:36.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 0x7fea48155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 0x7fea480b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea480cb070 con 0x7fea480ae6e0 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea480caef0 con 0x7fea4815ab10 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480cb1f0 con 0x7fea480cee90 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45788/0 (socket says 172.21.15.164:45788) 2024-05-08T04:41:36.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 -- 172.21.15.164:0/1476565138 learned_addr learned my addr 172.21.15.164:0/1476565138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:36.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4d800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 0x7fea480b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea47e00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 0x7fea48155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 -- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 msgr2=0x7fea48155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 0x7fea48155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 -- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 msgr2=0x7fea480b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 0x7fea480b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 -- 172.21.15.164:0/1476565138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48156170 con 0x7fea480ae6e0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4d800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 0x7fea480b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea47e00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 0x7fea48155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4ce00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00d620 tx=0x7fea3c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0125f0 con 0x7fea480ae6e0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c019070 con 0x7fea480ae6e0 2024-05-08T04:41:36.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0129f0 con 0x7fea480ae6e0 2024-05-08T04:41:36.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea480b0f00 con 0x7fea480ae6e0 2024-05-08T04:41:36.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea48154e70 con 0x7fea480ae6e0 2024-05-08T04:41:36.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.500+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48072d30 con 0x7fea480ae6e0 2024-05-08T04:41:36.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.504+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fea3c012b90 con 0x7fea480ae6e0 2024-05-08T04:41:36.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.504+0000 7fea44c00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 0x7fea280699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.504+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fea3c085b50 con 0x7fea480ae6e0 2024-05-08T04:41:36.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.504+0000 7fea47e00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 0x7fea280699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.504+0000 7fea47e00640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 0x7fea280699c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea480b0730 tx=0x7fea34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.508+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea48156170 con 0x7fea480ae6e0 2024-05-08T04:41:36.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea4804f010 con 0x7fea480ae6e0 2024-05-08T04:41:36.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea44c00640 1 -- 172.21.15.164:0/1476565138 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea3c055f60 con 0x7fea480ae6e0 2024-05-08T04:41:36.668 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:36.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 msgr2=0x7fea280699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 0x7fea280699c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea480b0730 tx=0x7fea34009040 comp rx=0 tx=0).stop 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 msgr2=0x7fea4814c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00d620 tx=0x7fea3c00dae0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 shutdown_connections 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fea28067500 0x7fea280699c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480cee90 0x7fea48155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4815ab10 0x7fea480b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 --2- 172.21.15.164:0/1476565138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea480ae6e0 0x7fea4814c830 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 >> 172.21.15.164:0/1476565138 conn(0x7fea480cf770 msgr2=0x7fea480cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:36.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 shutdown_connections 2024-05-08T04:41:36.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.664+0000 7fea4f800640 1 -- 172.21.15.164:0/1476565138 wait complete. 2024-05-08T04:41:36.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2024-05-08T04:41:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.820+0000 7f4000400640 1 Processor -- start 2024-05-08T04:41:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.820+0000 7f4000400640 1 -- start start 2024-05-08T04:41:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f4000400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f4000400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ff815be50 0x7f3ff80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f4000400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff80cd9f0 con 0x7f3ff815ba70 2024-05-08T04:41:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f4000400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff815fe70 con 0x7f3ff815be50 2024-05-08T04:41:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f4000400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80cdcd0 con 0x7f3ff80cfda0 2024-05-08T04:41:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffda00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3ff815ba70 0x7f3ff80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39996/0 (socket says 172.21.15.164:39996) 2024-05-08T04:41:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffda00640 1 -- 172.21.15.164:0/4261549811 learned_addr learned my addr 172.21.15.164:0/4261549811 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffd000640 1 --2- 172.21.15.164:0/4261549811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffe400640 1 --2- 172.21.15.164:0/4261549811 >> v2:172.21.15.172:3300/0 conn(0x7f3ff815be50 0x7f3ff80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143878334 0 0) 0x7f3ff80cd9f0 con 0x7f3ff815ba70 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe0003530 con 0x7f3ff815ba70 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffd000640 1 -- 172.21.15.164:0/4261549811 >> v2:172.21.15.172:3300/0 conn(0x7f3ff815be50 msgr2=0x7f3ff80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffd000640 1 --2- 172.21.15.164:0/4261549811 >> v2:172.21.15.172:3300/0 conn(0x7f3ff815be50 0x7f3ff80b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231502599 0 0) 0x7f3fe0003530 con 0x7f3ff815ba70 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffd000640 1 -- 172.21.15.164:0/4261549811 >> v1:172.21.15.164:6789/0 conn(0x7f3ff815ba70 legacy=0x7f3ff80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.824+0000 7f3ffd000640 1 -- 172.21.15.164:0/4261549811 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff815f820 con 0x7f3ff80cfda0 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f3ffd000640 1 --2- 172.21.15.164:0/4261549811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8160300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40097c0 tx=0x7f3fe401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731e1b952ed2f665 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b0070 con 0x7f3ff80cfda0 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe40ab400 con 0x7f3ff80cfda0 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f3ff7e00640 1 -- 172.21.15.164:0/4261549811 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec001680 con 0x7f3ff80cfda0 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- 172.21.15.164:0/4261549811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 msgr2=0x7f3ff8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 --2- 172.21.15.164:0/4261549811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8160300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40097c0 tx=0x7f3fe401c390 comp rx=0 tx=0).stop 2024-05-08T04:41:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- 172.21.15.164:0/4261549811 shutdown_connections 2024-05-08T04:41:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 --2- 172.21.15.164:0/4261549811 >> v2:172.21.15.172:3300/0 conn(0x7f3ff815be50 0x7f3ff80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 --2- 172.21.15.164:0/4261549811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8160300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- 172.21.15.164:0/4261549811 >> 172.21.15.164:0/4261549811 conn(0x7f3ff806ac90 msgr2=0x7f3ff806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- 172.21.15.164:0/4261549811 shutdown_connections 2024-05-08T04:41:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- 172.21.15.164:0/4261549811 wait complete. 2024-05-08T04:41:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 Processor -- start 2024-05-08T04:41:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.828+0000 7f4000400640 1 -- start start 2024-05-08T04:41:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ff815ba70 0x7f3ff81f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8168d40 con 0x7f3ff815be50 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8168bc0 con 0x7f3ff815ba70 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8168ec0 con 0x7f3ff80cfda0 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45796/0 (socket says 172.21.15.164:45796) 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 -- 172.21.15.164:0/3654159678 learned_addr learned my addr 172.21.15.164:0/3654159678 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffd000640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ff815ba70 0x7f3ff81f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffda00640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 -- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 msgr2=0x7f3ff8071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 -- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ff815ba70 msgr2=0x7f3ff81f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ff815ba70 0x7f3ff81f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 -- 172.21.15.164:0/3654159678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff81f8860 con 0x7f3ff815be50 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ffe400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009d30 tx=0x7f3ff000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0018020 con 0x7f3ff815be50 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0014480 con 0x7f3ff815be50 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0014870 con 0x7f3ff815be50 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff8149730 con 0x7f3ff815be50 2024-05-08T04:41:36.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff80b08c0 con 0x7f3ff815be50 2024-05-08T04:41:36.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8067820 con 0x7f3ff815be50 2024-05-08T04:41:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.832+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f3ff001b020 con 0x7f3ff815be50 2024-05-08T04:41:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.836+0000 7f3ff5600640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 0x7f3fd80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.836+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f3ff00862f0 con 0x7f3ff815be50 2024-05-08T04:41:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.836+0000 7f3ffda00640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 0x7f3fd80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.836+0000 7f3ffda00640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 0x7f3fd80699c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007a10 tx=0x7f3fec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:36.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:36.840+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ff0022070 con 0x7f3ff815be50 2024-05-08T04:41:37.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.048+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f3ff8069d60 con 0x7f3ff815be50 2024-05-08T04:41:37.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.812+0000 7f3ff5600640 1 -- 172.21.15.164:0/3654159678 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v204) v1 ==== 167+0+0 (secure 0 0 0) 0x7f3ff0010040 con 0x7f3ff815be50 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 msgr2=0x7f3fd80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 0x7f3fd80699c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007a10 tx=0x7f3fec009040 comp rx=0 tx=0).stop 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 msgr2=0x7f3ff8159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009d30 tx=0x7f3ff000c490 comp rx=0 tx=0).stop 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 shutdown_connections 2024-05-08T04:41:37.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3fd8067500 0x7f3fd80699c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ff815be50 0x7f3ff8159550 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ff815ba70 0x7f3ff81f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 --2- 172.21.15.164:0/3654159678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ff80cfda0 0x7f3ff8071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 >> 172.21.15.164:0/3654159678 conn(0x7f3ff806ac90 msgr2=0x7f3ff806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 shutdown_connections 2024-05-08T04:41:37.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.816+0000 7f4000400640 1 -- 172.21.15.164:0/3654159678 wait complete. 2024-05-08T04:41:37.841 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:41:37.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 Processor -- start 2024-05-08T04:41:37.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 -- start start 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b0c0af200 0x7f0b0c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0b49b0 0x7f0b0c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b0c0cff30 0x7f0b0c161960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0b4e10 con 0x7f0b0c0af200 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0d0720 con 0x7f0b0c0cff30 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b11400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c15f2d0 con 0x7f0b0c0b49b0 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b0c0af200 0x7f0b0c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b0c0af200 0x7f0b0c0b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45810/0 (socket says 172.21.15.164:45810) 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 -- 172.21.15.164:0/468457888 learned_addr learned my addr 172.21.15.164:0/468457888 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:37.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 -- 172.21.15.164:0/468457888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0b49b0 msgr2=0x7f0b0c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 --2- 172.21.15.164:0/468457888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0b49b0 0x7f0b0c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 -- 172.21.15.164:0/468457888 >> v2:172.21.15.172:3300/0 conn(0x7f0b0c0cff30 msgr2=0x7f0b0c161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 --2- 172.21.15.164:0/468457888 >> v2:172.21.15.172:3300/0 conn(0x7f0b0c0cff30 0x7f0b0c161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.976+0000 7f0b0a800640 1 -- 172.21.15.164:0/468457888 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c15e510 con 0x7f0b0c0af200 2024-05-08T04:41:37.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b0a800640 1 --2- 172.21.15.164:0/468457888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b4470 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b5030 tx=0x7f0afc01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f2caecca086753 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:37.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b09e00640 1 -- 172.21.15.164:0/468457888 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0ac070 con 0x7f0b0c0af200 2024-05-08T04:41:37.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b09e00640 1 -- 172.21.15.164:0/468457888 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc0a8420 con 0x7f0b0c0af200 2024-05-08T04:41:37.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b09e00640 1 -- 172.21.15.164:0/468457888 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a87f0 con 0x7f0b0c0af200 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 -- 172.21.15.164:0/468457888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 msgr2=0x7f0b0c0b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 --2- 172.21.15.164:0/468457888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b4470 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b5030 tx=0x7f0afc01c730 comp rx=0 tx=0).stop 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 -- 172.21.15.164:0/468457888 shutdown_connections 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 --2- 172.21.15.164:0/468457888 >> v2:172.21.15.172:3300/0 conn(0x7f0b0c0cff30 0x7f0b0c161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 --2- 172.21.15.164:0/468457888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0b49b0 0x7f0b0c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 --2- 172.21.15.164:0/468457888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b4470 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 -- 172.21.15.164:0/468457888 >> 172.21.15.164:0/468457888 conn(0x7f0b0c06ac90 msgr2=0x7f0b0c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:37.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 -- 172.21.15.164:0/468457888 shutdown_connections 2024-05-08T04:41:37.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.980+0000 7f0b11400640 1 -- 172.21.15.164:0/468457888 wait complete. 2024-05-08T04:41:37.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 Processor -- start 2024-05-08T04:41:37.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- start start 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c0b49b0 0x7f0b0c149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0cff30 0x7f0b0c149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0d0360 con 0x7f0b0c0af200 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0af5e0 con 0x7f0b0c0b49b0 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0d04e0 con 0x7f0b0c0cff30 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45818/0 (socket says 172.21.15.164:45818) 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 -- 172.21.15.164:0/1281182067 learned_addr learned my addr 172.21.15.164:0/1281182067 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 -- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0cff30 msgr2=0x7f0b0c149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:37.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0cff30 0x7f0b0c149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 -- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c0b49b0 msgr2=0x7f0b0c149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:37.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c0b49b0 0x7f0b0c149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:37.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 -- 172.21.15.164:0/1281182067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c150e00 con 0x7f0b0c0af200 2024-05-08T04:41:37.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b0a800640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01ade0 tx=0x7f0afc0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0ac040 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc0bd650 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1510e0 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0bda40 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1516a0 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.984+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c04f010 con 0x7f0b0c0af200 2024-05-08T04:41:37.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.988+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f0afc0af070 con 0x7f0b0c0af200 2024-05-08T04:41:37.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.988+0000 7f0b03400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 0x7f0ae80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:37.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.988+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0afc120410 con 0x7f0b0c0af200 2024-05-08T04:41:37.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.988+0000 7f0b01e00640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 0x7f0ae80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:37.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.988+0000 7f0b01e00640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 0x7f0ae80699c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c157d10 tx=0x7f0af4006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:37.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:37.992+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0afc0165e0 con 0x7f0b0c0af200 2024-05-08T04:41:38.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.144+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b0c072750 con 0x7f0b0c0af200 2024-05-08T04:41:38.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.148+0000 7f0b03400640 1 -- 172.21.15.164:0/1281182067 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0afc0f0e30 con 0x7f0b0c0af200 2024-05-08T04:41:38.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 msgr2=0x7f0ae80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 0x7f0ae80699c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c157d10 tx=0x7f0af4006ca0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 msgr2=0x7f0b0c0b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01ade0 tx=0x7f0afc0194d0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 shutdown_connections 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ae8067500 0x7f0ae80699c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b0c0cff30 0x7f0b0c149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c0b49b0 0x7f0b0c149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 --2- 172.21.15.164:0/1281182067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c0af200 0x7f0b0c0b2c50 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 >> 172.21.15.164:0/1281182067 conn(0x7f0b0c06ac90 msgr2=0x7f0b0c15c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 shutdown_connections 2024-05-08T04:41:38.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.152+0000 7f0b11400640 1 -- 172.21.15.164:0/1281182067 wait complete. 2024-05-08T04:41:38.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:41:38.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:38.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:41:38.345 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:41:38.345 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:38.345 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:38.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:41:38.354 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:38.354 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:38.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:41:38.363 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:38.363 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:38.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:41:38.403 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:38.403 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:38.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:41:38.418 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:41:38.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:38.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 Processor -- start 2024-05-08T04:41:38.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 -- start start 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc161fe0 con 0x7fbddc144200 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc1622c0 con 0x7fbddc161950 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbde0c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc162880 con 0x7fbddc0cfd60 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbdda800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbdd9e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbddc144200 0x7fbddc0cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40010/0 (socket says 172.21.15.164:40010) 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbdd9e00640 1 -- 172.21.15.164:0/2472546303 learned_addr learned my addr 172.21.15.164:0/2472546303 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:38.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.572+0000 7fbdda800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45824/0 (socket says 172.21.15.164:45824) 2024-05-08T04:41:38.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdda800640 1 -- 172.21.15.164:0/2472546303 >> v1:172.21.15.172:6789/0 conn(0x7fbddc161950 legacy=0x7fbddc15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdda800640 1 -- 172.21.15.164:0/2472546303 >> v1:172.21.15.164:6789/0 conn(0x7fbddc144200 legacy=0x7fbddc0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdda800640 1 -- 172.21.15.164:0/2472546303 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc1625a0 con 0x7fbddc0cfd60 2024-05-08T04:41:38.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdd9400640 1 -- 172.21.15.164:0/2472546303 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956927815 0 0) 0x7fbddc161fe0 con 0x7fbddc144200 2024-05-08T04:41:38.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdda800640 1 --2- 172.21.15.164:0/2472546303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40097c0 tx=0x7fbdd401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3258d5e6034abb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:38.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdd9400640 1 -- 172.21.15.164:0/2472546303 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40b0070 con 0x7fbddc0cfd60 2024-05-08T04:41:38.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdd9400640 1 -- 172.21.15.164:0/2472546303 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd40ab600 con 0x7fbddc0cfd60 2024-05-08T04:41:38.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbdd9400640 1 -- 172.21.15.164:0/2472546303 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40ab9d0 con 0x7fbddc0cfd60 2024-05-08T04:41:38.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 -- 172.21.15.164:0/2472546303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0cfd60 msgr2=0x7fbddc0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 --2- 172.21.15.164:0/2472546303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40097c0 tx=0x7fbdd401c410 comp rx=0 tx=0).stop 2024-05-08T04:41:38.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 -- 172.21.15.164:0/2472546303 shutdown_connections 2024-05-08T04:41:38.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 --2- 172.21.15.164:0/2472546303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0cfd60 0x7fbddc0ce9e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 -- 172.21.15.164:0/2472546303 >> 172.21.15.164:0/2472546303 conn(0x7fbddc06ac90 msgr2=0x7fbddc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:38.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.576+0000 7fbde0c00640 1 -- 172.21.15.164:0/2472546303 shutdown_connections 2024-05-08T04:41:38.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- 172.21.15.164:0/2472546303 wait complete. 2024-05-08T04:41:38.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 Processor -- start 2024-05-08T04:41:38.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- start start 2024-05-08T04:41:38.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbddc0cfd60 0x7fbddc13b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:38.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:38.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbddc161950 0x7fbddc140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0cc7c0 con 0x7fbddc144200 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc161d80 con 0x7fbddc0cfd60 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0cc940 con 0x7fbddc161950 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45832/0 (socket says 172.21.15.164:45832) 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 -- 172.21.15.164:0/1355526029 learned_addr learned my addr 172.21.15.164:0/1355526029 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd9e00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbddc0cfd60 0x7fbddc13b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdda800640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbddc161950 0x7fbddc140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 -- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbddc161950 msgr2=0x7fbddc140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbddc161950 0x7fbddc140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 -- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbddc0cfd60 msgr2=0x7fbddc13b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbddc0cfd60 0x7fbddc13b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 -- 172.21.15.164:0/1355526029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc13b690 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd1400640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c7f0 tx=0x7fbdc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4003b80 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc4003d20 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0b2550 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400aa90 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.580+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc0b2b10 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc069d60 con 0x7fbddc144200 2024-05-08T04:41:38.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fbdc400ac30 con 0x7fbddc144200 2024-05-08T04:41:38.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbdd2a00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 0x7fbdb80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:38.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fbdc40857c0 con 0x7fbddc144200 2024-05-08T04:41:38.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbdd9e00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 0x7fbdb80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:38.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.584+0000 7fbdd9e00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 0x7fbdb80699c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc007fd0 tx=0x7fbdcc007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:38.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.588+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdc40561e0 con 0x7fbddc144200 2024-05-08T04:41:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.776+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbddc0add30 con 0x7fbddc144200 2024-05-08T04:41:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.776+0000 7fbdd2a00640 1 -- 172.21.15.164:0/1355526029 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fbdc4055ae0 con 0x7fbddc144200 2024-05-08T04:41:38.783 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:38.783 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1910284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1910284},{"type":"v1","addr":"172.21.15.172:6834","nonce":1910284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4138743834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4138743834},{"type":"v1","addr":"172.21.15.172:6835","nonce":4138743834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3847701140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":3847701140},{"type":"v1","addr":"172.21.15.164:6836","nonce":3847701140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2563870132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2563870132},{"type":"v1","addr":"172.21.15.164:6837","nonce":2563870132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-05-08T04:41:38.783 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 46 2024-05-08T04:41:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 msgr2=0x7fbdb80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 0x7fbdb80699c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc007fd0 tx=0x7fbdcc007ba0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 msgr2=0x7fbddc0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c7f0 tx=0x7fbdc400ccb0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 shutdown_connections 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbdb8067500 0x7fbdb80699c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbddc161950 0x7fbddc140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc144200 0x7fbddc0b35f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 --2- 172.21.15.164:0/1355526029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbddc0cfd60 0x7fbddc13b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 >> 172.21.15.164:0/1355526029 conn(0x7fbddc06ac90 msgr2=0x7fbddc070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 shutdown_connections 2024-05-08T04:41:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:38.780+0000 7fbde0c00640 1 -- 172.21.15.164:0/1355526029 wait complete. 2024-05-08T04:41:38.803 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:41:38.803 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:41:38.803 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:41:38.804 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:38.804 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:41:38.804 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:41:38.804 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:41:38.804 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:38.804 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:41:38.804 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:41:38.804 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:41:38.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:38.805 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:41:38.805 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:41:38.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:41:38.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:41:38.816 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:41:38.816 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:41:38.817 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:41:38.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:41:38.820 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:41:38.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:41:38.821 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:41:38.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:41:38.865 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:41:38.866 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:38.867 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:41:38.867 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:38.868 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:41:38.868 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:38.870 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:41:38.870 DEBUG:teuthology.parallel:result is None 2024-05-08T04:41:38.870 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:41:38.952 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:41:38.953 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:41:38.955 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:41:38.956 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:41:39.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 Processor -- start 2024-05-08T04:41:39.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 -- start start 2024-05-08T04:41:39.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f67b00b26c0 0x7f67b00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b006c770 con 0x7f67b00cfbd0 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b006ca50 con 0x7f67b015ab60 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67b4a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00d0060 con 0x7f67b00b26c0 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67ae800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f67b00cfbd0 0x7f67b00cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40046/0 (socket says 172.21.15.164:40046) 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67ae800640 1 -- 172.21.15.164:0/4283921468 learned_addr learned my addr 172.21.15.164:0/4283921468 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67a5400640 1 --2- 172.21.15.164:0/4283921468 >> v2:172.21.15.172:3300/0 conn(0x7f67b00b26c0 0x7f67b00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67a5400640 1 -- 172.21.15.164:0/4283921468 >> v1:172.21.15.172:6789/0 conn(0x7f67b015ab60 legacy=0x7f67b0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67a5400640 1 -- 172.21.15.164:0/4283921468 >> v1:172.21.15.164:6789/0 conn(0x7f67b00cfbd0 legacy=0x7f67b00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67ad400640 1 -- 172.21.15.164:0/4283921468 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82153920 0 0) 0x7f67b006c770 con 0x7f67b00cfbd0 2024-05-08T04:41:39.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.012+0000 7f67a5400640 1 -- 172.21.15.164:0/4283921468 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b006cd30 con 0x7f67b00b26c0 2024-05-08T04:41:39.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67a5400640 1 --2- 172.21.15.164:0/4283921468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00b26c0 0x7f67b00cf690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67980097c0 tx=0x7f679801e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aacb58116cf1f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:39.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67ad400640 1 -- 172.21.15.164:0/4283921468 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980ac620 con 0x7f67b00b26c0 2024-05-08T04:41:39.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67ad400640 1 -- 172.21.15.164:0/4283921468 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67980b1070 con 0x7f67b00b26c0 2024-05-08T04:41:39.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67ad400640 1 -- 172.21.15.164:0/4283921468 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980ac9e0 con 0x7f67b00b26c0 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 -- 172.21.15.164:0/4283921468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00b26c0 msgr2=0x7f67b00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 --2- 172.21.15.164:0/4283921468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00b26c0 0x7f67b00cf690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67980097c0 tx=0x7f679801e6b0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 -- 172.21.15.164:0/4283921468 shutdown_connections 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 --2- 172.21.15.164:0/4283921468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00b26c0 0x7f67b00cf690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 -- 172.21.15.164:0/4283921468 >> 172.21.15.164:0/4283921468 conn(0x7f67b006ac90 msgr2=0x7f67b006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 -- 172.21.15.164:0/4283921468 shutdown_connections 2024-05-08T04:41:39.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 -- 172.21.15.164:0/4283921468 wait complete. 2024-05-08T04:41:39.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.016+0000 7f67b4a00640 1 Processor -- start 2024-05-08T04:41:39.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- start start 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 0x7f67b0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 0x7f67b014aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00aeeb0 con 0x7f67b015ab60 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00aed30 con 0x7f67b00cfbd0 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00cc1b0 con 0x7f67b00b26c0 2024-05-08T04:41:39.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45880/0 (socket says 172.21.15.164:45880) 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ae800640 1 -- 172.21.15.164:0/145328450 learned_addr learned my addr 172.21.15.164:0/145328450 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 0x7f67b014aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ade00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 0x7f67b0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 -- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 msgr2=0x7f67b0154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 0x7f67b0154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 -- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 msgr2=0x7f67b015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 -- 172.21.15.164:0/145328450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b014f990 con 0x7f67b00cfbd0 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ae800640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67ade00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 0x7f67b0154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a5400640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 0x7f67b014aa60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f67980ad000 tx=0x7f679801cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980aca80 con 0x7f67b00cfbd0 2024-05-08T04:41:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67980acc60 con 0x7f67b00cfbd0 2024-05-08T04:41:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b014fc70 con 0x7f67b00cfbd0 2024-05-08T04:41:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980b5660 con 0x7f67b00cfbd0 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b014b5c0 con 0x7f67b00cfbd0 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.020+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b015c990 con 0x7f67b00cfbd0 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.024+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f67980c0020 con 0x7f67b00cfbd0 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.024+0000 7f67a6a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 0x7f678c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.024+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f679811ef50 con 0x7f67b00cfbd0 2024-05-08T04:41:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.024+0000 7f67ade00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 0x7f678c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:39.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.024+0000 7f67ade00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 0x7f678c0699c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67b01f9e00 tx=0x7f67a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:39.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.028+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67980182a0 con 0x7f67b00cfbd0 2024-05-08T04:41:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.196+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67b006d9a0 con 0x7f67b00cfbd0 2024-05-08T04:41:39.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.836+0000 7f67a6a00640 1 -- 172.21.15.164:0/145328450 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f67980ef490 con 0x7f67b00cfbd0 2024-05-08T04:41:39.843 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:41:39.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 msgr2=0x7f678c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 0x7f678c0699c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67b01f9e00 tx=0x7f67a0009040 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 msgr2=0x7f67b014aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 0x7f67b014aa60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f67980ad000 tx=0x7f679801cef0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 shutdown_connections 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f678c067500 0x7f678c0699c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67b015ab60 0x7f67b015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67b00cfbd0 0x7f67b014aa60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 --2- 172.21.15.164:0/145328450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67b00b26c0 0x7f67b0154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 >> 172.21.15.164:0/145328450 conn(0x7f67b006ac90 msgr2=0x7f67b015e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 shutdown_connections 2024-05-08T04:41:39.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.840+0000 7f67b4a00640 1 -- 172.21.15.164:0/145328450 wait complete. 2024-05-08T04:41:39.867 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:41:40.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 Processor -- start 2024-05-08T04:41:40.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 -- start start 2024-05-08T04:41:40.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b806bda0 0x7f22b80c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22b80c4ed0 0x7f22b80d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b80c52e0 con 0x7f22b8162050 2024-05-08T04:41:40.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80b4310 con 0x7f22b80c4ed0 2024-05-08T04:41:40.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:39.996+0000 7f22bd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b806c260 con 0x7f22b806bda0 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22b80c4ed0 0x7f22b80d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f22b8162050 0x7f22b8166710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:59448/0 (socket says 172.21.15.164:59448) 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5e00640 1 -- 172.21.15.164:0/2260174122 learned_addr learned my addr 172.21.15.164:0/2260174122 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22afe00640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b806bda0 0x7f22b80c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 -- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b806bda0 msgr2=0x7f22b80c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b806bda0 0x7f22b80c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 -- 172.21.15.164:0/2260174122 >> v1:172.21.15.172:6789/0 conn(0x7f22b8162050 legacy=0x7f22b8166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 -- 172.21.15.164:0/2260174122 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b806c540 con 0x7f22b80c4ed0 2024-05-08T04:41:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5400640 1 -- 172.21.15.164:0/2260174122 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573378219 0 0) 0x7f22a8001120 con 0x7f22b8162050 2024-05-08T04:41:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b6800640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b80d0110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009d90 tx=0x7f22b001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df12e18f24e5174a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5400640 1 -- 172.21.15.164:0/2260174122 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00b2070 con 0x7f22b80c4ed0 2024-05-08T04:41:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5400640 1 -- 172.21.15.164:0/2260174122 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00a5970 con 0x7f22b80c4ed0 2024-05-08T04:41:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.000+0000 7f22b5400640 1 -- 172.21.15.164:0/2260174122 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00a5d40 con 0x7f22b80c4ed0 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 msgr2=0x7f22b80d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b80d0110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009d90 tx=0x7f22b001c8b0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- 172.21.15.164:0/2260174122 shutdown_connections 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b806bda0 0x7f22b80c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- 172.21.15.164:0/2260174122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b80d0110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- 172.21.15.164:0/2260174122 >> 172.21.15.164:0/2260174122 conn(0x7f22b80b34f0 msgr2=0x7f22b80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- 172.21.15.164:0/2260174122 shutdown_connections 2024-05-08T04:41:40.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- 172.21.15.164:0/2260174122 wait complete. 2024-05-08T04:41:40.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 Processor -- start 2024-05-08T04:41:40.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- start start 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b8162050 0x7f22b815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b81635c0 con 0x7f22b806bda0 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8163440 con 0x7f22b80c4ed0 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22bd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80d0e60 con 0x7f22b8162050 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57416/0 (socket says 172.21.15.164:57416) 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 -- 172.21.15.164:0/1343989242 learned_addr learned my addr 172.21.15.164:0/1343989242 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:40.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b6800640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b8162050 0x7f22b815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22afe00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 -- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b8162050 msgr2=0x7f22b815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b8162050 0x7f22b815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 -- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 msgr2=0x7f22b815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.004+0000 7f22b5e00640 1 -- 172.21.15.164:0/1343989242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80cd780 con 0x7f22b806bda0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22b5e00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c4b0 tx=0x7f22a800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8018070 con 0x7f22b806bda0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8007500 con 0x7f22b806bda0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004a30 con 0x7f22b806bda0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b80cda60 con 0x7f22b806bda0 2024-05-08T04:41:40.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b80ce020 con 0x7f22b806bda0 2024-05-08T04:41:40.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b8069d60 con 0x7f22b806bda0 2024-05-08T04:41:40.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f22a8004bd0 con 0x7f22b806bda0 2024-05-08T04:41:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 0x7f22940699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f22a8084910 con 0x7f22b806bda0 2024-05-08T04:41:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.008+0000 7f22afe00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 0x7f22940699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.012+0000 7f22afe00640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 0x7f22940699f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f22b80cb200 tx=0x7f22a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.012+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22b80c52e0 con 0x7f22b806bda0 2024-05-08T04:41:40.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.220+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f22b80bbf60 con 0x7f22b806bda0 2024-05-08T04:41:40.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.220+0000 7f22ae000640 1 -- 172.21.15.164:0/1343989242 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f22a8055410 con 0x7f22b806bda0 2024-05-08T04:41:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:40.227 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:41:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:41:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:41:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:41:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:41:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:41:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:41:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:41:40.241 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 msgr2=0x7f22940699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 0x7f22940699f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f22b80cb200 tx=0x7f22a0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 msgr2=0x7f22b80b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c4b0 tx=0x7f22a800c970 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 shutdown_connections 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2294067530 0x7f22940699f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22b8162050 0x7f22b815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22b80c4ed0 0x7f22b815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 --2- 172.21.15.164:0/1343989242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22b806bda0 0x7f22b80b5fc0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 >> 172.21.15.164:0/1343989242 conn(0x7f22b80b34f0 msgr2=0x7f22b80d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 shutdown_connections 2024-05-08T04:41:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.224+0000 7f22bd000640 1 -- 172.21.15.164:0/1343989242 wait complete. 2024-05-08T04:41:40.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:40.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.380+0000 7f0d82200640 1 Processor -- start 2024-05-08T04:41:40.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.380+0000 7f0d82200640 1 -- start start 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d82200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0d7c159760 0x7f0d7c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d82200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0d7c15fd30 0x7f0d7c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d82200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0c49e0 con 0x7f0d7c0c4420 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d82200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c4cc0 con 0x7f0d7c159760 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d82200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c4fa0 con 0x7f0d7c15fd30 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0d7c159760 0x7f0d7c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0d7c159760 0x7f0d7c0c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57426/0 (socket says 172.21.15.164:57426) 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 -- 172.21.15.164:0/3902089931 learned_addr learned my addr 172.21.15.164:0/3902089931 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d81800640 1 --2- 172.21.15.164:0/3902089931 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c15fd30 0x7f0d7c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 -- 172.21.15.164:0/3902089931 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c15fd30 msgr2=0x7f0d7c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 --2- 172.21.15.164:0/3902089931 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c15fd30 0x7f0d7c15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 -- 172.21.15.164:0/3902089931 >> v1:172.21.15.172:6789/0 conn(0x7f0d7c0c4420 legacy=0x7f0d7c15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.384+0000 7f0d80e00640 1 -- 172.21.15.164:0/3902089931 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0c5280 con 0x7f0d7c159760 2024-05-08T04:41:40.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d80e00640 1 --2- 172.21.15.164:0/3902089931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c159760 0x7f0d7c0c5e70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b1140 tx=0x7f0d7001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc700122e8f89f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d7b400640 1 -- 172.21.15.164:0/3902089931 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700a56c0 con 0x7f0d7c159760 2024-05-08T04:41:40.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d7b400640 1 -- 172.21.15.164:0/3902089931 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d700a5860 con 0x7f0d7c159760 2024-05-08T04:41:40.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d7b400640 1 -- 172.21.15.164:0/3902089931 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700a5c30 con 0x7f0d7c159760 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- 172.21.15.164:0/3902089931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c159760 msgr2=0x7f0d7c0c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 --2- 172.21.15.164:0/3902089931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c159760 0x7f0d7c0c5e70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b1140 tx=0x7f0d7001c9f0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- 172.21.15.164:0/3902089931 shutdown_connections 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 --2- 172.21.15.164:0/3902089931 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c15fd30 0x7f0d7c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 --2- 172.21.15.164:0/3902089931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c159760 0x7f0d7c0c5e70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- 172.21.15.164:0/3902089931 >> 172.21.15.164:0/3902089931 conn(0x7f0d7c06b670 msgr2=0x7f0d7c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- 172.21.15.164:0/3902089931 shutdown_connections 2024-05-08T04:41:40.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- 172.21.15.164:0/3902089931 wait complete. 2024-05-08T04:41:40.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 Processor -- start 2024-05-08T04:41:40.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.388+0000 7f0d82200640 1 -- start start 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c159760 0x7f0d7c0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 0x7f0d7c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15e830 con 0x7f0d7c0c4420 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15e6b0 con 0x7f0d7c15fd30 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15e9b0 con 0x7f0d7c159760 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57436/0 (socket says 172.21.15.164:57436) 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 -- 172.21.15.164:0/1030762326 learned_addr learned my addr 172.21.15.164:0/1030762326 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d81800640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 0x7f0d7c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d7be00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c159760 0x7f0d7c0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 -- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c159760 msgr2=0x7f0d7c0c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c159760 0x7f0d7c0c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 -- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 msgr2=0x7f0d7c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 0x7f0d7c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 -- 172.21.15.164:0/1030762326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0b2430 con 0x7f0d7c0c4420 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d81800640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 0x7f0d7c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d80e00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0d700af040 tx=0x7f0d70019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700bc020 con 0x7f0d7c0c4420 2024-05-08T04:41:40.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d700b8670 con 0x7f0d7c0c4420 2024-05-08T04:41:40.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700b8a60 con 0x7f0d7c0c4420 2024-05-08T04:41:40.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c0c6fa0 con 0x7f0d7c0c4420 2024-05-08T04:41:40.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0c7560 con 0x7f0d7c0c4420 2024-05-08T04:41:40.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c04ef30 con 0x7f0d7c0c4420 2024-05-08T04:41:40.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.392+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f0d700bf020 con 0x7f0d7c0c4420 2024-05-08T04:41:40.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.396+0000 7f0d78c00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 0x7f0d5c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.396+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f0d701208e0 con 0x7f0d7c0c4420 2024-05-08T04:41:40.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.396+0000 7f0d7be00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 0x7f0d5c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.396+0000 7f0d7be00640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 0x7f0d5c0699c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b56f0 tx=0x7f0d68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.400+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d700c6030 con 0x7f0d7c0c4420 2024-05-08T04:41:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d7c068170 con 0x7f0d7c0c4420 2024-05-08T04:41:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d78c00640 1 -- 172.21.15.164:0/1030762326 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+13040 (secure 0 0 0) 0x7f0d700165e0 con 0x7f0d7c0c4420 2024-05-08T04:41:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:40.564 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:39.848140+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:40.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 msgr2=0x7f0d5c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 0x7f0d5c0699c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b56f0 tx=0x7f0d68009040 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 msgr2=0x7f0d7c14c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0d700af040 tx=0x7f0d70019790 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 shutdown_connections 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0d5c067500 0x7f0d5c0699c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c159760 0x7f0d7c0c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15fd30 0x7f0d7c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 --2- 172.21.15.164:0/1030762326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d7c0c4420 0x7f0d7c14c440 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 >> 172.21.15.164:0/1030762326 conn(0x7f0d7c06b670 msgr2=0x7f0d7c159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 shutdown_connections 2024-05-08T04:41:40.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.560+0000 7f0d82200640 1 -- 172.21.15.164:0/1030762326 wait complete. 2024-05-08T04:41:40.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:41:40.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:41:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 Processor -- start 2024-05-08T04:41:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 -- start start 2024-05-08T04:41:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440b1960 0x7f8f44071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f440b1d40 0x7f8f440ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4415c900 con 0x7f8f440cec40 2024-05-08T04:41:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4415cec0 con 0x7f8f440b1d40 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415c6a0 con 0x7f8f440b1960 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f43400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f440b1d40 0x7f8f440ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f42000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8f440cec40 0x7f8f4415bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60216/0 (socket says 172.21.15.164:60216) 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.720+0000 7f8f42000640 1 -- 172.21.15.164:0/2898229538 learned_addr learned my addr 172.21.15.164:0/2898229538 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f42a00640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440b1960 0x7f8f44071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43400640 1 -- 172.21.15.164:0/2898229538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440b1960 msgr2=0x7f8f44071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43400640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440b1960 0x7f8f44071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43400640 1 -- 172.21.15.164:0/2898229538 >> v1:172.21.15.164:6789/0 conn(0x7f8f440cec40 legacy=0x7f8f4415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43400640 1 -- 172.21.15.164:0/2898229538 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4415cbe0 con 0x7f8f440b1d40 2024-05-08T04:41:40.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f41600640 1 -- 172.21.15.164:0/2898229538 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804329265 0 0) 0x7f8f4415c900 con 0x7f8f440cec40 2024-05-08T04:41:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43400640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1d40 0x7f8f440ccca0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0097c0 tx=0x7f8f3c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d70b4970effd546 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f41600640 1 -- 172.21.15.164:0/2898229538 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0b0070 con 0x7f8f440b1d40 2024-05-08T04:41:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f41600640 1 -- 172.21.15.164:0/2898229538 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c004230 con 0x7f8f440b1d40 2024-05-08T04:41:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f41600640 1 -- 172.21.15.164:0/2898229538 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003620 con 0x7f8f440b1d40 2024-05-08T04:41:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 -- 172.21.15.164:0/2898229538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1d40 msgr2=0x7f8f440ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1d40 0x7f8f440ccca0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0097c0 tx=0x7f8f3c01c350 comp rx=0 tx=0).stop 2024-05-08T04:41:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 -- 172.21.15.164:0/2898229538 shutdown_connections 2024-05-08T04:41:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1d40 0x7f8f440ccca0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 --2- 172.21.15.164:0/2898229538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440b1960 0x7f8f44071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.724+0000 7f8f43e00640 1 -- 172.21.15.164:0/2898229538 >> 172.21.15.164:0/2898229538 conn(0x7f8f4406ac90 msgr2=0x7f8f4406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- 172.21.15.164:0/2898229538 shutdown_connections 2024-05-08T04:41:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- 172.21.15.164:0/2898229538 wait complete. 2024-05-08T04:41:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 Processor -- start 2024-05-08T04:41:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- start start 2024-05-08T04:41:40.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f440b1d40 0x7f8f440afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440cec40 0x7f8f440b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b3bf0 con 0x7f8f440b1960 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44071fe0 con 0x7f8f440b1d40 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b3d70 con 0x7f8f440cec40 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57454/0 (socket says 172.21.15.164:57454) 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 -- 172.21.15.164:0/1789349977 learned_addr learned my addr 172.21.15.164:0/1789349977 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 -- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440cec40 msgr2=0x7f8f440b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440cec40 0x7f8f440b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 -- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f440b1d40 msgr2=0x7f8f440afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f440b1d40 0x7f8f440afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 -- 172.21.15.164:0/1789349977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440cc440 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f42a00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8f34007f20 tx=0x7f8f3400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34014620 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34019070 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f340149e0 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f441fac20 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.728+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f441fb1e0 con 0x7f8f440b1960 2024-05-08T04:41:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4404ef30 con 0x7f8f440b1960 2024-05-08T04:41:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f8f34014b80 con 0x7f8f440b1960 2024-05-08T04:41:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f3aa00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 0x7f8f240699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f8f340860c0 con 0x7f8f440b1960 2024-05-08T04:41:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f42000640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 0x7f8f240699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.732+0000 7f8f42000640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 0x7f8f240699c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b11c0 tx=0x7f8f30007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:40.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.736+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f34056b30 con 0x7f8f440b1960 2024-05-08T04:41:40.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.892+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f44167700 con 0x7f8f440b1960 2024-05-08T04:41:40.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f3aa00640 1 -- 172.21.15.164:0/1789349977 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f34056430 con 0x7f8f440b1960 2024-05-08T04:41:40.900 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 msgr2=0x7f8f240699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 0x7f8f240699c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b11c0 tx=0x7f8f30007990 comp rx=0 tx=0).stop 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 msgr2=0x7f8f4414f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8f34007f20 tx=0x7f8f3400e570 comp rx=0 tx=0).stop 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 shutdown_connections 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8f24067500 0x7f8f240699c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f440cec40 0x7f8f440b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f440b1d40 0x7f8f440afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 --2- 172.21.15.164:0/1789349977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f440b1960 0x7f8f4414f450 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:40.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 >> 172.21.15.164:0/1789349977 conn(0x7f8f4406ac90 msgr2=0x7f8f4406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:40.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 shutdown_connections 2024-05-08T04:41:40.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:40.896+0000 7f8f43e00640 1 -- 172.21.15.164:0/1789349977 wait complete. 2024-05-08T04:41:40.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:41:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.048+0000 7f6352200640 1 Processor -- start 2024-05-08T04:41:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 -- start start 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634c15ba70 0x7f634c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15be50 0x7f634c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c0cd9f0 con 0x7f634c0aeda0 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634c15fe70 con 0x7f634c15ba70 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f6352200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0cdcd0 con 0x7f634c15be50 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634c15ba70 0x7f634c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634c15ba70 0x7f634c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57466/0 (socket says 172.21.15.164:57466) 2024-05-08T04:41:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 -- 172.21.15.164:0/3081923073 learned_addr learned my addr 172.21.15.164:0/3081923073 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 -- 172.21.15.164:0/3081923073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15be50 msgr2=0x7f634c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 --2- 172.21.15.164:0/3081923073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15be50 0x7f634c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 -- 172.21.15.164:0/3081923073 >> v1:172.21.15.164:6789/0 conn(0x7f634c0aeda0 legacy=0x7f634c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.052+0000 7f634b400640 1 -- 172.21.15.164:0/3081923073 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c15f820 con 0x7f634c15ba70 2024-05-08T04:41:41.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f634b400640 1 --2- 172.21.15.164:0/3081923073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15ba70 0x7f634c160300 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5f8b4117e22ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f634aa00640 1 -- 172.21.15.164:0/3081923073 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0ab640 con 0x7f634c15ba70 2024-05-08T04:41:41.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f634aa00640 1 -- 172.21.15.164:0/3081923073 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c0b0070 con 0x7f634c15ba70 2024-05-08T04:41:41.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f634aa00640 1 -- 172.21.15.164:0/3081923073 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0aba00 con 0x7f634c15ba70 2024-05-08T04:41:41.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- 172.21.15.164:0/3081923073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15ba70 msgr2=0x7f634c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 --2- 172.21.15.164:0/3081923073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15ba70 0x7f634c160300 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c01c530 comp rx=0 tx=0).stop 2024-05-08T04:41:41.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- 172.21.15.164:0/3081923073 shutdown_connections 2024-05-08T04:41:41.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 --2- 172.21.15.164:0/3081923073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15be50 0x7f634c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 --2- 172.21.15.164:0/3081923073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15ba70 0x7f634c160300 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- 172.21.15.164:0/3081923073 >> 172.21.15.164:0/3081923073 conn(0x7f634c06ac90 msgr2=0x7f634c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:41.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- 172.21.15.164:0/3081923073 shutdown_connections 2024-05-08T04:41:41.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- 172.21.15.164:0/3081923073 wait complete. 2024-05-08T04:41:41.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 Processor -- start 2024-05-08T04:41:41.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.056+0000 7f6352200640 1 -- start start 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634c0aeda0 0x7f634c147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15ba70 0x7f634c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c168d40 con 0x7f634c15be50 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c168bc0 con 0x7f634c0aeda0 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c168ec0 con 0x7f634c15ba70 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57476/0 (socket says 172.21.15.164:57476) 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 -- 172.21.15.164:0/617736445 learned_addr learned my addr 172.21.15.164:0/617736445 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634b400640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634c0aeda0 0x7f634c147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6342a00640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15ba70 0x7f634c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 -- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15ba70 msgr2=0x7f634c1471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15ba70 0x7f634c1471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 -- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634c0aeda0 msgr2=0x7f634c147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634c0aeda0 0x7f634c147980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 -- 172.21.15.164:0/617736445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c071fb0 con 0x7f634c15be50 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f634be00640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f634400c7a0 tx=0x7f634400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344003b80 con 0x7f634c15be50 2024-05-08T04:41:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344003d20 con 0x7f634c15be50 2024-05-08T04:41:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c072290 con 0x7f634c15be50 2024-05-08T04:41:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634400aa30 con 0x7f634c15be50 2024-05-08T04:41:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c147700 con 0x7f634c15be50 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c069d60 con 0x7f634c15be50 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.060+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f634400abd0 con 0x7f634c15be50 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.064+0000 7f6343e00640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 0x7f63280699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.064+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f6344085590 con 0x7f634c15be50 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.064+0000 7f634b400640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 0x7f63280699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.064+0000 7f634b400640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 0x7f63280699c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af000 tx=0x7f633c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.068+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6344056090 con 0x7f634c15be50 2024-05-08T04:41:41.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f634c06c580 con 0x7f634c15be50 2024-05-08T04:41:41.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6343e00640 1 -- 172.21.15.164:0/617736445 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6344085020 con 0x7f634c15be50 2024-05-08T04:41:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:41.228 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:41:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:41:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:41:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:41.229 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:41:41.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 msgr2=0x7f63280699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 0x7f63280699c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af000 tx=0x7f633c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 msgr2=0x7f634c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f634400c7a0 tx=0x7f634400cc60 comp rx=0 tx=0).stop 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 shutdown_connections 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6328067500 0x7f63280699c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634c15ba70 0x7f634c1471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634c15be50 0x7f634c06d400 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 --2- 172.21.15.164:0/617736445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634c0aeda0 0x7f634c147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 >> 172.21.15.164:0/617736445 conn(0x7f634c06ac90 msgr2=0x7f634c06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:41.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 shutdown_connections 2024-05-08T04:41:41.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.224+0000 7f6352200640 1 -- 172.21.15.164:0/617736445 wait complete. 2024-05-08T04:41:41.248 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:41:41.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:41:41.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 Processor -- start 2024-05-08T04:41:41.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 -- start start 2024-05-08T04:41:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd8161780 0x7f6fd8161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fd80cdef0 0x7f6fd80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd80b4270 con 0x7f6fd815db10 2024-05-08T04:41:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd8162120 con 0x7f6fd80cdef0 2024-05-08T04:41:41.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fdca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd806eed0 con 0x7f6fd8161780 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fd80cdef0 0x7f6fd80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fcde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd8161780 0x7f6fd8161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6fd80cdef0 0x7f6fd80ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:43152/0 (socket says 172.21.15.164:43152) 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 -- 172.21.15.164:0/3870897677 learned_addr learned my addr 172.21.15.164:0/3870897677 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 -- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd8161780 msgr2=0x7f6fd8161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 --2- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd8161780 0x7f6fd8161be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 -- 172.21.15.164:0/3870897677 >> v1:172.21.15.172:6789/0 conn(0x7f6fd815db10 legacy=0x7f6fd80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.384+0000 7f6fd7200640 1 -- 172.21.15.164:0/3870897677 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd806f1b0 con 0x7f6fd80cdef0 2024-05-08T04:41:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fd7200640 1 --2- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd80cdef0 0x7f6fd80ce380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009870 tx=0x7f6fd001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a11bf9abe2f2734 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fd5e00640 1 -- 172.21.15.164:0/3870897677 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00a9610 con 0x7f6fd80cdef0 2024-05-08T04:41:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fd5e00640 1 -- 172.21.15.164:0/3870897677 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd00a97b0 con 0x7f6fd80cdef0 2024-05-08T04:41:41.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fd5e00640 1 -- 172.21.15.164:0/3870897677 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00a9b80 con 0x7f6fd80cdef0 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 -- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd80cdef0 msgr2=0x7f6fd80ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 --2- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd80cdef0 0x7f6fd80ce380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009870 tx=0x7f6fd001c530 comp rx=0 tx=0).stop 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 -- 172.21.15.164:0/3870897677 shutdown_connections 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 --2- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd8161780 0x7f6fd8161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 --2- 172.21.15.164:0/3870897677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd80cdef0 0x7f6fd80ce380 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 -- 172.21.15.164:0/3870897677 >> 172.21.15.164:0/3870897677 conn(0x7f6fd809aa20 msgr2=0x7f6fd80b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 -- 172.21.15.164:0/3870897677 shutdown_connections 2024-05-08T04:41:41.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.388+0000 7f6fdca00640 1 -- 172.21.15.164:0/3870897677 wait complete. 2024-05-08T04:41:41.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 Processor -- start 2024-05-08T04:41:41.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- start start 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 0x7f6fd815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 0x7f6fd8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd815def0 con 0x7f6fd80cdef0 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd806f770 con 0x7f6fd8161780 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd815e070 con 0x7f6fd815db10 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57490/0 (socket says 172.21.15.164:57490) 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 -- 172.21.15.164:0/2318231859 learned_addr learned my addr 172.21.15.164:0/2318231859 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fcde00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 0x7f6fd815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd7200640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 0x7f6fd8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 -- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 msgr2=0x7f6fd815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 0x7f6fd815e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 -- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 msgr2=0x7f6fd8151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 0x7f6fd8151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 -- 172.21.15.164:0/2318231859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd815e790 con 0x7f6fd80cdef0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fcde00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 0x7f6fd815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd7200640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 0x7f6fd8151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fd6800640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800c790 tx=0x7f6fc800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8016020 con 0x7f6fd80cdef0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8006d60 con 0x7f6fd80cdef0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8003bc0 con 0x7f6fd80cdef0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd815ea60 con 0x7f6fd80cdef0 2024-05-08T04:41:41.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd815f020 con 0x7f6fd80cdef0 2024-05-08T04:41:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.392+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd804ef30 con 0x7f6fd80cdef0 2024-05-08T04:41:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.396+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f6fc800a850 con 0x7f6fd80cdef0 2024-05-08T04:41:41.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.396+0000 7f6fcf400640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 0x7f6fb40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:41.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.396+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f6fc8085510 con 0x7f6fd80cdef0 2024-05-08T04:41:41.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.396+0000 7f6fcde00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 0x7f6fb40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:41.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.396+0000 7f6fcde00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 0x7f6fb40699c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8067ea0 tx=0x7f6fc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:41.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.400+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fc8020020 con 0x7f6fd80cdef0 2024-05-08T04:41:41.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.556+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6fd80d1e70 con 0x7f6fd80cdef0 2024-05-08T04:41:41.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fcf400640 1 -- 172.21.15.164:0/2318231859 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fc8019310 con 0x7f6fd80cdef0 2024-05-08T04:41:41.845 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:41:41.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 msgr2=0x7f6fb40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 0x7f6fb40699c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8067ea0 tx=0x7f6fc0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:41.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 msgr2=0x7f6fd81593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:41.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800c790 tx=0x7f6fc800cc50 comp rx=0 tx=0).stop 2024-05-08T04:41:41.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 shutdown_connections 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6fb4067500 0x7f6fb40699c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fd815db10 0x7f6fd815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd8161780 0x7f6fd8151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 --2- 172.21.15.164:0/2318231859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fd80cdef0 0x7f6fd81593e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 >> 172.21.15.164:0/2318231859 conn(0x7f6fd809aa20 msgr2=0x7f6fd806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 shutdown_connections 2024-05-08T04:41:41.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:41.840+0000 7f6fdca00640 1 -- 172.21.15.164:0/2318231859 wait complete. 2024-05-08T04:41:41.873 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:41:42.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 Processor -- start 2024-05-08T04:41:42.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 -- start start 2024-05-08T04:41:42.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 0x7f7558072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 0x7f755806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75580cdd60 con 0x7f755806d1e0 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580ce970 con 0x7f755815db40 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580ce040 con 0x7f75580adaa0 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 0x7f7558072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f754fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 0x7f755806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 0x7f7558072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:43174/0 (socket says 172.21.15.164:43174) 2024-05-08T04:41:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f755cc00640 1 -- 172.21.15.164:0/518222336 learned_addr learned my addr 172.21.15.164:0/518222336 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:42.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f754fe00640 1 -- 172.21.15.164:0/518222336 >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 msgr2=0x7f7558072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:42.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f754fe00640 1 --2- 172.21.15.164:0/518222336 >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 0x7f7558072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f754fe00640 1 -- 172.21.15.164:0/518222336 >> v1:172.21.15.172:6789/0 conn(0x7f755806d1e0 legacy=0x7f75580cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.012+0000 7f754fe00640 1 -- 172.21.15.164:0/518222336 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580ce320 con 0x7f75580adaa0 2024-05-08T04:41:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f754fe00640 1 --2- 172.21.15.164:0/518222336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 0x7f755806cca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75400097c0 tx=0x7f754001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef286f5652ad39f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f7557e00640 1 -- 172.21.15.164:0/518222336 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400b1070 con 0x7f75580adaa0 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f7557e00640 1 -- 172.21.15.164:0/518222336 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75400ac630 con 0x7f75580adaa0 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f7557e00640 1 -- 172.21.15.164:0/518222336 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a8050 con 0x7f75580adaa0 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 -- 172.21.15.164:0/518222336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 msgr2=0x7f755806cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 --2- 172.21.15.164:0/518222336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 0x7f755806cca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75400097c0 tx=0x7f754001c470 comp rx=0 tx=0).stop 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 -- 172.21.15.164:0/518222336 shutdown_connections 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 --2- 172.21.15.164:0/518222336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75580adaa0 0x7f755806cca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 --2- 172.21.15.164:0/518222336 >> v2:172.21.15.172:3300/0 conn(0x7f755815db40 0x7f7558072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 -- 172.21.15.164:0/518222336 >> 172.21.15.164:0/518222336 conn(0x7f755806ac90 msgr2=0x7f755806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:42.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 -- 172.21.15.164:0/518222336 shutdown_connections 2024-05-08T04:41:42.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.016+0000 7f755e000640 1 -- 172.21.15.164:0/518222336 wait complete. 2024-05-08T04:41:42.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 Processor -- start 2024-05-08T04:41:42.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- start start 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755806d1e0 0x7f755813e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f755815db40 0x7f755813bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580cf690 con 0x7f75580adaa0 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580cf510 con 0x7f755806d1e0 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755816ab10 con 0x7f755815db40 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755806d1e0 0x7f755813e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57492/0 (socket says 172.21.15.164:57492) 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 -- 172.21.15.164:0/3020086675 learned_addr learned my addr 172.21.15.164:0/3020086675 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755d600640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f755815db40 0x7f755813bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 -- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f755815db40 msgr2=0x7f755813bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f755815db40 0x7f755813bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 -- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755806d1e0 msgr2=0x7f755813e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755806d1e0 0x7f755813e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 -- 172.21.15.164:0/3020086675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755813ee00 con 0x7f75580adaa0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f754fe00640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75400af040 tx=0x7f7540019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400b1040 con 0x7f75580adaa0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75400bb070 con 0x7f75580adaa0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a8040 con 0x7f75580adaa0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755813f0e0 con 0x7f75580adaa0 2024-05-08T04:41:42.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.020+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755813f6a0 con 0x7f75580adaa0 2024-05-08T04:41:42.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.024+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755804ef30 con 0x7f75580adaa0 2024-05-08T04:41:42.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.024+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f75400b5650 con 0x7f75580adaa0 2024-05-08T04:41:42.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.024+0000 7f7555600640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 0x7f75340699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:42.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.024+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f754011f560 con 0x7f75580adaa0 2024-05-08T04:41:42.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.024+0000 7f755cc00640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 0x7f75340699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:42.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.028+0000 7f755cc00640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 0x7f75340699c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7548009ab0 tx=0x7f7548009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:42.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.028+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7540016090 con 0x7f75580adaa0 2024-05-08T04:41:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.184+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7558068170 con 0x7f75580adaa0 2024-05-08T04:41:42.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.844+0000 7f7555600640 1 -- 172.21.15.164:0/3020086675 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f75400ef900 con 0x7f75580adaa0 2024-05-08T04:41:42.848 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 msgr2=0x7f75340699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 0x7f75340699c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7548009ab0 tx=0x7f7548009240 comp rx=0 tx=0).stop 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 msgr2=0x7f7558159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75400af040 tx=0x7f7540019210 comp rx=0 tx=0).stop 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 shutdown_connections 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7534067500 0x7f75340699c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f755815db40 0x7f755813bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580adaa0 0x7f7558159350 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 --2- 172.21.15.164:0/3020086675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755806d1e0 0x7f755813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:42.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 >> 172.21.15.164:0/3020086675 conn(0x7f755806ac90 msgr2=0x7f75580b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:42.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 shutdown_connections 2024-05-08T04:41:42.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:42.848+0000 7f755e000640 1 -- 172.21.15.164:0/3020086675 wait complete. 2024-05-08T04:41:42.886 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:41:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 Processor -- start 2024-05-08T04:41:43.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 -- start start 2024-05-08T04:41:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2ea815ab10 0x7f2ea806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2ea80cee90 0x7f2ea80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea815e870 con 0x7f2ea80ae6e0 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea806bb80 con 0x7f2ea815ab10 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eaf800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea815eb50 con 0x7f2ea80cee90 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2ea815ab10 0x7f2ea806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2ea815ab10 0x7f2ea806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57504/0 (socket says 172.21.15.164:57504) 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 -- 172.21.15.164:0/2315676323 learned_addr learned my addr 172.21.15.164:0/2315676323 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2ead800640 1 --2- 172.21.15.164:0/2315676323 >> v2:172.21.15.172:3300/0 conn(0x7f2ea80cee90 0x7f2ea80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 -- 172.21.15.164:0/2315676323 >> v2:172.21.15.172:3300/0 conn(0x7f2ea80cee90 msgr2=0x7f2ea80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 --2- 172.21.15.164:0/2315676323 >> v2:172.21.15.172:3300/0 conn(0x7f2ea80cee90 0x7f2ea80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 -- 172.21.15.164:0/2315676323 >> v1:172.21.15.172:6789/0 conn(0x7f2ea80ae6e0 legacy=0x7f2ea80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.020+0000 7f2eace00640 1 -- 172.21.15.164:0/2315676323 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea815ee30 con 0x7f2ea815ab10 2024-05-08T04:41:43.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eace00640 1 --2- 172.21.15.164:0/2315676323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea815ab10 0x7f2ea806cce0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80ca1f0 tx=0x7f2e9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac045333effd614 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:43.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2ea7e00640 1 -- 172.21.15.164:0/2315676323 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab660 con 0x7f2ea815ab10 2024-05-08T04:41:43.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2ea7e00640 1 -- 172.21.15.164:0/2315676323 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980b0070 con 0x7f2ea815ab10 2024-05-08T04:41:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2ea7e00640 1 -- 172.21.15.164:0/2315676323 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980aba20 con 0x7f2ea815ab10 2024-05-08T04:41:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 -- 172.21.15.164:0/2315676323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea815ab10 msgr2=0x7f2ea806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 --2- 172.21.15.164:0/2315676323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea815ab10 0x7f2ea806cce0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80ca1f0 tx=0x7f2e9801c460 comp rx=0 tx=0).stop 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 -- 172.21.15.164:0/2315676323 shutdown_connections 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 --2- 172.21.15.164:0/2315676323 >> v2:172.21.15.172:3300/0 conn(0x7f2ea80cee90 0x7f2ea80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 --2- 172.21.15.164:0/2315676323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea815ab10 0x7f2ea806cce0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 -- 172.21.15.164:0/2315676323 >> 172.21.15.164:0/2315676323 conn(0x7f2ea80cf770 msgr2=0x7f2ea80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 -- 172.21.15.164:0/2315676323 shutdown_connections 2024-05-08T04:41:43.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.024+0000 7f2eaf800640 1 -- 172.21.15.164:0/2315676323 wait complete. 2024-05-08T04:41:43.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 Processor -- start 2024-05-08T04:41:43.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- start start 2024-05-08T04:41:43.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ea80ae6e0 0x7f2ea814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ea815ab10 0x7f2ea814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cb070 con 0x7f2ea80cee90 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80caef0 con 0x7f2ea815ab10 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cb1f0 con 0x7f2ea80ae6e0 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57518/0 (socket says 172.21.15.164:57518) 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 -- 172.21.15.164:0/3877807863 learned_addr learned my addr 172.21.15.164:0/3877807863 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eace00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ea80ae6e0 0x7f2ea814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2ead800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ea815ab10 0x7f2ea814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 -- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ea80ae6e0 msgr2=0x7f2ea814e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ea80ae6e0 0x7f2ea814e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 -- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ea815ab10 msgr2=0x7f2ea814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ea815ab10 0x7f2ea814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 -- 172.21.15.164:0/3877807863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80c8200 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2e9fe00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2e90007fd0 tx=0x7f2e9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90017070 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e900052d0 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90003840 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80c84e0 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.028+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea80c8aa0 con 0x7f2ea80cee90 2024-05-08T04:41:43.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8072d30 con 0x7f2ea80cee90 2024-05-08T04:41:43.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f2e9001a070 con 0x7f2ea80cee90 2024-05-08T04:41:43.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2ea5600640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 0x7f2e840699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:43.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f2e90085180 con 0x7f2ea80cee90 2024-05-08T04:41:43.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2eace00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 0x7f2e840699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:43.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.032+0000 7f2eace00640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 0x7f2e840699c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2e980af000 tx=0x7f2e980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:43.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.036+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e90021020 con 0x7f2ea80cee90 2024-05-08T04:41:43.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.220+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2ea804f010 con 0x7f2ea80cee90 2024-05-08T04:41:43.876 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 12 and data pool 13 2024-05-08T04:41:43.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.868+0000 7f2ea5600640 1 -- 172.21.15.164:0/3877807863 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2e90005480 con 0x7f2ea80cee90 2024-05-08T04:41:43.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.876+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 msgr2=0x7f2e840699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.876+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 0x7f2e840699c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2e980af000 tx=0x7f2e980a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:43.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.876+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 msgr2=0x7f2ea80c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:43.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.876+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2e90007fd0 tx=0x7f2e9000c890 comp rx=0 tx=0).stop 2024-05-08T04:41:43.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 shutdown_connections 2024-05-08T04:41:43.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2e84067500 0x7f2e840699c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ea815ab10 0x7f2ea814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea80cee90 0x7f2ea80c92a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 --2- 172.21.15.164:0/3877807863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ea80ae6e0 0x7f2ea814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:43.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 >> 172.21.15.164:0/3877807863 conn(0x7f2ea80cf770 msgr2=0x7f2ea80c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:43.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.880+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 shutdown_connections 2024-05-08T04:41:43.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:43.884+0000 7f2eaf800640 1 -- 172.21.15.164:0/3877807863 wait complete. 2024-05-08T04:41:43.918 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:44.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 Processor -- start 2024-05-08T04:41:44.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 -- start start 2024-05-08T04:41:44.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98015ab60 0x7fd980072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd9800cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd980072640 con 0x7fd9800b26c0 2024-05-08T04:41:44.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98006c770 con 0x7fd98015ab60 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd985000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98006ca50 con 0x7fd9800cfda0 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.048+0000 7fd97de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98015ab60 0x7fd980072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98015ab60 0x7fd980072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57520/0 (socket says 172.21.15.164:57520) 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 -- 172.21.15.164:0/2885691566 learned_addr learned my addr 172.21.15.164:0/2885691566 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97e800640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd9800cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 -- 172.21.15.164:0/2885691566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 msgr2=0x7fd9800cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:44.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd9800cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 -- 172.21.15.164:0/2885691566 >> v1:172.21.15.164:6789/0 conn(0x7fd9800b26c0 legacy=0x7fd9800cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 -- 172.21.15.164:0/2885691566 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98006cd30 con 0x7fd98015ab60 2024-05-08T04:41:44.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97de00640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98015ab60 0x7fd980072100 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bb10 tx=0x7fd97001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329f814bc145fb32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97d400640 1 -- 172.21.15.164:0/2885691566 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b4070 con 0x7fd98015ab60 2024-05-08T04:41:44.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97d400640 1 -- 172.21.15.164:0/2885691566 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700a7b00 con 0x7fd98015ab60 2024-05-08T04:41:44.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.052+0000 7fd97d400640 1 -- 172.21.15.164:0/2885691566 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700af440 con 0x7fd98015ab60 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- 172.21.15.164:0/2885691566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98015ab60 msgr2=0x7fd980072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98015ab60 0x7fd980072100 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bb10 tx=0x7fd97001ec30 comp rx=0 tx=0).stop 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- 172.21.15.164:0/2885691566 shutdown_connections 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd9800cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- 172.21.15.164:0/2885691566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98015ab60 0x7fd980072100 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- 172.21.15.164:0/2885691566 >> 172.21.15.164:0/2885691566 conn(0x7fd98006ac90 msgr2=0x7fd98006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:44.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- 172.21.15.164:0/2885691566 shutdown_connections 2024-05-08T04:41:44.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- 172.21.15.164:0/2885691566 wait complete. 2024-05-08T04:41:44.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 Processor -- start 2024-05-08T04:41:44.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- start start 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd980147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98015ab60 0x7fd980153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800aeeb0 con 0x7fd9800b26c0 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800aed30 con 0x7fd98015ab60 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd985000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800cc1b0 con 0x7fd9800cfda0 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57530/0 (socket says 172.21.15.164:57530) 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 -- 172.21.15.164:0/2467115459 learned_addr learned my addr 172.21.15.164:0/2467115459 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd975400640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd980147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97e800640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98015ab60 0x7fd980153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 -- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 msgr2=0x7fd980147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd980147350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 -- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98015ab60 msgr2=0x7fd980153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98015ab60 0x7fd980153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 -- 172.21.15.164:0/2467115459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98015b180 con 0x7fd9800b26c0 2024-05-08T04:41:44.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd975400640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd980147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.056+0000 7fd97de00640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b2c0 tx=0x7fd97001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700be020 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700afce0 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b4040 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98015b460 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98014b5f0 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980069d60 con 0x7fd9800b26c0 2024-05-08T04:41:44.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fd9700c1020 con 0x7fd9800b26c0 2024-05-08T04:41:44.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 0x7fd95c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd970122b90 con 0x7fd9800b26c0 2024-05-08T04:41:44.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd975400640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 0x7fd95c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.060+0000 7fd975400640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 0x7fd95c0699c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd9800b0970 tx=0x7fd968009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.064+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9700c8030 con 0x7fd9800b26c0 2024-05-08T04:41:44.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.220+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd98006d9a0 con 0x7fd9800b26c0 2024-05-08T04:41:44.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd976a00640 1 -- 172.21.15.164:0/2467115459 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fd970018820 con 0x7fd9800b26c0 2024-05-08T04:41:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:44.228 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":85,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:43.227305+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 msgr2=0x7fd95c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 0x7fd95c0699c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd9800b0970 tx=0x7fd968009040 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 msgr2=0x7fd9801f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b2c0 tx=0x7fd97001b9d0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 shutdown_connections 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd95c067500 0x7fd95c0699c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9800cfda0 0x7fd980147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98015ab60 0x7fd980153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 --2- 172.21.15.164:0/2467115459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9800b26c0 0x7fd9801f27e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 >> 172.21.15.164:0/2467115459 conn(0x7fd98006ac90 msgr2=0x7fd9800cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:44.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 shutdown_connections 2024-05-08T04:41:44.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.224+0000 7fd985000640 1 -- 172.21.15.164:0/2467115459 wait complete. 2024-05-08T04:41:44.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:44.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.376+0000 7f88fcc00640 1 Processor -- start 2024-05-08T04:41:44.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 -- start start 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88f815de60 0x7f88f806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88f8161e50 0x7f88f806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f806bda0 con 0x7f88f815da80 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f806c9b0 con 0x7f88f815de60 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88fcc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f806c080 con 0x7f88f8161e50 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88f815de60 0x7f88f806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88f815de60 0x7f88f806eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57542/0 (socket says 172.21.15.164:57542) 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 -- 172.21.15.164:0/569494600 learned_addr learned my addr 172.21.15.164:0/569494600 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88f7e00640 1 --2- 172.21.15.164:0/569494600 >> v2:172.21.15.172:3300/0 conn(0x7f88f8161e50 0x7f88f806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 -- 172.21.15.164:0/569494600 >> v2:172.21.15.172:3300/0 conn(0x7f88f8161e50 msgr2=0x7f88f806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 --2- 172.21.15.164:0/569494600 >> v2:172.21.15.172:3300/0 conn(0x7f88f8161e50 0x7f88f806f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 -- 172.21.15.164:0/569494600 >> v1:172.21.15.164:6789/0 conn(0x7f88f815da80 legacy=0x7f88f8161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.380+0000 7f88eea00640 1 -- 172.21.15.164:0/569494600 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f806c360 con 0x7f88f815de60 2024-05-08T04:41:44.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88eea00640 1 --2- 172.21.15.164:0/569494600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815de60 0x7f88f806eed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f88f80b7940 tx=0x7f88e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb21b1e235ef10e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88f6a00640 1 -- 172.21.15.164:0/569494600 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00a9400 con 0x7f88f815de60 2024-05-08T04:41:44.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88f6a00640 1 -- 172.21.15.164:0/569494600 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e00a95a0 con 0x7f88f815de60 2024-05-08T04:41:44.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88f6a00640 1 -- 172.21.15.164:0/569494600 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00a9970 con 0x7f88f815de60 2024-05-08T04:41:44.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- 172.21.15.164:0/569494600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815de60 msgr2=0x7f88f806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 --2- 172.21.15.164:0/569494600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815de60 0x7f88f806eed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f88f80b7940 tx=0x7f88e001c520 comp rx=0 tx=0).stop 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- 172.21.15.164:0/569494600 shutdown_connections 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 --2- 172.21.15.164:0/569494600 >> v2:172.21.15.172:3300/0 conn(0x7f88f8161e50 0x7f88f806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 --2- 172.21.15.164:0/569494600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815de60 0x7f88f806eed0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- 172.21.15.164:0/569494600 >> 172.21.15.164:0/569494600 conn(0x7f88f80b32e0 msgr2=0x7f88f80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- 172.21.15.164:0/569494600 shutdown_connections 2024-05-08T04:41:44.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- 172.21.15.164:0/569494600 wait complete. 2024-05-08T04:41:44.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 Processor -- start 2024-05-08T04:41:44.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.384+0000 7f88fcc00640 1 -- start start 2024-05-08T04:41:44.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f815de60 0x7f88f80ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f8161e50 0x7f88f80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806d9c0 con 0x7f88f815da80 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806d840 con 0x7f88f815de60 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806db40 con 0x7f88f8161e50 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57556/0 (socket says 172.21.15.164:57556) 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 -- 172.21.15.164:0/879766129 learned_addr learned my addr 172.21.15.164:0/879766129 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88eea00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f815de60 0x7f88f80ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7e00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f8161e50 0x7f88f80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 -- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f8161e50 msgr2=0x7f88f80ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f8161e50 0x7f88f80ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 -- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f815de60 msgr2=0x7f88f80ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f815de60 0x7f88f80ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 -- 172.21.15.164:0/879766129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f80c6200 con 0x7f88f815da80 2024-05-08T04:41:44.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88f7400640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c790 tx=0x7f88e800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8016020 con 0x7f88f815da80 2024-05-08T04:41:44.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8006d60 con 0x7f88f815da80 2024-05-08T04:41:44.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8003bc0 con 0x7f88f815da80 2024-05-08T04:41:44.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f80c64e0 con 0x7f88f815da80 2024-05-08T04:41:44.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f80c6aa0 con 0x7f88f815da80 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f804ef30 con 0x7f88f815da80 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.388+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f88e80050d0 con 0x7f88f815da80 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.392+0000 7f88efe00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 0x7f88d40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.392+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f88e80858a0 con 0x7f88f815da80 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.392+0000 7f88eea00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 0x7f88d40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.392+0000 7f88eea00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 0x7f88d40699c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f88e00ae000 tx=0x7f88e00b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.396+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88e8020020 con 0x7f88f815da80 2024-05-08T04:41:44.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.548+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88f8074aa0 con 0x7f88f815da80 2024-05-08T04:41:44.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88efe00640 1 -- 172.21.15.164:0/879766129 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f88e8056050 con 0x7f88f815da80 2024-05-08T04:41:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:44.557 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":85,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:43.227305+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 msgr2=0x7f88d40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 0x7f88d40699c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f88e00ae000 tx=0x7f88e00b3040 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 msgr2=0x7f88f8155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c790 tx=0x7f88e800cb90 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 shutdown_connections 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f88d4067500 0x7f88d40699c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f8161e50 0x7f88f80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f815de60 0x7f88f80ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 --2- 172.21.15.164:0/879766129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f815da80 0x7f88f8155ec0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 >> 172.21.15.164:0/879766129 conn(0x7f88f80b32e0 msgr2=0x7f88f80c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:44.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 shutdown_connections 2024-05-08T04:41:44.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.552+0000 7f88fcc00640 1 -- 172.21.15.164:0/879766129 wait complete. 2024-05-08T04:41:44.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:41:44.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.712+0000 7eff36200640 1 Processor -- start 2024-05-08T04:41:44.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.712+0000 7eff36200640 1 -- start start 2024-05-08T04:41:44.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff36200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff30071880 0x7eff3015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff36200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3015b020 con 0x7eff300cc0f0 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff36200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3015b300 con 0x7eff300af920 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff36200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff300cf460 con 0x7eff30071880 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff34e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7eff300cc0f0 0x7eff300b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60256/0 (socket says 172.21.15.164:60256) 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff34e00640 1 -- 172.21.15.164:0/2126735617 learned_addr learned my addr 172.21.15.164:0/2126735617 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff35800640 1 --2- 172.21.15.164:0/2126735617 >> v2:172.21.15.172:3300/0 conn(0x7eff30071880 0x7eff3015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff2f400640 1 -- 172.21.15.164:0/2126735617 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801095413 0 0) 0x7eff3015b020 con 0x7eff300cc0f0 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff2f400640 1 -- 172.21.15.164:0/2126735617 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14003530 con 0x7eff300cc0f0 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff35800640 1 -- 172.21.15.164:0/2126735617 >> v1:172.21.15.172:6789/0 conn(0x7eff300af920 legacy=0x7eff30071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff35800640 1 -- 172.21.15.164:0/2126735617 >> v1:172.21.15.164:6789/0 conn(0x7eff300cc0f0 legacy=0x7eff300b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.716+0000 7eff35800640 1 -- 172.21.15.164:0/2126735617 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3015b5e0 con 0x7eff30071880 2024-05-08T04:41:44.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff35800640 1 --2- 172.21.15.164:0/2126735617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff30071880 0x7eff3015e630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff28009d90 tx=0x7eff2801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c6ae630cae6bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff2f400640 1 -- 172.21.15.164:0/2126735617 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280b2070 con 0x7eff30071880 2024-05-08T04:41:44.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff2f400640 1 -- 172.21.15.164:0/2126735617 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff280a58b0 con 0x7eff30071880 2024-05-08T04:41:44.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff2f400640 1 -- 172.21.15.164:0/2126735617 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280a5c80 con 0x7eff30071880 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- 172.21.15.164:0/2126735617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff30071880 msgr2=0x7eff3015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 --2- 172.21.15.164:0/2126735617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff30071880 0x7eff3015e630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff28009d90 tx=0x7eff2801c8e0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- 172.21.15.164:0/2126735617 shutdown_connections 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 --2- 172.21.15.164:0/2126735617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff30071880 0x7eff3015e630 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- 172.21.15.164:0/2126735617 >> 172.21.15.164:0/2126735617 conn(0x7eff3006ac90 msgr2=0x7eff3006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- 172.21.15.164:0/2126735617 shutdown_connections 2024-05-08T04:41:44.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- 172.21.15.164:0/2126735617 wait complete. 2024-05-08T04:41:44.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 Processor -- start 2024-05-08T04:41:44.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.720+0000 7eff36200640 1 -- start start 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300af920 0x7eff301577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300cc0f0 0x7eff30159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3015ac60 con 0x7eff30071880 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3015bba0 con 0x7eff300cc0f0 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3015ade0 con 0x7eff300af920 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57566/0 (socket says 172.21.15.164:57566) 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 -- 172.21.15.164:0/3282349901 learned_addr learned my addr 172.21.15.164:0/3282349901 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 -- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300af920 msgr2=0x7eff301577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300af920 0x7eff301577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 -- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300cc0f0 msgr2=0x7eff30159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300cc0f0 0x7eff30159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 -- 172.21.15.164:0/3282349901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff30158f10 con 0x7eff30071880 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff34e00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c420 tx=0x7eff2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20017070 con 0x7eff30071880 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20005cf0 con 0x7eff30071880 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff200072f0 con 0x7eff30071880 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff301591f0 con 0x7eff30071880 2024-05-08T04:41:44.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff301597b0 con 0x7eff30071880 2024-05-08T04:41:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3004ef30 con 0x7eff30071880 2024-05-08T04:41:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.724+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7eff2001a070 con 0x7eff30071880 2024-05-08T04:41:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.728+0000 7eff2cc00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 0x7eff100699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:44.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.728+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5765+0+0 (secure 0 0 0) 0x7eff20084f60 con 0x7eff30071880 2024-05-08T04:41:44.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.728+0000 7eff2fe00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 0x7eff100699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:44.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.728+0000 7eff2fe00640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 0x7eff100699c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f70 tx=0x7eff1c0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:44.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.732+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff20055680 con 0x7eff30071880 2024-05-08T04:41:44.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:44.924+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7eff30166280 con 0x7eff30071880 2024-05-08T04:41:45.845 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-05-08T04:41:45.840+0000 7f7b8ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:41:45.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.876+0000 7eff2cc00640 1 -- 172.21.15.164:0/3282349901 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7eff20086020 con 0x7eff30071880 2024-05-08T04:41:45.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 msgr2=0x7eff100699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:45.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 0x7eff100699c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f70 tx=0x7eff1c0079e0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 msgr2=0x7eff30146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:45.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c420 tx=0x7eff2000c8e0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 shutdown_connections 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff10067500 0x7eff100699c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff300af920 0x7eff301577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff300cc0f0 0x7eff30159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 --2- 172.21.15.164:0/3282349901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff30071880 0x7eff30146130 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 >> 172.21.15.164:0/3282349901 conn(0x7eff3006ac90 msgr2=0x7eff300b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 shutdown_connections 2024-05-08T04:41:45.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:45.880+0000 7eff36200640 1 -- 172.21.15.164:0/3282349901 wait complete. 2024-05-08T04:41:45.908 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:41:45.908 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:41:46.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.044+0000 7ff772600640 1 Processor -- start 2024-05-08T04:41:46.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.044+0000 7ff772600640 1 -- start start 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff772600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff76c0b1960 0x7ff76c071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff772600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c15c900 con 0x7ff76c0cec40 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff772600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c15cbe0 con 0x7ff76c0b1d40 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff772600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c15cec0 con 0x7ff76c0b1960 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76be00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff76c0cec40 0x7ff76c15bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60270/0 (socket says 172.21.15.164:60270) 2024-05-08T04:41:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76be00640 1 -- 172.21.15.164:0/2491066796 learned_addr learned my addr 172.21.15.164:0/2491066796 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:46.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 --2- 172.21.15.164:0/2491066796 >> v2:172.21.15.172:3300/0 conn(0x7ff76c0b1960 0x7ff76c071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639313969 0 0) 0x7ff76c15c900 con 0x7ff76c0cec40 2024-05-08T04:41:46.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff750003530 con 0x7ff76c0cec40 2024-05-08T04:41:46.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895023110 0 0) 0x7ff76c15cbe0 con 0x7ff76c0b1d40 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff76c15c900 con 0x7ff76c0b1d40 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4075261897 0 0) 0x7ff750003530 con 0x7ff76c0cec40 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 >> v2:172.21.15.172:3300/0 conn(0x7ff76c0b1960 msgr2=0x7ff76c071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 --2- 172.21.15.164:0/2491066796 >> v2:172.21.15.172:3300/0 conn(0x7ff76c0b1960 0x7ff76c071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 >> v1:172.21.15.172:6789/0 conn(0x7ff76c0b1d40 legacy=0x7ff76c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c0712a0 con 0x7ff76c0cec40 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7ff758001680 con 0x7ff76c0cec40 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 >> v1:172.21.15.164:6789/0 conn(0x7ff76c0cec40 legacy=0x7ff76c15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 0x7ff750003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750004810 0x7ff750004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff750005290 0x7ff75000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15cec0 con 0x7ff750003b50 2024-05-08T04:41:46.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000c1a0 con 0x7ff750005290 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000c3a0 con 0x7ff750004810 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 0x7ff750003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771c00640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff750005290 0x7ff75000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff76be00640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750004810 0x7ff750004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 -- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750004810 msgr2=0x7ff750004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750004810 0x7ff750004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 -- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff750005290 msgr2=0x7ff75000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff750005290 0x7ff75000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771200640 1 -- 172.21.15.164:0/2491066796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff750003530 con 0x7ff750003b50 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771c00640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff750005290 0x7ff75000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:46.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.048+0000 7ff771c00640 1 -- 172.21.15.164:0/2491066796 reap_dead start 2024-05-08T04:41:46.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff771200640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 0x7ff750003f70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009ae0 tx=0x7ff75c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c6500f844e716a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:46.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0096b0 con 0x7ff750003b50 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c0a6510 con 0x7ff750003b50 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff76b400640 1 -- 172.21.15.164:0/2491066796 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0a6900 con 0x7ff750003b50 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 -- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 msgr2=0x7ff750003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 0x7ff750003f70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009ae0 tx=0x7ff75c01ca20 comp rx=0 tx=0).stop 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 -- 172.21.15.164:0/2491066796 shutdown_connections 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 --2- 172.21.15.164:0/2491066796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff750003b50 0x7ff750003f70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 -- 172.21.15.164:0/2491066796 >> 172.21.15.164:0/2491066796 conn(0x7ff76c06ac90 msgr2=0x7ff76c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 -- 172.21.15.164:0/2491066796 shutdown_connections 2024-05-08T04:41:46.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.052+0000 7ff772600640 1 -- 172.21.15.164:0/2491066796 wait complete. 2024-05-08T04:41:46.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 Processor -- start 2024-05-08T04:41:46.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- start start 2024-05-08T04:41:46.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 0x7ff76c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 0x7ff76c1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cd030 con 0x7ff76c1565a0 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cceb0 con 0x7ff76c157d60 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cd1b0 con 0x7ff750003b50 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57596/0 (socket says 172.21.15.164:57596) 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 -- 172.21.15.164:0/1576495974 learned_addr learned my addr 172.21.15.164:0/1576495974 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff76be00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 0x7ff76c1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771200640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 0x7ff76c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 -- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 msgr2=0x7ff76c14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 0x7ff76c14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 -- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 msgr2=0x7ff76c1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 0x7ff76c1581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 -- 172.21.15.164:0/1576495974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c154de0 con 0x7ff76c1565a0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771200640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 0x7ff76c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff76be00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 0x7ff76c1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff771c00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0aefc0 tx=0x7ff76400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764019070 con 0x7ff76c1565a0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764006b60 con 0x7ff76c1565a0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c158890 con 0x7ff76c1565a0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764003980 con 0x7ff76c1565a0 2024-05-08T04:41:46.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c158e50 con 0x7ff76c1565a0 2024-05-08T04:41:46.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.056+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c04ef30 con 0x7ff76c1565a0 2024-05-08T04:41:46.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.060+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7ff76401c070 con 0x7ff76c1565a0 2024-05-08T04:41:46.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.060+0000 7ff768c00640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 0x7ff74c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:46.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.060+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff764084dd0 con 0x7ff76c1565a0 2024-05-08T04:41:46.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.060+0000 7ff771200640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 0x7ff74c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:46.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.060+0000 7ff771200640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 0x7ff74c0699c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0af000 tx=0x7ff75c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:46.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.064+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff764055580 con 0x7ff76c1565a0 2024-05-08T04:41:46.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.252+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff76c06b1c0 con 0x7ff76c1565a0 2024-05-08T04:41:46.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.880+0000 7ff768c00640 1 -- 172.21.15.164:0/1576495974 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff764054e80 con 0x7ff76c1565a0 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 msgr2=0x7ff74c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 0x7ff74c0699c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0af000 tx=0x7ff75c0046d0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 msgr2=0x7ff76c156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0aefc0 tx=0x7ff76400eb90 comp rx=0 tx=0).stop 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 shutdown_connections 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff74c067500 0x7ff74c0699c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff76c1565a0 0x7ff76c156a20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c157d60 0x7ff76c1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 --2- 172.21.15.164:0/1576495974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff750003b50 0x7ff76c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:46.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 >> 172.21.15.164:0/1576495974 conn(0x7ff76c06ac90 msgr2=0x7ff76c15f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:46.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 shutdown_connections 2024-05-08T04:41:46.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:46.884+0000 7ff772600640 1 -- 172.21.15.164:0/1576495974 wait complete. 2024-05-08T04:41:46.910 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:41:46.910 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:47.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 Processor -- start 2024-05-08T04:41:47.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 -- start start 2024-05-08T04:41:47.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f867c0cfda0 0x7f867c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0cd9f0 con 0x7f867c15ba70 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0cdcd0 con 0x7f867c15be50 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8680e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c15fe70 con 0x7f867c0cfda0 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8679e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f867c15ba70 0x7f867c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60278/0 (socket says 172.21.15.164:60278) 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8679e00640 1 -- 172.21.15.164:0/590177068 learned_addr learned my addr 172.21.15.164:0/590177068 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8671400640 1 --2- 172.21.15.164:0/590177068 >> v2:172.21.15.164:3300/0 conn(0x7f867c0cfda0 0x7f867c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8671400640 1 -- 172.21.15.164:0/590177068 >> v1:172.21.15.172:6789/0 conn(0x7f867c15be50 legacy=0x7f867c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8671400640 1 -- 172.21.15.164:0/590177068 >> v1:172.21.15.164:6789/0 conn(0x7f867c15ba70 legacy=0x7f867c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8671400640 1 -- 172.21.15.164:0/590177068 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15f820 con 0x7f867c0cfda0 2024-05-08T04:41:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.044+0000 7f8679400640 1 -- 172.21.15.164:0/590177068 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936789939 0 0) 0x7f867c0cd9f0 con 0x7f867c15ba70 2024-05-08T04:41:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8671400640 1 --2- 172.21.15.164:0/590177068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c160300 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f86640097c0 tx=0x7f866401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511ea1ca1fa67c32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8679400640 1 -- 172.21.15.164:0/590177068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640b0070 con 0x7f867c0cfda0 2024-05-08T04:41:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8679400640 1 -- 172.21.15.164:0/590177068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86640ab600 con 0x7f867c0cfda0 2024-05-08T04:41:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8679400640 1 -- 172.21.15.164:0/590177068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640ab9d0 con 0x7f867c0cfda0 2024-05-08T04:41:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 -- 172.21.15.164:0/590177068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 msgr2=0x7f867c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 --2- 172.21.15.164:0/590177068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c160300 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f86640097c0 tx=0x7f866401c430 comp rx=0 tx=0).stop 2024-05-08T04:41:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 -- 172.21.15.164:0/590177068 shutdown_connections 2024-05-08T04:41:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 --2- 172.21.15.164:0/590177068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c160300 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 -- 172.21.15.164:0/590177068 >> 172.21.15.164:0/590177068 conn(0x7f867c06ac90 msgr2=0x7f867c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 -- 172.21.15.164:0/590177068 shutdown_connections 2024-05-08T04:41:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.048+0000 7f8680e00640 1 -- 172.21.15.164:0/590177068 wait complete. 2024-05-08T04:41:47.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 Processor -- start 2024-05-08T04:41:47.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- start start 2024-05-08T04:41:47.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c15ba70 0x7f867c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15be50 0x7f867c14a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c169f60 con 0x7f867c0cfda0 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c169de0 con 0x7f867c15ba70 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c16a0e0 con 0x7f867c15be50 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57614/0 (socket says 172.21.15.164:57614) 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 -- 172.21.15.164:0/1804387547 learned_addr learned my addr 172.21.15.164:0/1804387547 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 -- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15be50 msgr2=0x7f867c14a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15be50 0x7f867c14a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 -- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c15ba70 msgr2=0x7f867c15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c15ba70 0x7f867c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 -- 172.21.15.164:0/1804387547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15db30 con 0x7f867c0cfda0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8679e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f866c0011f0 tx=0x7f866c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c018070 con 0x7f867c0cfda0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0074e0 con 0x7f867c0cfda0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c004a10 con 0x7f867c0cfda0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c15de10 con 0x7f867c0cfda0 2024-05-08T04:41:47.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c15e3d0 con 0x7f867c0cfda0 2024-05-08T04:41:47.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.052+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c069d60 con 0x7f867c0cfda0 2024-05-08T04:41:47.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.056+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f866c004bb0 con 0x7f867c0cfda0 2024-05-08T04:41:47.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.056+0000 7f8672a00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 0x7f86580699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.056+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f866c084e70 con 0x7f867c0cfda0 2024-05-08T04:41:47.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.056+0000 7f8671400640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 0x7f86580699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.056+0000 7f8671400640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 0x7f86580699c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f867c06bc70 tx=0x7f8664004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.060+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f866c055590 con 0x7f867c0cfda0 2024-05-08T04:41:47.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.244+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f867c1542e0 con 0x7f867c0cfda0 2024-05-08T04:41:47.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8672a00640 1 -- 172.21.15.164:0/1804387547 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f866c054e90 con 0x7f867c0cfda0 2024-05-08T04:41:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:47.252 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:41:47.253 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 52 2024-05-08T04:41:47.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 msgr2=0x7f86580699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 0x7f86580699c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f867c06bc70 tx=0x7f8664004740 comp rx=0 tx=0).stop 2024-05-08T04:41:47.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 msgr2=0x7f867c15d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f866c0011f0 tx=0x7f866c00c9b0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 shutdown_connections 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8658067500 0x7f86580699c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15be50 0x7f867c14a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c15ba70 0x7f867c15ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 --2- 172.21.15.164:0/1804387547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f867c0cfda0 0x7f867c15d490 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 >> 172.21.15.164:0/1804387547 conn(0x7f867c06ac90 msgr2=0x7f867c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 shutdown_connections 2024-05-08T04:41:47.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.248+0000 7f8680e00640 1 -- 172.21.15.164:0/1804387547 wait complete. 2024-05-08T04:41:47.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:41:47.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.404+0000 7f03bb200640 1 Processor -- start 2024-05-08T04:41:47.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.404+0000 7f03bb200640 1 -- start start 2024-05-08T04:41:47.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03bb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f03bc0c4420 0x7f03bc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03bb200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0c49e0 con 0x7f03bc159760 2024-05-08T04:41:47.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03bb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0c4cc0 con 0x7f03bc15fb60 2024-05-08T04:41:47.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03bb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc15ffd0 con 0x7f03bc0c4420 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f03bc159760 0x7f03bc0c5e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60290/0 (socket says 172.21.15.164:60290) 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9e00640 1 -- 172.21.15.164:0/4089602976 learned_addr learned my addr 172.21.15.164:0/4089602976 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b1400640 1 --2- 172.21.15.164:0/4089602976 >> v2:172.21.15.172:3300/0 conn(0x7f03bc0c4420 0x7f03bc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684930960 0 0) 0x7f03bc0c49e0 con 0x7f03bc159760 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a0003530 con 0x7f03bc159760 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426667169 0 0) 0x7f03a0003530 con 0x7f03bc159760 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 >> v2:172.21.15.172:3300/0 conn(0x7f03bc0c4420 msgr2=0x7f03bc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 --2- 172.21.15.164:0/4089602976 >> v2:172.21.15.172:3300/0 conn(0x7f03bc0c4420 0x7f03bc15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 >> v1:172.21.15.172:6789/0 conn(0x7f03bc15fb60 legacy=0x7f03bc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0c4fa0 con 0x7f03bc159760 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f03ac001a00 con 0x7f03bc159760 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 >> v1:172.21.15.164:6789/0 conn(0x7f03bc159760 legacy=0x7f03bc0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03a00039c0 0x7f03a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a00043c0 0x7f03a0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 0x7f03a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15ffd0 con 0x7f03a0005130 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000c070 con 0x7f03a00043c0 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000c270 con 0x7f03a00039c0 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 0x7f03a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b9e00640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03a00039c0 0x7f03a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03b1400640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a00043c0 0x7f03a0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 -- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03a00039c0 msgr2=0x7f03a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03a00039c0 0x7f03a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 -- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a00043c0 msgr2=0x7f03a0004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a00043c0 0x7f03a0004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.408+0000 7f03ba800640 1 -- 172.21.15.164:0/4089602976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0003530 con 0x7f03a0005130 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03ba800640 1 -- 172.21.15.164:0/4089602976 reap_dead start 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03ba800640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 0x7f03a000b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001860 tx=0x7f03b401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc1c65dd688e72e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40b0070 con 0x7f03a0005130 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b40ab600 con 0x7f03a0005130 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03b9400640 1 -- 172.21.15.164:0/4089602976 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4001460 con 0x7f03a0005130 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 -- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 msgr2=0x7f03a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 0x7f03a000b9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001860 tx=0x7f03b401c500 comp rx=0 tx=0).stop 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 -- 172.21.15.164:0/4089602976 shutdown_connections 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 --2- 172.21.15.164:0/4089602976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03a0005130 0x7f03a000b9d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 -- 172.21.15.164:0/4089602976 >> 172.21.15.164:0/4089602976 conn(0x7f03bc06b670 msgr2=0x7f03bc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 -- 172.21.15.164:0/4089602976 shutdown_connections 2024-05-08T04:41:47.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.412+0000 7f03bb200640 1 -- 172.21.15.164:0/4089602976 wait complete. 2024-05-08T04:41:47.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 Processor -- start 2024-05-08T04:41:47.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- start start 2024-05-08T04:41:47.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0005130 0x7f03bc14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0afe40 0x7f03bc0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15f9b0 con 0x7f03bc0c6630 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15f830 con 0x7f03a0005130 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15e630 con 0x7f03bc0afe40 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57634/0 (socket says 172.21.15.164:57634) 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 -- 172.21.15.164:0/2283700207 learned_addr learned my addr 172.21.15.164:0/2283700207 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03ba800640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0afe40 0x7f03bc0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b9e00640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0005130 0x7f03bc14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 -- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0afe40 msgr2=0x7f03bc0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0afe40 0x7f03bc0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 -- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0005130 msgr2=0x7f03bc14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0005130 0x7f03bc14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 -- 172.21.15.164:0/2283700207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0c6b30 con 0x7f03bc0c6630 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b1400640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f03a4011960 tx=0x7f03a4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400faf0 con 0x7f03bc0c6630 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a400fc90 con 0x7f03bc0c6630 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4014680 con 0x7f03bc0c6630 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc15bf40 con 0x7f03bc0c6630 2024-05-08T04:41:47.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc15c500 con 0x7f03bc0c6630 2024-05-08T04:41:47.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.416+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc04ef30 con 0x7f03bc0c6630 2024-05-08T04:41:47.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.420+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f03a4012040 con 0x7f03bc0c6630 2024-05-08T04:41:47.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.420+0000 7f03b2a00640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 0x7f03980699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.420+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f03a4086120 con 0x7f03bc0c6630 2024-05-08T04:41:47.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.420+0000 7f03b9e00640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 0x7f03980699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.420+0000 7f03b9e00640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 0x7f03980699c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03ac001470 tx=0x7f03ac006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.424+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03a4019030 con 0x7f03bc0c6630 2024-05-08T04:41:47.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.580+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03bc068170 con 0x7f03bc0c6630 2024-05-08T04:41:47.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03b2a00640 1 -- 172.21.15.164:0/2283700207 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16808 (secure 0 0 0) 0x7f03a405c070 con 0x7f03bc0c6630 2024-05-08T04:41:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:47.588 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":86,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:44.861638+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:41:47.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 msgr2=0x7f03980699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 0x7f03980699c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03ac001470 tx=0x7f03ac006040 comp rx=0 tx=0).stop 2024-05-08T04:41:47.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 msgr2=0x7f03bc0af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f03a4011960 tx=0x7f03a4011e20 comp rx=0 tx=0).stop 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 shutdown_connections 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0398067500 0x7f03980699c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03bc0afe40 0x7f03bc0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03bc0c6630 0x7f03bc0af900 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 --2- 172.21.15.164:0/2283700207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03a0005130 0x7f03bc14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 >> 172.21.15.164:0/2283700207 conn(0x7f03bc06b670 msgr2=0x7f03bc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 shutdown_connections 2024-05-08T04:41:47.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.584+0000 7f03bb200640 1 -- 172.21.15.164:0/2283700207 wait complete. 2024-05-08T04:41:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:41:47.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:47.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.740+0000 7fde0f800640 1 Processor -- start 2024-05-08T04:41:47.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.740+0000 7fde0f800640 1 -- start start 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0f800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fde080b26c0 0x7fde080cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0f800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0806c770 con 0x7fde080cfbd0 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0f800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0806ca50 con 0x7fde0815ab60 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0f800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde080d0060 con 0x7fde080b26c0 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fde080cfbd0 0x7fde080cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60304/0 (socket says 172.21.15.164:60304) 2024-05-08T04:41:47.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 -- 172.21.15.164:0/681266852 learned_addr learned my addr 172.21.15.164:0/681266852 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:47.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fddffe00640 1 --2- 172.21.15.164:0/681266852 >> v2:172.21.15.172:3300/0 conn(0x7fde080b26c0 0x7fde080cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694611116 0 0) 0x7fde0806c770 con 0x7fde080cfbd0 2024-05-08T04:41:47.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddec003530 con 0x7fde080cfbd0 2024-05-08T04:41:47.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470866663 0 0) 0x7fde0806ca50 con 0x7fde0815ab60 2024-05-08T04:41:47.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde0806c770 con 0x7fde0815ab60 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769756107 0 0) 0x7fddec003530 con 0x7fde080cfbd0 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 >> v2:172.21.15.172:3300/0 conn(0x7fde080b26c0 msgr2=0x7fde080cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 --2- 172.21.15.164:0/681266852 >> v2:172.21.15.172:3300/0 conn(0x7fde080b26c0 0x7fde080cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 >> v1:172.21.15.172:6789/0 conn(0x7fde0815ab60 legacy=0x7fde08072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0806cd30 con 0x7fde080cfbd0 2024-05-08T04:41:47.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fde00001a00 con 0x7fde080cfbd0 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 >> v1:172.21.15.164:6789/0 conn(0x7fde080cfbd0 legacy=0x7fde080cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddec003b50 0x7fddec003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddec004810 0x7fddec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fddec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec00c020 con 0x7fddec0051d0 2024-05-08T04:41:47.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec00c1d0 con 0x7fddec003b50 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec00c3d0 con 0x7fddec004810 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fddec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0ce00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddec003b50 0x7fddec003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fddffe00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddec004810 0x7fddec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 -- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddec004810 msgr2=0x7fddec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddec004810 0x7fddec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 -- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddec003b50 msgr2=0x7fddec003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:47.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddec003b50 0x7fddec003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fde0d800640 1 -- 172.21.15.164:0/681266852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec003530 con 0x7fddec0051d0 2024-05-08T04:41:47.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.744+0000 7fddffe00640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddec004810 0x7fddec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:47.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0d800640 1 -- 172.21.15.164:0/681266852 reap_dead start 2024-05-08T04:41:47.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0d800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fddec00b9d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a7e0 tx=0x7fde0001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a840338b7cf20299 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000a430 con 0x7fddec0051d0 2024-05-08T04:41:47.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde000aa6b0 con 0x7fddec0051d0 2024-05-08T04:41:47.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde07e00640 1 -- 172.21.15.164:0/681266852 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000aaaa0 con 0x7fddec0051d0 2024-05-08T04:41:47.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 -- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 msgr2=0x7fddec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fddec00b9d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a7e0 tx=0x7fde0001fa40 comp rx=0 tx=0).stop 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 -- 172.21.15.164:0/681266852 shutdown_connections 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 --2- 172.21.15.164:0/681266852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fddec00b9d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 -- 172.21.15.164:0/681266852 >> 172.21.15.164:0/681266852 conn(0x7fde0806ac90 msgr2=0x7fde0806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 -- 172.21.15.164:0/681266852 shutdown_connections 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 -- 172.21.15.164:0/681266852 wait complete. 2024-05-08T04:41:47.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.748+0000 7fde0f800640 1 Processor -- start 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- start start 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde08150930 0x7fde08153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde080af900 0x7fde0815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080aed30 con 0x7fddec0051d0 2024-05-08T04:41:47.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0806d2f0 con 0x7fde08150930 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080aeeb0 con 0x7fde080af900 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57662/0 (socket says 172.21.15.164:57662) 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 -- 172.21.15.164:0/2324057153 learned_addr learned my addr 172.21.15.164:0/2324057153 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 -- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde080af900 msgr2=0x7fde0815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde080af900 0x7fde0815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 -- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde08150930 msgr2=0x7fde08153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde08150930 0x7fde08153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 -- 172.21.15.164:0/2324057153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080b0870 con 0x7fddec0051d0 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0ce00640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fddf800f7a0 tx=0x7fddf800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf801a070 con 0x7fddec0051d0 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800fe50 con 0x7fddec0051d0 2024-05-08T04:41:47.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde08152b40 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8007030 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde08153100 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde08152930 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.752+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fddf801d070 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.756+0000 7fde05600640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 0x7fdde40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.756+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fddf80852d0 con 0x7fddec0051d0 2024-05-08T04:41:47.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.756+0000 7fddffe00640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 0x7fdde40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:47.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.756+0000 7fddffe00640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 0x7fdde40699c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde08072760 tx=0x7fddf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:47.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.760+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddf80559f0 con 0x7fddec0051d0 2024-05-08T04:41:47.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.912+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde08069d60 con 0x7fdde4067500 2024-05-08T04:41:47.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.916+0000 7fde05600640 1 -- 172.21.15.164:0/2324057153 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240193 (secure 0 0 0) 0x7fde08069d60 con 0x7fdde4067500 2024-05-08T04:41:47.925 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:47.926 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:47.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 msgr2=0x7fdde40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 0x7fdde40699c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde08072760 tx=0x7fddf0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:47.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 msgr2=0x7fde080b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fddf800f7a0 tx=0x7fddf800fc60 comp rx=0 tx=0).stop 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 shutdown_connections 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdde4067500 0x7fdde40699c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde080af900 0x7fde0815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde08150930 0x7fde08153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 --2- 172.21.15.164:0/2324057153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddec0051d0 0x7fde080b0290 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 >> 172.21.15.164:0/2324057153 conn(0x7fde0806ac90 msgr2=0x7fde080b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 shutdown_connections 2024-05-08T04:41:47.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:47.920+0000 7fde0f800640 1 -- 172.21.15.164:0/2324057153 wait complete. 2024-05-08T04:41:47.945 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-08T04:41:47.470353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250284,"kb_used_data":9104,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724436,"statfs":{"total":773094113280,"available":772837822464,"internally_reserved":0,"allocated":9322496,"data_stored":4645182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619869"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856695+0000","last_change":"2024-05-08T04:41:41.856695+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856695+0000","last_undegraded":"2024-05-08T04:41:41.856695+0000","last_fullsized":"2024-05-08T04:41:41.856695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855972+0000","last_change":"2024-05-08T04:41:41.855972+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855972+0000","last_undegraded":"2024-05-08T04:41:41.855972+0000","last_fullsized":"2024-05-08T04:41:41.855972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854155+0000","last_change":"2024-05-08T04:41:41.854155+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854155+0000","last_undegraded":"2024-05-08T04:41:41.854155+0000","last_fullsized":"2024-05-08T04:41:41.854155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854901+0000","last_change":"2024-05-08T04:41:41.854901+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854901+0000","last_undegraded":"2024-05-08T04:41:41.854901+0000","last_fullsized":"2024-05-08T04:41:41.854901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856760+0000","last_change":"2024-05-08T04:41:41.856760+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856760+0000","last_undegraded":"2024-05-08T04:41:41.856760+0000","last_fullsized":"2024-05-08T04:41:41.856760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856885+0000","last_change":"2024-05-08T04:41:41.856885+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856885+0000","last_undegraded":"2024-05-08T04:41:41.856885+0000","last_fullsized":"2024-05-08T04:41:41.856885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854398+0000","last_change":"2024-05-08T04:41:41.854398+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854398+0000","last_undegraded":"2024-05-08T04:41:41.854398+0000","last_fullsized":"2024-05-08T04:41:41.854398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851663+0000","last_change":"2024-05-08T04:41:41.851663+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851663+0000","last_undegraded":"2024-05-08T04:41:41.851663+0000","last_fullsized":"2024-05-08T04:41:41.851663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856219+0000","last_change":"2024-05-08T04:41:41.856219+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856219+0000","last_undegraded":"2024-05-08T04:41:41.856219+0000","last_fullsized":"2024-05-08T04:41:41.856219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855707+0000","last_change":"2024-05-08T04:41:41.855707+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855707+0000","last_undegraded":"2024-05-08T04:41:41.855707+0000","last_fullsized":"2024-05-08T04:41:41.855707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.857601+0000","last_change":"2024-05-08T04:41:41.857601+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.857601+0000","last_undegraded":"2024-05-08T04:41:41.857601+0000","last_fullsized":"2024-05-08T04:41:41.857601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851671+0000","last_change":"2024-05-08T04:41:41.851671+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851671+0000","last_undegraded":"2024-05-08T04:41:41.851671+0000","last_fullsized":"2024-05-08T04:41:41.851671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854185+0000","last_change":"2024-05-08T04:41:41.854185+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854185+0000","last_undegraded":"2024-05-08T04:41:41.854185+0000","last_fullsized":"2024-05-08T04:41:41.854185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855729+0000","last_change":"2024-05-08T04:41:41.855729+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855729+0000","last_undegraded":"2024-05-08T04:41:41.855729+0000","last_fullsized":"2024-05-08T04:41:41.855729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854833+0000","last_change":"2024-05-08T04:41:41.854833+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854833+0000","last_undegraded":"2024-05-08T04:41:41.854833+0000","last_fullsized":"2024-05-08T04:41:41.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854497+0000","last_change":"2024-05-08T04:41:41.854497+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854497+0000","last_undegraded":"2024-05-08T04:41:41.854497+0000","last_fullsized":"2024-05-08T04:41:41.854497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849579+0000","last_change":"2024-05-08T04:41:41.849579+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849579+0000","last_undegraded":"2024-05-08T04:41:41.849579+0000","last_fullsized":"2024-05-08T04:41:41.849579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853253+0000","last_change":"2024-05-08T04:41:41.853253+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853253+0000","last_undegraded":"2024-05-08T04:41:41.853253+0000","last_fullsized":"2024-05-08T04:41:41.853253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851801+0000","last_change":"2024-05-08T04:41:41.851801+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851801+0000","last_undegraded":"2024-05-08T04:41:41.851801+0000","last_fullsized":"2024-05-08T04:41:41.851801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853284+0000","last_change":"2024-05-08T04:41:41.853284+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853284+0000","last_undegraded":"2024-05-08T04:41:41.853284+0000","last_fullsized":"2024-05-08T04:41:41.853284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851679+0000","last_change":"2024-05-08T04:41:41.851679+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851679+0000","last_undegraded":"2024-05-08T04:41:41.851679+0000","last_fullsized":"2024-05-08T04:41:41.851679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856286+0000","last_change":"2024-05-08T04:41:41.856286+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856286+0000","last_undegraded":"2024-05-08T04:41:41.856286+0000","last_fullsized":"2024-05-08T04:41:41.856286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855160+0000","last_change":"2024-05-08T04:41:41.855160+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855160+0000","last_undegraded":"2024-05-08T04:41:41.855160+0000","last_fullsized":"2024-05-08T04:41:41.855160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854066+0000","last_change":"2024-05-08T04:41:41.854066+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854066+0000","last_undegraded":"2024-05-08T04:41:41.854066+0000","last_fullsized":"2024-05-08T04:41:41.854066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856353+0000","last_change":"2024-05-08T04:41:41.856353+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856353+0000","last_undegraded":"2024-05-08T04:41:41.856353+0000","last_fullsized":"2024-05-08T04:41:41.856353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855123+0000","last_change":"2024-05-08T04:41:41.855123+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855123+0000","last_undegraded":"2024-05-08T04:41:41.855123+0000","last_fullsized":"2024-05-08T04:41:41.855123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856524+0000","last_change":"2024-05-08T04:41:41.856524+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856524+0000","last_undegraded":"2024-05-08T04:41:41.856524+0000","last_fullsized":"2024-05-08T04:41:41.856524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852743+0000","last_change":"2024-05-08T04:41:41.852743+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852743+0000","last_undegraded":"2024-05-08T04:41:41.852743+0000","last_fullsized":"2024-05-08T04:41:41.852743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T04:41:41.852517+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:41.852517+0000","last_peered":"2024-05-08T04:41:41.852517+0000","last_clean":"2024-05-08T04:41:41.852517+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:41.852517+0000","last_undegraded":"2024-05-08T04:41:41.852517+0000","last_fullsized":"2024-05-08T04:41:41.852517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:47.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 Processor -- start 2024-05-08T04:41:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 -- start start 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 0x7f473c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f473c0b26c0 0x7f473c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 0x7f473c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c0d0060 con 0x7f473c0b26c0 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c072640 con 0x7f473c15ab60 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4742a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06c770 con 0x7f473c0cfbd0 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4733400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f473c0b26c0 0x7f473c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4733400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f473c0b26c0 0x7f473c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57672/0 (socket says 172.21.15.164:57672) 2024-05-08T04:41:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.120+0000 7f4733400640 1 -- 172.21.15.164:0/2180350190 learned_addr learned my addr 172.21.15.164:0/2180350190 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f473be00640 1 --2- 172.21.15.164:0/2180350190 >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 0x7f473c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4740a00640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 0x7f473c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 -- 172.21.15.164:0/2180350190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 msgr2=0x7f473c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 0x7f473c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 -- 172.21.15.164:0/2180350190 >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 msgr2=0x7f473c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 --2- 172.21.15.164:0/2180350190 >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 0x7f473c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 -- 172.21.15.164:0/2180350190 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c06ca50 con 0x7f473c0b26c0 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f473be00640 1 --2- 172.21.15.164:0/2180350190 >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 0x7f473c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4740a00640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 0x7f473c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4733400640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0b26c0 0x7f473c0cf690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cd420 tx=0x7f472401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f06b180c4de89e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f473b400640 1 -- 172.21.15.164:0/2180350190 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240b4070 con 0x7f473c0b26c0 2024-05-08T04:41:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f473b400640 1 -- 172.21.15.164:0/2180350190 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47240a7a80 con 0x7f473c0b26c0 2024-05-08T04:41:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f473b400640 1 -- 172.21.15.164:0/2180350190 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240a7e50 con 0x7f473c0b26c0 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4742a00640 1 -- 172.21.15.164:0/2180350190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0b26c0 msgr2=0x7f473c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.124+0000 7f4742a00640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0b26c0 0x7f473c0cf690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cd420 tx=0x7f472401ebf0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- 172.21.15.164:0/2180350190 shutdown_connections 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c0cfbd0 0x7f473c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- 172.21.15.164:0/2180350190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0b26c0 0x7f473c0cf690 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- 172.21.15.164:0/2180350190 >> v2:172.21.15.172:3300/0 conn(0x7f473c15ab60 0x7f473c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- 172.21.15.164:0/2180350190 >> 172.21.15.164:0/2180350190 conn(0x7f473c06ac90 msgr2=0x7f473c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- 172.21.15.164:0/2180350190 shutdown_connections 2024-05-08T04:41:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- 172.21.15.164:0/2180350190 wait complete. 2024-05-08T04:41:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 Processor -- start 2024-05-08T04:41:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- start start 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c0b26c0 0x7f473c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c15ab60 0x7f473c1411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06c550 con 0x7f473c0cfbd0 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06c3d0 con 0x7f473c0b26c0 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4742a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0aed30 con 0x7f473c15ab60 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57688/0 (socket says 172.21.15.164:57688) 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f473be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c0b26c0 0x7f473c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 -- 172.21.15.164:0/2795468096 learned_addr learned my addr 172.21.15.164:0/2795468096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4740a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c15ab60 0x7f473c1411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 -- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c15ab60 msgr2=0x7f473c1411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c15ab60 0x7f473c1411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 -- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c0b26c0 msgr2=0x7f473c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c0b26c0 0x7f473c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 -- 172.21.15.164:0/2795468096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c155c90 con 0x7f473c0cfbd0 2024-05-08T04:41:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.128+0000 7f4733400640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f472401d2a0 tx=0x7f472401b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240b4040 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47240bf070 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c155f70 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240ba6a0 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c156530 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c069d60 con 0x7f473c0cfbd0 2024-05-08T04:41:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f47240ba840 con 0x7f473c0cfbd0 2024-05-08T04:41:48.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 0x7f47180699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4724123130 con 0x7f473c0cfbd0 2024-05-08T04:41:48.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f473be00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 0x7f47180699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.132+0000 7f473be00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 0x7f47180699c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f472c006fd0 tx=0x7f472c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.136+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47240187e0 con 0x7f473c0cfbd0 2024-05-08T04:41:48.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.288+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f473c06d9a0 con 0x7f4718067500 2024-05-08T04:41:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.292+0000 7f4738c00640 1 -- 172.21.15.164:0/2795468096 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240193 (secure 0 0 0) 0x7f473c06d9a0 con 0x7f4718067500 2024-05-08T04:41:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:48.303 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 msgr2=0x7f47180699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 0x7f47180699c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f472c006fd0 tx=0x7f472c008040 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 msgr2=0x7f473c140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f472401d2a0 tx=0x7f472401b990 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 shutdown_connections 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4718067500 0x7f47180699c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f473c15ab60 0x7f473c1411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f473c0cfbd0 0x7f473c140c60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 --2- 172.21.15.164:0/2795468096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c0b26c0 0x7f473c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 >> 172.21.15.164:0/2795468096 conn(0x7f473c06ac90 msgr2=0x7f473c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 shutdown_connections 2024-05-08T04:41:48.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.296+0000 7f4742a00640 1 -- 172.21.15.164:0/2795468096 wait complete. 2024-05-08T04:41:48.323 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-08T04:41:47.470353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250284,"kb_used_data":9104,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724436,"statfs":{"total":773094113280,"available":772837822464,"internally_reserved":0,"allocated":9322496,"data_stored":4645182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619869"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856695+0000","last_change":"2024-05-08T04:41:41.856695+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856695+0000","last_undegraded":"2024-05-08T04:41:41.856695+0000","last_fullsized":"2024-05-08T04:41:41.856695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855972+0000","last_change":"2024-05-08T04:41:41.855972+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855972+0000","last_undegraded":"2024-05-08T04:41:41.855972+0000","last_fullsized":"2024-05-08T04:41:41.855972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854155+0000","last_change":"2024-05-08T04:41:41.854155+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854155+0000","last_undegraded":"2024-05-08T04:41:41.854155+0000","last_fullsized":"2024-05-08T04:41:41.854155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854901+0000","last_change":"2024-05-08T04:41:41.854901+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854901+0000","last_undegraded":"2024-05-08T04:41:41.854901+0000","last_fullsized":"2024-05-08T04:41:41.854901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856760+0000","last_change":"2024-05-08T04:41:41.856760+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856760+0000","last_undegraded":"2024-05-08T04:41:41.856760+0000","last_fullsized":"2024-05-08T04:41:41.856760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856885+0000","last_change":"2024-05-08T04:41:41.856885+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856885+0000","last_undegraded":"2024-05-08T04:41:41.856885+0000","last_fullsized":"2024-05-08T04:41:41.856885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854398+0000","last_change":"2024-05-08T04:41:41.854398+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854398+0000","last_undegraded":"2024-05-08T04:41:41.854398+0000","last_fullsized":"2024-05-08T04:41:41.854398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851663+0000","last_change":"2024-05-08T04:41:41.851663+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851663+0000","last_undegraded":"2024-05-08T04:41:41.851663+0000","last_fullsized":"2024-05-08T04:41:41.851663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856219+0000","last_change":"2024-05-08T04:41:41.856219+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856219+0000","last_undegraded":"2024-05-08T04:41:41.856219+0000","last_fullsized":"2024-05-08T04:41:41.856219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855707+0000","last_change":"2024-05-08T04:41:41.855707+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855707+0000","last_undegraded":"2024-05-08T04:41:41.855707+0000","last_fullsized":"2024-05-08T04:41:41.855707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.857601+0000","last_change":"2024-05-08T04:41:41.857601+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.857601+0000","last_undegraded":"2024-05-08T04:41:41.857601+0000","last_fullsized":"2024-05-08T04:41:41.857601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851671+0000","last_change":"2024-05-08T04:41:41.851671+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851671+0000","last_undegraded":"2024-05-08T04:41:41.851671+0000","last_fullsized":"2024-05-08T04:41:41.851671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854185+0000","last_change":"2024-05-08T04:41:41.854185+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854185+0000","last_undegraded":"2024-05-08T04:41:41.854185+0000","last_fullsized":"2024-05-08T04:41:41.854185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855729+0000","last_change":"2024-05-08T04:41:41.855729+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855729+0000","last_undegraded":"2024-05-08T04:41:41.855729+0000","last_fullsized":"2024-05-08T04:41:41.855729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854833+0000","last_change":"2024-05-08T04:41:41.854833+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854833+0000","last_undegraded":"2024-05-08T04:41:41.854833+0000","last_fullsized":"2024-05-08T04:41:41.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854497+0000","last_change":"2024-05-08T04:41:41.854497+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854497+0000","last_undegraded":"2024-05-08T04:41:41.854497+0000","last_fullsized":"2024-05-08T04:41:41.854497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849579+0000","last_change":"2024-05-08T04:41:41.849579+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849579+0000","last_undegraded":"2024-05-08T04:41:41.849579+0000","last_fullsized":"2024-05-08T04:41:41.849579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853253+0000","last_change":"2024-05-08T04:41:41.853253+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853253+0000","last_undegraded":"2024-05-08T04:41:41.853253+0000","last_fullsized":"2024-05-08T04:41:41.853253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851801+0000","last_change":"2024-05-08T04:41:41.851801+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851801+0000","last_undegraded":"2024-05-08T04:41:41.851801+0000","last_fullsized":"2024-05-08T04:41:41.851801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853284+0000","last_change":"2024-05-08T04:41:41.853284+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853284+0000","last_undegraded":"2024-05-08T04:41:41.853284+0000","last_fullsized":"2024-05-08T04:41:41.853284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851679+0000","last_change":"2024-05-08T04:41:41.851679+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851679+0000","last_undegraded":"2024-05-08T04:41:41.851679+0000","last_fullsized":"2024-05-08T04:41:41.851679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856286+0000","last_change":"2024-05-08T04:41:41.856286+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856286+0000","last_undegraded":"2024-05-08T04:41:41.856286+0000","last_fullsized":"2024-05-08T04:41:41.856286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855160+0000","last_change":"2024-05-08T04:41:41.855160+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855160+0000","last_undegraded":"2024-05-08T04:41:41.855160+0000","last_fullsized":"2024-05-08T04:41:41.855160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854066+0000","last_change":"2024-05-08T04:41:41.854066+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854066+0000","last_undegraded":"2024-05-08T04:41:41.854066+0000","last_fullsized":"2024-05-08T04:41:41.854066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856353+0000","last_change":"2024-05-08T04:41:41.856353+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856353+0000","last_undegraded":"2024-05-08T04:41:41.856353+0000","last_fullsized":"2024-05-08T04:41:41.856353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855123+0000","last_change":"2024-05-08T04:41:41.855123+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855123+0000","last_undegraded":"2024-05-08T04:41:41.855123+0000","last_fullsized":"2024-05-08T04:41:41.855123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856524+0000","last_change":"2024-05-08T04:41:41.856524+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856524+0000","last_undegraded":"2024-05-08T04:41:41.856524+0000","last_fullsized":"2024-05-08T04:41:41.856524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852743+0000","last_change":"2024-05-08T04:41:41.852743+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852743+0000","last_undegraded":"2024-05-08T04:41:41.852743+0000","last_fullsized":"2024-05-08T04:41:41.852743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T04:41:41.852517+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:41.852517+0000","last_peered":"2024-05-08T04:41:41.852517+0000","last_clean":"2024-05-08T04:41:41.852517+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:41.852517+0000","last_undegraded":"2024-05-08T04:41:41.852517+0000","last_fullsized":"2024-05-08T04:41:41.852517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:48.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:48.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 Processor -- start 2024-05-08T04:41:48.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 -- start start 2024-05-08T04:41:48.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 0x7f80e4160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80e415be50 0x7f80e40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40cd9f0 con 0x7f80e40aeda0 2024-05-08T04:41:48.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e415fe70 con 0x7f80e415be50 2024-05-08T04:41:48.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40cdcd0 con 0x7f80e415ba70 2024-05-08T04:41:48.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80e415be50 0x7f80e40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 0x7f80e4160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80e415be50 0x7f80e40cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:43348/0 (socket says 172.21.15.164:43348) 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 -- 172.21.15.164:0/838493496 learned_addr learned my addr 172.21.15.164:0/838493496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 0x7f80e4160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42424/0 (socket says 172.21.15.164:42424) 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e2800640 1 --1- 172.21.15.164:0/838493496 >> v1:172.21.15.172:6789/0 conn(0x7f80e40aeda0 0x7f80e40b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:59500/0 (socket says 172.21.15.164:59500) 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 -- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 msgr2=0x7f80e4160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 --2- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 0x7f80e4160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 -- 172.21.15.164:0/838493496 >> v1:172.21.15.172:6789/0 conn(0x7f80e40aeda0 legacy=0x7f80e40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.500+0000 7f80e1400640 1 -- 172.21.15.164:0/838493496 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e415f820 con 0x7f80e415be50 2024-05-08T04:41:48.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e1400640 1 --2- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e415be50 0x7f80e40cf690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009870 tx=0x7f80d001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e985c98a76d1e335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e0a00640 1 -- 172.21.15.164:0/838493496 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00ab640 con 0x7f80e415be50 2024-05-08T04:41:48.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e0a00640 1 -- 172.21.15.164:0/838493496 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d00b0070 con 0x7f80e415be50 2024-05-08T04:41:48.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e0a00640 1 -- 172.21.15.164:0/838493496 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00aba00 con 0x7f80e415be50 2024-05-08T04:41:48.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 -- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e415be50 msgr2=0x7f80e40cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 --2- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e415be50 0x7f80e40cf690 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009870 tx=0x7f80d001c530 comp rx=0 tx=0).stop 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 -- 172.21.15.164:0/838493496 shutdown_connections 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 --2- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e415be50 0x7f80e40cf690 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 --2- 172.21.15.164:0/838493496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415ba70 0x7f80e4160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 -- 172.21.15.164:0/838493496 >> 172.21.15.164:0/838493496 conn(0x7f80e406ac90 msgr2=0x7f80e406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 -- 172.21.15.164:0/838493496 shutdown_connections 2024-05-08T04:41:48.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.504+0000 7f80e9200640 1 -- 172.21.15.164:0/838493496 wait complete. 2024-05-08T04:41:48.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 Processor -- start 2024-05-08T04:41:48.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- start start 2024-05-08T04:41:48.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e40aeda0 0x7f80e40ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415be50 0x7f80e415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4168d40 con 0x7f80e415ba70 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4168bc0 con 0x7f80e40aeda0 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4168ec0 con 0x7f80e415be50 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57698/0 (socket says 172.21.15.164:57698) 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 -- 172.21.15.164:0/3277098012 learned_addr learned my addr 172.21.15.164:0/3277098012 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1e00640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e40aeda0 0x7f80e40ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e2800640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415be50 0x7f80e415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 -- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415be50 msgr2=0x7f80e415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415be50 0x7f80e415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 -- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e40aeda0 msgr2=0x7f80e40ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e40aeda0 0x7f80e40ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 -- 172.21.15.164:0/3277098012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4150190 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e1400640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ad040 tx=0x7f80d001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00be3f0 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d00be590 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e4150470 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00be980 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.508+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e4150a30 con 0x7f80e415ba70 2024-05-08T04:41:48.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4069d60 con 0x7f80e415ba70 2024-05-08T04:41:48.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f80d00b43f0 con 0x7f80e415ba70 2024-05-08T04:41:48.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80da000640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 0x7f80c40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f80d0120090 con 0x7f80e415ba70 2024-05-08T04:41:48.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80e1e00640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 0x7f80c40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.512+0000 7f80e1e00640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 0x7f80c40699c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80d40113e0 tx=0x7f80d400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.516+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80d00c4060 con 0x7f80e415ba70 2024-05-08T04:41:48.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.668+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80e40cf9b0 con 0x7f80c4067500 2024-05-08T04:41:48.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.676+0000 7f80da000640 1 -- 172.21.15.164:0/3277098012 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240193 (secure 0 0 0) 0x7f80e40cf9b0 con 0x7f80c4067500 2024-05-08T04:41:48.683 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:48.684 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 msgr2=0x7f80c40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 0x7f80c40699c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80d40113e0 tx=0x7f80d400b040 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 msgr2=0x7f80e4146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ad040 tx=0x7f80d001ad70 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 shutdown_connections 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f80c4067500 0x7f80c40699c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80e415be50 0x7f80e415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e415ba70 0x7f80e4146b40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 --2- 172.21.15.164:0/3277098012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80e40aeda0 0x7f80e40ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 >> 172.21.15.164:0/3277098012 conn(0x7f80e406ac90 msgr2=0x7f80e40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:48.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 shutdown_connections 2024-05-08T04:41:48.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.680+0000 7f80e9200640 1 -- 172.21.15.164:0/3277098012 wait complete. 2024-05-08T04:41:48.703 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-08T04:41:47.470353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250284,"kb_used_data":9104,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724436,"statfs":{"total":773094113280,"available":772837822464,"internally_reserved":0,"allocated":9322496,"data_stored":4645182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619869"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856695+0000","last_change":"2024-05-08T04:41:41.856695+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856695+0000","last_undegraded":"2024-05-08T04:41:41.856695+0000","last_fullsized":"2024-05-08T04:41:41.856695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855972+0000","last_change":"2024-05-08T04:41:41.855972+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855972+0000","last_undegraded":"2024-05-08T04:41:41.855972+0000","last_fullsized":"2024-05-08T04:41:41.855972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854155+0000","last_change":"2024-05-08T04:41:41.854155+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854155+0000","last_undegraded":"2024-05-08T04:41:41.854155+0000","last_fullsized":"2024-05-08T04:41:41.854155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854901+0000","last_change":"2024-05-08T04:41:41.854901+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854901+0000","last_undegraded":"2024-05-08T04:41:41.854901+0000","last_fullsized":"2024-05-08T04:41:41.854901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856760+0000","last_change":"2024-05-08T04:41:41.856760+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856760+0000","last_undegraded":"2024-05-08T04:41:41.856760+0000","last_fullsized":"2024-05-08T04:41:41.856760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856885+0000","last_change":"2024-05-08T04:41:41.856885+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856885+0000","last_undegraded":"2024-05-08T04:41:41.856885+0000","last_fullsized":"2024-05-08T04:41:41.856885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854398+0000","last_change":"2024-05-08T04:41:41.854398+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854398+0000","last_undegraded":"2024-05-08T04:41:41.854398+0000","last_fullsized":"2024-05-08T04:41:41.854398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851663+0000","last_change":"2024-05-08T04:41:41.851663+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851663+0000","last_undegraded":"2024-05-08T04:41:41.851663+0000","last_fullsized":"2024-05-08T04:41:41.851663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856219+0000","last_change":"2024-05-08T04:41:41.856219+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856219+0000","last_undegraded":"2024-05-08T04:41:41.856219+0000","last_fullsized":"2024-05-08T04:41:41.856219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855707+0000","last_change":"2024-05-08T04:41:41.855707+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855707+0000","last_undegraded":"2024-05-08T04:41:41.855707+0000","last_fullsized":"2024-05-08T04:41:41.855707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.857601+0000","last_change":"2024-05-08T04:41:41.857601+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.857601+0000","last_undegraded":"2024-05-08T04:41:41.857601+0000","last_fullsized":"2024-05-08T04:41:41.857601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851671+0000","last_change":"2024-05-08T04:41:41.851671+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851671+0000","last_undegraded":"2024-05-08T04:41:41.851671+0000","last_fullsized":"2024-05-08T04:41:41.851671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854185+0000","last_change":"2024-05-08T04:41:41.854185+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854185+0000","last_undegraded":"2024-05-08T04:41:41.854185+0000","last_fullsized":"2024-05-08T04:41:41.854185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855729+0000","last_change":"2024-05-08T04:41:41.855729+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855729+0000","last_undegraded":"2024-05-08T04:41:41.855729+0000","last_fullsized":"2024-05-08T04:41:41.855729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854833+0000","last_change":"2024-05-08T04:41:41.854833+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854833+0000","last_undegraded":"2024-05-08T04:41:41.854833+0000","last_fullsized":"2024-05-08T04:41:41.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854497+0000","last_change":"2024-05-08T04:41:41.854497+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854497+0000","last_undegraded":"2024-05-08T04:41:41.854497+0000","last_fullsized":"2024-05-08T04:41:41.854497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849579+0000","last_change":"2024-05-08T04:41:41.849579+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849579+0000","last_undegraded":"2024-05-08T04:41:41.849579+0000","last_fullsized":"2024-05-08T04:41:41.849579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853253+0000","last_change":"2024-05-08T04:41:41.853253+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853253+0000","last_undegraded":"2024-05-08T04:41:41.853253+0000","last_fullsized":"2024-05-08T04:41:41.853253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851801+0000","last_change":"2024-05-08T04:41:41.851801+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851801+0000","last_undegraded":"2024-05-08T04:41:41.851801+0000","last_fullsized":"2024-05-08T04:41:41.851801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853284+0000","last_change":"2024-05-08T04:41:41.853284+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853284+0000","last_undegraded":"2024-05-08T04:41:41.853284+0000","last_fullsized":"2024-05-08T04:41:41.853284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851679+0000","last_change":"2024-05-08T04:41:41.851679+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851679+0000","last_undegraded":"2024-05-08T04:41:41.851679+0000","last_fullsized":"2024-05-08T04:41:41.851679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856286+0000","last_change":"2024-05-08T04:41:41.856286+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856286+0000","last_undegraded":"2024-05-08T04:41:41.856286+0000","last_fullsized":"2024-05-08T04:41:41.856286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855160+0000","last_change":"2024-05-08T04:41:41.855160+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855160+0000","last_undegraded":"2024-05-08T04:41:41.855160+0000","last_fullsized":"2024-05-08T04:41:41.855160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854066+0000","last_change":"2024-05-08T04:41:41.854066+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854066+0000","last_undegraded":"2024-05-08T04:41:41.854066+0000","last_fullsized":"2024-05-08T04:41:41.854066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856353+0000","last_change":"2024-05-08T04:41:41.856353+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856353+0000","last_undegraded":"2024-05-08T04:41:41.856353+0000","last_fullsized":"2024-05-08T04:41:41.856353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855123+0000","last_change":"2024-05-08T04:41:41.855123+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855123+0000","last_undegraded":"2024-05-08T04:41:41.855123+0000","last_fullsized":"2024-05-08T04:41:41.855123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856524+0000","last_change":"2024-05-08T04:41:41.856524+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856524+0000","last_undegraded":"2024-05-08T04:41:41.856524+0000","last_fullsized":"2024-05-08T04:41:41.856524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852743+0000","last_change":"2024-05-08T04:41:41.852743+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852743+0000","last_undegraded":"2024-05-08T04:41:41.852743+0000","last_fullsized":"2024-05-08T04:41:41.852743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T04:41:41.852517+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:41.852517+0000","last_peered":"2024-05-08T04:41:41.852517+0000","last_clean":"2024-05-08T04:41:41.852517+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:41.852517+0000","last_undegraded":"2024-05-08T04:41:41.852517+0000","last_fullsized":"2024-05-08T04:41:41.852517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2024-05-08T04:41:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-05-08T04:41:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2024-05-08T04:41:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-05-08T04:41:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-05-08T04:41:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856695+0000', 'last_change': '2024-05-08T04:41:41.856695+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856695+0000', 'last_undegraded': '2024-05-08T04:41:41.856695+0000', 'last_fullsized': '2024-05-08T04:41:41.856695+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.855972+0000', 'last_change': '2024-05-08T04:41:41.855972+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.855972+0000', 'last_undegraded': '2024-05-08T04:41:41.855972+0000', 'last_fullsized': '2024-05-08T04:41:41.855972+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854155+0000', 'last_change': '2024-05-08T04:41:41.854155+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854155+0000', 'last_undegraded': '2024-05-08T04:41:41.854155+0000', 'last_fullsized': '2024-05-08T04:41:41.854155+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-05-08T04:41:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854901+0000', 'last_change': '2024-05-08T04:41:41.854901+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854901+0000', 'last_undegraded': '2024-05-08T04:41:41.854901+0000', 'last_fullsized': '2024-05-08T04:41:41.854901+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856760+0000', 'last_change': '2024-05-08T04:41:41.856760+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856760+0000', 'last_undegraded': '2024-05-08T04:41:41.856760+0000', 'last_fullsized': '2024-05-08T04:41:41.856760+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856885+0000', 'last_change': '2024-05-08T04:41:41.856885+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856885+0000', 'last_undegraded': '2024-05-08T04:41:41.856885+0000', 'last_fullsized': '2024-05-08T04:41:41.856885+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854398+0000', 'last_change': '2024-05-08T04:41:41.854398+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854398+0000', 'last_undegraded': '2024-05-08T04:41:41.854398+0000', 'last_fullsized': '2024-05-08T04:41:41.854398+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.851663+0000', 'last_change': '2024-05-08T04:41:41.851663+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.851663+0000', 'last_undegraded': '2024-05-08T04:41:41.851663+0000', 'last_fullsized': '2024-05-08T04:41:41.851663+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856219+0000', 'last_change': '2024-05-08T04:41:41.856219+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856219+0000', 'last_undegraded': '2024-05-08T04:41:41.856219+0000', 'last_fullsized': '2024-05-08T04:41:41.856219+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-05-08T04:41:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.855707+0000', 'last_change': '2024-05-08T04:41:41.855707+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.855707+0000', 'last_undegraded': '2024-05-08T04:41:41.855707+0000', 'last_fullsized': '2024-05-08T04:41:41.855707+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.857601+0000', 'last_change': '2024-05-08T04:41:41.857601+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.857601+0000', 'last_undegraded': '2024-05-08T04:41:41.857601+0000', 'last_fullsized': '2024-05-08T04:41:41.857601+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.852780+0000', 'last_change': '2024-05-08T04:41:41.852780+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.852780+0000', 'last_undegraded': '2024-05-08T04:41:41.852780+0000', 'last_fullsized': '2024-05-08T04:41:41.852780+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-05-08T04:41:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.851671+0000', 'last_change': '2024-05-08T04:41:41.851671+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.851671+0000', 'last_undegraded': '2024-05-08T04:41:41.851671+0000', 'last_fullsized': '2024-05-08T04:41:41.851671+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.849578+0000', 'last_change': '2024-05-08T04:41:41.849578+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.849578+0000', 'last_undegraded': '2024-05-08T04:41:41.849578+0000', 'last_fullsized': '2024-05-08T04:41:41.849578+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.852780+0000', 'last_change': '2024-05-08T04:41:41.852780+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.852780+0000', 'last_undegraded': '2024-05-08T04:41:41.852780+0000', 'last_fullsized': '2024-05-08T04:41:41.852780+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T04:41:48.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854185+0000', 'last_change': '2024-05-08T04:41:41.854185+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854185+0000', 'last_undegraded': '2024-05-08T04:41:41.854185+0000', 'last_fullsized': '2024-05-08T04:41:41.854185+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.855729+0000', 'last_change': '2024-05-08T04:41:41.855729+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.855729+0000', 'last_undegraded': '2024-05-08T04:41:41.855729+0000', 'last_fullsized': '2024-05-08T04:41:41.855729+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854833+0000', 'last_change': '2024-05-08T04:41:41.854833+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854833+0000', 'last_undegraded': '2024-05-08T04:41:41.854833+0000', 'last_fullsized': '2024-05-08T04:41:41.854833+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854497+0000', 'last_change': '2024-05-08T04:41:41.854497+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854497+0000', 'last_undegraded': '2024-05-08T04:41:41.854497+0000', 'last_fullsized': '2024-05-08T04:41:41.854497+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.849579+0000', 'last_change': '2024-05-08T04:41:41.849579+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.849579+0000', 'last_undegraded': '2024-05-08T04:41:41.849579+0000', 'last_fullsized': '2024-05-08T04:41:41.849579+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.853253+0000', 'last_change': '2024-05-08T04:41:41.853253+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.853253+0000', 'last_undegraded': '2024-05-08T04:41:41.853253+0000', 'last_fullsized': '2024-05-08T04:41:41.853253+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-05-08T04:41:48.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.851801+0000', 'last_change': '2024-05-08T04:41:41.851801+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.851801+0000', 'last_undegraded': '2024-05-08T04:41:41.851801+0000', 'last_fullsized': '2024-05-08T04:41:41.851801+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.849578+0000', 'last_change': '2024-05-08T04:41:41.849578+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.849578+0000', 'last_undegraded': '2024-05-08T04:41:41.849578+0000', 'last_fullsized': '2024-05-08T04:41:41.849578+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.853284+0000', 'last_change': '2024-05-08T04:41:41.853284+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.853284+0000', 'last_undegraded': '2024-05-08T04:41:41.853284+0000', 'last_fullsized': '2024-05-08T04:41:41.853284+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:48.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.851679+0000', 'last_change': '2024-05-08T04:41:41.851679+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.851679+0000', 'last_undegraded': '2024-05-08T04:41:41.851679+0000', 'last_fullsized': '2024-05-08T04:41:41.851679+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856286+0000', 'last_change': '2024-05-08T04:41:41.856286+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856286+0000', 'last_undegraded': '2024-05-08T04:41:41.856286+0000', 'last_fullsized': '2024-05-08T04:41:41.856286+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-05-08T04:41:48.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.855160+0000', 'last_change': '2024-05-08T04:41:41.855160+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.855160+0000', 'last_undegraded': '2024-05-08T04:41:41.855160+0000', 'last_fullsized': '2024-05-08T04:41:41.855160+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.854066+0000', 'last_change': '2024-05-08T04:41:41.854066+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.854066+0000', 'last_undegraded': '2024-05-08T04:41:41.854066+0000', 'last_fullsized': '2024-05-08T04:41:41.854066+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856353+0000', 'last_change': '2024-05-08T04:41:41.856353+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856353+0000', 'last_undegraded': '2024-05-08T04:41:41.856353+0000', 'last_fullsized': '2024-05-08T04:41:41.856353+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-05-08T04:41:48.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.855123+0000', 'last_change': '2024-05-08T04:41:41.855123+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.855123+0000', 'last_undegraded': '2024-05-08T04:41:41.855123+0000', 'last_fullsized': '2024-05-08T04:41:41.855123+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.856524+0000', 'last_change': '2024-05-08T04:41:41.856524+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.856524+0000', 'last_undegraded': '2024-05-08T04:41:41.856524+0000', 'last_fullsized': '2024-05-08T04:41:41.856524+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:41:42.845526+0000', 'last_change': '2024-05-08T04:41:42.845526+0000', 'last_active': '2024-05-08T04:41:42.845526+0000', 'last_peered': '2024-05-08T04:41:42.845526+0000', 'last_clean': '2024-05-08T04:41:42.845526+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:42.845526+0000', 'last_undegraded': '2024-05-08T04:41:42.845526+0000', 'last_fullsized': '2024-05-08T04:41:42.845526+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:42.845526+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-05-08T04:41:48.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 83, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:41:41.852743+0000', 'last_change': '2024-05-08T04:41:41.852743+0000', 'last_active': '2024-05-08T04:41:41.840202+0000', 'last_peered': '2024-05-08T04:41:41.840202+0000', 'last_clean': '2024-05-08T04:41:41.840202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:41:41.852743+0000', 'last_undegraded': '2024-05-08T04:41:41.852743+0000', 'last_fullsized': '2024-05-08T04:41:41.852743+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'last_clean_scrub_stamp': '2024-05-08T04:41:41.840202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T04:41:48.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T04:41:48.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.876+0000 7f36e9200640 1 Processor -- start 2024-05-08T04:41:48.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.876+0000 7f36e9200640 1 -- start start 2024-05-08T04:41:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 0x7f36e40af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e9200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 0x7f36e406b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e9200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e415ba10 con 0x7f36e415b630 2024-05-08T04:41:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e9200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e406b640 con 0x7f36e406ac90 2024-05-08T04:41:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e415e730 con 0x7f36e415ebc0 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f36e415b630 0x7f36e40b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60318/0 (socket says 172.21.15.164:60318) 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e3400640 1 -- 172.21.15.164:0/375747824 learned_addr learned my addr 172.21.15.164:0/375747824 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2a00640 1 --2- 172.21.15.164:0/375747824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 0x7f36e40af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e3e00640 1 --2- 172.21.15.164:0/375747824 >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 0x7f36e406b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2000640 1 -- 172.21.15.164:0/375747824 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377290285 0 0) 0x7f36e415ba10 con 0x7f36e415b630 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2000640 1 -- 172.21.15.164:0/375747824 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c8003530 con 0x7f36e415b630 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2a00640 1 -- 172.21.15.164:0/375747824 >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 msgr2=0x7f36e406b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2a00640 1 --2- 172.21.15.164:0/375747824 >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 0x7f36e406b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2a00640 1 -- 172.21.15.164:0/375747824 >> v1:172.21.15.164:6789/0 conn(0x7f36e415b630 legacy=0x7f36e40b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e2a00640 1 -- 172.21.15.164:0/375747824 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40afbb0 con 0x7f36e415ebc0 2024-05-08T04:41:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.880+0000 7f36e3e00640 1 --2- 172.21.15.164:0/375747824 >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 0x7f36e406b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:48.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e2a00640 1 --2- 172.21.15.164:0/375747824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 0x7f36e40af540 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36d00097c0 tx=0x7f36d001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830e861dd0d62f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e2000640 1 -- 172.21.15.164:0/375747824 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00b1070 con 0x7f36e415ebc0 2024-05-08T04:41:48.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e2000640 1 -- 172.21.15.164:0/375747824 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d00a8050 con 0x7f36e415ebc0 2024-05-08T04:41:48.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e2000640 1 -- 172.21.15.164:0/375747824 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00ac860 con 0x7f36e415ebc0 2024-05-08T04:41:48.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- 172.21.15.164:0/375747824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 msgr2=0x7f36e40af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 --2- 172.21.15.164:0/375747824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 0x7f36e40af540 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36d00097c0 tx=0x7f36d001c3e0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- 172.21.15.164:0/375747824 shutdown_connections 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 --2- 172.21.15.164:0/375747824 >> v2:172.21.15.172:3300/0 conn(0x7f36e406ac90 0x7f36e406b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 --2- 172.21.15.164:0/375747824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e415ebc0 0x7f36e40af540 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- 172.21.15.164:0/375747824 >> 172.21.15.164:0/375747824 conn(0x7f36e4066fa0 msgr2=0x7f36e406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- 172.21.15.164:0/375747824 shutdown_connections 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- 172.21.15.164:0/375747824 wait complete. 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 Processor -- start 2024-05-08T04:41:48.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.884+0000 7f36e9200640 1 -- start start 2024-05-08T04:41:48.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e406ac90 0x7f36e414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e415b630 0x7f36e414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40d0cc0 con 0x7f36e415ebc0 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40d0b40 con 0x7f36e415b630 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40d0e40 con 0x7f36e406ac90 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57714/0 (socket says 172.21.15.164:57714) 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 -- 172.21.15.164:0/3950459449 learned_addr learned my addr 172.21.15.164:0/3950459449 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3400640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e406ac90 0x7f36e414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e2a00640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e415b630 0x7f36e414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 -- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e406ac90 msgr2=0x7f36e414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e406ac90 0x7f36e414f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 -- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e415b630 msgr2=0x7f36e414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e415b630 0x7f36e414dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 -- 172.21.15.164:0/3950459449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4071a40 con 0x7f36e415ebc0 2024-05-08T04:41:48.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e3e00640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009f00 tx=0x7f36dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc018030 con 0x7f36e415ebc0 2024-05-08T04:41:48.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc00ce50 con 0x7f36e415ebc0 2024-05-08T04:41:48.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc003bc0 con 0x7f36e415ebc0 2024-05-08T04:41:48.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e414e410 con 0x7f36e415ebc0 2024-05-08T04:41:48.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e40cd0a0 con 0x7f36e415ebc0 2024-05-08T04:41:48.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e404ef30 con 0x7f36e415ebc0 2024-05-08T04:41:48.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.888+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f36dc01b070 con 0x7f36e415ebc0 2024-05-08T04:41:48.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.892+0000 7f36db400640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 0x7f36c40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:48.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.892+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f36dc086510 con 0x7f36e415ebc0 2024-05-08T04:41:48.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.892+0000 7f36e3400640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 0x7f36c40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:48.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.892+0000 7f36e3400640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 0x7f36c40699c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36d4007fd0 tx=0x7f36d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:48.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:48.896+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36dc022070 con 0x7f36e415ebc0 2024-05-08T04:41:49.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36e40b3ca0 con 0x7f36e415ebc0 2024-05-08T04:41:49.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36db400640 1 -- 172.21.15.164:0/3950459449 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1528 (secure 0 0 0) 0x7f36dc010040 con 0x7f36e415ebc0 2024-05-08T04:41:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:49.109 INFO:teuthology.orchestra.run.smithi164.stdout:{"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":224,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1715143096,"num_in_osds":8,"osd_in_since":1715143086,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":256290816,"bytes_avail":772837822464,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":52,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":6951}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-08T04:40:55.460197+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 msgr2=0x7f36c40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 0x7f36c40699c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36d4007fd0 tx=0x7f36d4009040 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 msgr2=0x7f36e40ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009f00 tx=0x7f36dc00c490 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 shutdown_connections 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f36c4067500 0x7f36c40699c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e415ebc0 0x7f36e40ae150 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e415b630 0x7f36e414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 --2- 172.21.15.164:0/3950459449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e406ac90 0x7f36e414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 >> 172.21.15.164:0/3950459449 conn(0x7f36e4066fa0 msgr2=0x7f36e415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 shutdown_connections 2024-05-08T04:41:49.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.104+0000 7f36e9200640 1 -- 172.21.15.164:0/3950459449 wait complete. 2024-05-08T04:41:49.130 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T04:41:49.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:49.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 Processor -- start 2024-05-08T04:41:49.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 -- start start 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634415ba70 0x7f6344160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63440cd9f0 con 0x7f634415be50 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63440cdcd0 con 0x7f63440cfda0 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6348c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634415fe70 con 0x7f634415ba70 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6341e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634415ba70 0x7f6344160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6341e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f634415ba70 0x7f6344160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36564/0 (socket says 172.21.15.164:36564) 2024-05-08T04:41:49.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.260+0000 7f6341e00640 1 -- 172.21.15.164:0/4107546717 learned_addr learned my addr 172.21.15.164:0/4107546717 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:49.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6341e00640 1 -- 172.21.15.164:0/4107546717 >> v1:172.21.15.172:6789/0 conn(0x7f63440cfda0 legacy=0x7f63440b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6341e00640 1 -- 172.21.15.164:0/4107546717 >> v1:172.21.15.164:6789/0 conn(0x7f634415be50 legacy=0x7f63440cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6341e00640 1 -- 172.21.15.164:0/4107546717 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634415f820 con 0x7f634415ba70 2024-05-08T04:41:49.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6340a00640 1 -- 172.21.15.164:0/4107546717 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556502914 0 0) 0x7f63440cd9f0 con 0x7f634415be50 2024-05-08T04:41:49.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6341e00640 1 --2- 172.21.15.164:0/4107546717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634415ba70 0x7f6344160300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f633401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3643fcb87b2f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:49.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6340a00640 1 -- 172.21.15.164:0/4107546717 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340a57e0 con 0x7f634415ba70 2024-05-08T04:41:49.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6340a00640 1 -- 172.21.15.164:0/4107546717 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63340a5980 con 0x7f634415ba70 2024-05-08T04:41:49.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6340a00640 1 -- 172.21.15.164:0/4107546717 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340a5d50 con 0x7f634415ba70 2024-05-08T04:41:49.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6348c00640 1 -- 172.21.15.164:0/4107546717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634415ba70 msgr2=0x7f6344160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6348c00640 1 --2- 172.21.15.164:0/4107546717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634415ba70 0x7f6344160300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f633401ca00 comp rx=0 tx=0).stop 2024-05-08T04:41:49.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6348c00640 1 -- 172.21.15.164:0/4107546717 shutdown_connections 2024-05-08T04:41:49.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6348c00640 1 --2- 172.21.15.164:0/4107546717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f634415ba70 0x7f6344160300 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.264+0000 7f6348c00640 1 -- 172.21.15.164:0/4107546717 >> 172.21.15.164:0/4107546717 conn(0x7f634406ac90 msgr2=0x7f634406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:49.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- 172.21.15.164:0/4107546717 shutdown_connections 2024-05-08T04:41:49.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- 172.21.15.164:0/4107546717 wait complete. 2024-05-08T04:41:49.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 Processor -- start 2024-05-08T04:41:49.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- start start 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634415ba70 0x7f634415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 0x7f6344152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344169f60 con 0x7f63440cfda0 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344169de0 con 0x7f634415ba70 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634416a0e0 con 0x7f634415be50 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36574/0 (socket says 172.21.15.164:36574) 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 -- 172.21.15.164:0/2568081493 learned_addr learned my addr 172.21.15.164:0/2568081493 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341400640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634415ba70 0x7f634415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6342800640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 0x7f6344152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 -- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 msgr2=0x7f6344152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 0x7f6344152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 -- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634415ba70 msgr2=0x7f634415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634415ba70 0x7f634415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 -- 172.21.15.164:0/2568081493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634415cee0 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6342800640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 0x7f6344152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6341e00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f63340197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340bc030 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63340b8680 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340b8a50 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.268+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634415d1c0 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634415d780 con 0x7f63440cfda0 2024-05-08T04:41:49.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6344067820 con 0x7f63440cfda0 2024-05-08T04:41:49.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f63340bf070 con 0x7f63440cfda0 2024-05-08T04:41:49.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f633a000640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 0x7f63240699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:49.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6334120550 con 0x7f63440cfda0 2024-05-08T04:41:49.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f6341400640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 0x7f63240699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:49.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.272+0000 7f6341400640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 0x7f63240699c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f634415a940 tx=0x7f6330009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:49.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.276+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63340c7020 con 0x7f63440cfda0 2024-05-08T04:41:49.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.428+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6344069d60 con 0x7f6324067500 2024-05-08T04:41:49.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.436+0000 7f633a000640 1 -- 172.21.15.164:0/2568081493 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240193 (secure 0 0 0) 0x7f6344069d60 con 0x7f6324067500 2024-05-08T04:41:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:49.446 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:49.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 msgr2=0x7f63240699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 0x7f63240699c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f634415a940 tx=0x7f6330009040 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 msgr2=0x7f634414ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f63340197a0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 shutdown_connections 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6324067500 0x7f63240699c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f634415be50 0x7f6344152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f634415ba70 0x7f634415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 --2- 172.21.15.164:0/2568081493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63440cfda0 0x7f634414ce00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 >> 172.21.15.164:0/2568081493 conn(0x7f634406ac90 msgr2=0x7f634406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 shutdown_connections 2024-05-08T04:41:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:49.440+0000 7f6348c00640 1 -- 172.21.15.164:0/2568081493 wait complete. 2024-05-08T04:41:49.464 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-08T04:41:47.470353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250284,"kb_used_data":9104,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724436,"statfs":{"total":773094113280,"available":772837822464,"internally_reserved":0,"allocated":9322496,"data_stored":4645182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619869"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856695+0000","last_change":"2024-05-08T04:41:41.856695+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856695+0000","last_undegraded":"2024-05-08T04:41:41.856695+0000","last_fullsized":"2024-05-08T04:41:41.856695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855972+0000","last_change":"2024-05-08T04:41:41.855972+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855972+0000","last_undegraded":"2024-05-08T04:41:41.855972+0000","last_fullsized":"2024-05-08T04:41:41.855972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854155+0000","last_change":"2024-05-08T04:41:41.854155+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854155+0000","last_undegraded":"2024-05-08T04:41:41.854155+0000","last_fullsized":"2024-05-08T04:41:41.854155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854901+0000","last_change":"2024-05-08T04:41:41.854901+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854901+0000","last_undegraded":"2024-05-08T04:41:41.854901+0000","last_fullsized":"2024-05-08T04:41:41.854901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856760+0000","last_change":"2024-05-08T04:41:41.856760+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856760+0000","last_undegraded":"2024-05-08T04:41:41.856760+0000","last_fullsized":"2024-05-08T04:41:41.856760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856885+0000","last_change":"2024-05-08T04:41:41.856885+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856885+0000","last_undegraded":"2024-05-08T04:41:41.856885+0000","last_fullsized":"2024-05-08T04:41:41.856885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854398+0000","last_change":"2024-05-08T04:41:41.854398+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854398+0000","last_undegraded":"2024-05-08T04:41:41.854398+0000","last_fullsized":"2024-05-08T04:41:41.854398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851663+0000","last_change":"2024-05-08T04:41:41.851663+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851663+0000","last_undegraded":"2024-05-08T04:41:41.851663+0000","last_fullsized":"2024-05-08T04:41:41.851663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856219+0000","last_change":"2024-05-08T04:41:41.856219+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856219+0000","last_undegraded":"2024-05-08T04:41:41.856219+0000","last_fullsized":"2024-05-08T04:41:41.856219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855707+0000","last_change":"2024-05-08T04:41:41.855707+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855707+0000","last_undegraded":"2024-05-08T04:41:41.855707+0000","last_fullsized":"2024-05-08T04:41:41.855707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.857601+0000","last_change":"2024-05-08T04:41:41.857601+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.857601+0000","last_undegraded":"2024-05-08T04:41:41.857601+0000","last_fullsized":"2024-05-08T04:41:41.857601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851671+0000","last_change":"2024-05-08T04:41:41.851671+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851671+0000","last_undegraded":"2024-05-08T04:41:41.851671+0000","last_fullsized":"2024-05-08T04:41:41.851671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852780+0000","last_change":"2024-05-08T04:41:41.852780+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852780+0000","last_undegraded":"2024-05-08T04:41:41.852780+0000","last_fullsized":"2024-05-08T04:41:41.852780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854185+0000","last_change":"2024-05-08T04:41:41.854185+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854185+0000","last_undegraded":"2024-05-08T04:41:41.854185+0000","last_fullsized":"2024-05-08T04:41:41.854185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855729+0000","last_change":"2024-05-08T04:41:41.855729+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855729+0000","last_undegraded":"2024-05-08T04:41:41.855729+0000","last_fullsized":"2024-05-08T04:41:41.855729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854833+0000","last_change":"2024-05-08T04:41:41.854833+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854833+0000","last_undegraded":"2024-05-08T04:41:41.854833+0000","last_fullsized":"2024-05-08T04:41:41.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854497+0000","last_change":"2024-05-08T04:41:41.854497+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854497+0000","last_undegraded":"2024-05-08T04:41:41.854497+0000","last_fullsized":"2024-05-08T04:41:41.854497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849579+0000","last_change":"2024-05-08T04:41:41.849579+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849579+0000","last_undegraded":"2024-05-08T04:41:41.849579+0000","last_fullsized":"2024-05-08T04:41:41.849579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853253+0000","last_change":"2024-05-08T04:41:41.853253+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853253+0000","last_undegraded":"2024-05-08T04:41:41.853253+0000","last_fullsized":"2024-05-08T04:41:41.853253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851801+0000","last_change":"2024-05-08T04:41:41.851801+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851801+0000","last_undegraded":"2024-05-08T04:41:41.851801+0000","last_fullsized":"2024-05-08T04:41:41.851801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.849578+0000","last_change":"2024-05-08T04:41:41.849578+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.849578+0000","last_undegraded":"2024-05-08T04:41:41.849578+0000","last_fullsized":"2024-05-08T04:41:41.849578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.853284+0000","last_change":"2024-05-08T04:41:41.853284+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.853284+0000","last_undegraded":"2024-05-08T04:41:41.853284+0000","last_fullsized":"2024-05-08T04:41:41.853284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.851679+0000","last_change":"2024-05-08T04:41:41.851679+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.851679+0000","last_undegraded":"2024-05-08T04:41:41.851679+0000","last_fullsized":"2024-05-08T04:41:41.851679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856286+0000","last_change":"2024-05-08T04:41:41.856286+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856286+0000","last_undegraded":"2024-05-08T04:41:41.856286+0000","last_fullsized":"2024-05-08T04:41:41.856286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855160+0000","last_change":"2024-05-08T04:41:41.855160+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855160+0000","last_undegraded":"2024-05-08T04:41:41.855160+0000","last_fullsized":"2024-05-08T04:41:41.855160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.854066+0000","last_change":"2024-05-08T04:41:41.854066+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.854066+0000","last_undegraded":"2024-05-08T04:41:41.854066+0000","last_fullsized":"2024-05-08T04:41:41.854066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856353+0000","last_change":"2024-05-08T04:41:41.856353+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856353+0000","last_undegraded":"2024-05-08T04:41:41.856353+0000","last_fullsized":"2024-05-08T04:41:41.856353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.855123+0000","last_change":"2024-05-08T04:41:41.855123+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.855123+0000","last_undegraded":"2024-05-08T04:41:41.855123+0000","last_fullsized":"2024-05-08T04:41:41.855123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.856524+0000","last_change":"2024-05-08T04:41:41.856524+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.856524+0000","last_undegraded":"2024-05-08T04:41:41.856524+0000","last_fullsized":"2024-05-08T04:41:41.856524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:41:42.845526+0000","last_change":"2024-05-08T04:41:42.845526+0000","last_active":"2024-05-08T04:41:42.845526+0000","last_peered":"2024-05-08T04:41:42.845526+0000","last_clean":"2024-05-08T04:41:42.845526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:42.845526+0000","last_undegraded":"2024-05-08T04:41:42.845526+0000","last_fullsized":"2024-05-08T04:41:42.845526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-05-08T04:41:41.852743+0000","last_change":"2024-05-08T04:41:41.852743+0000","last_active":"2024-05-08T04:41:41.840202+0000","last_peered":"2024-05-08T04:41:41.840202+0000","last_clean":"2024-05-08T04:41:41.840202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:41:41.852743+0000","last_undegraded":"2024-05-08T04:41:41.852743+0000","last_fullsized":"2024-05-08T04:41:41.852743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T04:41:41.852517+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:41.852517+0000","last_peered":"2024-05-08T04:41:41.852517+0000","last_clean":"2024-05-08T04:41:41.852517+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:41.852517+0000","last_undegraded":"2024-05-08T04:41:41.852517+0000","last_fullsized":"2024-05-08T04:41:41.852517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":434524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":428651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1613824,"data_stored":1024892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:52.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:41:52.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 Processor -- start 2024-05-08T04:41:52.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 -- start start 2024-05-08T04:41:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95fc144200 0x7f95fc161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc161fe0 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc162880 con 0x7f95fc144200 2024-05-08T04:41:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9604400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc1622c0 con 0x7f95fc0cfd60 2024-05-08T04:41:52.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95fc144200 0x7f95fc161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95fc144200 0x7f95fc161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36588/0 (socket says 172.21.15.164:36588) 2024-05-08T04:41:52.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 -- 172.21.15.164:0/1293670000 learned_addr learned my addr 172.21.15.164:0/1293670000 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:52.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 -- 172.21.15.164:0/1293670000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 msgr2=0x7f95fc0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:52.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601000640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 -- 172.21.15.164:0/1293670000 >> v1:172.21.15.172:6789/0 conn(0x7f95fc0cc5f0 legacy=0x7f95fc15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:52.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601a00640 1 -- 172.21.15.164:0/1293670000 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc1625a0 con 0x7f95fc144200 2024-05-08T04:41:52.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.652+0000 7f9601000640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:52.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9601a00640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc144200 0x7f95fc161950 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009870 tx=0x7f95f001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e9b4c7d51365b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:52.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f95fbe00640 1 -- 172.21.15.164:0/1293670000 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b0070 con 0x7f95fc144200 2024-05-08T04:41:52.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f95fbe00640 1 -- 172.21.15.164:0/1293670000 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00ab600 con 0x7f95fc144200 2024-05-08T04:41:52.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f95fbe00640 1 -- 172.21.15.164:0/1293670000 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ab9d0 con 0x7f95fc144200 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 -- 172.21.15.164:0/1293670000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc144200 msgr2=0x7f95fc161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc144200 0x7f95fc161950 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009870 tx=0x7f95f001c490 comp rx=0 tx=0).stop 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 -- 172.21.15.164:0/1293670000 shutdown_connections 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 --2- 172.21.15.164:0/1293670000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc144200 0x7f95fc161950 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 -- 172.21.15.164:0/1293670000 >> 172.21.15.164:0/1293670000 conn(0x7f95fc06ac90 msgr2=0x7f95fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 -- 172.21.15.164:0/1293670000 shutdown_connections 2024-05-08T04:41:52.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.656+0000 7f9604400640 1 -- 172.21.15.164:0/1293670000 wait complete. 2024-05-08T04:41:52.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 Processor -- start 2024-05-08T04:41:52.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- start start 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc1424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 0x7f95fc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc15f2b0 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc15f130 con 0x7f95fc144200 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0cf1f0 con 0x7f95fc0cfd60 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36594/0 (socket says 172.21.15.164:36594) 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 -- 172.21.15.164:0/732677285 learned_addr learned my addr 172.21.15.164:0/732677285 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9602400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 0x7f95fc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601000640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc1424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 -- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 msgr2=0x7f95fc1424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc1424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 -- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 msgr2=0x7f95fc15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 0x7f95fc15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 -- 172.21.15.164:0/732677285 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0712b0 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9602400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 0x7f95fc15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9601a00640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ae040 tx=0x7f95f001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ba020 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00b6620 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b0040 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc071590 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc071b50 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.660+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc168af0 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.664+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f95f00bd020 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.664+0000 7f95f9600640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 0x7f95dc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.664+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f95f011fe60 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.664+0000 7f9601000640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 0x7f95dc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:52.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.664+0000 7f9601000640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 0x7f95dc0699c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cbd0 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:52.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.668+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95f00c4030 con 0x7f95fc0cc5f0 2024-05-08T04:41:52.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.816+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95fc15bbb0 con 0x7f95dc067500 2024-05-08T04:41:52.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.824+0000 7f95f9600640 1 -- 172.21.15.164:0/732677285 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252085 (secure 0 0 0) 0x7f95fc15bbb0 con 0x7f95dc067500 2024-05-08T04:41:52.836 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:52.839 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 msgr2=0x7f95dc0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 0x7f95dc0699c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cbd0 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 msgr2=0x7f95fc15b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ae040 tx=0x7f95f001acd0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 shutdown_connections 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f95dc067500 0x7f95dc0699c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f95fc0cfd60 0x7f95fc1424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95fc144200 0x7f95fc15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 --2- 172.21.15.164:0/732677285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95fc0cc5f0 0x7f95fc15b670 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:52.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 >> 172.21.15.164:0/732677285 conn(0x7f95fc06ac90 msgr2=0x7f95fc0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:52.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 shutdown_connections 2024-05-08T04:41:52.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:52.828+0000 7f9604400640 1 -- 172.21.15.164:0/732677285 wait complete. 2024-05-08T04:41:52.853 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-05-08T04:41:51.471410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254780,"kb_used_data":9468,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719940,"statfs":{"total":773094113280,"available":772833218560,"internally_reserved":0,"allocated":9695232,"data_stored":4811536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001866"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112882+0000","last_change":"2024-05-08T04:41:43.892556+0000","last_active":"2024-05-08T04:41:45.112882+0000","last_peered":"2024-05-08T04:41:45.112882+0000","last_clean":"2024-05-08T04:41:45.112882+0000","last_became_active":"2024-05-08T04:41:43.891329+0000","last_became_peered":"2024-05-08T04:41:43.891329+0000","last_unstale":"2024-05-08T04:41:45.112882+0000","last_undegraded":"2024-05-08T04:41:45.112882+0000","last_fullsized":"2024-05-08T04:41:45.112882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:45.950574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.888019+0000","last_change":"2024-05-08T04:41:43.889531+0000","last_active":"2024-05-08T04:41:44.888019+0000","last_peered":"2024-05-08T04:41:44.888019+0000","last_clean":"2024-05-08T04:41:44.888019+0000","last_became_active":"2024-05-08T04:41:43.889281+0000","last_became_peered":"2024-05-08T04:41:43.889281+0000","last_unstale":"2024-05-08T04:41:44.888019+0000","last_undegraded":"2024-05-08T04:41:44.888019+0000","last_fullsized":"2024-05-08T04:41:44.888019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:33.436174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868178+0000","last_change":"2024-05-08T04:41:43.889464+0000","last_active":"2024-05-08T04:41:44.868178+0000","last_peered":"2024-05-08T04:41:44.868178+0000","last_clean":"2024-05-08T04:41:44.868178+0000","last_became_active":"2024-05-08T04:41:43.889121+0000","last_became_peered":"2024-05-08T04:41:43.889121+0000","last_unstale":"2024-05-08T04:41:44.868178+0000","last_undegraded":"2024-05-08T04:41:44.868178+0000","last_fullsized":"2024-05-08T04:41:44.868178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:12.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866918+0000","last_change":"2024-05-08T04:41:43.885372+0000","last_active":"2024-05-08T04:41:44.866918+0000","last_peered":"2024-05-08T04:41:44.866918+0000","last_clean":"2024-05-08T04:41:44.866918+0000","last_became_active":"2024-05-08T04:41:43.885096+0000","last_became_peered":"2024-05-08T04:41:43.885096+0000","last_unstale":"2024-05-08T04:41:44.866918+0000","last_undegraded":"2024-05-08T04:41:44.866918+0000","last_fullsized":"2024-05-08T04:41:44.866918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:15:10.135857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031957+0000","last_change":"2024-05-08T04:41:43.874315+0000","last_active":"2024-05-08T04:41:45.031957+0000","last_peered":"2024-05-08T04:41:45.031957+0000","last_clean":"2024-05-08T04:41:45.031957+0000","last_became_active":"2024-05-08T04:41:43.873954+0000","last_became_peered":"2024-05-08T04:41:43.873954+0000","last_unstale":"2024-05-08T04:41:45.031957+0000","last_undegraded":"2024-05-08T04:41:45.031957+0000","last_fullsized":"2024-05-08T04:41:45.031957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:21.022218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868711+0000","last_change":"2024-05-08T04:41:43.880819+0000","last_active":"2024-05-08T04:41:44.868711+0000","last_peered":"2024-05-08T04:41:44.868711+0000","last_clean":"2024-05-08T04:41:44.868711+0000","last_became_active":"2024-05-08T04:41:43.880382+0000","last_became_peered":"2024-05-08T04:41:43.880382+0000","last_unstale":"2024-05-08T04:41:44.868711+0000","last_undegraded":"2024-05-08T04:41:44.868711+0000","last_fullsized":"2024-05-08T04:41:44.868711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:48.141818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866366+0000","last_change":"2024-05-08T04:41:43.883671+0000","last_active":"2024-05-08T04:41:44.866366+0000","last_peered":"2024-05-08T04:41:44.866366+0000","last_clean":"2024-05-08T04:41:44.866366+0000","last_became_active":"2024-05-08T04:41:43.882767+0000","last_became_peered":"2024-05-08T04:41:43.882767+0000","last_unstale":"2024-05-08T04:41:44.866366+0000","last_undegraded":"2024-05-08T04:41:44.866366+0000","last_fullsized":"2024-05-08T04:41:44.866366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:30.645045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868608+0000","last_change":"2024-05-08T04:41:43.870589+0000","last_active":"2024-05-08T04:41:44.868608+0000","last_peered":"2024-05-08T04:41:44.868608+0000","last_clean":"2024-05-08T04:41:44.868608+0000","last_became_active":"2024-05-08T04:41:43.870246+0000","last_became_peered":"2024-05-08T04:41:43.870246+0000","last_unstale":"2024-05-08T04:41:44.868608+0000","last_undegraded":"2024-05-08T04:41:44.868608+0000","last_fullsized":"2024-05-08T04:41:44.868608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:13:23.623707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868806+0000","last_change":"2024-05-08T04:41:43.880308+0000","last_active":"2024-05-08T04:41:44.868806+0000","last_peered":"2024-05-08T04:41:44.868806+0000","last_clean":"2024-05-08T04:41:44.868806+0000","last_became_active":"2024-05-08T04:41:43.880024+0000","last_became_peered":"2024-05-08T04:41:43.880024+0000","last_unstale":"2024-05-08T04:41:44.868806+0000","last_undegraded":"2024-05-08T04:41:44.868806+0000","last_fullsized":"2024-05-08T04:41:44.868806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:08.818996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112661+0000","last_change":"2024-05-08T04:41:43.887330+0000","last_active":"2024-05-08T04:41:45.112661+0000","last_peered":"2024-05-08T04:41:45.112661+0000","last_clean":"2024-05-08T04:41:45.112661+0000","last_became_active":"2024-05-08T04:41:43.887057+0000","last_became_peered":"2024-05-08T04:41:43.887057+0000","last_unstale":"2024-05-08T04:41:45.112661+0000","last_undegraded":"2024-05-08T04:41:45.112661+0000","last_fullsized":"2024-05-08T04:41:45.112661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:49:07.772182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.888066+0000","last_change":"2024-05-08T04:41:43.874445+0000","last_active":"2024-05-08T04:41:44.888066+0000","last_peered":"2024-05-08T04:41:44.888066+0000","last_clean":"2024-05-08T04:41:44.888066+0000","last_became_active":"2024-05-08T04:41:43.874111+0000","last_became_peered":"2024-05-08T04:41:43.874111+0000","last_unstale":"2024-05-08T04:41:44.888066+0000","last_undegraded":"2024-05-08T04:41:44.888066+0000","last_fullsized":"2024-05-08T04:41:44.888066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:41:21.812153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112289+0000","last_change":"2024-05-08T04:41:43.887833+0000","last_active":"2024-05-08T04:41:45.112289+0000","last_peered":"2024-05-08T04:41:45.112289+0000","last_clean":"2024-05-08T04:41:45.112289+0000","last_became_active":"2024-05-08T04:41:43.887453+0000","last_became_peered":"2024-05-08T04:41:43.887453+0000","last_unstale":"2024-05-08T04:41:45.112289+0000","last_undegraded":"2024-05-08T04:41:45.112289+0000","last_fullsized":"2024-05-08T04:41:45.112289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:48.141818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868007+0000","last_change":"2024-05-08T04:41:43.885699+0000","last_active":"2024-05-08T04:41:44.868007+0000","last_peered":"2024-05-08T04:41:44.868007+0000","last_clean":"2024-05-08T04:41:44.868007+0000","last_became_active":"2024-05-08T04:41:43.885432+0000","last_became_peered":"2024-05-08T04:41:43.885432+0000","last_unstale":"2024-05-08T04:41:44.868007+0000","last_undegraded":"2024-05-08T04:41:44.868007+0000","last_fullsized":"2024-05-08T04:41:44.868007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:05.516389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866663+0000","last_change":"2024-05-08T04:41:43.883175+0000","last_active":"2024-05-08T04:41:44.866663+0000","last_peered":"2024-05-08T04:41:44.866663+0000","last_clean":"2024-05-08T04:41:44.866663+0000","last_became_active":"2024-05-08T04:41:43.882392+0000","last_became_peered":"2024-05-08T04:41:43.882392+0000","last_unstale":"2024-05-08T04:41:44.866663+0000","last_undegraded":"2024-05-08T04:41:44.866663+0000","last_fullsized":"2024-05-08T04:41:44.866663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:53.416885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954564+0000","last_change":"2024-05-08T04:41:43.880166+0000","last_active":"2024-05-08T04:41:46.954564+0000","last_peered":"2024-05-08T04:41:46.954564+0000","last_clean":"2024-05-08T04:41:46.954564+0000","last_became_active":"2024-05-08T04:41:43.879792+0000","last_became_peered":"2024-05-08T04:41:43.879792+0000","last_unstale":"2024-05-08T04:41:46.954564+0000","last_undegraded":"2024-05-08T04:41:46.954564+0000","last_fullsized":"2024-05-08T04:41:46.954564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:03.645103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031752+0000","last_change":"2024-05-08T04:41:43.887547+0000","last_active":"2024-05-08T04:41:45.031752+0000","last_peered":"2024-05-08T04:41:45.031752+0000","last_clean":"2024-05-08T04:41:45.031752+0000","last_became_active":"2024-05-08T04:41:43.887280+0000","last_became_peered":"2024-05-08T04:41:43.887280+0000","last_unstale":"2024-05-08T04:41:45.031752+0000","last_undegraded":"2024-05-08T04:41:45.031752+0000","last_fullsized":"2024-05-08T04:41:45.031752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:13:23.623707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032356+0000","last_change":"2024-05-08T04:41:43.885964+0000","last_active":"2024-05-08T04:41:45.032356+0000","last_peered":"2024-05-08T04:41:45.032356+0000","last_clean":"2024-05-08T04:41:45.032356+0000","last_became_active":"2024-05-08T04:41:43.884673+0000","last_became_peered":"2024-05-08T04:41:43.884673+0000","last_unstale":"2024-05-08T04:41:45.032356+0000","last_undegraded":"2024-05-08T04:41:45.032356+0000","last_fullsized":"2024-05-08T04:41:45.032356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:36.816141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866945+0000","last_change":"2024-05-08T04:41:43.890549+0000","last_active":"2024-05-08T04:41:44.866945+0000","last_peered":"2024-05-08T04:41:44.866945+0000","last_clean":"2024-05-08T04:41:44.866945+0000","last_became_active":"2024-05-08T04:41:43.889584+0000","last_became_peered":"2024-05-08T04:41:43.889584+0000","last_unstale":"2024-05-08T04:41:44.866945+0000","last_undegraded":"2024-05-08T04:41:44.866945+0000","last_fullsized":"2024-05-08T04:41:44.866945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:35:19.759196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890598+0000","last_change":"2024-05-08T04:41:43.886584+0000","last_active":"2024-05-08T04:41:44.890598+0000","last_peered":"2024-05-08T04:41:44.890598+0000","last_clean":"2024-05-08T04:41:44.890598+0000","last_became_active":"2024-05-08T04:41:43.884693+0000","last_became_peered":"2024-05-08T04:41:43.884693+0000","last_unstale":"2024-05-08T04:41:44.890598+0000","last_undegraded":"2024-05-08T04:41:44.890598+0000","last_fullsized":"2024-05-08T04:41:44.890598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:25.980994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866897+0000","last_change":"2024-05-08T04:41:43.890285+0000","last_active":"2024-05-08T04:41:44.866897+0000","last_peered":"2024-05-08T04:41:44.866897+0000","last_clean":"2024-05-08T04:41:44.866897+0000","last_became_active":"2024-05-08T04:41:43.889807+0000","last_became_peered":"2024-05-08T04:41:43.889807+0000","last_unstale":"2024-05-08T04:41:44.866897+0000","last_undegraded":"2024-05-08T04:41:44.866897+0000","last_fullsized":"2024-05-08T04:41:44.866897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:01.080762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866778+0000","last_change":"2024-05-08T04:41:43.888279+0000","last_active":"2024-05-08T04:41:44.866778+0000","last_peered":"2024-05-08T04:41:44.866778+0000","last_clean":"2024-05-08T04:41:44.866778+0000","last_became_active":"2024-05-08T04:41:43.888012+0000","last_became_peered":"2024-05-08T04:41:43.888012+0000","last_unstale":"2024-05-08T04:41:44.866778+0000","last_undegraded":"2024-05-08T04:41:44.866778+0000","last_fullsized":"2024-05-08T04:41:44.866778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:01.055238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887720+0000","last_change":"2024-05-08T04:41:43.881257+0000","last_active":"2024-05-08T04:41:44.887720+0000","last_peered":"2024-05-08T04:41:44.887720+0000","last_clean":"2024-05-08T04:41:44.887720+0000","last_became_active":"2024-05-08T04:41:43.880979+0000","last_became_peered":"2024-05-08T04:41:43.880979+0000","last_unstale":"2024-05-08T04:41:44.887720+0000","last_undegraded":"2024-05-08T04:41:44.887720+0000","last_fullsized":"2024-05-08T04:41:44.887720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:52:37.477624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112216+0000","last_change":"2024-05-08T04:41:43.892671+0000","last_active":"2024-05-08T04:41:45.112216+0000","last_peered":"2024-05-08T04:41:45.112216+0000","last_clean":"2024-05-08T04:41:45.112216+0000","last_became_active":"2024-05-08T04:41:43.892385+0000","last_became_peered":"2024-05-08T04:41:43.892385+0000","last_unstale":"2024-05-08T04:41:45.112216+0000","last_undegraded":"2024-05-08T04:41:45.112216+0000","last_fullsized":"2024-05-08T04:41:45.112216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:13:23.623707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866587+0000","last_change":"2024-05-08T04:41:43.888242+0000","last_active":"2024-05-08T04:41:44.866587+0000","last_peered":"2024-05-08T04:41:44.866587+0000","last_clean":"2024-05-08T04:41:44.866587+0000","last_became_active":"2024-05-08T04:41:43.887987+0000","last_became_peered":"2024-05-08T04:41:43.887987+0000","last_unstale":"2024-05-08T04:41:44.866587+0000","last_undegraded":"2024-05-08T04:41:44.866587+0000","last_fullsized":"2024-05-08T04:41:44.866587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:11:33.815998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.888090+0000","last_change":"2024-05-08T04:41:43.889323+0000","last_active":"2024-05-08T04:41:44.888090+0000","last_peered":"2024-05-08T04:41:44.888090+0000","last_clean":"2024-05-08T04:41:44.888090+0000","last_became_active":"2024-05-08T04:41:43.888943+0000","last_became_peered":"2024-05-08T04:41:43.888943+0000","last_unstale":"2024-05-08T04:41:44.888090+0000","last_undegraded":"2024-05-08T04:41:44.888090+0000","last_fullsized":"2024-05-08T04:41:44.888090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:57:06.975336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868037+0000","last_change":"2024-05-08T04:41:43.883861+0000","last_active":"2024-05-08T04:41:44.868037+0000","last_peered":"2024-05-08T04:41:44.868037+0000","last_clean":"2024-05-08T04:41:44.868037+0000","last_became_active":"2024-05-08T04:41:43.883543+0000","last_became_peered":"2024-05-08T04:41:43.883543+0000","last_unstale":"2024-05-08T04:41:44.868037+0000","last_undegraded":"2024-05-08T04:41:44.868037+0000","last_fullsized":"2024-05-08T04:41:44.868037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:25.980994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032019+0000","last_change":"2024-05-08T04:41:43.874901+0000","last_active":"2024-05-08T04:41:45.032019+0000","last_peered":"2024-05-08T04:41:45.032019+0000","last_clean":"2024-05-08T04:41:45.032019+0000","last_became_active":"2024-05-08T04:41:43.874549+0000","last_became_peered":"2024-05-08T04:41:43.874549+0000","last_unstale":"2024-05-08T04:41:45.032019+0000","last_undegraded":"2024-05-08T04:41:45.032019+0000","last_fullsized":"2024-05-08T04:41:45.032019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:08.818996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112366+0000","last_change":"2024-05-08T04:41:43.891170+0000","last_active":"2024-05-08T04:41:45.112366+0000","last_peered":"2024-05-08T04:41:45.112366+0000","last_clean":"2024-05-08T04:41:45.112366+0000","last_became_active":"2024-05-08T04:41:43.890870+0000","last_became_peered":"2024-05-08T04:41:43.890870+0000","last_unstale":"2024-05-08T04:41:45.112366+0000","last_undegraded":"2024-05-08T04:41:45.112366+0000","last_fullsized":"2024-05-08T04:41:45.112366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.664761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887923+0000","last_change":"2024-05-08T04:41:43.874064+0000","last_active":"2024-05-08T04:41:44.887923+0000","last_peered":"2024-05-08T04:41:44.887923+0000","last_clean":"2024-05-08T04:41:44.887923+0000","last_became_active":"2024-05-08T04:41:43.873022+0000","last_became_peered":"2024-05-08T04:41:43.873022+0000","last_unstale":"2024-05-08T04:41:44.887923+0000","last_undegraded":"2024-05-08T04:41:44.887923+0000","last_fullsized":"2024-05-08T04:41:44.887923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:12:00.952968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887970+0000","last_change":"2024-05-08T04:41:43.888024+0000","last_active":"2024-05-08T04:41:44.887970+0000","last_peered":"2024-05-08T04:41:44.887970+0000","last_clean":"2024-05-08T04:41:44.887970+0000","last_became_active":"2024-05-08T04:41:43.887735+0000","last_became_peered":"2024-05-08T04:41:43.887735+0000","last_unstale":"2024-05-08T04:41:44.887970+0000","last_undegraded":"2024-05-08T04:41:44.887970+0000","last_fullsized":"2024-05-08T04:41:44.887970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:46.022165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031749+0000","last_change":"2024-05-08T04:41:43.883079+0000","last_active":"2024-05-08T04:41:45.031749+0000","last_peered":"2024-05-08T04:41:45.031749+0000","last_clean":"2024-05-08T04:41:45.031749+0000","last_became_active":"2024-05-08T04:41:43.882219+0000","last_became_peered":"2024-05-08T04:41:43.882219+0000","last_unstale":"2024-05-08T04:41:45.031749+0000","last_undegraded":"2024-05-08T04:41:45.031749+0000","last_fullsized":"2024-05-08T04:41:45.031749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:42:59.123397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112282+0000","last_change":"2024-05-08T04:41:43.896255+0000","last_active":"2024-05-08T04:41:45.112282+0000","last_peered":"2024-05-08T04:41:45.112282+0000","last_clean":"2024-05-08T04:41:45.112282+0000","last_became_active":"2024-05-08T04:41:43.895901+0000","last_became_peered":"2024-05-08T04:41:43.895901+0000","last_unstale":"2024-05-08T04:41:45.112282+0000","last_undegraded":"2024-05-08T04:41:45.112282+0000","last_fullsized":"2024-05-08T04:41:45.112282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:21.022218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032244+0000","last_change":"2024-05-08T04:41:43.890486+0000","last_active":"2024-05-08T04:41:45.032244+0000","last_peered":"2024-05-08T04:41:45.032244+0000","last_clean":"2024-05-08T04:41:45.032244+0000","last_became_active":"2024-05-08T04:41:43.889128+0000","last_became_peered":"2024-05-08T04:41:43.889128+0000","last_unstale":"2024-05-08T04:41:45.032244+0000","last_undegraded":"2024-05-08T04:41:45.032244+0000","last_fullsized":"2024-05-08T04:41:45.032244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:42.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112949+0000","last_change":"2024-05-08T04:41:42.871380+0000","last_active":"2024-05-08T04:41:45.112949+0000","last_peered":"2024-05-08T04:41:45.112949+0000","last_clean":"2024-05-08T04:41:45.112949+0000","last_became_active":"2024-05-08T04:41:42.870899+0000","last_became_peered":"2024-05-08T04:41:42.870899+0000","last_unstale":"2024-05-08T04:41:45.112949+0000","last_undegraded":"2024-05-08T04:41:45.112949+0000","last_fullsized":"2024-05-08T04:41:45.112949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:21.054164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112882+0000","last_change":"2024-05-08T04:41:43.898411+0000","last_active":"2024-05-08T04:41:45.112882+0000","last_peered":"2024-05-08T04:41:45.112882+0000","last_clean":"2024-05-08T04:41:45.112882+0000","last_became_active":"2024-05-08T04:41:43.898143+0000","last_became_peered":"2024-05-08T04:41:43.898143+0000","last_unstale":"2024-05-08T04:41:45.112882+0000","last_undegraded":"2024-05-08T04:41:45.112882+0000","last_fullsized":"2024-05-08T04:41:45.112882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:36.816141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032349+0000","last_change":"2024-05-08T04:41:42.873707+0000","last_active":"2024-05-08T04:41:45.032349+0000","last_peered":"2024-05-08T04:41:45.032349+0000","last_clean":"2024-05-08T04:41:45.032349+0000","last_became_active":"2024-05-08T04:41:42.873381+0000","last_became_peered":"2024-05-08T04:41:42.873381+0000","last_unstale":"2024-05-08T04:41:45.032349+0000","last_undegraded":"2024-05-08T04:41:45.032349+0000","last_fullsized":"2024-05-08T04:41:45.032349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:47:30.028762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112519+0000","last_change":"2024-05-08T04:41:43.894744+0000","last_active":"2024-05-08T04:41:45.112519+0000","last_peered":"2024-05-08T04:41:45.112519+0000","last_clean":"2024-05-08T04:41:45.112519+0000","last_became_active":"2024-05-08T04:41:43.894428+0000","last_became_peered":"2024-05-08T04:41:43.894428+0000","last_unstale":"2024-05-08T04:41:45.112519+0000","last_undegraded":"2024-05-08T04:41:45.112519+0000","last_fullsized":"2024-05-08T04:41:45.112519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:42.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890530+0000","last_change":"2024-05-08T04:41:42.885672+0000","last_active":"2024-05-08T04:41:44.890530+0000","last_peered":"2024-05-08T04:41:44.890530+0000","last_clean":"2024-05-08T04:41:44.890530+0000","last_became_active":"2024-05-08T04:41:42.885390+0000","last_became_peered":"2024-05-08T04:41:42.885390+0000","last_unstale":"2024-05-08T04:41:44.890530+0000","last_undegraded":"2024-05-08T04:41:44.890530+0000","last_fullsized":"2024-05-08T04:41:44.890530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:57.232712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868087+0000","last_change":"2024-05-08T04:41:43.883735+0000","last_active":"2024-05-08T04:41:44.868087+0000","last_peered":"2024-05-08T04:41:44.868087+0000","last_clean":"2024-05-08T04:41:44.868087+0000","last_became_active":"2024-05-08T04:41:43.883432+0000","last_became_peered":"2024-05-08T04:41:43.883432+0000","last_unstale":"2024-05-08T04:41:44.868087+0000","last_undegraded":"2024-05-08T04:41:44.868087+0000","last_fullsized":"2024-05-08T04:41:44.868087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:31.243192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890495+0000","last_change":"2024-05-08T04:41:42.885685+0000","last_active":"2024-05-08T04:41:44.890495+0000","last_peered":"2024-05-08T04:41:44.890495+0000","last_clean":"2024-05-08T04:41:44.890495+0000","last_became_active":"2024-05-08T04:41:42.885417+0000","last_became_peered":"2024-05-08T04:41:42.885417+0000","last_unstale":"2024-05-08T04:41:44.890495+0000","last_undegraded":"2024-05-08T04:41:44.890495+0000","last_fullsized":"2024-05-08T04:41:44.890495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:35.810817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.888063+0000","last_change":"2024-05-08T04:41:43.887947+0000","last_active":"2024-05-08T04:41:44.888063+0000","last_peered":"2024-05-08T04:41:44.888063+0000","last_clean":"2024-05-08T04:41:44.888063+0000","last_became_active":"2024-05-08T04:41:43.887614+0000","last_became_peered":"2024-05-08T04:41:43.887614+0000","last_unstale":"2024-05-08T04:41:44.888063+0000","last_undegraded":"2024-05-08T04:41:44.888063+0000","last_fullsized":"2024-05-08T04:41:44.888063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:42.992833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032065+0000","last_change":"2024-05-08T04:41:42.873878+0000","last_active":"2024-05-08T04:41:45.032065+0000","last_peered":"2024-05-08T04:41:45.032065+0000","last_clean":"2024-05-08T04:41:45.032065+0000","last_became_active":"2024-05-08T04:41:42.873356+0000","last_became_peered":"2024-05-08T04:41:42.873356+0000","last_unstale":"2024-05-08T04:41:45.032065+0000","last_undegraded":"2024-05-08T04:41:45.032065+0000","last_fullsized":"2024-05-08T04:41:45.032065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:52:15.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031974+0000","last_change":"2024-05-08T04:41:43.886393+0000","last_active":"2024-05-08T04:41:45.031974+0000","last_peered":"2024-05-08T04:41:45.031974+0000","last_clean":"2024-05-08T04:41:45.031974+0000","last_became_active":"2024-05-08T04:41:43.886107+0000","last_became_peered":"2024-05-08T04:41:43.886107+0000","last_unstale":"2024-05-08T04:41:45.031974+0000","last_undegraded":"2024-05-08T04:41:45.031974+0000","last_fullsized":"2024-05-08T04:41:45.031974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:48.141818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031635+0000","last_change":"2024-05-08T04:41:42.872128+0000","last_active":"2024-05-08T04:41:45.031635+0000","last_peered":"2024-05-08T04:41:45.031635+0000","last_clean":"2024-05-08T04:41:45.031635+0000","last_became_active":"2024-05-08T04:41:42.871840+0000","last_became_peered":"2024-05-08T04:41:42.871840+0000","last_unstale":"2024-05-08T04:41:45.031635+0000","last_undegraded":"2024-05-08T04:41:45.031635+0000","last_fullsized":"2024-05-08T04:41:45.031635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:32.158729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.867538+0000","last_change":"2024-05-08T04:41:43.890232+0000","last_active":"2024-05-08T04:41:44.867538+0000","last_peered":"2024-05-08T04:41:44.867538+0000","last_clean":"2024-05-08T04:41:44.867538+0000","last_became_active":"2024-05-08T04:41:43.889926+0000","last_became_peered":"2024-05-08T04:41:43.889926+0000","last_unstale":"2024-05-08T04:41:44.867538+0000","last_undegraded":"2024-05-08T04:41:44.867538+0000","last_fullsized":"2024-05-08T04:41:44.867538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:55.329766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890274+0000","last_change":"2024-05-08T04:41:42.886316+0000","last_active":"2024-05-08T04:41:44.890274+0000","last_peered":"2024-05-08T04:41:44.890274+0000","last_clean":"2024-05-08T04:41:44.890274+0000","last_became_active":"2024-05-08T04:41:42.885814+0000","last_became_peered":"2024-05-08T04:41:42.885814+0000","last_unstale":"2024-05-08T04:41:44.890274+0000","last_undegraded":"2024-05-08T04:41:44.890274+0000","last_fullsized":"2024-05-08T04:41:44.890274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:59.940626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954379+0000","last_change":"2024-05-08T04:41:43.873065+0000","last_active":"2024-05-08T04:41:46.954379+0000","last_peered":"2024-05-08T04:41:46.954379+0000","last_clean":"2024-05-08T04:41:46.954379+0000","last_became_active":"2024-05-08T04:41:43.872806+0000","last_became_peered":"2024-05-08T04:41:43.872806+0000","last_unstale":"2024-05-08T04:41:46.954379+0000","last_undegraded":"2024-05-08T04:41:46.954379+0000","last_fullsized":"2024-05-08T04:41:46.954379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:24.022429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887732+0000","last_change":"2024-05-08T04:41:42.873271+0000","last_active":"2024-05-08T04:41:44.887732+0000","last_peered":"2024-05-08T04:41:44.887732+0000","last_clean":"2024-05-08T04:41:44.887732+0000","last_became_active":"2024-05-08T04:41:42.873014+0000","last_became_peered":"2024-05-08T04:41:42.873014+0000","last_unstale":"2024-05-08T04:41:44.887732+0000","last_undegraded":"2024-05-08T04:41:44.887732+0000","last_fullsized":"2024-05-08T04:41:44.887732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:06.848370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954621+0000","last_change":"2024-05-08T04:41:43.874751+0000","last_active":"2024-05-08T04:41:46.954621+0000","last_peered":"2024-05-08T04:41:46.954621+0000","last_clean":"2024-05-08T04:41:46.954621+0000","last_became_active":"2024-05-08T04:41:43.873995+0000","last_became_peered":"2024-05-08T04:41:43.873995+0000","last_unstale":"2024-05-08T04:41:46.954621+0000","last_undegraded":"2024-05-08T04:41:46.954621+0000","last_fullsized":"2024-05-08T04:41:46.954621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:52.932175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.113048+0000","last_change":"2024-05-08T04:41:42.871169+0000","last_active":"2024-05-08T04:41:45.113048+0000","last_peered":"2024-05-08T04:41:45.113048+0000","last_clean":"2024-05-08T04:41:45.113048+0000","last_became_active":"2024-05-08T04:41:42.870616+0000","last_became_peered":"2024-05-08T04:41:42.870616+0000","last_unstale":"2024-05-08T04:41:45.113048+0000","last_undegraded":"2024-05-08T04:41:45.113048+0000","last_fullsized":"2024-05-08T04:41:45.113048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:57.864027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868981+0000","last_change":"2024-05-08T04:41:43.875732+0000","last_active":"2024-05-08T04:41:44.868981+0000","last_peered":"2024-05-08T04:41:44.868981+0000","last_clean":"2024-05-08T04:41:44.868981+0000","last_became_active":"2024-05-08T04:41:43.875372+0000","last_became_peered":"2024-05-08T04:41:43.875372+0000","last_unstale":"2024-05-08T04:41:44.868981+0000","last_undegraded":"2024-05-08T04:41:44.868981+0000","last_fullsized":"2024-05-08T04:41:44.868981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:42.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112640+0000","last_change":"2024-05-08T04:41:42.875013+0000","last_active":"2024-05-08T04:41:45.112640+0000","last_peered":"2024-05-08T04:41:45.112640+0000","last_clean":"2024-05-08T04:41:45.112640+0000","last_became_active":"2024-05-08T04:41:42.874750+0000","last_became_peered":"2024-05-08T04:41:42.874750+0000","last_unstale":"2024-05-08T04:41:45.112640+0000","last_undegraded":"2024-05-08T04:41:45.112640+0000","last_fullsized":"2024-05-08T04:41:45.112640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:03:20.200213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890445+0000","last_change":"2024-05-08T04:41:43.890561+0000","last_active":"2024-05-08T04:41:44.890445+0000","last_peered":"2024-05-08T04:41:44.890445+0000","last_clean":"2024-05-08T04:41:44.890445+0000","last_became_active":"2024-05-08T04:41:43.890246+0000","last_became_peered":"2024-05-08T04:41:43.890246+0000","last_unstale":"2024-05-08T04:41:44.890445+0000","last_undegraded":"2024-05-08T04:41:44.890445+0000","last_fullsized":"2024-05-08T04:41:44.890445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:05.516389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112202+0000","last_change":"2024-05-08T04:41:42.871506+0000","last_active":"2024-05-08T04:41:45.112202+0000","last_peered":"2024-05-08T04:41:45.112202+0000","last_clean":"2024-05-08T04:41:45.112202+0000","last_became_active":"2024-05-08T04:41:42.870887+0000","last_became_peered":"2024-05-08T04:41:42.870887+0000","last_unstale":"2024-05-08T04:41:45.112202+0000","last_undegraded":"2024-05-08T04:41:45.112202+0000","last_fullsized":"2024-05-08T04:41:45.112202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:57:05.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032087+0000","last_change":"2024-05-08T04:41:43.889286+0000","last_active":"2024-05-08T04:41:45.032087+0000","last_peered":"2024-05-08T04:41:45.032087+0000","last_clean":"2024-05-08T04:41:45.032087+0000","last_became_active":"2024-05-08T04:41:43.888589+0000","last_became_peered":"2024-05-08T04:41:43.888589+0000","last_unstale":"2024-05-08T04:41:45.032087+0000","last_undegraded":"2024-05-08T04:41:45.032087+0000","last_fullsized":"2024-05-08T04:41:45.032087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.664761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868744+0000","last_change":"2024-05-08T04:41:42.871241+0000","last_active":"2024-05-08T04:41:44.868744+0000","last_peered":"2024-05-08T04:41:44.868744+0000","last_clean":"2024-05-08T04:41:44.868744+0000","last_became_active":"2024-05-08T04:41:42.868401+0000","last_became_peered":"2024-05-08T04:41:42.868401+0000","last_unstale":"2024-05-08T04:41:44.868744+0000","last_undegraded":"2024-05-08T04:41:44.868744+0000","last_fullsized":"2024-05-08T04:41:44.868744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:07.567005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890357+0000","last_change":"2024-05-08T04:41:43.880447+0000","last_active":"2024-05-08T04:41:44.890357+0000","last_peered":"2024-05-08T04:41:44.890357+0000","last_clean":"2024-05-08T04:41:44.890357+0000","last_became_active":"2024-05-08T04:41:43.880141+0000","last_became_peered":"2024-05-08T04:41:43.880141+0000","last_unstale":"2024-05-08T04:41:44.890357+0000","last_undegraded":"2024-05-08T04:41:44.890357+0000","last_fullsized":"2024-05-08T04:41:44.890357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:53.752602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887855+0000","last_change":"2024-05-08T04:41:42.886962+0000","last_active":"2024-05-08T04:41:44.887855+0000","last_peered":"2024-05-08T04:41:44.887855+0000","last_clean":"2024-05-08T04:41:44.887855+0000","last_became_active":"2024-05-08T04:41:42.886642+0000","last_became_peered":"2024-05-08T04:41:42.886642+0000","last_unstale":"2024-05-08T04:41:44.887855+0000","last_undegraded":"2024-05-08T04:41:44.887855+0000","last_fullsized":"2024-05-08T04:41:44.887855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:11:32.810674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868278+0000","last_change":"2024-05-08T04:41:43.892830+0000","last_active":"2024-05-08T04:41:44.868278+0000","last_peered":"2024-05-08T04:41:44.868278+0000","last_clean":"2024-05-08T04:41:44.868278+0000","last_became_active":"2024-05-08T04:41:43.892520+0000","last_became_peered":"2024-05-08T04:41:43.892520+0000","last_unstale":"2024-05-08T04:41:44.868278+0000","last_undegraded":"2024-05-08T04:41:44.868278+0000","last_fullsized":"2024-05-08T04:41:44.868278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:16.444641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954356+0000","last_change":"2024-05-08T04:41:42.874956+0000","last_active":"2024-05-08T04:41:46.954356+0000","last_peered":"2024-05-08T04:41:46.954356+0000","last_clean":"2024-05-08T04:41:46.954356+0000","last_became_active":"2024-05-08T04:41:42.874687+0000","last_became_peered":"2024-05-08T04:41:42.874687+0000","last_unstale":"2024-05-08T04:41:46.954356+0000","last_undegraded":"2024-05-08T04:41:46.954356+0000","last_fullsized":"2024-05-08T04:41:46.954356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:11.195117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887677+0000","last_change":"2024-05-08T04:41:43.882660+0000","last_active":"2024-05-08T04:41:44.887677+0000","last_peered":"2024-05-08T04:41:44.887677+0000","last_clean":"2024-05-08T04:41:44.887677+0000","last_became_active":"2024-05-08T04:41:43.882424+0000","last_became_peered":"2024-05-08T04:41:43.882424+0000","last_unstale":"2024-05-08T04:41:44.887677+0000","last_undegraded":"2024-05-08T04:41:44.887677+0000","last_fullsized":"2024-05-08T04:41:44.887677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:11.488069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868836+0000","last_change":"2024-05-08T04:41:42.873490+0000","last_active":"2024-05-08T04:41:44.868836+0000","last_peered":"2024-05-08T04:41:44.868836+0000","last_clean":"2024-05-08T04:41:44.868836+0000","last_became_active":"2024-05-08T04:41:42.873206+0000","last_became_peered":"2024-05-08T04:41:42.873206+0000","last_unstale":"2024-05-08T04:41:44.868836+0000","last_undegraded":"2024-05-08T04:41:44.868836+0000","last_fullsized":"2024-05-08T04:41:44.868836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:49:14.068593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887603+0000","last_change":"2024-05-08T04:41:43.890169+0000","last_active":"2024-05-08T04:41:44.887603+0000","last_peered":"2024-05-08T04:41:44.887603+0000","last_clean":"2024-05-08T04:41:44.887603+0000","last_became_active":"2024-05-08T04:41:43.889002+0000","last_became_peered":"2024-05-08T04:41:43.889002+0000","last_unstale":"2024-05-08T04:41:44.887603+0000","last_undegraded":"2024-05-08T04:41:44.887603+0000","last_fullsized":"2024-05-08T04:41:44.887603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:42:28.040815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890134+0000","last_change":"2024-05-08T04:41:42.884638+0000","last_active":"2024-05-08T04:41:44.890134+0000","last_peered":"2024-05-08T04:41:44.890134+0000","last_clean":"2024-05-08T04:41:44.890134+0000","last_became_active":"2024-05-08T04:41:42.884290+0000","last_became_peered":"2024-05-08T04:41:42.884290+0000","last_unstale":"2024-05-08T04:41:44.890134+0000","last_undegraded":"2024-05-08T04:41:44.890134+0000","last_fullsized":"2024-05-08T04:41:44.890134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:23.911856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868021+0000","last_change":"2024-05-08T04:41:43.882480+0000","last_active":"2024-05-08T04:41:44.868021+0000","last_peered":"2024-05-08T04:41:44.868021+0000","last_clean":"2024-05-08T04:41:44.868021+0000","last_became_active":"2024-05-08T04:41:43.882145+0000","last_became_peered":"2024-05-08T04:41:43.882145+0000","last_unstale":"2024-05-08T04:41:44.868021+0000","last_undegraded":"2024-05-08T04:41:44.868021+0000","last_fullsized":"2024-05-08T04:41:44.868021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:24.660248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.113154+0000","last_change":"2024-05-08T04:41:42.870314+0000","last_active":"2024-05-08T04:41:45.113154+0000","last_peered":"2024-05-08T04:41:45.113154+0000","last_clean":"2024-05-08T04:41:45.113154+0000","last_became_active":"2024-05-08T04:41:42.870011+0000","last_became_peered":"2024-05-08T04:41:42.870011+0000","last_unstale":"2024-05-08T04:41:45.113154+0000","last_undegraded":"2024-05-08T04:41:45.113154+0000","last_fullsized":"2024-05-08T04:41:45.113154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:52:15.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.113034+0000","last_change":"2024-05-08T04:41:43.892206+0000","last_active":"2024-05-08T04:41:45.113034+0000","last_peered":"2024-05-08T04:41:45.113034+0000","last_clean":"2024-05-08T04:41:45.113034+0000","last_became_active":"2024-05-08T04:41:43.891834+0000","last_became_peered":"2024-05-08T04:41:43.891834+0000","last_unstale":"2024-05-08T04:41:45.113034+0000","last_undegraded":"2024-05-08T04:41:45.113034+0000","last_fullsized":"2024-05-08T04:41:45.113034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:07.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890689+0000","last_change":"2024-05-08T04:41:42.886046+0000","last_active":"2024-05-08T04:41:44.890689+0000","last_peered":"2024-05-08T04:41:44.890689+0000","last_clean":"2024-05-08T04:41:44.890689+0000","last_became_active":"2024-05-08T04:41:42.885749+0000","last_became_peered":"2024-05-08T04:41:42.885749+0000","last_unstale":"2024-05-08T04:41:44.890689+0000","last_undegraded":"2024-05-08T04:41:44.890689+0000","last_fullsized":"2024-05-08T04:41:44.890689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:49:06.766858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032243+0000","last_change":"2024-05-08T04:41:43.875903+0000","last_active":"2024-05-08T04:41:45.032243+0000","last_peered":"2024-05-08T04:41:45.032243+0000","last_clean":"2024-05-08T04:41:45.032243+0000","last_became_active":"2024-05-08T04:41:43.875618+0000","last_became_peered":"2024-05-08T04:41:43.875618+0000","last_unstale":"2024-05-08T04:41:45.032243+0000","last_undegraded":"2024-05-08T04:41:45.032243+0000","last_fullsized":"2024-05-08T04:41:45.032243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:46.225513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890620+0000","last_change":"2024-05-08T04:41:42.885644+0000","last_active":"2024-05-08T04:41:44.890620+0000","last_peered":"2024-05-08T04:41:44.890620+0000","last_clean":"2024-05-08T04:41:44.890620+0000","last_became_active":"2024-05-08T04:41:42.885366+0000","last_became_peered":"2024-05-08T04:41:42.885366+0000","last_unstale":"2024-05-08T04:41:44.890620+0000","last_undegraded":"2024-05-08T04:41:44.890620+0000","last_fullsized":"2024-05-08T04:41:44.890620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:06.914165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866723+0000","last_change":"2024-05-08T04:41:43.875252+0000","last_active":"2024-05-08T04:41:44.866723+0000","last_peered":"2024-05-08T04:41:44.866723+0000","last_clean":"2024-05-08T04:41:44.866723+0000","last_became_active":"2024-05-08T04:41:43.874984+0000","last_became_peered":"2024-05-08T04:41:43.874984+0000","last_unstale":"2024-05-08T04:41:44.866723+0000","last_undegraded":"2024-05-08T04:41:44.866723+0000","last_fullsized":"2024-05-08T04:41:44.866723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:55.375643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954464+0000","last_change":"2024-05-08T04:41:42.868312+0000","last_active":"2024-05-08T04:41:46.954464+0000","last_peered":"2024-05-08T04:41:46.954464+0000","last_clean":"2024-05-08T04:41:46.954464+0000","last_became_active":"2024-05-08T04:41:42.867480+0000","last_became_peered":"2024-05-08T04:41:42.867480+0000","last_unstale":"2024-05-08T04:41:46.954464+0000","last_undegraded":"2024-05-08T04:41:46.954464+0000","last_fullsized":"2024-05-08T04:41:46.954464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:30.237868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868644+0000","last_change":"2024-05-08T04:41:43.874434+0000","last_active":"2024-05-08T04:41:44.868644+0000","last_peered":"2024-05-08T04:41:44.868644+0000","last_clean":"2024-05-08T04:41:44.868644+0000","last_became_active":"2024-05-08T04:41:43.874085+0000","last_became_peered":"2024-05-08T04:41:43.874085+0000","last_unstale":"2024-05-08T04:41:44.868644+0000","last_undegraded":"2024-05-08T04:41:44.868644+0000","last_fullsized":"2024-05-08T04:41:44.868644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:21.022218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866721+0000","last_change":"2024-05-08T04:41:42.886166+0000","last_active":"2024-05-08T04:41:44.866721+0000","last_peered":"2024-05-08T04:41:44.866721+0000","last_clean":"2024-05-08T04:41:44.866721+0000","last_became_active":"2024-05-08T04:41:42.885906+0000","last_became_peered":"2024-05-08T04:41:42.885906+0000","last_unstale":"2024-05-08T04:41:44.866721+0000","last_undegraded":"2024-05-08T04:41:44.866721+0000","last_fullsized":"2024-05-08T04:41:44.866721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:50.384763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866585+0000","last_change":"2024-05-08T04:41:43.885920+0000","last_active":"2024-05-08T04:41:44.866585+0000","last_peered":"2024-05-08T04:41:44.866585+0000","last_clean":"2024-05-08T04:41:44.866585+0000","last_became_active":"2024-05-08T04:41:43.885633+0000","last_became_peered":"2024-05-08T04:41:43.885633+0000","last_unstale":"2024-05-08T04:41:44.866585+0000","last_undegraded":"2024-05-08T04:41:44.866585+0000","last_fullsized":"2024-05-08T04:41:44.866585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:07.853694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.866343+0000","last_change":"2024-05-08T04:41:42.871377+0000","last_active":"2024-05-08T04:41:44.866343+0000","last_peered":"2024-05-08T04:41:44.866343+0000","last_clean":"2024-05-08T04:41:44.866343+0000","last_became_active":"2024-05-08T04:41:42.870529+0000","last_became_peered":"2024-05-08T04:41:42.870529+0000","last_unstale":"2024-05-08T04:41:44.866343+0000","last_undegraded":"2024-05-08T04:41:44.866343+0000","last_fullsized":"2024-05-08T04:41:44.866343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:55.024373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.869023+0000","last_change":"2024-05-08T04:41:43.887499+0000","last_active":"2024-05-08T04:41:44.869023+0000","last_peered":"2024-05-08T04:41:44.869023+0000","last_clean":"2024-05-08T04:41:44.869023+0000","last_became_active":"2024-05-08T04:41:43.887134+0000","last_became_peered":"2024-05-08T04:41:43.887134+0000","last_unstale":"2024-05-08T04:41:44.869023+0000","last_undegraded":"2024-05-08T04:41:44.869023+0000","last_fullsized":"2024-05-08T04:41:44.869023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:46.225513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:46.954153+0000","last_change":"2024-05-08T04:41:42.868198+0000","last_active":"2024-05-08T04:41:46.954153+0000","last_peered":"2024-05-08T04:41:46.954153+0000","last_clean":"2024-05-08T04:41:46.954153+0000","last_became_active":"2024-05-08T04:41:42.867905+0000","last_became_peered":"2024-05-08T04:41:42.867905+0000","last_unstale":"2024-05-08T04:41:46.954153+0000","last_undegraded":"2024-05-08T04:41:46.954153+0000","last_fullsized":"2024-05-08T04:41:46.954153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:24.975669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032302+0000","last_change":"2024-05-08T04:41:43.882361+0000","last_active":"2024-05-08T04:41:45.032302+0000","last_peered":"2024-05-08T04:41:45.032302+0000","last_clean":"2024-05-08T04:41:45.032302+0000","last_became_active":"2024-05-08T04:41:43.882072+0000","last_became_peered":"2024-05-08T04:41:43.882072+0000","last_unstale":"2024-05-08T04:41:45.032302+0000","last_undegraded":"2024-05-08T04:41:45.032302+0000","last_fullsized":"2024-05-08T04:41:45.032302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:45.950574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887992+0000","last_change":"2024-05-08T04:41:42.870288+0000","last_active":"2024-05-08T04:41:44.887992+0000","last_peered":"2024-05-08T04:41:44.887992+0000","last_clean":"2024-05-08T04:41:44.887992+0000","last_became_active":"2024-05-08T04:41:42.870017+0000","last_became_peered":"2024-05-08T04:41:42.870017+0000","last_unstale":"2024-05-08T04:41:44.887992+0000","last_undegraded":"2024-05-08T04:41:44.887992+0000","last_fullsized":"2024-05-08T04:41:44.887992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:52.411561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868328+0000","last_change":"2024-05-08T04:41:43.888621+0000","last_active":"2024-05-08T04:41:44.868328+0000","last_peered":"2024-05-08T04:41:44.868328+0000","last_clean":"2024-05-08T04:41:44.868328+0000","last_became_active":"2024-05-08T04:41:43.888281+0000","last_became_peered":"2024-05-08T04:41:43.888281+0000","last_unstale":"2024-05-08T04:41:44.868328+0000","last_undegraded":"2024-05-08T04:41:44.868328+0000","last_fullsized":"2024-05-08T04:41:44.868328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:41:24.260326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.888126+0000","last_change":"2024-05-08T04:41:42.887618+0000","last_active":"2024-05-08T04:41:44.888126+0000","last_peered":"2024-05-08T04:41:44.888126+0000","last_clean":"2024-05-08T04:41:44.888126+0000","last_became_active":"2024-05-08T04:41:42.887317+0000","last_became_peered":"2024-05-08T04:41:42.887317+0000","last_unstale":"2024-05-08T04:41:44.888126+0000","last_undegraded":"2024-05-08T04:41:44.888126+0000","last_fullsized":"2024-05-08T04:41:44.888126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:54.370318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112615+0000","last_change":"2024-05-08T04:41:43.887995+0000","last_active":"2024-05-08T04:41:45.112615+0000","last_peered":"2024-05-08T04:41:45.112615+0000","last_clean":"2024-05-08T04:41:45.112615+0000","last_became_active":"2024-05-08T04:41:43.887671+0000","last_became_peered":"2024-05-08T04:41:43.887671+0000","last_unstale":"2024-05-08T04:41:45.112615+0000","last_undegraded":"2024-05-08T04:41:45.112615+0000","last_fullsized":"2024-05-08T04:41:45.112615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:46.225513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112511+0000","last_change":"2024-05-08T04:41:42.871173+0000","last_active":"2024-05-08T04:41:45.112511+0000","last_peered":"2024-05-08T04:41:45.112511+0000","last_clean":"2024-05-08T04:41:45.112511+0000","last_became_active":"2024-05-08T04:41:42.870414+0000","last_became_peered":"2024-05-08T04:41:42.870414+0000","last_unstale":"2024-05-08T04:41:45.112511+0000","last_undegraded":"2024-05-08T04:41:45.112511+0000","last_fullsized":"2024-05-08T04:41:45.112511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:32.158729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.032304+0000","last_change":"2024-05-08T04:41:43.889440+0000","last_active":"2024-05-08T04:41:45.032304+0000","last_peered":"2024-05-08T04:41:45.032304+0000","last_clean":"2024-05-08T04:41:45.032304+0000","last_became_active":"2024-05-08T04:41:43.888992+0000","last_became_peered":"2024-05-08T04:41:43.888992+0000","last_unstale":"2024-05-08T04:41:45.032304+0000","last_undegraded":"2024-05-08T04:41:45.032304+0000","last_fullsized":"2024-05-08T04:41:45.032304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:49:07.772182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112587+0000","last_change":"2024-05-08T04:41:42.872160+0000","last_active":"2024-05-08T04:41:45.112587+0000","last_peered":"2024-05-08T04:41:45.112587+0000","last_clean":"2024-05-08T04:41:45.112587+0000","last_became_active":"2024-05-08T04:41:42.870594+0000","last_became_peered":"2024-05-08T04:41:42.870594+0000","last_unstale":"2024-05-08T04:41:45.112587+0000","last_undegraded":"2024-05-08T04:41:45.112587+0000","last_fullsized":"2024-05-08T04:41:45.112587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:47:30.028762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868845+0000","last_change":"2024-05-08T04:41:43.874589+0000","last_active":"2024-05-08T04:41:44.868845+0000","last_peered":"2024-05-08T04:41:44.868845+0000","last_clean":"2024-05-08T04:41:44.868845+0000","last_became_active":"2024-05-08T04:41:43.873980+0000","last_became_peered":"2024-05-08T04:41:43.873980+0000","last_unstale":"2024-05-08T04:41:44.868845+0000","last_undegraded":"2024-05-08T04:41:44.868845+0000","last_fullsized":"2024-05-08T04:41:44.868845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.664761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890500+0000","last_change":"2024-05-08T04:41:42.885472+0000","last_active":"2024-05-08T04:41:44.890500+0000","last_peered":"2024-05-08T04:41:44.890500+0000","last_clean":"2024-05-08T04:41:44.890500+0000","last_became_active":"2024-05-08T04:41:42.885199+0000","last_became_peered":"2024-05-08T04:41:42.885199+0000","last_unstale":"2024-05-08T04:41:44.890500+0000","last_undegraded":"2024-05-08T04:41:44.890500+0000","last_fullsized":"2024-05-08T04:41:44.890500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.945250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.890528+0000","last_change":"2024-05-08T04:41:43.879144+0000","last_active":"2024-05-08T04:41:44.890528+0000","last_peered":"2024-05-08T04:41:44.890528+0000","last_clean":"2024-05-08T04:41:44.890528+0000","last_became_active":"2024-05-08T04:41:43.878895+0000","last_became_peered":"2024-05-08T04:41:43.878895+0000","last_unstale":"2024-05-08T04:41:44.890528+0000","last_undegraded":"2024-05-08T04:41:44.890528+0000","last_fullsized":"2024-05-08T04:41:44.890528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:24.660248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031739+0000","last_change":"2024-05-08T04:41:42.873472+0000","last_active":"2024-05-08T04:41:45.031739+0000","last_peered":"2024-05-08T04:41:45.031739+0000","last_clean":"2024-05-08T04:41:45.031739+0000","last_became_active":"2024-05-08T04:41:42.872143+0000","last_became_peered":"2024-05-08T04:41:42.872143+0000","last_unstale":"2024-05-08T04:41:45.031739+0000","last_undegraded":"2024-05-08T04:41:45.031739+0000","last_fullsized":"2024-05-08T04:41:45.031739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:21.054164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.867585+0000","last_change":"2024-05-08T04:41:43.894935+0000","last_active":"2024-05-08T04:41:44.867585+0000","last_peered":"2024-05-08T04:41:44.867585+0000","last_clean":"2024-05-08T04:41:44.867585+0000","last_became_active":"2024-05-08T04:41:43.894551+0000","last_became_peered":"2024-05-08T04:41:43.894551+0000","last_unstale":"2024-05-08T04:41:44.867585+0000","last_undegraded":"2024-05-08T04:41:44.867585+0000","last_fullsized":"2024-05-08T04:41:44.867585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:53.752602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112391+0000","last_change":"2024-05-08T04:41:42.870431+0000","last_active":"2024-05-08T04:41:45.112391+0000","last_peered":"2024-05-08T04:41:45.112391+0000","last_clean":"2024-05-08T04:41:45.112391+0000","last_became_active":"2024-05-08T04:41:42.870136+0000","last_became_peered":"2024-05-08T04:41:42.870136+0000","last_unstale":"2024-05-08T04:41:45.112391+0000","last_undegraded":"2024-05-08T04:41:45.112391+0000","last_fullsized":"2024-05-08T04:41:45.112391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:41:20.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.112326+0000","last_change":"2024-05-08T04:41:43.886236+0000","last_active":"2024-05-08T04:41:45.112326+0000","last_peered":"2024-05-08T04:41:45.112326+0000","last_clean":"2024-05-08T04:41:45.112326+0000","last_became_active":"2024-05-08T04:41:43.885940+0000","last_became_peered":"2024-05-08T04:41:43.885940+0000","last_unstale":"2024-05-08T04:41:45.112326+0000","last_undegraded":"2024-05-08T04:41:45.112326+0000","last_fullsized":"2024-05-08T04:41:45.112326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:08.818996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:45.031581+0000","last_change":"2024-05-08T04:41:42.876426+0000","last_active":"2024-05-08T04:41:45.031581+0000","last_peered":"2024-05-08T04:41:45.031581+0000","last_clean":"2024-05-08T04:41:45.031581+0000","last_became_active":"2024-05-08T04:41:42.876141+0000","last_became_peered":"2024-05-08T04:41:42.876141+0000","last_unstale":"2024-05-08T04:41:45.031581+0000","last_undegraded":"2024-05-08T04:41:45.031581+0000","last_fullsized":"2024-05-08T04:41:45.031581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:17:57.864027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.887920+0000","last_change":"2024-05-08T04:41:43.877816+0000","last_active":"2024-05-08T04:41:44.887920+0000","last_peered":"2024-05-08T04:41:44.887920+0000","last_clean":"2024-05-08T04:41:44.887920+0000","last_became_active":"2024-05-08T04:41:43.876942+0000","last_became_peered":"2024-05-08T04:41:43.876942+0000","last_unstale":"2024-05-08T04:41:44.887920+0000","last_undegraded":"2024-05-08T04:41:44.887920+0000","last_fullsized":"2024-05-08T04:41:44.887920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:42.845526+0000","last_clean_scrub_stamp":"2024-05-08T04:41:42.845526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:20.252023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.868149+0000","last_change":"2024-05-08T04:41:42.868782+0000","last_active":"2024-05-08T04:41:44.868149+0000","last_peered":"2024-05-08T04:41:44.868149+0000","last_clean":"2024-05-08T04:41:44.868149+0000","last_became_active":"2024-05-08T04:41:42.868478+0000","last_became_peered":"2024-05-08T04:41:42.868478+0000","last_unstale":"2024-05-08T04:41:44.868149+0000","last_undegraded":"2024-05-08T04:41:44.868149+0000","last_fullsized":"2024-05-08T04:41:44.868149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:41:41.840202+0000","last_clean_scrub_stamp":"2024-05-08T04:41:41.840202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:49:06.766858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T04:41:44.867599+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:41:44.867599+0000","last_peered":"2024-05-08T04:41:44.867599+0000","last_clean":"2024-05-08T04:41:44.867599+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:41:44.867599+0000","last_undegraded":"2024-05-08T04:41:44.867599+0000","last_fullsized":"2024-05-08T04:41:44.867599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1654784,"data_stored":1044480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":454736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":455676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1056768,"data_stored":454042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":449298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":454624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1069056,"data_stored":454236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1044444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:41:52.858 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:41:52.858 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:41:53.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.032+0000 7f2f74600640 1 Processor -- start 2024-05-08T04:41:53.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.032+0000 7f2f74600640 1 -- start start 2024-05-08T04:41:53.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f74600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f74600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f74600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0cfce0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f74600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c071be0 con 0x7f2f6c072720 2024-05-08T04:41:53.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f74600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c071060 con 0x7f2f6c070bd0 2024-05-08T04:41:53.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f72600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:59872/0 (socket says 172.21.15.164:59872) 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f72600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:57168/0 (socket says 172.21.15.164:57168) 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 -- 172.21.15.164:0/564870281 learned_addr learned my addr 172.21.15.164:0/564870281 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 -- 172.21.15.164:0/564870281 >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 msgr2=0x7f2f6c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 --2- 172.21.15.164:0/564870281 >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 -- 172.21.15.164:0/564870281 >> v1:172.21.15.172:6789/0 conn(0x7f2f6c0cf900 legacy=0x7f2f6c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f71200640 1 -- 172.21.15.164:0/564870281 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c071340 con 0x7f2f6c070bd0 2024-05-08T04:41:53.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.036+0000 7f2f72600640 1 --2- 172.21.15.164:0/564870281 >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:53.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f71200640 1 --2- 172.21.15.164:0/564870281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009870 tx=0x7f2f5801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afeae4680cecdff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f6be00640 1 -- 172.21.15.164:0/564870281 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c152f90 con 0x7f2f6c070bd0 2024-05-08T04:41:53.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f6be00640 1 -- 172.21.15.164:0/564870281 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f580b0070 con 0x7f2f6c070bd0 2024-05-08T04:41:53.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f6be00640 1 -- 172.21.15.164:0/564870281 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580ab870 con 0x7f2f6c070bd0 2024-05-08T04:41:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 -- 172.21.15.164:0/564870281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 msgr2=0x7f2f6c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 --2- 172.21.15.164:0/564870281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009870 tx=0x7f2f5801c560 comp rx=0 tx=0).stop 2024-05-08T04:41:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 -- 172.21.15.164:0/564870281 shutdown_connections 2024-05-08T04:41:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 --2- 172.21.15.164:0/564870281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c070bd0 0x7f2f6c15cb20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 --2- 172.21.15.164:0/564870281 >> v2:172.21.15.172:3300/0 conn(0x7f2f6c072720 0x7f2f6c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 -- 172.21.15.164:0/564870281 >> 172.21.15.164:0/564870281 conn(0x7f2f6c06cca0 msgr2=0x7f2f6c06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 -- 172.21.15.164:0/564870281 shutdown_connections 2024-05-08T04:41:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 -- 172.21.15.164:0/564870281 wait complete. 2024-05-08T04:41:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.040+0000 7f2f74600640 1 Processor -- start 2024-05-08T04:41:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- start start 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c072720 0x7f2f6c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f6c1faad0 0x7f2f6c146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c06b0f0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c06af70 con 0x7f2f6c1faad0 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c06b270 con 0x7f2f6c072720 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36606/0 (socket says 172.21.15.164:36606) 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 -- 172.21.15.164:0/348307493 learned_addr learned my addr 172.21.15.164:0/348307493 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71c00640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c072720 0x7f2f6c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f72600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f6c1faad0 0x7f2f6c146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 -- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c072720 msgr2=0x7f2f6c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c072720 0x7f2f6c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 -- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f6c1faad0 msgr2=0x7f2f6c146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f6c1faad0 0x7f2f6c146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 -- 172.21.15.164:0/348307493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c1544d0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f71200640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2f5801a190 tx=0x7f2f5801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580be5f0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f580be7b0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c1547b0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580beba0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c154d70 con 0x7f2f6c0cf900 2024-05-08T04:41:53.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.044+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c06ddf0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.048+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f2f580b43f0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.048+0000 7f2f69600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 0x7f2f4c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.048+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2f581203e0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.048+0000 7f2f71c00640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 0x7f2f4c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.048+0000 7f2f71c00640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 0x7f2f4c0699c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2f60001770 tx=0x7f2f60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.052+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f58016150 con 0x7f2f6c0cf900 2024-05-08T04:41:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.260+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2f6c1604c0 con 0x7f2f6c0cf900 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.264+0000 7f2f69600640 1 -- 172.21.15.164:0/348307493 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2f580f0430 con 0x7f2f6c0cf900 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:41:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:41:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:41:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 msgr2=0x7f2f4c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 0x7f2f4c0699c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2f60001770 tx=0x7f2f60009040 comp rx=0 tx=0).stop 2024-05-08T04:41:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 msgr2=0x7f2f6c153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2f5801a190 tx=0x7f2f5801ada0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 shutdown_connections 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f4c067500 0x7f2f4c0699c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f6c1faad0 0x7f2f6c146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f6c0cf900 0x7f2f6c153df0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 --2- 172.21.15.164:0/348307493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f6c072720 0x7f2f6c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 >> 172.21.15.164:0/348307493 conn(0x7f2f6c06cca0 msgr2=0x7f2f6c1509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 shutdown_connections 2024-05-08T04:41:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.268+0000 7f2f74600640 1 -- 172.21.15.164:0/348307493 wait complete. 2024-05-08T04:41:53.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:41:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 Processor -- start 2024-05-08T04:41:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 -- start start 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f41a40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f41a406bca0 0x7f41a406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a4071980 con 0x7f41a40cdfa0 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a4161810 con 0x7f41a406bca0 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41aaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a406eed0 con 0x7f41a40af480 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f419b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f41a40cdfa0 0x7f41a406cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51260/0 (socket says 172.21.15.164:51260) 2024-05-08T04:41:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f419b400640 1 -- 172.21.15.164:0/2985601099 learned_addr learned my addr 172.21.15.164:0/2985601099 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3e00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f41a40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a8a00640 1 --2- 172.21.15.164:0/2985601099 >> v2:172.21.15.172:3300/0 conn(0x7f41a406bca0 0x7f41a406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266556726 0 0) 0x7f418c001120 con 0x7f41a40cdfa0 2024-05-08T04:41:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4188003530 con 0x7f41a40cdfa0 2024-05-08T04:41:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181170676 0 0) 0x7f4188003530 con 0x7f41a40cdfa0 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 msgr2=0x7f41a40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f41a40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 >> v2:172.21.15.172:3300/0 conn(0x7f41a406bca0 msgr2=0x7f41a406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 --2- 172.21.15.164:0/2985601099 >> v2:172.21.15.172:3300/0 conn(0x7f41a406bca0 0x7f41a406c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4071c00 con 0x7f41a40cdfa0 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f418c001850 con 0x7f41a40cdfa0 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 >> v1:172.21.15.164:6789/0 conn(0x7f41a40cdfa0 legacy=0x7f41a406cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880039c0 0x7f4188003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f4188004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4188004d50 0x7f41880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4161810 con 0x7f41880039c0 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a406eed0 con 0x7f4188004d50 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418800b9d0 con 0x7f41a40af480 2024-05-08T04:41:53.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.424+0000 7f41a3e00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880039c0 0x7f4188003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a8a00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4188004d50 0x7f41880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f419b400640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f4188004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 -- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 msgr2=0x7f4188004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40af480 0x7f4188004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 -- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4188004d50 msgr2=0x7f41880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4188004d50 0x7f41880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 -- 172.21.15.164:0/2985601099 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188003530 con 0x7f41880039c0 2024-05-08T04:41:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3e00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880039c0 0x7f4188003de0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4194009ae0 tx=0x7f419401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281f569fa263550f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940b2070 con 0x7f41880039c0 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41940a5550 con 0x7f41880039c0 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a3400640 1 -- 172.21.15.164:0/2985601099 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940096b0 con 0x7f41880039c0 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41aaa00640 1 -- 172.21.15.164:0/2985601099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880039c0 msgr2=0x7f4188003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41aaa00640 1 --2- 172.21.15.164:0/2985601099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880039c0 0x7f4188003de0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4194009ae0 tx=0x7f419401ca70 comp rx=0 tx=0).stop 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.428+0000 7f41a8a00640 1 -- 172.21.15.164:0/2985601099 reap_dead start 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- 172.21.15.164:0/2985601099 shutdown_connections 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- 172.21.15.164:0/2985601099 >> 172.21.15.164:0/2985601099 conn(0x7f41a40ccab0 msgr2=0x7f41a40ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- 172.21.15.164:0/2985601099 shutdown_connections 2024-05-08T04:41:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- 172.21.15.164:0/2985601099 wait complete. 2024-05-08T04:41:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 Processor -- start 2024-05-08T04:41:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- start start 2024-05-08T04:41:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41880039c0 0x7f41a40caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40c5ea0 0x7f41a40c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4072bd0 con 0x7f41a40c3240 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4071f50 con 0x7f41880039c0 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41aaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4072d50 con 0x7f41a40c5ea0 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36622/0 (socket says 172.21.15.164:36622) 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 -- 172.21.15.164:0/1813613618 learned_addr learned my addr 172.21.15.164:0/1813613618 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41a3e00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41880039c0 0x7f41a40caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41a8a00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40c5ea0 0x7f41a40c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 -- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40c5ea0 msgr2=0x7f41a40c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40c5ea0 0x7f41a40c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 -- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41880039c0 msgr2=0x7f41a40caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41880039c0 0x7f41a40caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 -- 172.21.15.164:0/1813613618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4151930 con 0x7f41a40c3240 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f419b400640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f418c007330 tx=0x7f418c0077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00e950 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.432+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c018070 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00ed50 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a40cb5f0 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a40cbbb0 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a4069d60 con 0x7f41a40c3240 2024-05-08T04:41:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f418c01d460 con 0x7f41a40c3240 2024-05-08T04:41:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a0c00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 0x7f41800699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f418c085af0 con 0x7f41a40c3240 2024-05-08T04:41:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a3e00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 0x7f41800699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.436+0000 7f41a3e00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 0x7f41800699c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f41940af000 tx=0x7f41940046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:53.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.440+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41a4071980 con 0x7f41a40c3240 2024-05-08T04:41:53.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.628+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41a4066df0 con 0x7f41a40c3240 2024-05-08T04:41:53.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.628+0000 7f41a0c00640 1 -- 172.21.15.164:0/1813613618 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f418c056210 con 0x7f41a40c3240 2024-05-08T04:41:53.633 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:41:53.634 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:41:53.634 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 52 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 msgr2=0x7f41800699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 0x7f41800699c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f41940af000 tx=0x7f41940046d0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 msgr2=0x7f41a40caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f418c007330 tx=0x7f418c0077f0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 shutdown_connections 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4180067500 0x7f41800699c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41a40c5ea0 0x7f41a40c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41a40c3240 0x7f41a40caf50 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 --2- 172.21.15.164:0/1813613618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41880039c0 0x7f41a40caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 >> 172.21.15.164:0/1813613618 conn(0x7f41a40ccab0 msgr2=0x7f41a4162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 shutdown_connections 2024-05-08T04:41:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:53.632+0000 7f41aaa00640 1 -- 172.21.15.164:0/1813613618 wait complete. 2024-05-08T04:41:54.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:41:43.227654+0000', 'modified': '2024-05-08T04:41:46.260430+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6951}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6951': {'gid': 6951, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/1815840371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1815840371}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1815840371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:41:54.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:41:54.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T04:41:54.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.188+0000 7fb3cf400640 1 Processor -- start 2024-05-08T04:41:54.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.188+0000 7fb3cf400640 1 -- start start 2024-05-08T04:41:54.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cf400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3c815bc50 0x7fb3c80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cf400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c80b42a0 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cf400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c80cd4e0 con 0x7fb3c80d0780 2024-05-08T04:41:54.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cf400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c80cd7c0 con 0x7fb3c815bc50 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3c815bc50 0x7fb3c80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3c7e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb3c80ccfe0 0x7fb3c80d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51270/0 (socket says 172.21.15.164:51270) 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3c7e00640 1 -- 172.21.15.164:0/76951014 learned_addr learned my addr 172.21.15.164:0/76951014 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cca00640 1 -- 172.21.15.164:0/76951014 >> v1:172.21.15.172:6789/0 conn(0x7fb3c80d0780 legacy=0x7fb3c815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cca00640 1 -- 172.21.15.164:0/76951014 >> v1:172.21.15.164:6789/0 conn(0x7fb3c80ccfe0 legacy=0x7fb3c80d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3cca00640 1 -- 172.21.15.164:0/76951014 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80cf590 con 0x7fb3c815bc50 2024-05-08T04:41:54.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.192+0000 7fb3c7400640 1 -- 172.21.15.164:0/76951014 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655047372 0 0) 0x7fb3c80b42a0 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cca00640 1 --2- 172.21.15.164:0/76951014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c815bc50 0x7fb3c80b4810 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009d90 tx=0x7fb3b801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42a01c773104fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3c7400640 1 -- 172.21.15.164:0/76951014 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b2070 con 0x7fb3c815bc50 2024-05-08T04:41:54.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3c7400640 1 -- 172.21.15.164:0/76951014 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80a5930 con 0x7fb3c815bc50 2024-05-08T04:41:54.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3c7400640 1 -- 172.21.15.164:0/76951014 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a5d00 con 0x7fb3c815bc50 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- 172.21.15.164:0/76951014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c815bc50 msgr2=0x7fb3c80b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 --2- 172.21.15.164:0/76951014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c815bc50 0x7fb3c80b4810 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009d90 tx=0x7fb3b801c870 comp rx=0 tx=0).stop 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- 172.21.15.164:0/76951014 shutdown_connections 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 --2- 172.21.15.164:0/76951014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c815bc50 0x7fb3c80b4810 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- 172.21.15.164:0/76951014 >> 172.21.15.164:0/76951014 conn(0x7fb3c80b34f0 msgr2=0x7fb3c80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- 172.21.15.164:0/76951014 shutdown_connections 2024-05-08T04:41:54.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- 172.21.15.164:0/76951014 wait complete. 2024-05-08T04:41:54.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 Processor -- start 2024-05-08T04:41:54.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.196+0000 7fb3cf400640 1 -- start start 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 0x7fb3c815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 0x7fb3c81524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8169c70 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8169af0 con 0x7fb3c815bc50 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8169df0 con 0x7fb3c80d0780 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36644/0 (socket says 172.21.15.164:36644) 2024-05-08T04:41:54.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 -- 172.21.15.164:0/1940971585 learned_addr learned my addr 172.21.15.164:0/1940971585 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 0x7fb3c815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cd400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 0x7fb3c81524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 -- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 msgr2=0x7fb3c815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 0x7fb3c815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 -- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 msgr2=0x7fb3c81524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 0x7fb3c81524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 -- 172.21.15.164:0/1940971585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c815ea40 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 0x7fb3c815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cd400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 0x7fb3c81524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b0040 tx=0x7fb3b8019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3c4c00640 1 -- 172.21.15.164:0/1940971585 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80bd020 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3c4c00640 1 -- 172.21.15.164:0/1940971585 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80b8670 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3c4c00640 1 -- 172.21.15.164:0/1940971585 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b2040 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c80cb560 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.200+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c815c2a0 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c4c00640 1 -- 172.21.15.164:0/1940971585 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fb3b80c0020 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c4c00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 0x7fb3a80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c4c00640 1 -- 172.21.15.164:0/1940971585 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb3b8120d10 con 0x7fb3c80ccfe0 2024-05-08T04:41:54.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 0x7fb3a80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 Processor -- start 2024-05-08T04:41:54.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 -- start start 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 0x7fb3a80699c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8153040 tx=0x7fb3b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba4f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2ba50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b9375f0 con 0x55bb2ba99920 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b89eda0 con 0x55bb2ba9a240 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b827150 con 0x55bb2ba4ff30 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36660/0 (socket says 172.21.15.164:36660) 2024-05-08T04:41:54.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 -- 172.21.15.164:0/1602293220 learned_addr learned my addr 172.21.15.164:0/1602293220 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba4f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cd400640 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2ba50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 -- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 msgr2=0x55bb2ba50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2ba50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 -- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 msgr2=0x55bb2ba4f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba4f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 -- 172.21.15.164:0/1602293220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2b840ca0 con 0x55bb2ba99920 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3cca00640 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8098450 tx=0x7fb3b80f1620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e01725389ec7920 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3bd600640 1 -- 172.21.15.164:0/1602293220 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80f9570 con 0x55bb2ba99920 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3bd600640 1 -- 172.21.15.164:0/1602293220 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80ed4d0 con 0x55bb2ba99920 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.204+0000 7fb3bd600640 1 -- 172.21.15.164:0/1602293220 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80f70b0 con 0x55bb2ba99920 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/1602293220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 msgr2=0x55bb2ba99d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8098450 tx=0x7fb3b80f1620 comp rx=0 tx=0).stop 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/1602293220 shutdown_connections 2024-05-08T04:41:54.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2ba50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba4f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/1602293220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2ba99d00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/1602293220 >> 172.21.15.164:0/1602293220 conn(0x55bb2ba562a0 msgr2=0x55bb2ba4e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/1602293220 shutdown_connections 2024-05-08T04:41:54.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/1602293220 wait complete. 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- start start 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 Processor -- start 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baa0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b859990 con 0x55bb2ba99920 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b89ae40 con 0x55bb2ba9a240 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b82a900 con 0x55bb2ba4ff30 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 10 client.0 _ll_register_callbacks cb 0x55bb2b9521c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36664/0 (socket says 172.21.15.164:36664) 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 -- 172.21.15.164:0/414387113 learned_addr learned my addr 172.21.15.164:0/414387113 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3cca00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baa0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3cd400640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba53aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 -- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 msgr2=0x55bb2baa0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baa0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 -- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 msgr2=0x55bb2ba53aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba53aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 -- 172.21.15.164:0/414387113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2b857630 con 0x55bb2ba99920 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4007350 tx=0x7fb3b4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00cbc462809a421 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3b2000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4026060 con 0x55bb2ba99920 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb3b4021ad0 con 0x55bb2ba99920 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3b80ed4d0 con 0x55bb2ba99920 2024-05-08T04:41:54.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b4021cd0 con 0x55bb2ba99920 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 10 client.7107 fetch_fsmap learned FSMap version 52 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.208+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55bb2b8273a0 con 0x55bb2ba99920 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80ed4d0 con 0x55bb2ba99920 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb3b4012410 con 0x55bb2ba99920 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 10 client.7107 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 10 client.7107 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 20 client.7107 populate_metadata read hostname 'smithi164' 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2ba55160 0x55bb2baaace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 4 client.7107 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 --> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb2b83aa50 con 0x55bb2ba55160 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3cca00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2ba55160 0x55bb2baaace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 10 client.7107 ms_handle_connect on v2:172.21.15.164:6836/1815840371 2024-05-08T04:41:54.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3cca00640 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2ba55160 0x55bb2baaace0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628b0eb724bb9339 server_cookie=35b2830fdeabcd01 in_seq=0 out_seq=0 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 1 -- 172.21.15.164:0/414387113 <== mds.0 v2:172.21.15.164:6836/1815840371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb2b83aa50 con 0x55bb2ba55160 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 10 client.7107 handle_command_reply: tid=0 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 client.7107 shutdown 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 msgr2=0x55bb2baa11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4007350 tx=0x7fb3b4000ee0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 shutdown_connections 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2ba55160 0x55bb2baaace0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba9a240 0x55bb2ba53aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba99920 0x55bb2baa11c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/414387113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baa0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3b2000640 0 client.7107 ms_handle_reset on v2:172.21.15.164:6836/1815840371 2024-05-08T04:41:54.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 >> 172.21.15.164:0/414387113 conn(0x55bb2ba562a0 msgr2=0x55bb2ba4e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 shutdown_connections 2024-05-08T04:41:54.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 1 -- 172.21.15.164:0/414387113 wait complete. 2024-05-08T04:41:54.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.212+0000 7fb3d27b7000 20 client.7107 trim_cache size 0 max 16384 2024-05-08T04:41:54.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 Processor -- start 2024-05-08T04:41:54.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 -- start start 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baf53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2baf58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c82024e0 con 0x55bb2ba9a240 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815d3b0 con 0x55bb2ba99920 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815d9b0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36668/0 (socket says 172.21.15.164:36668) 2024-05-08T04:41:54.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 -- 172.21.15.164:0/2891973108 learned_addr learned my addr 172.21.15.164:0/2891973108 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2baf58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cca00640 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baf53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 -- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 msgr2=0x55bb2baf53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baf53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 -- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 msgr2=0x55bb2baf58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2baf58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 -- 172.21.15.164:0/2891973108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2ba9a070 con 0x55bb2ba9a240 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3cd400640 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0004e60 tx=0x7fb3c0005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f63669b7060bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3bcc00640 1 -- 172.21.15.164:0/2891973108 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0019070 con 0x55bb2ba9a240 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3bcc00640 1 -- 172.21.15.164:0/2891973108 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0012ab0 con 0x55bb2ba9a240 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.228+0000 7fb3bcc00640 1 -- 172.21.15.164:0/2891973108 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0012e80 con 0x55bb2ba9a240 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/2891973108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 msgr2=0x55bb2baf5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0004e60 tx=0x7fb3c0005320 comp rx=0 tx=0).stop 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/2891973108 shutdown_connections 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba9a240 0x55bb2baf5e20 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2baf58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/2891973108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba4ff30 0x55bb2baf53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/2891973108 >> 172.21.15.164:0/2891973108 conn(0x55bb2ba562a0 msgr2=0x55bb2ba53220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/2891973108 shutdown_connections 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/2891973108 wait complete. 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- start start 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 Processor -- start 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2bade730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba9a240 0x55bb2badec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b8a33d0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b95ad50 con 0x55bb2ba99920 2024-05-08T04:41:54.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2b839980 con 0x55bb2ba9a240 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 10 client.0 _ll_register_callbacks cb 0x7fb3b80f0cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36670/0 (socket says 172.21.15.164:36670) 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 -- 172.21.15.164:0/3828739273 learned_addr learned my addr 172.21.15.164:0/3828739273 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 -- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba9a240 msgr2=0x55bb2badec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba9a240 0x55bb2badec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 -- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 msgr2=0x55bb2bade730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2bade730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 -- 172.21.15.164:0/3828739273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2b920600 con 0x55bb2ba4ff30 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3cca00640 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ef110 tx=0x7fb3b80c4070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95263e49af28f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b810c040 con 0x55bb2ba4ff30 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c0012ab0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb3b8005d20 con 0x55bb2ba4ff30 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b80f9570 con 0x55bb2ba4ff30 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 10 client.7113 fetch_fsmap learned FSMap version 52 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55bb2b8339b0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0012ab0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb3b80f70b0 con 0x55bb2ba4ff30 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 10 client.7113 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 10 client.7113 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 20 client.7113 populate_metadata read hostname 'smithi164' 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2badf1b0 0x55bb2badf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 4 client.7113 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 --> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb2b8ac2e0 con 0x55bb2badf1b0 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.232+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2badf1b0 0x55bb2badf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3c7e00640 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2badf1b0 0x55bb2badf590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01f17ed723e8710 server_cookie=ae1a756addbe71b5 in_seq=0 out_seq=0 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3b2000640 10 client.7113 ms_handle_connect on v2:172.21.15.164:6836/1815840371 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3b2000640 1 -- 172.21.15.164:0/3828739273 <== mds.0 v2:172.21.15.164:6836/1815840371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55bb2b8ac2e0 con 0x55bb2badf1b0 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3b2000640 10 client.7113 handle_command_reply: tid=0 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 client.7113 shutdown 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 msgr2=0x55bb2baddf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ef110 tx=0x7fb3b80c4070 comp rx=0 tx=0).stop 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 shutdown_connections 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:6836/1815840371,v1:172.21.15.164:6837/1815840371] conn(0x55bb2badf1b0 0x55bb2badf590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bb2ba9a240 0x55bb2badec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb2ba99920 0x55bb2bade730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 --2- 172.21.15.164:0/3828739273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb2ba4ff30 0x55bb2baddf30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3b2000640 0 client.7113 ms_handle_reset on v2:172.21.15.164:6836/1815840371 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 >> 172.21.15.164:0/3828739273 conn(0x55bb2ba562a0 msgr2=0x55bb2baa1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 shutdown_connections 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 1 -- 172.21.15.164:0/3828739273 wait complete. 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3d27b7000 20 client.7113 trim_cache size 0 max 16384 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 6951, 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:41:54.248 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 10.363251631000001, 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 52, 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 86, 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 86, 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 15.282612044 2024-05-08T04:41:54.249 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 msgr2=0x7fb3a80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 0x7fb3a80699c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8153040 tx=0x7fb3b4009040 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 msgr2=0x7fb3c80c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b0040 tx=0x7fb3b8019610 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 shutdown_connections 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3a8067500 0x7fb3a80699c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80d0780 0x7fb3c815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c815bc50 0x7fb3c81524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 --2- 172.21.15.164:0/1940971585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3c80ccfe0 0x7fb3c80c8d10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 >> 172.21.15.164:0/1940971585 conn(0x7fb3c80b34f0 msgr2=0x7fb3c806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 shutdown_connections 2024-05-08T04:41:54.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.236+0000 7fb3cf400640 1 -- 172.21.15.164:0/1940971585 wait complete. 2024-05-08T04:41:54.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:41:54.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 Processor -- start 2024-05-08T04:41:54.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 -- start start 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f65180cfda0 0x7f6518160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f65180b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180cd9f0 con 0x7f651815ba70 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651815fe70 con 0x7f65180cfda0 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cdcd0 con 0x7f651815be50 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651d400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f651815ba70 0x7f65180cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51286/0 (socket says 172.21.15.164:51286) 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651d400640 1 -- 172.21.15.164:0/691029686 learned_addr learned my addr 172.21.15.164:0/691029686 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651ca00640 1 --2- 172.21.15.164:0/691029686 >> v2:172.21.15.164:3300/0 conn(0x7f65180cfda0 0x7f6518160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651de00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f65180b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f6517e00640 1 -- 172.21.15.164:0/691029686 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032737263 0 0) 0x7f65180cd9f0 con 0x7f651815ba70 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f6517e00640 1 -- 172.21.15.164:0/691029686 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003530 con 0x7f651815ba70 2024-05-08T04:41:54.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651ca00640 1 -- 172.21.15.164:0/691029686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 msgr2=0x7f65180b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:54.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651ca00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f65180b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651ca00640 1 -- 172.21.15.164:0/691029686 >> v1:172.21.15.164:6789/0 conn(0x7f651815ba70 legacy=0x7f65180cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651ca00640 1 -- 172.21.15.164:0/691029686 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815f820 con 0x7f65180cfda0 2024-05-08T04:41:54.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.392+0000 7f651de00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f65180b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:54.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651ca00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f6518160300 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f65040097c0 tx=0x7f650401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659942bdcb75f3bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f6517e00640 1 -- 172.21.15.164:0/691029686 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040b2070 con 0x7f65180cfda0 2024-05-08T04:41:54.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f6517e00640 1 -- 172.21.15.164:0/691029686 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65040ad680 con 0x7f65180cfda0 2024-05-08T04:41:54.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f6517e00640 1 -- 172.21.15.164:0/691029686 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040ada30 con 0x7f65180cfda0 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 -- 172.21.15.164:0/691029686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 msgr2=0x7f6518160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f6518160300 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f65040097c0 tx=0x7f650401c470 comp rx=0 tx=0).stop 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 -- 172.21.15.164:0/691029686 shutdown_connections 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f65180b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 --2- 172.21.15.164:0/691029686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f6518160300 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 -- 172.21.15.164:0/691029686 >> 172.21.15.164:0/691029686 conn(0x7f651806ac90 msgr2=0x7f651806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 -- 172.21.15.164:0/691029686 shutdown_connections 2024-05-08T04:41:54.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.396+0000 7f651fe00640 1 -- 172.21.15.164:0/691029686 wait complete. 2024-05-08T04:41:54.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 Processor -- start 2024-05-08T04:41:54.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- start start 2024-05-08T04:41:54.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f651815ba70 0x7f651815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f6518070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518168d40 con 0x7f65180cfda0 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518168bc0 con 0x7f651815ba70 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6518168ec0 con 0x7f651815be50 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36688/0 (socket says 172.21.15.164:36688) 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 -- 172.21.15.164:0/2153347805 learned_addr learned my addr 172.21.15.164:0/2153347805 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651ca00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f651815ba70 0x7f651815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651de00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f6518070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 -- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 msgr2=0x7f6518070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f6518070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 -- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f651815ba70 msgr2=0x7f651815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f651815ba70 0x7f651815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 -- 172.21.15.164:0/2153347805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815a4c0 con 0x7f65180cfda0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651d400640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c450 tx=0x7f650c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c017070 con 0x7f65180cfda0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c005cf0 con 0x7f65180cfda0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c007290 con 0x7f65180cfda0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65180b0350 con 0x7f65180cfda0 2024-05-08T04:41:54.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.400+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65180b05f0 con 0x7f65180cfda0 2024-05-08T04:41:54.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6518069d60 con 0x7f65180cfda0 2024-05-08T04:41:54.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f650c01a070 con 0x7f65180cfda0 2024-05-08T04:41:54.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f6515600640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 0x7f64f80699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:54.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f650c084dd0 con 0x7f65180cfda0 2024-05-08T04:41:54.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f651ca00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 0x7f64f80699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:54.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.404+0000 7f651ca00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 0x7f64f80699f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f65040af000 tx=0x7f65040a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:54.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.408+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f650c055580 con 0x7f65180cfda0 2024-05-08T04:41:54.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.596+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f65180ae4c0 con 0x7f65180cfda0 2024-05-08T04:41:54.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.596+0000 7f6515600640 1 -- 172.21.15.164:0/2153347805 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f650c054e80 con 0x7f65180cfda0 2024-05-08T04:41:54.602 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 msgr2=0x7f64f80699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 0x7f64f80699f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f65040af000 tx=0x7f65040a7000 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 msgr2=0x7f651814a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c450 tx=0x7f650c00c910 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 shutdown_connections 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64f8067530 0x7f64f80699f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f651815be50 0x7f6518070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f651815ba70 0x7f651815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 --2- 172.21.15.164:0/2153347805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65180cfda0 0x7f651814a5f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:54.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 >> 172.21.15.164:0/2153347805 conn(0x7f651806ac90 msgr2=0x7f65180ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:54.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 shutdown_connections 2024-05-08T04:41:54.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:54.600+0000 7f651fe00640 1 -- 172.21.15.164:0/2153347805 wait complete. 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:41:54.623 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:41:54.624 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:54.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.624 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:41:54.632 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:41:54.632 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:41:54.633 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42152sec preferred_lft 42152sec 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.634 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:41:54.635 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:54.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.636 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:41:54.696 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:41:54.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.699 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:41:54.751 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:41:54.751 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:41:54.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.752 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:41:54.753 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:41:54.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:41:54.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:41:54.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:41:54.753 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:41:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.847 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:41:54.904 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:41:54.904 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:41:54.904 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:54.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:54.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:41:54.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:54.966 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:55.019 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:41:55.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:55.019 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:55.070 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:41:55.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.070 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:55.162 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:55.162 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:55.162 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:55.177 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:55.177 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:55.177 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:55.177 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:55.177 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:41:55.189 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:41:55.193 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:41:55.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:41:55.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.213 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:41:55.276 INFO:teuthology.orchestra.run.smithi164.stdout:client6829 2024-05-08T04:41:55.276 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:41:55.276 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:55.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:41:55.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 Processor -- start 2024-05-08T04:41:55.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 -- start start 2024-05-08T04:41:55.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f50a806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50a80aee30 0x7f50a80cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a806e0e0 con 0x7f50a80cef80 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a815e8f0 con 0x7f50a80aee30 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50aec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815a8c0 con 0x7f50a815b390 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50acc00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f50a80cef80 0x7f50a80cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51296/0 (socket says 172.21.15.164:51296) 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.504+0000 7f50acc00640 1 -- 172.21.15.164:0/4214300030 learned_addr learned my addr 172.21.15.164:0/4214300030 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f509f400640 1 --2- 172.21.15.164:0/4214300030 >> v2:172.21.15.172:3300/0 conn(0x7f50a80aee30 0x7f50a80cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7e00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f50a806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267686116 0 0) 0x7f50a806e0e0 con 0x7f50a80cef80 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c003530 con 0x7f50a80cef80 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421198788 0 0) 0x7f508c003530 con 0x7f50a80cef80 2024-05-08T04:41:55.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 msgr2=0x7f50a806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f50a806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 >> v2:172.21.15.172:3300/0 conn(0x7f50a80aee30 msgr2=0x7f50a80cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 --2- 172.21.15.164:0/4214300030 >> v2:172.21.15.172:3300/0 conn(0x7f50a80aee30 0x7f50a80cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a815ed80 con 0x7f50a80cef80 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f50a0001a00 con 0x7f50a80cef80 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 >> v1:172.21.15.164:6789/0 conn(0x7f50a80cef80 legacy=0x7f50a80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f508c0039c0 0x7f508c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f508c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f508c004d50 0x7f508c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815e8f0 con 0x7f508c004d50 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815a8c0 con 0x7f508c0039c0 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c00b9d0 con 0x7f50a815b390 2024-05-08T04:41:55.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f508c004d50 0x7f508c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f509f400640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f508c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7e00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f508c0039c0 0x7f508c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 -- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 msgr2=0x7f508c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a815b390 0x7f508c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 -- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f508c0039c0 msgr2=0x7f508c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f508c0039c0 0x7f508c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50acc00640 1 -- 172.21.15.164:0/4214300030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c003530 con 0x7f508c004d50 2024-05-08T04:41:55.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.508+0000 7f50a7e00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f508c0039c0 0x7f508c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:55.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50acc00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f508c004d50 0x7f508c0051d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f50a000a8a0 tx=0x7f50a001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec402abda06f17bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:55.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a000a4c0 con 0x7f508c004d50 2024-05-08T04:41:55.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a00a9020 con 0x7f508c004d50 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50a7400640 1 -- 172.21.15.164:0/4214300030 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a00072d0 con 0x7f508c004d50 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- 172.21.15.164:0/4214300030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f508c004d50 msgr2=0x7f508c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 --2- 172.21.15.164:0/4214300030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f508c004d50 0x7f508c0051d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f50a000a8a0 tx=0x7f50a001d3c0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50acc00640 1 -- 172.21.15.164:0/4214300030 reap_dead start 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- 172.21.15.164:0/4214300030 shutdown_connections 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- 172.21.15.164:0/4214300030 >> 172.21.15.164:0/4214300030 conn(0x7f50a806ac90 msgr2=0x7f50a806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- 172.21.15.164:0/4214300030 shutdown_connections 2024-05-08T04:41:55.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- 172.21.15.164:0/4214300030 wait complete. 2024-05-08T04:41:55.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 Processor -- start 2024-05-08T04:41:55.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- start start 2024-05-08T04:41:55.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508c004d50 0x7f50a81495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a814ca40 0x7f50a81516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815e580 con 0x7f50a814f400 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815e2a0 con 0x7f50a814ca40 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50aec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815b770 con 0x7f508c004d50 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f509f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f509f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36700/0 (socket says 172.21.15.164:36700) 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f509f400640 1 -- 172.21.15.164:0/23236123 learned_addr learned my addr 172.21.15.164:0/23236123 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50a7e00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508c004d50 0x7f50a81495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.512+0000 7f50acc00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a814ca40 0x7f50a81516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 -- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508c004d50 msgr2=0x7f50a81495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508c004d50 0x7f50a81495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 -- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a814ca40 msgr2=0x7f50a81516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a814ca40 0x7f50a81516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 -- 172.21.15.164:0/23236123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a814f880 con 0x7f50a814f400 2024-05-08T04:41:55.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f509f400640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5090011ad0 tx=0x7f5090011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:55.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000fbb0 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509000fd50 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a8152e40 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090014680 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a81530e0 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a8069d60 con 0x7f50a814f400 2024-05-08T04:41:55.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f5090012040 con 0x7f50a814f400 2024-05-08T04:41:55.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 0x7f50840699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:55.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5090086070 con 0x7f50a814f400 2024-05-08T04:41:55.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.516+0000 7f50a7e00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 0x7f50840699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:55.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.520+0000 7f50a7e00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 0x7f50840699c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50a814b340 tx=0x7f5098008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:55.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.520+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50a814f880 con 0x7f50a814f400 2024-05-08T04:41:55.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.712+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50a80d0580 con 0x7f50a814f400 2024-05-08T04:41:55.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50a4c00640 1 -- 172.21.15.164:0/23236123 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5090019030 con 0x7f50a814f400 2024-05-08T04:41:55.720 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:41:55.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 msgr2=0x7f50840699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 0x7f50840699c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50a814b340 tx=0x7f5098008040 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 msgr2=0x7f50a81484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5090011ad0 tx=0x7f5090011f90 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 shutdown_connections 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5084067500 0x7f50840699c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a814ca40 0x7f50a81516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a814f400 0x7f50a81484f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 --2- 172.21.15.164:0/23236123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508c004d50 0x7f50a81495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 >> 172.21.15.164:0/23236123 conn(0x7f50a806ac90 msgr2=0x7f50a80cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:55.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 shutdown_connections 2024-05-08T04:41:55.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:55.716+0000 7f50aec00640 1 -- 172.21.15.164:0/23236123 wait complete. 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:41:55.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:55.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.741 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.751 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.752 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:41:55.753 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:41:55.754 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:41:55.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.755 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:41:55.817 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:41:55.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.820 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:41:55.875 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:41:55.876 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:55.876 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:55.876 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:41:55.876 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:41:55.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.876 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:41:55.876 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:41:55.876 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:41:55.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:41:55.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:41:55.877 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:41:55.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.972 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:41:55.981 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:41:55.981 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:41:55.981 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:55.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:55.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:41:56.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:56.042 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:56.094 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:41:56.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:41:56.094 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:56.145 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:41:56.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:56.145 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:56.236 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:56.236 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:56.236 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:41:56.263 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:41:56.267 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:41:56.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:56.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:41:56.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:56.288 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:41:56.345 INFO:teuthology.orchestra.run.smithi172.stdout:client7131 2024-05-08T04:41:56.345 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:41:56.345 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:41:56.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:41:56.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.532+0000 7f915b600640 1 Processor -- start 2024-05-08T04:41:56.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.532+0000 7f915b600640 1 -- start start 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f915b600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f915406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f915b600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f91540ae6e0 0x7f91540aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f915b600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915415e870 con 0x7f91540ce780 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f915b600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915406bb80 con 0x7f91540ae6e0 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f915b600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915415eb50 con 0x7f915415ab10 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9159600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f91540ce780 0x7f91540c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51312/0 (socket says 172.21.15.164:51312) 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9159600640 1 -- 172.21.15.164:0/2423398293 learned_addr learned my addr 172.21.15.164:0/2423398293 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153e00640 1 --2- 172.21.15.164:0/2423398293 >> v2:172.21.15.172:3300/0 conn(0x7f91540ae6e0 0x7f91540aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9158c00640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f915406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700648349 0 0) 0x7f915415e870 con 0x7f91540ce780 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9138003530 con 0x7f91540ce780 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153e00640 1 -- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 msgr2=0x7f915406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032229761 0 0) 0x7f9138003530 con 0x7f91540ce780 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153e00640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f915406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153e00640 1 -- 172.21.15.164:0/2423398293 >> v1:172.21.15.164:6789/0 conn(0x7f91540ce780 legacy=0x7f91540c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9153e00640 1 -- 172.21.15.164:0/2423398293 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915415ee30 con 0x7f91540ae6e0 2024-05-08T04:41:56.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.536+0000 7f9158c00640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f915406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:41:56.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f9153e00640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ae6e0 0x7f91540aeb40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91400097c0 tx=0x7f914001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22feb08cf1f7954e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:56.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91400b0070 con 0x7f91540ae6e0 2024-05-08T04:41:56.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91400ab400 con 0x7f91540ae6e0 2024-05-08T04:41:56.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f9153400640 1 -- 172.21.15.164:0/2423398293 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c001680 con 0x7f91540ae6e0 2024-05-08T04:41:56.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ae6e0 msgr2=0x7f91540aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:56.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ae6e0 0x7f91540aeb40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91400097c0 tx=0x7f914001c300 comp rx=0 tx=0).stop 2024-05-08T04:41:56.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- 172.21.15.164:0/2423398293 shutdown_connections 2024-05-08T04:41:56.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ae6e0 0x7f91540aeb40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 --2- 172.21.15.164:0/2423398293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f915406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- 172.21.15.164:0/2423398293 >> 172.21.15.164:0/2423398293 conn(0x7f91540cf770 msgr2=0x7f91540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- 172.21.15.164:0/2423398293 shutdown_connections 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- 172.21.15.164:0/2423398293 wait complete. 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 Processor -- start 2024-05-08T04:41:56.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.540+0000 7f915b600640 1 -- start start 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ce780 0x7f91540b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f91540b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9154167ae0 con 0x7f91540ae6e0 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915415aef0 con 0x7f91540ce780 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9154167c60 con 0x7f915415ab10 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36708/0 (socket says 172.21.15.164:36708) 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 -- 172.21.15.164:0/1551218866 learned_addr learned my addr 172.21.15.164:0/1551218866 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9153e00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ce780 0x7f91540b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9159600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f91540b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 -- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 msgr2=0x7f91540b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f91540b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 -- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ce780 msgr2=0x7f91540b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ce780 0x7f91540b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 -- 172.21.15.164:0/1551218866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540b1120 con 0x7f91540ae6e0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9158c00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9144009ec0 tx=0x7f914400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144018020 con 0x7f91540ae6e0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144014680 con 0x7f91540ae6e0 2024-05-08T04:41:56.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91540b35d0 con 0x7f91540ae6e0 2024-05-08T04:41:56.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144014a50 con 0x7f91540ae6e0 2024-05-08T04:41:56.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91540c6840 con 0x7f91540ae6e0 2024-05-08T04:41:56.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9154072d30 con 0x7f91540ae6e0 2024-05-08T04:41:56.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.544+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f914401b020 con 0x7f91540ae6e0 2024-05-08T04:41:56.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.548+0000 7f9150c00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 0x7f91340699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:56.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.548+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9144086730 con 0x7f91540ae6e0 2024-05-08T04:41:56.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.548+0000 7f9153e00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 0x7f91340699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:56.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.548+0000 7f9153e00640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 0x7f91340699c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91400ad000 tx=0x7f9140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:56.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.552+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9144022030 con 0x7f91540ae6e0 2024-05-08T04:41:56.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:56.736+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f915404f010 con 0x7f91540ae6e0 2024-05-08T04:41:57.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f9150c00640 1 -- 172.21.15.164:0/1551218866 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9144010040 con 0x7f91540ae6e0 2024-05-08T04:41:57.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 msgr2=0x7f91340699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 0x7f91340699c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91400ad000 tx=0x7f9140008040 comp rx=0 tx=0).stop 2024-05-08T04:41:57.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 msgr2=0x7f915414f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9144009ec0 tx=0x7f914400c490 comp rx=0 tx=0).stop 2024-05-08T04:41:57.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 shutdown_connections 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9134067500 0x7f91340699c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f915415ab10 0x7f91540b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91540ce780 0x7f91540b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 --2- 172.21.15.164:0/1551218866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91540ae6e0 0x7f915414f2f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 >> 172.21.15.164:0/1551218866 conn(0x7f91540cf770 msgr2=0x7f915406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 shutdown_connections 2024-05-08T04:41:57.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.276+0000 7f915b600640 1 -- 172.21.15.164:0/1551218866 wait complete. 2024-05-08T04:41:57.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:41:57.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.436+0000 7fee81000640 1 Processor -- start 2024-05-08T04:41:57.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee81000640 1 -- start start 2024-05-08T04:41:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee81000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee7c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee81000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0cd9f0 con 0x7fee7c0cfda0 2024-05-08T04:41:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee81000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0cdcd0 con 0x7fee7c15ba70 2024-05-08T04:41:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee81000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c15fe70 con 0x7fee7c15be50 2024-05-08T04:41:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee71400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fee7c0cfda0 0x7fee7c0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51328/0 (socket says 172.21.15.164:51328) 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee71400640 1 -- 172.21.15.164:0/127667262 learned_addr learned my addr 172.21.15.164:0/127667262 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee7c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824596047 0 0) 0x7fee7c0cd9f0 con 0x7fee7c0cfda0 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee60003530 con 0x7fee7c0cfda0 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079375708 0 0) 0x7fee60003530 con 0x7fee7c0cfda0 2024-05-08T04:41:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 msgr2=0x7fee7c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee7c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 >> v1:172.21.15.172:6789/0 conn(0x7fee7c15ba70 legacy=0x7fee7c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c15f820 con 0x7fee7c0cfda0 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fee64001680 con 0x7fee7c0cfda0 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 >> v1:172.21.15.164:6789/0 conn(0x7fee7c0cfda0 legacy=0x7fee7c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee600039f0 0x7fee60003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee60004d50 0x7fee600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c15fe70 con 0x7fee60004d50 2024-05-08T04:41:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6000ba80 con 0x7fee600039f0 2024-05-08T04:41:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6000bc00 con 0x7fee7c15be50 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee60004d50 0x7fee600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee71400640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee79e00640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee600039f0 0x7fee60003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 -- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 msgr2=0x7fee60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee7c15be50 0x7fee60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 -- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee600039f0 msgr2=0x7fee60003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee600039f0 0x7fee60003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.440+0000 7fee7a800640 1 -- 172.21.15.164:0/127667262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee60003530 con 0x7fee60004d50 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee7a800640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee60004d50 0x7fee600051d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fee74009ab0 tx=0x7fee7401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c389dcce14060a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740b2070 con 0x7fee60004d50 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee740a5640 con 0x7fee60004d50 2024-05-08T04:41:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee79400640 1 -- 172.21.15.164:0/127667262 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740096b0 con 0x7fee60004d50 2024-05-08T04:41:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 -- 172.21.15.164:0/127667262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee60004d50 msgr2=0x7fee600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 --2- 172.21.15.164:0/127667262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee60004d50 0x7fee600051d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fee74009ab0 tx=0x7fee7401ca50 comp rx=0 tx=0).stop 2024-05-08T04:41:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee7a800640 1 -- 172.21.15.164:0/127667262 reap_dead start 2024-05-08T04:41:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 -- 172.21.15.164:0/127667262 shutdown_connections 2024-05-08T04:41:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 -- 172.21.15.164:0/127667262 >> 172.21.15.164:0/127667262 conn(0x7fee7c06ac90 msgr2=0x7fee7c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 -- 172.21.15.164:0/127667262 shutdown_connections 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.444+0000 7fee81000640 1 -- 172.21.15.164:0/127667262 wait complete. 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 Processor -- start 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- start start 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee60004d50 0x7fee7c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee7c15e6e0 0x7fee7c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0cd7b0 con 0x7fee7c153020 2024-05-08T04:41:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0cd630 con 0x7fee7c15e6e0 2024-05-08T04:41:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0aeda0 con 0x7fee60004d50 2024-05-08T04:41:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36718/0 (socket says 172.21.15.164:36718) 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 -- 172.21.15.164:0/4040839241 learned_addr learned my addr 172.21.15.164:0/4040839241 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee79e00640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee60004d50 0x7fee7c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee71400640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee7c15e6e0 0x7fee7c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 -- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee60004d50 msgr2=0x7fee7c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee60004d50 0x7fee7c155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 -- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee7c15e6e0 msgr2=0x7fee7c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee7c15e6e0 0x7fee7c15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 -- 172.21.15.164:0/4040839241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c070510 con 0x7fee7c153020 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee7a800640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fee740b0040 tx=0x7fee60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740bd070 con 0x7fee7c153020 2024-05-08T04:41:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee740a5cf0 con 0x7fee7c153020 2024-05-08T04:41:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c0b1140 con 0x7fee7c153020 2024-05-08T04:41:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740b2040 con 0x7fee7c153020 2024-05-08T04:41:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.448+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c153a10 con 0x7fee7c153020 2024-05-08T04:41:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c069d60 con 0x7fee7c153020 2024-05-08T04:41:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7fee740c0070 con 0x7fee7c153020 2024-05-08T04:41:57.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee72a00640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 0x7fee580699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fee7411f630 con 0x7fee7c153020 2024-05-08T04:41:57.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee79e00640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 0x7fee580699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.452+0000 7fee79e00640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 0x7fee580699c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000f90 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.456+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee74016760 con 0x7fee7c153020 2024-05-08T04:41:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.632+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fee7c06e210 con 0x7fee58067500 2024-05-08T04:41:57.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.632+0000 7fee72a00640 1 -- 172.21.15.164:0/4040839241 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fee7c06e210 con 0x7fee58067500 2024-05-08T04:41:57.639 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:41:57.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:41:57.639 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:41:57.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:41:57.640 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:41:57.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 msgr2=0x7fee580699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 0x7fee580699c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000f90 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 msgr2=0x7fee7c1534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fee740b0040 tx=0x7fee60005840 comp rx=0 tx=0).stop 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 shutdown_connections 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fee58067500 0x7fee580699c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee7c153020 0x7fee7c1534d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee7c15e6e0 0x7fee7c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 --2- 172.21.15.164:0/4040839241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee60004d50 0x7fee7c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 >> 172.21.15.164:0/4040839241 conn(0x7fee7c06ac90 msgr2=0x7fee7c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 shutdown_connections 2024-05-08T04:41:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.636+0000 7fee81000640 1 -- 172.21.15.164:0/4040839241 wait complete. 2024-05-08T04:41:57.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:41:57.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 Processor -- start 2024-05-08T04:41:57.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 -- start start 2024-05-08T04:41:57.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ecc15ba20 0x7f3ecc160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc0c6600 con 0x7f3ecc06de50 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc0c6040 con 0x7f3ecc0aedf0 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ed1400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc15fbe0 con 0x7f3ecc15ba20 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec1e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3ecc06de50 0x7f3ecc0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51338/0 (socket says 172.21.15.164:51338) 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec1e00640 1 -- 172.21.15.164:0/1040084253 learned_addr learned my addr 172.21.15.164:0/1040084253 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3eca800640 1 --2- 172.21.15.164:0/1040084253 >> v2:172.21.15.172:3300/0 conn(0x7f3ecc15ba20 0x7f3ecc160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931084468 0 0) 0x7f3ecc0c6600 con 0x7f3ecc06de50 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb0003530 con 0x7f3ecc06de50 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977326720 0 0) 0x7f3eb0003530 con 0x7f3ecc06de50 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 >> v2:172.21.15.172:3300/0 conn(0x7f3ecc15ba20 msgr2=0x7f3ecc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 --2- 172.21.15.164:0/1040084253 >> v2:172.21.15.172:3300/0 conn(0x7f3ecc15ba20 0x7f3ecc160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 >> v1:172.21.15.172:6789/0 conn(0x7f3ecc0aedf0 legacy=0x7f3ecc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0c6320 con 0x7f3ecc06de50 2024-05-08T04:41:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f3eb4001680 con 0x7f3ecc06de50 2024-05-08T04:41:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 >> v1:172.21.15.164:6789/0 conn(0x7f3ecc06de50 legacy=0x7f3ecc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb0004810 0x7f3eb0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb00051d0 0x7f3eb000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc15fbe0 con 0x7f3eb00039c0 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb000c070 con 0x7f3eb0004810 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb000c270 con 0x7f3eb00051d0 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3eca800640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ec1e00640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb0004810 0x7f3eb0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.796+0000 7f3ecb200640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb00051d0 0x7f3eb000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 -- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb00051d0 msgr2=0x7f3eb000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3eb00051d0 0x7f3eb000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 -- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb0004810 msgr2=0x7f3eb0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb0004810 0x7f3eb0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 -- 172.21.15.164:0/1040084253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0003530 con 0x7f3eb00039c0 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ec1e00640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb0004810 0x7f3eb0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ecb200640 1 -- 172.21.15.164:0/1040084253 reap_dead start 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3eca800640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009ae0 tx=0x7f3ebc01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86589eab2ed1b3c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0b2070 con 0x7f3eb00039c0 2024-05-08T04:41:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0a55f0 con 0x7f3eb00039c0 2024-05-08T04:41:57.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ec9e00640 1 -- 172.21.15.164:0/1040084253 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0096b0 con 0x7f3eb00039c0 2024-05-08T04:41:57.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 -- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 msgr2=0x7f3eb0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009ae0 tx=0x7f3ebc01ca60 comp rx=0 tx=0).stop 2024-05-08T04:41:57.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 -- 172.21.15.164:0/1040084253 shutdown_connections 2024-05-08T04:41:57.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 --2- 172.21.15.164:0/1040084253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb00039c0 0x7f3eb0003de0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 -- 172.21.15.164:0/1040084253 >> 172.21.15.164:0/1040084253 conn(0x7f3ecc0cf770 msgr2=0x7f3ecc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:57.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 -- 172.21.15.164:0/1040084253 shutdown_connections 2024-05-08T04:41:57.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.800+0000 7f3ed1400640 1 -- 172.21.15.164:0/1040084253 wait complete. 2024-05-08T04:41:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 Processor -- start 2024-05-08T04:41:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- start start 2024-05-08T04:41:57.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb00039c0 0x7f3ecc0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ecc1eb080 0x7f3ecc147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0aec70 con 0x7f3ecc1eac00 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0aeaf0 con 0x7f3eb00039c0 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0cd670 con 0x7f3ecc1eb080 2024-05-08T04:41:57.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36750/0 (socket says 172.21.15.164:36750) 2024-05-08T04:41:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 -- 172.21.15.164:0/3407701812 learned_addr learned my addr 172.21.15.164:0/3407701812 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 -- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ecc1eb080 msgr2=0x7f3ecc147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ecc1eb080 0x7f3ecc147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 -- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb00039c0 msgr2=0x7f3ecc0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb00039c0 0x7f3ecc0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 -- 172.21.15.164:0/3407701812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0ce300 con 0x7f3ecc1eac00 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec1e00640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400e8f0 tx=0x7f3eb400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400c850 con 0x7f3ecc1eac00 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb4018070 con 0x7f3ecc1eac00 2024-05-08T04:41:57.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400cc50 con 0x7f3ecc1eac00 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc1479d0 con 0x7f3ecc1eac00 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.804+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc147f70 con 0x7f3ecc1eac00 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc072d30 con 0x7f3ecc1eac00 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f3eb401d460 con 0x7f3ecc1eac00 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3ec3400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 0x7f3ea80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:57.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3eb40859c0 con 0x7f3ecc1eac00 2024-05-08T04:41:57.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3eca800640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 0x7f3ea80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:57.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.808+0000 7f3eca800640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 0x7f3ea80699c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0af000 tx=0x7f3ebc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:57.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.812+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3eb40561b0 con 0x7f3ecc1eac00 2024-05-08T04:41:57.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.968+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3ecc0cea50 con 0x7f3ecc1eac00 2024-05-08T04:41:57.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.968+0000 7f3ec3400640 1 -- 172.21.15.164:0/3407701812 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3eb4055ab0 con 0x7f3ecc1eac00 2024-05-08T04:41:57.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 msgr2=0x7f3ea80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 0x7f3ea80699c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0af000 tx=0x7f3ebc008040 comp rx=0 tx=0).stop 2024-05-08T04:41:57.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 msgr2=0x7f3ecc146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:57.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400e8f0 tx=0x7f3eb400edb0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 shutdown_connections 2024-05-08T04:41:57.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ea8067500 0x7f3ea80699c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ecc1eb080 0x7f3ecc147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ecc1eac00 0x7f3ecc146d00 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 --2- 172.21.15.164:0/3407701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb00039c0 0x7f3ecc0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:57.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 >> 172.21.15.164:0/3407701812 conn(0x7f3ecc0cf770 msgr2=0x7f3ecc15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:57.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 shutdown_connections 2024-05-08T04:41:57.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:57.972+0000 7f3ed1400640 1 -- 172.21.15.164:0/3407701812 wait complete. 2024-05-08T04:41:57.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp35820 2024-05-08T04:41:58.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 Processor -- start 2024-05-08T04:41:58.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 -- start start 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 0x7f39b415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b406da30 con 0x7f39b40affe0 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b406dcd0 con 0x7f39b406e7b0 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39bbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b406e320 con 0x7f39b406b3a0 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f39b40affe0 0x7f39b406f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51344/0 (socket says 172.21.15.164:51344) 2024-05-08T04:41:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9400640 1 -- 172.21.15.164:0/1791640494 learned_addr learned my addr 172.21.15.164:0/1791640494 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9e00640 1 --2- 172.21.15.164:0/1791640494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 0x7f39b415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971897523 0 0) 0x7f39b406da30 con 0x7f39b40affe0 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f399c003530 con 0x7f39b40affe0 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871022593 0 0) 0x7f39b406dcd0 con 0x7f39b406e7b0 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b406da30 con 0x7f39b406e7b0 2024-05-08T04:41:58.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9e00640 1 -- 172.21.15.164:0/1791640494 >> v1:172.21.15.172:6789/0 conn(0x7f39b406e7b0 legacy=0x7f39b406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9e00640 1 -- 172.21.15.164:0/1791640494 >> v1:172.21.15.164:6789/0 conn(0x7f39b40affe0 legacy=0x7f39b406f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.132+0000 7f39b9e00640 1 -- 172.21.15.164:0/1791640494 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b406dfb0 con 0x7f39b406b3a0 2024-05-08T04:41:58.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39b9e00640 1 --2- 172.21.15.164:0/1791640494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 0x7f39b415cbd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009dc0 tx=0x7f39ac01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b64250dea06174 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0b2070 con 0x7f39b406b3a0 2024-05-08T04:41:58.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac0a5960 con 0x7f39b406b3a0 2024-05-08T04:41:58.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39b3e00640 1 -- 172.21.15.164:0/1791640494 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0a5d10 con 0x7f39b406b3a0 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 -- 172.21.15.164:0/1791640494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 msgr2=0x7f39b415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 --2- 172.21.15.164:0/1791640494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 0x7f39b415cbd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009dc0 tx=0x7f39ac01c990 comp rx=0 tx=0).stop 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 -- 172.21.15.164:0/1791640494 shutdown_connections 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 --2- 172.21.15.164:0/1791640494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406b3a0 0x7f39b415cbd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 -- 172.21.15.164:0/1791640494 >> 172.21.15.164:0/1791640494 conn(0x7f39b40b2d00 msgr2=0x7f39b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:58.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 -- 172.21.15.164:0/1791640494 shutdown_connections 2024-05-08T04:41:58.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.136+0000 7f39bbe00640 1 -- 172.21.15.164:0/1791640494 wait complete. 2024-05-08T04:41:58.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 Processor -- start 2024-05-08T04:41:58.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- start start 2024-05-08T04:41:58.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b406b3a0 0x7f39b413e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 0x7f39b4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41691b0 con 0x7f39b40affe0 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4169030 con 0x7f39b406b3a0 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4169330 con 0x7f39b406e7b0 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36762/0 (socket says 172.21.15.164:36762) 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 -- 172.21.15.164:0/2561747339 learned_addr learned my addr 172.21.15.164:0/2561747339 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9400640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b406b3a0 0x7f39b413e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b8a00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 0x7f39b4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 -- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 msgr2=0x7f39b4154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 0x7f39b4154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 -- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b406b3a0 msgr2=0x7f39b413e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b406b3a0 0x7f39b413e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 -- 172.21.15.164:0/2561747339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b414dc90 con 0x7f39b40affe0 2024-05-08T04:41:58.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b8a00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 0x7f39b4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:58.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b9e00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0b0040 tx=0x7f39ac019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0b2050 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac0be070 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac009450 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b4151b50 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4151e30 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.140+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b4072e80 con 0x7f39b40affe0 2024-05-08T04:41:58.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.144+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f39ac004280 con 0x7f39b40affe0 2024-05-08T04:41:58.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.144+0000 7f39b1600640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 0x7f39940699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.144+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f39ac120d20 con 0x7f39b40affe0 2024-05-08T04:41:58.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.144+0000 7f39b9400640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 0x7f39940699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.144+0000 7f39b9400640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 0x7f39940699f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007fd0 tx=0x7f39a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.148+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39ac016580 con 0x7f39b40affe0 2024-05-08T04:41:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.328+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b41536e0 con 0x7f3994067530 2024-05-08T04:41:58.340 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:58.336+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:58.340 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:58.336+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:58.340 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:58.336+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:58.340 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:58.336+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:58.340 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:41:58.336+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:41:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.348+0000 7f39b1600640 1 -- 172.21.15.164:0/2561747339 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f39b41536e0 con 0x7f3994067530 2024-05-08T04:41:58.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 msgr2=0x7f39940699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 0x7f39940699f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007fd0 tx=0x7f39a8009040 comp rx=0 tx=0).stop 2024-05-08T04:41:58.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 msgr2=0x7f39b4154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0b0040 tx=0x7f39ac019730 comp rx=0 tx=0).stop 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 shutdown_connections 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3994067530 0x7f39940699f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b406e7b0 0x7f39b4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b40affe0 0x7f39b4154fe0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 --2- 172.21.15.164:0/2561747339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b406b3a0 0x7f39b413e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 >> 172.21.15.164:0/2561747339 conn(0x7f39b40b2d00 msgr2=0x7f39b415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 shutdown_connections 2024-05-08T04:41:58.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.352+0000 7f39bbe00640 1 -- 172.21.15.164:0/2561747339 wait complete. 2024-05-08T04:41:58.375 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol18070 --group_name subvol_grp35820 2024-05-08T04:41:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 Processor -- start 2024-05-08T04:41:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 -- start start 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f520015da10 0x7f520006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52000aeb80 0x7f52000aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52000adae0 con 0x7f5200070640 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52000ad890 con 0x7f520015da10 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5207200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52000ae6f0 con 0x7f52000aeb80 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5206800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5200070640 0x7f52000b29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51346/0 (socket says 172.21.15.164:51346) 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5206800640 1 -- 172.21.15.164:0/3839799698 learned_addr learned my addr 172.21.15.164:0/3839799698 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205e00640 1 --2- 172.21.15.164:0/3839799698 >> v2:172.21.15.164:3300/0 conn(0x7f520015da10 0x7f520006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205400640 1 --2- 172.21.15.164:0/3839799698 >> v2:172.21.15.172:3300/0 conn(0x7f52000aeb80 0x7f52000aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205e00640 1 -- 172.21.15.164:0/3839799698 >> v2:172.21.15.172:3300/0 conn(0x7f52000aeb80 msgr2=0x7f52000aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205e00640 1 --2- 172.21.15.164:0/3839799698 >> v2:172.21.15.172:3300/0 conn(0x7f52000aeb80 0x7f52000aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205e00640 1 -- 172.21.15.164:0/3839799698 >> v1:172.21.15.164:6789/0 conn(0x7f5200070640 legacy=0x7f52000b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5205e00640 1 -- 172.21.15.164:0/3839799698 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000addc0 con 0x7f520015da10 2024-05-08T04:41:58.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.512+0000 7f5204a00640 1 -- 172.21.15.164:0/3839799698 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995729039 0 0) 0x7f52000adae0 con 0x7f5200070640 2024-05-08T04:41:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5205e00640 1 --2- 172.21.15.164:0/3839799698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520015da10 0x7f520006d6e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5200074730 tx=0x7f51f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa93acd8003f318 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5204a00640 1 -- 172.21.15.164:0/3839799698 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40ab650 con 0x7f520015da10 2024-05-08T04:41:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5204a00640 1 -- 172.21.15.164:0/3839799698 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40b0070 con 0x7f520015da10 2024-05-08T04:41:58.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5204a00640 1 -- 172.21.15.164:0/3839799698 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40aba10 con 0x7f520015da10 2024-05-08T04:41:58.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 -- 172.21.15.164:0/3839799698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520015da10 msgr2=0x7f520006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 --2- 172.21.15.164:0/3839799698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520015da10 0x7f520006d6e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5200074730 tx=0x7f51f401c450 comp rx=0 tx=0).stop 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 -- 172.21.15.164:0/3839799698 shutdown_connections 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 --2- 172.21.15.164:0/3839799698 >> v2:172.21.15.172:3300/0 conn(0x7f52000aeb80 0x7f52000aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 --2- 172.21.15.164:0/3839799698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520015da10 0x7f520006d6e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 -- 172.21.15.164:0/3839799698 >> 172.21.15.164:0/3839799698 conn(0x7f52000b16f0 msgr2=0x7f52000b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 -- 172.21.15.164:0/3839799698 shutdown_connections 2024-05-08T04:41:58.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.516+0000 7f5207200640 1 -- 172.21.15.164:0/3839799698 wait complete. 2024-05-08T04:41:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 Processor -- start 2024-05-08T04:41:58.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- start start 2024-05-08T04:41:58.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 0x7f52001667d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 0x7f5200166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520006cb00 con 0x7f5200070640 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520006c980 con 0x7f520015da10 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520015ddf0 con 0x7f52000aeb80 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36796/0 (socket says 172.21.15.164:36796) 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 -- 172.21.15.164:0/4186003965 learned_addr learned my addr 172.21.15.164:0/4186003965 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 -- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 msgr2=0x7f52001667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205400640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 0x7f52001667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 0x7f52001667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5206800640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 0x7f5200166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 -- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 msgr2=0x7f5200166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 0x7f5200166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 -- 172.21.15.164:0/4186003965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200163e20 con 0x7f5200070640 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205400640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 0x7f52001667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5206800640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 0x7f5200166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:41:58.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5205e00640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51f40ad040 tx=0x7f51f401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40bf610 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40bf7d0 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5200164100 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40bfbc0 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.520+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52001646c0 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520004ef30 con 0x7f5200070640 2024-05-08T04:41:58.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f51f40bd020 con 0x7f5200070640 2024-05-08T04:41:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f51fa000640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 0x7f51e40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f51f4120350 con 0x7f5200070640 2024-05-08T04:41:58.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f5205400640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 0x7f51e40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.524+0000 7f5205400640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 0x7f51e40699c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f52001f9a70 tx=0x7f51f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.528+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51f4016070 con 0x7f5200070640 2024-05-08T04:41:58.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:41:58.634 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:41:58.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.708+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol18070", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f5200072140 con 0x7f51e4067500 2024-05-08T04:41:58.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f51fa000640 1 -- 172.21.15.164:0/4186003965 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5200072140 con 0x7f51e4067500 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 msgr2=0x7f51e40699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 0x7f51e40699c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f52001f9a70 tx=0x7f51f0008040 comp rx=0 tx=0).stop 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 msgr2=0x7f520015ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51f40ad040 tx=0x7f51f401ac90 comp rx=0 tx=0).stop 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 shutdown_connections 2024-05-08T04:41:58.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f51e4067500 0x7f51e40699c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52000aeb80 0x7f52001667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f520015da10 0x7f5200166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 --2- 172.21.15.164:0/4186003965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5200070640 0x7f520015ff60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 >> 172.21.15.164:0/4186003965 conn(0x7f52000b16f0 msgr2=0x7f52000afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 shutdown_connections 2024-05-08T04:41:58.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.732+0000 7f5207200640 1 -- 172.21.15.164:0/4186003965 wait complete. 2024-05-08T04:41:58.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol18070 subvol_snap53280 subvol_grp35820 2024-05-08T04:41:58.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 Processor -- start 2024-05-08T04:41:58.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 -- start start 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5b9815ce10 0x7f5b9806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980c6600 con 0x7f5b9806e390 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980c6040 con 0x7f5b9815ba20 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b9ce00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9815fbe0 con 0x7f5b9815ce10 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b96800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5b9806e390 0x7f5b980ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51350/0 (socket says 172.21.15.164:51350) 2024-05-08T04:41:58.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b96800640 1 -- 172.21.15.164:0/1611030230 learned_addr learned my addr 172.21.15.164:0/1611030230 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b8d400640 1 --2- 172.21.15.164:0/1611030230 >> v2:172.21.15.172:3300/0 conn(0x7f5b9815ce10 0x7f5b9806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783293212 0 0) 0x7f5b980c6600 con 0x7f5b9806e390 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c003530 con 0x7f5b9806e390 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141335860 0 0) 0x7f5b980c6040 con 0x7f5b9815ba20 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b980c6600 con 0x7f5b9815ba20 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b8d400640 1 -- 172.21.15.164:0/1611030230 >> v1:172.21.15.172:6789/0 conn(0x7f5b9815ba20 legacy=0x7f5b98160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b8d400640 1 -- 172.21.15.164:0/1611030230 >> v1:172.21.15.164:6789/0 conn(0x7f5b9806e390 legacy=0x7f5b980ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.944+0000 7f5b8d400640 1 -- 172.21.15.164:0/1611030230 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980c6320 con 0x7f5b9815ce10 2024-05-08T04:41:58.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b8d400640 1 --2- 172.21.15.164:0/1611030230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9815ce10 0x7f5b9806de50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b800097c0 tx=0x7f5b8001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e2c45e000a38ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800b0070 con 0x7f5b9815ce10 2024-05-08T04:41:58.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b800ab600 con 0x7f5b9815ce10 2024-05-08T04:41:58.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b95400640 1 -- 172.21.15.164:0/1611030230 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800ab9d0 con 0x7f5b9815ce10 2024-05-08T04:41:58.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 -- 172.21.15.164:0/1611030230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9815ce10 msgr2=0x7f5b9806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/1611030230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9815ce10 0x7f5b9806de50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b800097c0 tx=0x7f5b8001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 -- 172.21.15.164:0/1611030230 shutdown_connections 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/1611030230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9815ce10 0x7f5b9806de50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 -- 172.21.15.164:0/1611030230 >> 172.21.15.164:0/1611030230 conn(0x7f5b980cf770 msgr2=0x7f5b980cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 -- 172.21.15.164:0/1611030230 shutdown_connections 2024-05-08T04:41:58.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 -- 172.21.15.164:0/1611030230 wait complete. 2024-05-08T04:41:58.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.948+0000 7f5b9ce00640 1 Processor -- start 2024-05-08T04:41:58.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- start start 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9806e390 0x7f5b9814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b9815ce10 0x7f5b98153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980ce000 con 0x7f5b9815ba20 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980cde80 con 0x7f5b9806e390 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98169b80 con 0x7f5b9815ce10 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36798/0 (socket says 172.21.15.164:36798) 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 -- 172.21.15.164:0/3636613116 learned_addr learned my addr 172.21.15.164:0/3636613116 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b95e00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9806e390 0x7f5b9814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b96800640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b9815ce10 0x7f5b98153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 -- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b9815ce10 msgr2=0x7f5b98153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b9815ce10 0x7f5b98153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 -- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9806e390 msgr2=0x7f5b9814d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:58.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9806e390 0x7f5b9814d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:58.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 -- 172.21.15.164:0/3636613116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980affb0 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8d400640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5b800190b0 tx=0x7f5b80019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800b8020 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b800b4410 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b980af150 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800b0040 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b980af710 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.952+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b980c8100 con 0x7f5b9815ba20 2024-05-08T04:41:58.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.956+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f5b800bb020 con 0x7f5b9815ba20 2024-05-08T04:41:58.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.956+0000 7f5b8ea00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 0x7f5b740699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:58.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.956+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5b8011f5a0 con 0x7f5b9815ba20 2024-05-08T04:41:58.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.956+0000 7f5b95e00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 0x7f5b740699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:58.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.956+0000 7f5b95e00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 0x7f5b740699c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b98154300 tx=0x7f5b88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:58.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:58.960+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b980c8100 con 0x7f5b9815ba20 2024-05-08T04:41:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.144+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap53280", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b98072d30 con 0x7f5b74067500 2024-05-08T04:41:59.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.152+0000 7f5b8ea00640 1 -- 172.21.15.164:0/3636613116 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5b98072d30 con 0x7f5b74067500 2024-05-08T04:41:59.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 msgr2=0x7f5b740699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 0x7f5b740699c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b98154300 tx=0x7f5b88009040 comp rx=0 tx=0).stop 2024-05-08T04:41:59.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 msgr2=0x7f5b980b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5b800190b0 tx=0x7f5b80019240 comp rx=0 tx=0).stop 2024-05-08T04:41:59.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 shutdown_connections 2024-05-08T04:41:59.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b74067500 0x7f5b740699c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b9815ce10 0x7f5b98153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b9815ba20 0x7f5b980b04e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 --2- 172.21.15.164:0/3636613116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b9806e390 0x7f5b9814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 >> 172.21.15.164:0/3636613116 conn(0x7f5b980cf770 msgr2=0x7f5b9815dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:59.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 shutdown_connections 2024-05-08T04:41:59.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.156+0000 7f5b9ce00640 1 -- 172.21.15.164:0/3636613116 wait complete. 2024-05-08T04:41:59.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol18070 subvol_snap3211 subvol_grp35820 2024-05-08T04:41:59.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.320+0000 7f3996800640 1 Processor -- start 2024-05-08T04:41:59.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 -- start start 2024-05-08T04:41:59.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f39900b4dc0 0x7f39900ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399015e300 con 0x7f39900addd0 2024-05-08T04:41:59.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399006bca0 con 0x7f39900b4dc0 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3996800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399015e570 con 0x7f39900af6d0 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:55748/0 (socket says 172.21.15.164:55748) 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398fe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f39900addd0 0x7f39901617e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:41474/0 (socket says 172.21.15.164:41474) 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 -- 172.21.15.164:0/3513703282 learned_addr learned my addr 172.21.15.164:0/3513703282 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:59.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f3986a00640 1 --2- 172.21.15.164:0/3513703282 >> v2:172.21.15.172:3300/0 conn(0x7f39900b4dc0 0x7f39900ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 -- 172.21.15.164:0/3513703282 >> v2:172.21.15.172:3300/0 conn(0x7f39900b4dc0 msgr2=0x7f39900ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:59.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 --2- 172.21.15.164:0/3513703282 >> v2:172.21.15.172:3300/0 conn(0x7f39900b4dc0 0x7f39900ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 -- 172.21.15.164:0/3513703282 >> v1:172.21.15.172:6789/0 conn(0x7f39900addd0 legacy=0x7f39901617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398f400640 1 -- 172.21.15.164:0/3513703282 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399015e850 con 0x7f39900af6d0 2024-05-08T04:41:59.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.324+0000 7f398ea00640 1 -- 172.21.15.164:0/3513703282 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746927458 0 0) 0x7f399015e300 con 0x7f39900addd0 2024-05-08T04:41:59.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f398f400640 1 --2- 172.21.15.164:0/3513703282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3980009e50 tx=0x7f398001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5383bf188fd2fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f398ea00640 1 -- 172.21.15.164:0/3513703282 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800a5580 con 0x7f39900af6d0 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f398ea00640 1 -- 172.21.15.164:0/3513703282 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39800a5720 con 0x7f39900af6d0 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f398ea00640 1 -- 172.21.15.164:0/3513703282 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800a5af0 con 0x7f39900af6d0 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 -- 172.21.15.164:0/3513703282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 msgr2=0x7f39900b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 --2- 172.21.15.164:0/3513703282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3980009e50 tx=0x7f398001c9a0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 -- 172.21.15.164:0/3513703282 shutdown_connections 2024-05-08T04:41:59.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 --2- 172.21.15.164:0/3513703282 >> v2:172.21.15.172:3300/0 conn(0x7f39900b4dc0 0x7f39900ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 --2- 172.21.15.164:0/3513703282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900af6d0 0x7f39900b2ff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 -- 172.21.15.164:0/3513703282 >> 172.21.15.164:0/3513703282 conn(0x7f399006cca0 msgr2=0x7f399006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:59.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 -- 172.21.15.164:0/3513703282 shutdown_connections 2024-05-08T04:41:59.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 -- 172.21.15.164:0/3513703282 wait complete. 2024-05-08T04:41:59.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.328+0000 7f3996800640 1 Processor -- start 2024-05-08T04:41:59.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- start start 2024-05-08T04:41:59.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900addd0 0x7f3990138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39900af6d0 0x7f3990138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399016aad0 con 0x7f39900b4dc0 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399016a950 con 0x7f39900af6d0 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399016ac50 con 0x7f39900addd0 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50350/0 (socket says 172.21.15.164:50350) 2024-05-08T04:41:59.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 -- 172.21.15.164:0/4160325383 learned_addr learned my addr 172.21.15.164:0/4160325383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:59.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3986a00640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39900af6d0 0x7f3990138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398f400640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900addd0 0x7f3990138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 -- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900addd0 msgr2=0x7f3990138530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900addd0 0x7f3990138530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 -- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39900af6d0 msgr2=0x7f3990138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39900af6d0 0x7f3990138d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 -- 172.21.15.164:0/4160325383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990138a70 con 0x7f39900b4dc0 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f398fe00640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f398800c3e0 tx=0x7f398800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880074e0 con 0x7f39900b4dc0 2024-05-08T04:41:59.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880047f0 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988004be0 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399015b4a0 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399014cfd0 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.332+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399006e5b0 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.336+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f3988011070 con 0x7f39900b4dc0 2024-05-08T04:41:59.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.336+0000 7f3987e00640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 0x7f396c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.336+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3988085450 con 0x7f39900b4dc0 2024-05-08T04:41:59.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.336+0000 7f398f400640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 0x7f396c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.336+0000 7f398f400640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 0x7f396c0699f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39800ae000 tx=0x7f39800a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.340+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3988055560 con 0x7f39900b4dc0 2024-05-08T04:41:59.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.524+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap3211", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f399004f010 con 0x7f396c067530 2024-05-08T04:41:59.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3987e00640 1 -- 172.21.15.164:0/4160325383 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f399004f010 con 0x7f396c067530 2024-05-08T04:41:59.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 msgr2=0x7f396c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 0x7f396c0699f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39800ae000 tx=0x7f39800a7070 comp rx=0 tx=0).stop 2024-05-08T04:41:59.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 msgr2=0x7f3990153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f398800c3e0 tx=0x7f398800c8a0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 shutdown_connections 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f396c067530 0x7f396c0699f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39900b4dc0 0x7f3990153b00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39900af6d0 0x7f3990138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 --2- 172.21.15.164:0/4160325383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39900addd0 0x7f3990138530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 >> 172.21.15.164:0/4160325383 conn(0x7f399006cca0 msgr2=0x7f399006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:59.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 shutdown_connections 2024-05-08T04:41:59.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.536+0000 7f3996800640 1 -- 172.21.15.164:0/4160325383 wait complete. 2024-05-08T04:41:59.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol18070 subvol_snap29022 subvol_grp35820 2024-05-08T04:41:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 Processor -- start 2024-05-08T04:41:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 -- start start 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8015ba20 0x7f3c80160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c8006de50 0x7f3c8006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c800c6600 con 0x7f3c800ae6e0 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8015fbe0 con 0x7f3c8006de50 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c86400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800c6040 con 0x7f3c8015ba20 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3c800ae6e0 0x7f3c800cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49732/0 (socket says 172.21.15.164:49732) 2024-05-08T04:41:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 -- 172.21.15.164:0/569860088 learned_addr learned my addr 172.21.15.164:0/569860088 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c76a00640 1 --2- 172.21.15.164:0/569860088 >> v2:172.21.15.172:3300/0 conn(0x7f3c8006de50 0x7f3c8006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7f400640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8015ba20 0x7f3c80160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337777277 0 0) 0x7f3c800c6600 con 0x7f3c800ae6e0 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c64003530 con 0x7f3c800ae6e0 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 793240450 0 0) 0x7f3c64003530 con 0x7f3c800ae6e0 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8015ba20 msgr2=0x7f3c80160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8015ba20 0x7f3c80160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 >> v2:172.21.15.172:3300/0 conn(0x7f3c8006de50 msgr2=0x7f3c8006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 --2- 172.21.15.164:0/569860088 >> v2:172.21.15.172:3300/0 conn(0x7f3c8006de50 0x7f3c8006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800c6320 con 0x7f3c800ae6e0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f3c78001680 con 0x7f3c800ae6e0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 >> v1:172.21.15.164:6789/0 conn(0x7f3c800ae6e0 legacy=0x7f3c800cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c640039c0 0x7f3c64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c64004810 0x7f3c64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c640051d0 0x7f3c6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8015fbe0 con 0x7f3c640051d0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800c6040 con 0x7f3c64004810 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6400bf10 con 0x7f3c640039c0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c640051d0 0x7f3c6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c76a00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c64004810 0x7f3c64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7f400640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c640039c0 0x7f3c64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 -- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c640039c0 msgr2=0x7f3c64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c640039c0 0x7f3c64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 -- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c64004810 msgr2=0x7f3c64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c64004810 0x7f3c64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.700+0000 7f3c7fe00640 1 -- 172.21.15.164:0/569860088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64003530 con 0x7f3c640051d0 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c640051d0 0x7f3c6400b9d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800c190 tx=0x7f3c7801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afb3cbbfcb57853 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78007c90 con 0x7f3c640051d0 2024-05-08T04:41:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c780a9020 con 0x7f3c640051d0 2024-05-08T04:41:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c7ea00640 1 -- 172.21.15.164:0/569860088 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78005ae0 con 0x7f3c640051d0 2024-05-08T04:41:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 -- 172.21.15.164:0/569860088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c640051d0 msgr2=0x7f3c6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 --2- 172.21.15.164:0/569860088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c640051d0 0x7f3c6400b9d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800c190 tx=0x7f3c7801cb80 comp rx=0 tx=0).stop 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c7fe00640 1 -- 172.21.15.164:0/569860088 reap_dead start 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 -- 172.21.15.164:0/569860088 shutdown_connections 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 -- 172.21.15.164:0/569860088 >> 172.21.15.164:0/569860088 conn(0x7f3c800cf770 msgr2=0x7f3c800cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 -- 172.21.15.164:0/569860088 shutdown_connections 2024-05-08T04:41:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.704+0000 7f3c86400640 1 -- 172.21.15.164:0/569860088 wait complete. 2024-05-08T04:41:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 Processor -- start 2024-05-08T04:41:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- start start 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 0x7f3c800b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c801f7960 0x7f3c8014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8015ce10 con 0x7f3c8014f430 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800cf410 con 0x7f3c801f7580 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8015cf90 con 0x7f3c801f7960 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50376/0 (socket says 172.21.15.164:50376) 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 -- 172.21.15.164:0/1382388195 learned_addr learned my addr 172.21.15.164:0/1382388195 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:41:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7f400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 0x7f3c800b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c76a00640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c801f7960 0x7f3c8014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 -- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c801f7960 msgr2=0x7f3c8014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c801f7960 0x7f3c8014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 -- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 msgr2=0x7f3c800b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 0x7f3c800b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 -- 172.21.15.164:0/1382388195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800af590 con 0x7f3c8014f430 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7f400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 0x7f3c800b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c7fe00640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3c78001ec0 tx=0x7f3c64005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78001bc0 con 0x7f3c8014f430 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c780b2030 con 0x7f3c8014f430 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c800b0570 con 0x7f3c8014f430 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78004ab0 con 0x7f3c8014f430 2024-05-08T04:41:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.708+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c800c6b20 con 0x7f3c8014f430 2024-05-08T04:41:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80072d30 con 0x7f3c8014f430 2024-05-08T04:41:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f3c78004c50 con 0x7f3c8014f430 2024-05-08T04:41:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c77e00640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 0x7f3c5c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:41:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3c7811edb0 con 0x7f3c8014f430 2024-05-08T04:41:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c7f400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 0x7f3c5c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:41:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.712+0000 7f3c7f400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 0x7f3c5c0699f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3c800cc0e0 tx=0x7f3c7000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:41:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.716+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c780168f0 con 0x7f3c8014f430 2024-05-08T04:41:59.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.904+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap29022", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c8004f010 con 0x7f3c5c067530 2024-05-08T04:41:59.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c77e00640 1 -- 172.21.15.164:0/1382388195 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3c8004f010 con 0x7f3c5c067530 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 msgr2=0x7f3c5c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 0x7f3c5c0699f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3c800cc0e0 tx=0x7f3c7000b040 comp rx=0 tx=0).stop 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 msgr2=0x7f3c800b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3c78001ec0 tx=0x7f3c64005840 comp rx=0 tx=0).stop 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 shutdown_connections 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3c5c067530 0x7f3c5c0699f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c801f7960 0x7f3c8014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c8014f430 0x7f3c800b26a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 --2- 172.21.15.164:0/1382388195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c801f7580 0x7f3c800b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:41:59.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 >> 172.21.15.164:0/1382388195 conn(0x7f3c800cf770 msgr2=0x7f3c8015d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:41:59.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 shutdown_connections 2024-05-08T04:41:59.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:41:59.916+0000 7f3c86400640 1 -- 172.21.15.164:0/1382388195 wait complete. 2024-05-08T04:41:59.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:41:59.940 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp35820/.snap/ancestral_snap_1 ./volumes/subvol_grp35820/.snap/ancestral_snap_2) 2024-05-08T04:41:59.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol18070 subvol_grp35820 2024-05-08T04:42:00.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.148+0000 7f790d400640 1 Processor -- start 2024-05-08T04:42:00.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.148+0000 7f790d400640 1 -- start start 2024-05-08T04:42:00.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f790d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f790815f2b0 0x7f79080d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f790d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79080cd560 0x7f790815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f790d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7908070280 con 0x7f79080b3fe0 2024-05-08T04:42:00.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f790d400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790815f690 con 0x7f790815f2b0 2024-05-08T04:42:00.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f790d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080cff80 con 0x7f79080cd560 2024-05-08T04:42:00.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f790815f2b0 0x7f79080d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f790815f2b0 0x7f79080d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50382/0 (socket says 172.21.15.164:50382) 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 -- 172.21.15.164:0/2781297450 learned_addr learned my addr 172.21.15.164:0/2781297450 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7907200640 1 --2- 172.21.15.164:0/2781297450 >> v2:172.21.15.172:3300/0 conn(0x7f79080cd560 0x7f790815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 -- 172.21.15.164:0/2781297450 >> v2:172.21.15.172:3300/0 conn(0x7f79080cd560 msgr2=0x7f790815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 --2- 172.21.15.164:0/2781297450 >> v2:172.21.15.172:3300/0 conn(0x7f79080cd560 0x7f790815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 -- 172.21.15.164:0/2781297450 >> v1:172.21.15.172:6789/0 conn(0x7f79080b3fe0 legacy=0x7f79080cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 -- 172.21.15.164:0/2781297450 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908070560 con 0x7f790815f2b0 2024-05-08T04:42:00.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.152+0000 7f7906800640 1 --2- 172.21.15.164:0/2781297450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080d0410 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7908072340 tx=0x7f78f801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c73f5719022809 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7905400640 1 -- 172.21.15.164:0/2781297450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80a5880 con 0x7f790815f2b0 2024-05-08T04:42:00.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7905400640 1 -- 172.21.15.164:0/2781297450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80a5a20 con 0x7f790815f2b0 2024-05-08T04:42:00.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7905400640 1 -- 172.21.15.164:0/2781297450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80a5df0 con 0x7f790815f2b0 2024-05-08T04:42:00.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- 172.21.15.164:0/2781297450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 msgr2=0x7f79080d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- 172.21.15.164:0/2781297450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080d0410 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7908072340 tx=0x7f78f801c9c0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- 172.21.15.164:0/2781297450 shutdown_connections 2024-05-08T04:42:00.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- 172.21.15.164:0/2781297450 >> v2:172.21.15.172:3300/0 conn(0x7f79080cd560 0x7f790815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- 172.21.15.164:0/2781297450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080d0410 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- 172.21.15.164:0/2781297450 >> 172.21.15.164:0/2781297450 conn(0x7f79080b34f0 msgr2=0x7f79080b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:00.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- 172.21.15.164:0/2781297450 shutdown_connections 2024-05-08T04:42:00.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- 172.21.15.164:0/2781297450 wait complete. 2024-05-08T04:42:00.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 Processor -- start 2024-05-08T04:42:00.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- start start 2024-05-08T04:42:00.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 0x7f7908155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cd560 0x7f79080cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908168d70 con 0x7f790815f2b0 2024-05-08T04:42:00.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908168bf0 con 0x7f79080b3fe0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f790d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7908168ef0 con 0x7f79080cd560 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7907200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7907200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50396/0 (socket says 172.21.15.164:50396) 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7906800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 0x7f7908155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.156+0000 7f7907200640 1 -- 172.21.15.164:0/3351488290 learned_addr learned my addr 172.21.15.164:0/3351488290 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7905e00640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cd560 0x7f79080cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 -- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cd560 msgr2=0x7f79080cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cd560 0x7f79080cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 -- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 msgr2=0x7f7908155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 0x7f7908155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 -- 172.21.15.164:0/3351488290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080c78f0 con 0x7f790815f2b0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7906800640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 0x7f7908155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f7907200640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f790000e930 tx=0x7f790000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900017070 con 0x7f790815f2b0 2024-05-08T04:42:00.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000c9a0 con 0x7f790815f2b0 2024-05-08T04:42:00.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000cd70 con 0x7f790815f2b0 2024-05-08T04:42:00.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79080c7330 con 0x7f790815f2b0 2024-05-08T04:42:00.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79080c54d0 con 0x7f790815f2b0 2024-05-08T04:42:00.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79080b4560 con 0x7f790815f2b0 2024-05-08T04:42:00.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 11) v1 ==== 87237+0+0 (secure 0 0 0) 0x7f790001a070 con 0x7f790815f2b0 2024-05-08T04:42:00.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 0x7f78e80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.160+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7900085db0 con 0x7f790815f2b0 2024-05-08T04:42:00.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.164+0000 7f7906800640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 0x7f78e80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.164+0000 7f7906800640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 0x7f78e80699c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f78f80ae000 tx=0x7f78f80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.164+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7900055fd0 con 0x7f790815f2b0 2024-05-08T04:42:00.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.300+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7f7900052640 con 0x7f790815f2b0 2024-05-08T04:42:00.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.348+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol18070", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f79080b7920 con 0x7f78e8067500 2024-05-08T04:42:00.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.356+0000 7f78fea00640 1 -- 172.21.15.164:0/3351488290 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+146 (secure 0 0 0) 0x7f79080b7920 con 0x7f78e8067500 2024-05-08T04:42:00.363 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subvol_snap53280" 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subvol_snap3211" 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subvol_snap29022" 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:00.364 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:42:00.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 msgr2=0x7f78e80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 0x7f78e80699c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f78f80ae000 tx=0x7f78f80a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:00.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 msgr2=0x7f79080cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f790000e930 tx=0x7f790000edf0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 shutdown_connections 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e8067500 0x7f78e80699c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cd560 0x7f79080cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790815f2b0 0x7f79080cb160 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 --2- 172.21.15.164:0/3351488290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79080b3fe0 0x7f7908155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 >> 172.21.15.164:0/3351488290 conn(0x7f79080b34f0 msgr2=0x7f79080cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 shutdown_connections 2024-05-08T04:42:00.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.360+0000 7f790d400640 1 -- 172.21.15.164:0/3351488290 wait complete. 2024-05-08T04:42:00.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:00.384 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp35820/.snap/ancestral_snap_1 ./volumes/subvol_grp35820/.snap/ancestral_snap_2) 2024-05-08T04:42:00.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol18070 subvol_snap53280 subvol_grp35820 2024-05-08T04:42:00.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 Processor -- start 2024-05-08T04:42:00.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 -- start start 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17c40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c415e870 con 0x7f17c415ab10 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c415eb50 con 0x7f17c40cecc0 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17ca400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40cf130 con 0x7f17c406cce0 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f17c415ab10 0x7f17c40ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49734/0 (socket says 172.21.15.164:49734) 2024-05-08T04:42:00.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c3400640 1 -- 172.21.15.164:0/2687732541 learned_addr learned my addr 172.21.15.164:0/2687732541 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187026066 0 0) 0x7f17c415e870 con 0x7f17c415ab10 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17a8003530 con 0x7f17c415ab10 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17baa00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17c40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885209302 0 0) 0x7f17a8003530 con 0x7f17c415ab10 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 msgr2=0x7f17c40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17c40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 >> v1:172.21.15.172:6789/0 conn(0x7f17c40cecc0 legacy=0x7f17c40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c415ee30 con 0x7f17c415ab10 2024-05-08T04:42:00.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f17b4001680 con 0x7f17c415ab10 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 >> v1:172.21.15.164:6789/0 conn(0x7f17c415ab10 legacy=0x7f17c40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17a80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8003fc0 0x7f17a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17a8004d50 0x7f17a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40cf130 con 0x7f17a8004d50 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a800bbc0 con 0x7f17a8003fc0 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a800bdc0 con 0x7f17c406cce0 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17baa00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8003fc0 0x7f17a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.600+0000 7f17c3e00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17a8004d50 0x7f17a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3400640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17a80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 -- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 msgr2=0x7f17a80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c406cce0 0x7f17a80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 -- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8003fc0 msgr2=0x7f17a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8003fc0 0x7f17a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 -- 172.21.15.164:0/2687732541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8003530 con 0x7f17a8004d50 2024-05-08T04:42:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17a8004d50 0x7f17a80051d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001e70 tx=0x7f17bc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ae416d99dd4e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc001a90 con 0x7f17a8004d50 2024-05-08T04:42:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc003b80 con 0x7f17a8004d50 2024-05-08T04:42:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c2a00640 1 -- 172.21.15.164:0/2687732541 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc005080 con 0x7f17a8004d50 2024-05-08T04:42:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17ca400640 1 -- 172.21.15.164:0/2687732541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17a8004d50 msgr2=0x7f17a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17ca400640 1 --2- 172.21.15.164:0/2687732541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17a8004d50 0x7f17a80051d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001e70 tx=0x7f17bc01c780 comp rx=0 tx=0).stop 2024-05-08T04:42:00.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.604+0000 7f17c3e00640 1 -- 172.21.15.164:0/2687732541 reap_dead start 2024-05-08T04:42:00.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- 172.21.15.164:0/2687732541 shutdown_connections 2024-05-08T04:42:00.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- 172.21.15.164:0/2687732541 >> 172.21.15.164:0/2687732541 conn(0x7f17c40cf770 msgr2=0x7f17c40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:00.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- 172.21.15.164:0/2687732541 shutdown_connections 2024-05-08T04:42:00.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- 172.21.15.164:0/2687732541 wait complete. 2024-05-08T04:42:00.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 Processor -- start 2024-05-08T04:42:00.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- start start 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8004d50 0x7f17c4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c41f0ba0 0x7f17c40c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c415f3f0 con 0x7f17c40c9600 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c406bb80 con 0x7f17a8004d50 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17ca400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c415aef0 con 0x7f17c41f0ba0 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50418/0 (socket says 172.21.15.164:50418) 2024-05-08T04:42:00.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 -- 172.21.15.164:0/671343168 learned_addr learned my addr 172.21.15.164:0/671343168 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8004d50 0x7f17c4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17baa00640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c41f0ba0 0x7f17c40c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 -- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c41f0ba0 msgr2=0x7f17c40c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c41f0ba0 0x7f17c40c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 -- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8004d50 msgr2=0x7f17c4155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8004d50 0x7f17c4155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 -- 172.21.15.164:0/671343168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c415bee0 con 0x7f17c40c9600 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17c3e00640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01cc60 tx=0x7f17a8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.608+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0ba070 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc0012e0 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc001a90 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c415c1c0 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c415c780 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c4072d30 con 0x7f17c40c9600 2024-05-08T04:42:00.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7f17bc0bd070 con 0x7f17c40c9600 2024-05-08T04:42:00.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17bbe00640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 0x7f17a0069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f17bc11f2f0 con 0x7f17c40c9600 2024-05-08T04:42:00.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.612+0000 7f17c3400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 0x7f17a0069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.616+0000 7f17c3400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 0x7f17a0069a80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f17c415bba0 tx=0x7f17b4000bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.620+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17bc0ac020 con 0x7f17c40c9600 2024-05-08T04:42:00.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.800+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap53280", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c40c4bf0 con 0x7f17a00675c0 2024-05-08T04:42:00.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.824+0000 7f17bbe00640 1 -- 172.21.15.164:0/671343168 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f17c40c4bf0 con 0x7f17a00675c0 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 msgr2=0x7f17a0069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 0x7f17a0069a80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f17c415bba0 tx=0x7f17b4000bd0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 msgr2=0x7f17c40cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01cc60 tx=0x7f17a8005840 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 shutdown_connections 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f17a00675c0 0x7f17a0069a80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17c41f0ba0 0x7f17c40c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17c40c9600 0x7f17c40cdf10 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 --2- 172.21.15.164:0/671343168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17a8004d50 0x7f17c4155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 >> 172.21.15.164:0/671343168 conn(0x7f17c40cf770 msgr2=0x7f17c406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:00.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 shutdown_connections 2024-05-08T04:42:00.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.828+0000 7f17ca400640 1 -- 172.21.15.164:0/671343168 wait complete. 2024-05-08T04:42:00.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol18070 subvol_snap3211 subvol_grp35820 2024-05-08T04:42:00.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.988+0000 7fcfdde00640 1 Processor -- start 2024-05-08T04:42:00.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.988+0000 7fcfdde00640 1 -- start start 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfdde00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcfd815ab10 0x7fcfd806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfdde00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfd80cee90 0x7fcfd80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfdde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd815e870 con 0x7fcfd80ae6e0 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfdde00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd806bb80 con 0x7fcfd815ab10 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfdde00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd815eb50 con 0x7fcfd80cee90 2024-05-08T04:42:00.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcfd815ab10 0x7fcfd806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcfd815ab10 0x7fcfd806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50426/0 (socket says 172.21.15.164:50426) 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 -- 172.21.15.164:0/3145579568 learned_addr learned my addr 172.21.15.164:0/3145579568 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd7400640 1 --2- 172.21.15.164:0/3145579568 >> v2:172.21.15.172:3300/0 conn(0x7fcfd80cee90 0x7fcfd80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 -- 172.21.15.164:0/3145579568 >> v2:172.21.15.172:3300/0 conn(0x7fcfd80cee90 msgr2=0x7fcfd80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/3145579568 >> v2:172.21.15.172:3300/0 conn(0x7fcfd80cee90 0x7fcfd80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 -- 172.21.15.164:0/3145579568 >> v1:172.21.15.172:6789/0 conn(0x7fcfd80ae6e0 legacy=0x7fcfd80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:00.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 -- 172.21.15.164:0/3145579568 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd815ee30 con 0x7fcfd815ab10 2024-05-08T04:42:00.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.992+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/3145579568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd806cce0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80ca1f0 tx=0x7fcfc801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12ac1c25785f0c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:00.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd6000640 1 -- 172.21.15.164:0/3145579568 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80ab660 con 0x7fcfd815ab10 2024-05-08T04:42:00.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd6000640 1 -- 172.21.15.164:0/3145579568 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc80b0070 con 0x7fcfd815ab10 2024-05-08T04:42:00.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd6000640 1 -- 172.21.15.164:0/3145579568 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80aba20 con 0x7fcfd815ab10 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- 172.21.15.164:0/3145579568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 msgr2=0x7fcfd806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- 172.21.15.164:0/3145579568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd806cce0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80ca1f0 tx=0x7fcfc801c460 comp rx=0 tx=0).stop 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- 172.21.15.164:0/3145579568 shutdown_connections 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- 172.21.15.164:0/3145579568 >> v2:172.21.15.172:3300/0 conn(0x7fcfd80cee90 0x7fcfd80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- 172.21.15.164:0/3145579568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd806cce0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- 172.21.15.164:0/3145579568 >> 172.21.15.164:0/3145579568 conn(0x7fcfd80cf770 msgr2=0x7fcfd80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- 172.21.15.164:0/3145579568 shutdown_connections 2024-05-08T04:42:01.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- 172.21.15.164:0/3145579568 wait complete. 2024-05-08T04:42:01.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 Processor -- start 2024-05-08T04:42:01.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- start start 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 0x7fcfd814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 0x7fcfd80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80cb070 con 0x7fcfd815ab10 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80caef0 con 0x7fcfd80cee90 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfdde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80cb1f0 con 0x7fcfd80ae6e0 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50440/0 (socket says 172.21.15.164:50440) 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:00.996+0000 7fcfd7400640 1 -- 172.21.15.164:0/4179716457 learned_addr learned my addr 172.21.15.164:0/4179716457 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 0x7fcfd814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 -- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 msgr2=0x7fcfd814e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 0x7fcfd814e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcfe00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 0x7fcfd80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 -- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 msgr2=0x7fcfd80c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 0x7fcfd80c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 -- 172.21.15.164:0/4179716457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80c8200 con 0x7fcfd815ab10 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 0x7fcfd814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcfe00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 0x7fcfd80c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd7400640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0007e30 tx=0x7fcfd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0014620 con 0x7fcfd815ab10 2024-05-08T04:42:01.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0019070 con 0x7fcfd815ab10 2024-05-08T04:42:01.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd80c84e0 con 0x7fcfd815ab10 2024-05-08T04:42:01.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0014a00 con 0x7fcfd815ab10 2024-05-08T04:42:01.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80c8aa0 con 0x7fcfd815ab10 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8072d30 con 0x7fcfd815ab10 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fcfd0014ba0 con 0x7fcfd815ab10 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 0x7fcfb4069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcfd0086430 con 0x7fcfd815ab10 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.000+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 0x7fcfb4069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.004+0000 7fcfd6a00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 0x7fcfb4069a50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80af000 tx=0x7fcfc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.004+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfd0056b10 con 0x7fcfd815ab10 2024-05-08T04:42:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.188+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap3211", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd814dac0 con 0x7fcfb4067590 2024-05-08T04:42:01.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfcf400640 1 -- 172.21.15.164:0/4179716457 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcfd814dac0 con 0x7fcfb4067590 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 msgr2=0x7fcfb4069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 0x7fcfb4069a50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80af000 tx=0x7fcfc8008040 comp rx=0 tx=0).stop 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 msgr2=0x7fcfd814d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0007e30 tx=0x7fcfd000e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 shutdown_connections 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcfb4067590 0x7fcfb4069a50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfd815ab10 0x7fcfd814d580 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80cee90 0x7fcfd80c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 --2- 172.21.15.164:0/4179716457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfd80ae6e0 0x7fcfd814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 >> 172.21.15.164:0/4179716457 conn(0x7fcfd80cf770 msgr2=0x7fcfd80c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 shutdown_connections 2024-05-08T04:42:01.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.212+0000 7fcfdde00640 1 -- 172.21.15.164:0/4179716457 wait complete. 2024-05-08T04:42:01.237 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol18070 subvol_snap29022 subvol_grp35820 2024-05-08T04:42:01.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 Processor -- start 2024-05-08T04:42:01.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 -- start start 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1900ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1900cecc0 0x7fe1900ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19015e870 con 0x7fe19015ab10 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900cf130 con 0x7fe1900cecc0 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe194c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19006bb80 con 0x7fe19006cce0 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe18de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe19015ab10 0x7fe1900ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49748/0 (socket says 172.21.15.164:49748) 2024-05-08T04:42:01.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.376+0000 7fe18de00640 1 -- 172.21.15.164:0/1927225680 learned_addr learned my addr 172.21.15.164:0/1927225680 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:01.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 --2- 172.21.15.164:0/1927225680 >> v2:172.21.15.172:3300/0 conn(0x7fe1900cecc0 0x7fe1900ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe185400640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1900ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18d400640 1 -- 172.21.15.164:0/1927225680 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116693139 0 0) 0x7fe19015e870 con 0x7fe19015ab10 2024-05-08T04:42:01.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18d400640 1 -- 172.21.15.164:0/1927225680 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe174003530 con 0x7fe19015ab10 2024-05-08T04:42:01.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 -- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 msgr2=0x7fe1900ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:01.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1900ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 -- 172.21.15.164:0/1927225680 >> v1:172.21.15.164:6789/0 conn(0x7fe19015ab10 legacy=0x7fe1900ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 -- 172.21.15.164:0/1927225680 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19015eb50 con 0x7fe1900cecc0 2024-05-08T04:42:01.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18e800640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1900ca6e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe188009d90 tx=0x7fe18801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f1c089436a6892 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18d400640 1 -- 172.21.15.164:0/1927225680 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880b2070 con 0x7fe1900cecc0 2024-05-08T04:42:01.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18d400640 1 -- 172.21.15.164:0/1927225680 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880a5880 con 0x7fe1900cecc0 2024-05-08T04:42:01.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.380+0000 7fe18d400640 1 -- 172.21.15.164:0/1927225680 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880a5c50 con 0x7fe1900cecc0 2024-05-08T04:42:01.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 msgr2=0x7fe1900ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1900ca6e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe188009d90 tx=0x7fe18801c8b0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- 172.21.15.164:0/1927225680 shutdown_connections 2024-05-08T04:42:01.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1900ca6e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- 172.21.15.164:0/1927225680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1900ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- 172.21.15.164:0/1927225680 >> 172.21.15.164:0/1927225680 conn(0x7fe1900cf770 msgr2=0x7fe1900cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- 172.21.15.164:0/1927225680 shutdown_connections 2024-05-08T04:42:01.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- 172.21.15.164:0/1927225680 wait complete. 2024-05-08T04:42:01.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 Processor -- start 2024-05-08T04:42:01.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- start start 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1901f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1901f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1900aedf0 con 0x7fe19015ab10 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19015aef0 con 0x7fe1900cecc0 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe194c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1900aef70 con 0x7fe19006cce0 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50446/0 (socket says 172.21.15.164:50446) 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 -- 172.21.15.164:0/168564806 learned_addr learned my addr 172.21.15.164:0/168564806 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18de00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1901f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe185400640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1901f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 -- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 msgr2=0x7fe1901f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1901f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 -- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 msgr2=0x7fe1901f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1901f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.384+0000 7fe18e800640 1 -- 172.21.15.164:0/168564806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1901fb030 con 0x7fe19015ab10 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe18e800640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe18801af60 tx=0x7fe188019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880b2050 con 0x7fe19015ab10 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880be070 con 0x7fe19015ab10 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880ad850 con 0x7fe19015ab10 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1901fb310 con 0x7fe19015ab10 2024-05-08T04:42:01.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1901fb8d0 con 0x7fe19015ab10 2024-05-08T04:42:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190072d30 con 0x7fe19015ab10 2024-05-08T04:42:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fe1880ad9f0 con 0x7fe19015ab10 2024-05-08T04:42:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 0x7fe16c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe188120b10 con 0x7fe19015ab10 2024-05-08T04:42:01.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.388+0000 7fe18de00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 0x7fe16c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.392+0000 7fe18de00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 0x7fe16c069a50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe180007fd0 tx=0x7fe180009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.392+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe188016560 con 0x7fe19015ab10 2024-05-08T04:42:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.580+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol18070", "snap_name": "subvol_snap29022", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7fe19004f010 con 0x7fe16c067590 2024-05-08T04:42:01.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe186a00640 1 -- 172.21.15.164:0/168564806 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe19004f010 con 0x7fe16c067590 2024-05-08T04:42:01.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 msgr2=0x7fe16c069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 0x7fe16c069a50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe180007fd0 tx=0x7fe180009040 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 msgr2=0x7fe1900aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe18801af60 tx=0x7fe188019650 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 shutdown_connections 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe16c067590 0x7fe16c069a50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe19015ab10 0x7fe1900aded0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1900cecc0 0x7fe1901f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 --2- 172.21.15.164:0/168564806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe19006cce0 0x7fe1901f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.604+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 >> 172.21.15.164:0/168564806 conn(0x7fe1900cf770 msgr2=0x7fe1900af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.608+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 shutdown_connections 2024-05-08T04:42:01.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.608+0000 7fe194c00640 1 -- 172.21.15.164:0/168564806 wait complete. 2024-05-08T04:42:01.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol18070 subvol_grp35820 2024-05-08T04:42:01.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.764+0000 7fc66ce00640 1 Processor -- start 2024-05-08T04:42:01.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.764+0000 7fc66ce00640 1 -- start start 2024-05-08T04:42:01.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc66ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc66815ba20 0x7fc668160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 0x7fc66806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc66ce00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680c6600 con 0x7fc6680ae6e0 2024-05-08T04:42:01.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc66ce00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66815fbe0 con 0x7fc66815ba20 2024-05-08T04:42:01.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc66ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680c6040 con 0x7fc66806de50 2024-05-08T04:42:01.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc666800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc6680ae6e0 0x7fc6680cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49758/0 (socket says 172.21.15.164:49758) 2024-05-08T04:42:01.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc666800640 1 -- 172.21.15.164:0/2772879534 learned_addr learned my addr 172.21.15.164:0/2772879534 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:01.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc665e00640 1 --2- 172.21.15.164:0/2772879534 >> v2:172.21.15.164:3300/0 conn(0x7fc66815ba20 0x7fc668160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc65d400640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 0x7fc66806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc665e00640 1 -- 172.21.15.164:0/2772879534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 msgr2=0x7fc66806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:01.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc665e00640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 0x7fc66806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc665e00640 1 -- 172.21.15.164:0/2772879534 >> v1:172.21.15.164:6789/0 conn(0x7fc6680ae6e0 legacy=0x7fc6680cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc665e00640 1 -- 172.21.15.164:0/2772879534 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680c6320 con 0x7fc66815ba20 2024-05-08T04:42:01.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.768+0000 7fc65d400640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 0x7fc66806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:01.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc665e00640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc668160070 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc66815fd60 tx=0x7fc65801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e2cee8ab738428 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc665400640 1 -- 172.21.15.164:0/2772879534 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580b2070 con 0x7fc66815ba20 2024-05-08T04:42:01.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc665400640 1 -- 172.21.15.164:0/2772879534 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6580a5980 con 0x7fc66815ba20 2024-05-08T04:42:01.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc665400640 1 -- 172.21.15.164:0/2772879534 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580a5d50 con 0x7fc66815ba20 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- 172.21.15.164:0/2772879534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 msgr2=0x7fc668160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc668160070 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc66815fd60 tx=0x7fc65801c920 comp rx=0 tx=0).stop 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- 172.21.15.164:0/2772879534 shutdown_connections 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806de50 0x7fc66806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 --2- 172.21.15.164:0/2772879534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc668160070 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- 172.21.15.164:0/2772879534 >> 172.21.15.164:0/2772879534 conn(0x7fc6680cf770 msgr2=0x7fc6680cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- 172.21.15.164:0/2772879534 shutdown_connections 2024-05-08T04:42:01.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- 172.21.15.164:0/2772879534 wait complete. 2024-05-08T04:42:01.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 Processor -- start 2024-05-08T04:42:01.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 -- start start 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66806de50 0x7fc668146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.772+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680ae6e0 0x7fc6680ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680aec40 con 0x7fc66815ba20 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680aeac0 con 0x7fc66806de50 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680aedc0 con 0x7fc6680ae6e0 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50470/0 (socket says 172.21.15.164:50470) 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc65d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680ae6e0 0x7fc6680ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 -- 172.21.15.164:0/1350380092 learned_addr learned my addr 172.21.15.164:0/1350380092 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc665e00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66806de50 0x7fc668146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 -- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680ae6e0 msgr2=0x7fc6680ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680ae6e0 0x7fc6680ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 -- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66806de50 msgr2=0x7fc668146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66806de50 0x7fc668146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 -- 172.21.15.164:0/1350380092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680cb650 con 0x7fc66815ba20 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc666800640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c400 tx=0x7fc66000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660017070 con 0x7fc66815ba20 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660005cf0 con 0x7fc66815ba20 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600072d0 con 0x7fc66815ba20 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6680cb930 con 0x7fc66815ba20 2024-05-08T04:42:01.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6680ca920 con 0x7fc66815ba20 2024-05-08T04:42:01.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc668072d30 con 0x7fc66815ba20 2024-05-08T04:42:01.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.776+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fc66001a070 con 0x7fc66815ba20 2024-05-08T04:42:01.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.780+0000 7fc65ea00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 0x7fc644069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:01.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.780+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc660084e10 con 0x7fc66815ba20 2024-05-08T04:42:01.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.780+0000 7fc665e00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 0x7fc644069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:01.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.780+0000 7fc665e00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 0x7fc644069a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6580af000 tx=0x7fc6580a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:01.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.784+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc660054ea0 con 0x7fc66815ba20 2024-05-08T04:42:01.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.960+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol18070", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7fc66815b1e0 con 0x7fc644067590 2024-05-08T04:42:01.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.972+0000 7fc65ea00640 1 -- 172.21.15.164:0/1350380092 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc66815b1e0 con 0x7fc644067590 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 msgr2=0x7fc644069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 0x7fc644069a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6580af000 tx=0x7fc6580a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 msgr2=0x7fc6680addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c400 tx=0x7fc66000c8c0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 shutdown_connections 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc644067590 0x7fc644069a50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680ae6e0 0x7fc6680ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc66815ba20 0x7fc6680addd0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 --2- 172.21.15.164:0/1350380092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66806de50 0x7fc668146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:01.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 >> 172.21.15.164:0/1350380092 conn(0x7fc6680cf770 msgr2=0x7fc6681517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:01.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 shutdown_connections 2024-05-08T04:42:01.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:01.976+0000 7fc66ce00640 1 -- 172.21.15.164:0/1350380092 wait complete. 2024-05-08T04:42:01.986 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.980+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:01.986 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.980+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:01.986 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.980+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:01.986 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.980+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:01.986 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.980+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.002 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.996+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.003 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.996+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.003 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.996+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.003 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.996+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.003 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:01.996+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:02.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:02.006 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:42:02.020 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:42:02.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp35820 2024-05-08T04:42:02.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 Processor -- start 2024-05-08T04:42:02.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 -- start start 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcca015ab10 0x7fcca006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcca00cee90 0x7fcca00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca015e870 con 0x7fcca00ae6e0 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca006bb80 con 0x7fcca015ab10 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcca6800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca015eb50 con 0x7fcca00cee90 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.196+0000 7fcc9f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcca015ab10 0x7fcca006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcca015ab10 0x7fcca006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50512/0 (socket says 172.21.15.164:50512) 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 -- 172.21.15.164:0/2079693707 learned_addr learned my addr 172.21.15.164:0/2079693707 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9fe00640 1 --2- 172.21.15.164:0/2079693707 >> v2:172.21.15.172:3300/0 conn(0x7fcca00cee90 0x7fcca00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 -- 172.21.15.164:0/2079693707 >> v2:172.21.15.172:3300/0 conn(0x7fcca00cee90 msgr2=0x7fcca00aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 --2- 172.21.15.164:0/2079693707 >> v2:172.21.15.172:3300/0 conn(0x7fcca00cee90 0x7fcca00aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 -- 172.21.15.164:0/2079693707 >> v1:172.21.15.172:6789/0 conn(0x7fcca00ae6e0 legacy=0x7fcca00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 -- 172.21.15.164:0/2079693707 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca015ee30 con 0x7fcca015ab10 2024-05-08T04:42:02.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9f400640 1 --2- 172.21.15.164:0/2079693707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca015ab10 0x7fcca006cce0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ca1f0 tx=0x7fcc9001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1942015093c0ee54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9ea00640 1 -- 172.21.15.164:0/2079693707 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc900ab660 con 0x7fcca015ab10 2024-05-08T04:42:02.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9ea00640 1 -- 172.21.15.164:0/2079693707 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc900b0070 con 0x7fcca015ab10 2024-05-08T04:42:02.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.200+0000 7fcc9ea00640 1 -- 172.21.15.164:0/2079693707 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc900aba20 con 0x7fcca015ab10 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- 172.21.15.164:0/2079693707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca015ab10 msgr2=0x7fcca006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- 172.21.15.164:0/2079693707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca015ab10 0x7fcca006cce0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ca1f0 tx=0x7fcc9001c460 comp rx=0 tx=0).stop 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- 172.21.15.164:0/2079693707 shutdown_connections 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- 172.21.15.164:0/2079693707 >> v2:172.21.15.172:3300/0 conn(0x7fcca00cee90 0x7fcca00aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- 172.21.15.164:0/2079693707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca015ab10 0x7fcca006cce0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- 172.21.15.164:0/2079693707 >> 172.21.15.164:0/2079693707 conn(0x7fcca00cf770 msgr2=0x7fcca00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- 172.21.15.164:0/2079693707 shutdown_connections 2024-05-08T04:42:02.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- 172.21.15.164:0/2079693707 wait complete. 2024-05-08T04:42:02.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 Processor -- start 2024-05-08T04:42:02.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- start start 2024-05-08T04:42:02.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 0x7fcca014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca015ab10 0x7fcca0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00cb070 con 0x7fcca00ae6e0 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00caef0 con 0x7fcca015ab10 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcca6800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca00cb1f0 con 0x7fcca00cee90 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50522/0 (socket says 172.21.15.164:50522) 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 -- 172.21.15.164:0/4222475818 learned_addr learned my addr 172.21.15.164:0/4222475818 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9fe00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca015ab10 0x7fcca0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc96a00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 0x7fcca014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 -- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 msgr2=0x7fcca014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 0x7fcca014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 -- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca015ab10 msgr2=0x7fcca0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca015ab10 0x7fcca0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 -- 172.21.15.164:0/4222475818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca014b7f0 con 0x7fcca00ae6e0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc96a00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 0x7fcca014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.204+0000 7fcc9f400640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcc90019070 tx=0x7fcc9001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc900be5f0 con 0x7fcca00ae6e0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc900be790 con 0x7fcca00ae6e0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc900beb80 con 0x7fcca00ae6e0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca00c8950 con 0x7fcca00ae6e0 2024-05-08T04:42:02.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca00c8bf0 con 0x7fcca00ae6e0 2024-05-08T04:42:02.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca0072d30 con 0x7fcca00ae6e0 2024-05-08T04:42:02.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fcc900b43f0 con 0x7fcca00ae6e0 2024-05-08T04:42:02.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 0x7fcc7c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc901206a0 con 0x7fcca00ae6e0 2024-05-08T04:42:02.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc96a00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 0x7fcc7c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.208+0000 7fcc96a00640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 0x7fcc7c069a50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcca01fae30 tx=0x7fcc88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.212+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc90016110 con 0x7fcca00ae6e0 2024-05-08T04:42:02.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.392+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp35820", "target": ["mon-mgr", ""]}) v1 -- 0x7fcca00b4ec0 con 0x7fcc7c067590 2024-05-08T04:42:02.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.396+0000 7fcc97e00640 1 -- 172.21.15.164:0/4222475818 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcca00b4ec0 con 0x7fcc7c067590 2024-05-08T04:42:02.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 msgr2=0x7fcc7c069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 0x7fcc7c069a50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcca01fae30 tx=0x7fcc88009040 comp rx=0 tx=0).stop 2024-05-08T04:42:02.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 msgr2=0x7fcca00c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcc90019070 tx=0x7fcc9001aca0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 shutdown_connections 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc7c067590 0x7fcc7c069a50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcca00cee90 0x7fcca014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca015ab10 0x7fcca0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 --2- 172.21.15.164:0/4222475818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca00ae6e0 0x7fcca00c93f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 >> 172.21.15.164:0/4222475818 conn(0x7fcca00cf770 msgr2=0x7fcca006d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:02.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 shutdown_connections 2024-05-08T04:42:02.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.400+0000 7fcca6800640 1 -- 172.21.15.164:0/4222475818 wait complete. 2024-05-08T04:42:02.425 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:42:02.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:02.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:02.605 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:42:02.605 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:02.605 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:02.615 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:02.615 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:02.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:02.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 Processor -- start 2024-05-08T04:42:02.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 -- start start 2024-05-08T04:42:02.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6cf815de60 0x7f6cf806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf806bda0 con 0x7f6cf8161e50 2024-05-08T04:42:02.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf806c080 con 0x7f6cf815da80 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfd600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf806c9b0 con 0x7f6cf815de60 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cf7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6cf815de60 0x7f6cf806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfcc00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6cf8161e50 0x7f6cf806ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49772/0 (socket says 172.21.15.164:49772) 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cfcc00640 1 -- 172.21.15.164:0/1242045172 learned_addr learned my addr 172.21.15.164:0/1242045172 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cf7400640 1 -- 172.21.15.164:0/1242045172 >> v1:172.21.15.172:6789/0 conn(0x7f6cf815da80 legacy=0x7f6cf8161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cf7400640 1 -- 172.21.15.164:0/1242045172 >> v1:172.21.15.164:6789/0 conn(0x7f6cf8161e50 legacy=0x7f6cf806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.840+0000 7f6cf7400640 1 -- 172.21.15.164:0/1242045172 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf806c360 con 0x7f6cf815de60 2024-05-08T04:42:02.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cf7400640 1 --2- 172.21.15.164:0/1242045172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf806eed0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40097c0 tx=0x7f6ce401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464a4d6ad7192efb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cf6a00640 1 -- 172.21.15.164:0/1242045172 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40b1070 con 0x7f6cf815de60 2024-05-08T04:42:02.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cf6a00640 1 -- 172.21.15.164:0/1242045172 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40ac410 con 0x7f6cf815de60 2024-05-08T04:42:02.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cf6a00640 1 -- 172.21.15.164:0/1242045172 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40ac7e0 con 0x7f6cf815de60 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 -- 172.21.15.164:0/1242045172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 msgr2=0x7f6cf806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1242045172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf806eed0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40097c0 tx=0x7f6ce401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 -- 172.21.15.164:0/1242045172 shutdown_connections 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1242045172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf806eed0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 -- 172.21.15.164:0/1242045172 >> 172.21.15.164:0/1242045172 conn(0x7f6cf80b32e0 msgr2=0x7f6cf80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 -- 172.21.15.164:0/1242045172 shutdown_connections 2024-05-08T04:42:02.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.844+0000 7f6cfd600640 1 -- 172.21.15.164:0/1242045172 wait complete. 2024-05-08T04:42:02.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 Processor -- start 2024-05-08T04:42:02.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- start start 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf815da80 0x7f6cf80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf8161e50 0x7f6cf80c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf816be20 con 0x7f6cf815de60 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf816bca0 con 0x7f6cf8161e50 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf816bfa0 con 0x7f6cf815da80 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50540/0 (socket says 172.21.15.164:50540) 2024-05-08T04:42:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 -- 172.21.15.164:0/1174622609 learned_addr learned my addr 172.21.15.164:0/1174622609 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7e00640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf815da80 0x7f6cf80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfcc00640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf8161e50 0x7f6cf80c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 -- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf815da80 msgr2=0x7f6cf80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf815da80 0x7f6cf80cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 -- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf8161e50 msgr2=0x7f6cf80c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf8161e50 0x7f6cf80c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 -- 172.21.15.164:0/1174622609 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf81537d0 con 0x7f6cf815de60 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cf7400640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40190b0 tx=0x7f6ce4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40b9020 con 0x7f6cf815de60 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40ace40 con 0x7f6cf815de60 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf80ccaa0 con 0x7f6cf815de60 2024-05-08T04:42:02.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40b1040 con 0x7f6cf815de60 2024-05-08T04:42:02.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.848+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80cc4e0 con 0x7f6cf815de60 2024-05-08T04:42:02.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf804ef30 con 0x7f6cf815de60 2024-05-08T04:42:02.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7f6ce40bc020 con 0x7f6cf815de60 2024-05-08T04:42:02.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cefe00640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 0x7f6cd8069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:02.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6ce411f6b0 con 0x7f6cf815de60 2024-05-08T04:42:02.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cf7e00640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 0x7f6cd8069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:02.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.852+0000 7f6cf7e00640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 0x7f6cd8069a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8001d50 tx=0x7f6ce8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:02.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:02.856+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ce40c3030 con 0x7f6cf815de60 2024-05-08T04:42:03.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.016+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf8074aa0 con 0x7f6cf815de60 2024-05-08T04:42:03.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.016+0000 7f6cefe00640 1 -- 172.21.15.164:0/1174622609 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16808 (secure 0 0 0) 0x7f6ce4016090 con 0x7f6cf815de60 2024-05-08T04:42:03.022 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:03.023 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":86,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:41:44.861638+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:03.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 msgr2=0x7f6cd8069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 0x7f6cd8069a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8001d50 tx=0x7f6ce8009040 comp rx=0 tx=0).stop 2024-05-08T04:42:03.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 msgr2=0x7f6cf8151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40190b0 tx=0x7f6ce4019240 comp rx=0 tx=0).stop 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 shutdown_connections 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6cd80675c0 0x7f6cd8069a80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf8161e50 0x7f6cf80c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf815de60 0x7f6cf8151cd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 --2- 172.21.15.164:0/1174622609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf815da80 0x7f6cf80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 >> 172.21.15.164:0/1174622609 conn(0x7f6cf80b32e0 msgr2=0x7f6cf8160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 shutdown_connections 2024-05-08T04:42:03.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.020+0000 7f6cfd600640 1 -- 172.21.15.164:0/1174622609 wait complete. 2024-05-08T04:42:03.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:41.603797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:42.235153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:03.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:03.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.176+0000 7f061ac00640 1 Processor -- start 2024-05-08T04:42:03.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061ac00640 1 -- start start 2024-05-08T04:42:03.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06140cec40 0x7f061415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061ac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061415c900 con 0x7f06140b1d40 2024-05-08T04:42:03.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061415cbe0 con 0x7f06140b1960 2024-05-08T04:42:03.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061415cec0 con 0x7f06140cec40 2024-05-08T04:42:03.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061a200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f06140b1d40 0x7f06140ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49776/0 (socket says 172.21.15.164:49776) 2024-05-08T04:42:03.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061a200640 1 -- 172.21.15.164:0/387241289 learned_addr learned my addr 172.21.15.164:0/387241289 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0618e00640 1 --2- 172.21.15.164:0/387241289 >> v2:172.21.15.172:3300/0 conn(0x7f06140cec40 0x7f061415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767388793 0 0) 0x7f061415c900 con 0x7f06140b1d40 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc003530 con 0x7f06140b1d40 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373105198 0 0) 0x7f061415cbe0 con 0x7f06140b1960 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061415c900 con 0x7f06140b1960 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1803371064 0 0) 0x7f05fc003530 con 0x7f06140b1d40 2024-05-08T04:42:03.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 >> v2:172.21.15.172:3300/0 conn(0x7f06140cec40 msgr2=0x7f061415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 --2- 172.21.15.164:0/387241289 >> v2:172.21.15.172:3300/0 conn(0x7f06140cec40 0x7f061415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 >> v1:172.21.15.172:6789/0 conn(0x7f06140b1960 legacy=0x7f0614071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140712a0 con 0x7f06140b1d40 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f060c001a00 con 0x7f06140b1d40 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 >> v1:172.21.15.164:6789/0 conn(0x7f06140b1d40 legacy=0x7f06140ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f05fc003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05fc004810 0x7f05fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05fc0052e0 0x7f05fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061415cec0 con 0x7f05fc003ba0 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc00c1f0 con 0x7f05fc004810 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc00c3f0 con 0x7f05fc0052e0 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f05fc003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0618e00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05fc004810 0x7f05fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061a200640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05fc0052e0 0x7f05fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 -- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05fc0052e0 msgr2=0x7f05fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05fc0052e0 0x7f05fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 -- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05fc004810 msgr2=0x7f05fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05fc004810 0x7f05fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f0619800640 1 -- 172.21.15.164:0/387241289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc003530 con 0x7f05fc003ba0 2024-05-08T04:42:03.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.180+0000 7f061a200640 1 -- 172.21.15.164:0/387241289 reap_dead start 2024-05-08T04:42:03.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f0619800640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f05fc003fc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f06080018d0 tx=0x7f060801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505fefd6d377b231 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608001520 con 0x7f05fc003ba0 2024-05-08T04:42:03.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06080052f0 con 0x7f05fc003ba0 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f0613e00640 1 -- 172.21.15.164:0/387241289 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608003840 con 0x7f05fc003ba0 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 -- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 msgr2=0x7f05fc003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f05fc003fc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f06080018d0 tx=0x7f060801c4d0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 -- 172.21.15.164:0/387241289 shutdown_connections 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 --2- 172.21.15.164:0/387241289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f05fc003fc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 -- 172.21.15.164:0/387241289 >> 172.21.15.164:0/387241289 conn(0x7f061406ac90 msgr2=0x7f061406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 -- 172.21.15.164:0/387241289 shutdown_connections 2024-05-08T04:42:03.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.184+0000 7f061ac00640 1 -- 172.21.15.164:0/387241289 wait complete. 2024-05-08T04:42:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 Processor -- start 2024-05-08T04:42:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- start start 2024-05-08T04:42:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 0x7f0614140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 0x7f0614153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140cd030 con 0x7f05fc003ba0 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06140cceb0 con 0x7f0614150bc0 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06140cd1b0 con 0x7f0614153300 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50566/0 (socket says 172.21.15.164:50566) 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 -- 172.21.15.164:0/925252642 learned_addr learned my addr 172.21.15.164:0/925252642 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0618e00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 0x7f0614140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 -- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 msgr2=0x7f0614153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061a200640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 0x7f0614153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 0x7f0614153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 -- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 msgr2=0x7f0614140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 0x7f0614140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 -- 172.21.15.164:0/925252642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140b4800 con 0x7f05fc003ba0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0618e00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 0x7f0614140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061a200640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 0x7f0614153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0619800640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0608000c10 tx=0x7f05fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080ba070 con 0x7f05fc003ba0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06080054a0 con 0x7f05fc003ba0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608001500 con 0x7f05fc003ba0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06141598b0 con 0x7f05fc003ba0 2024-05-08T04:42:03.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.188+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0614067df0 con 0x7f05fc003ba0 2024-05-08T04:42:03.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061404ef30 con 0x7f05fc003ba0 2024-05-08T04:42:03.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7f06080bd070 con 0x7f05fc003ba0 2024-05-08T04:42:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f0611600640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 0x7f05f4069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f060811f190 con 0x7f05fc003ba0 2024-05-08T04:42:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f0618e00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 0x7f05f4069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.192+0000 7f0618e00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 0x7f05f4069a50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0614144bf0 tx=0x7f0600008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.196+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0608016210 con 0x7f05fc003ba0 2024-05-08T04:42:03.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.352+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f061415f400 con 0x7f05fc003ba0 2024-05-08T04:42:03.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f0611600640 1 -- 172.21.15.164:0/925252642 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06080ef740 con 0x7f05fc003ba0 2024-05-08T04:42:03.360 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 msgr2=0x7f05f4069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 0x7f05f4069a50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0614144bf0 tx=0x7f0600008040 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 msgr2=0x7f061415b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0608000c10 tx=0x7f05fc005840 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 shutdown_connections 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05f4067590 0x7f05f4069a50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0614153300 0x7f0614153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0614150bc0 0x7f0614140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 --2- 172.21.15.164:0/925252642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05fc003ba0 0x7f061415b670 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 >> 172.21.15.164:0/925252642 conn(0x7f061406ac90 msgr2=0x7f061406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 shutdown_connections 2024-05-08T04:42:03.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.356+0000 7f061ac00640 1 -- 172.21.15.164:0/925252642 wait complete. 2024-05-08T04:42:03.380 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:03.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 Processor -- start 2024-05-08T04:42:03.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 -- start start 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c680ce900 0x7f5c680cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6806eea0 con 0x7f5c6806f6f0 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6806f120 con 0x7f5c680b0b90 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6de00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6806c490 con 0x7f5c680ce900 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c67e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5c6806f6f0 0x7f5c680ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49782/0 (socket says 172.21.15.164:49782) 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c67e00640 1 -- 172.21.15.164:0/1972343089 learned_addr learned my addr 172.21.15.164:0/1972343089 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:03.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.516+0000 7f5c6d400640 1 --2- 172.21.15.164:0/1972343089 >> v2:172.21.15.172:3300/0 conn(0x7f5c680ce900 0x7f5c680cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013806086 0 0) 0x7f5c54001120 con 0x7f5c6806f6f0 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c4c003530 con 0x7f5c6806f6f0 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148458935 0 0) 0x7f5c6806f120 con 0x7f5c680b0b90 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c54001120 con 0x7f5c680b0b90 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c6d400640 1 -- 172.21.15.164:0/1972343089 >> v1:172.21.15.172:6789/0 conn(0x7f5c680b0b90 legacy=0x7f5c6806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c6d400640 1 -- 172.21.15.164:0/1972343089 >> v1:172.21.15.164:6789/0 conn(0x7f5c6806f6f0 legacy=0x7f5c680ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c6d400640 1 -- 172.21.15.164:0/1972343089 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6806f400 con 0x7f5c680ce900 2024-05-08T04:42:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c6d400640 1 --2- 172.21.15.164:0/1972343089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680cf970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009d90 tx=0x7f5c6001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f8425521b5fd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c600b2070 con 0x7f5c680ce900 2024-05-08T04:42:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c600a5830 con 0x7f5c680ce900 2024-05-08T04:42:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.520+0000 7f5c67400640 1 -- 172.21.15.164:0/1972343089 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a5c00 con 0x7f5c680ce900 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- 172.21.15.164:0/1972343089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 msgr2=0x7f5c680cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1972343089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680cf970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009d90 tx=0x7f5c6001c970 comp rx=0 tx=0).stop 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- 172.21.15.164:0/1972343089 shutdown_connections 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1972343089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680cf970 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- 172.21.15.164:0/1972343089 >> 172.21.15.164:0/1972343089 conn(0x7f5c680b3370 msgr2=0x7f5c680b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- 172.21.15.164:0/1972343089 shutdown_connections 2024-05-08T04:42:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- 172.21.15.164:0/1972343089 wait complete. 2024-05-08T04:42:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 Processor -- start 2024-05-08T04:42:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- start start 2024-05-08T04:42:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 0x7f5c6815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680d02c0 con 0x7f5c680b0b90 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b0f70 con 0x7f5c680ce900 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68153130 con 0x7f5c6806f6f0 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50568/0 (socket says 172.21.15.164:50568) 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 -- 172.21.15.164:0/1951874227 learned_addr learned my addr 172.21.15.164:0/1951874227 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6ca00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 0x7f5c6815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6d400640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 -- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 msgr2=0x7f5c6815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 0x7f5c6815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 -- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 msgr2=0x7f5c680b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 -- 172.21.15.164:0/1951874227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68067ce0 con 0x7f5c680b0b90 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c6ca00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 0x7f5c6815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c67e00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c690 tx=0x7f5c5400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.524+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54005ae0 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c680b49d0 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54007070 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68154c90 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6806b900 con 0x7f5c680b0b90 2024-05-08T04:42:03.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7f5c5401a070 con 0x7f5c680b0b90 2024-05-08T04:42:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c64c00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 0x7f5c48069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5c540850f0 con 0x7f5c680b0b90 2024-05-08T04:42:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c6ca00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 0x7f5c48069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.528+0000 7f5c6ca00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 0x7f5c48069a50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5c68155d20 tx=0x7f5c58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.532+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c6806eea0 con 0x7f5c680b0b90 2024-05-08T04:42:03.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.688+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c6804ef30 con 0x7f5c680b0b90 2024-05-08T04:42:03.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.688+0000 7f5c64c00640 1 -- 172.21.15.164:0/1951874227 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c54055180 con 0x7f5c680b0b90 2024-05-08T04:42:03.694 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 msgr2=0x7f5c48069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 0x7f5c48069a50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5c68155d20 tx=0x7f5c58009040 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 msgr2=0x7f5c6814ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c690 tx=0x7f5c5400cb50 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 shutdown_connections 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c48067590 0x7f5c48069a50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ce900 0x7f5c680b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c680b0b90 0x7f5c6814ffe0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 --2- 172.21.15.164:0/1951874227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6806f6f0 0x7f5c6815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 >> 172.21.15.164:0/1951874227 conn(0x7f5c680b3370 msgr2=0x7f5c680b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 shutdown_connections 2024-05-08T04:42:03.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.692+0000 7f5c6de00640 1 -- 172.21.15.164:0/1951874227 wait complete. 2024-05-08T04:42:03.715 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 Processor -- start 2024-05-08T04:42:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 -- start start 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbfc1624d0 0x7fdbfc161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc162230 con 0x7fdbfc161910 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc06e810 con 0x7fdbfc0d06c0 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdc02a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc06eaf0 con 0x7fdbfc1624d0 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfbe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdbfc161910 0x7fdbfc06f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49790/0 (socket says 172.21.15.164:49790) 2024-05-08T04:42:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfbe00640 1 -- 172.21.15.164:0/2755940457 learned_addr learned my addr 172.21.15.164:0/2755940457 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfb400640 1 --2- 172.21.15.164:0/2755940457 >> v2:172.21.15.172:3300/0 conn(0x7fdbfc1624d0 0x7fdbfc161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464351236 0 0) 0x7fdbfc162230 con 0x7fdbfc161910 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe0003530 con 0x7fdbfc161910 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2266982277 0 0) 0x7fdbe0003530 con 0x7fdbfc161910 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 >> v2:172.21.15.172:3300/0 conn(0x7fdbfc1624d0 msgr2=0x7fdbfc161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 --2- 172.21.15.164:0/2755940457 >> v2:172.21.15.172:3300/0 conn(0x7fdbfc1624d0 0x7fdbfc161cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 >> v1:172.21.15.172:6789/0 conn(0x7fdbfc0d06c0 legacy=0x7fdbfc15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc06edd0 con 0x7fdbfc161910 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fdbec001a00 con 0x7fdbfc161910 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 >> v1:172.21.15.164:6789/0 conn(0x7fdbfc161910 legacy=0x7fdbfc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 0x7fdbe0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe0004810 0x7fdbe0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.848+0000 7fdbfaa00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00051d0 0x7fdbe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc06eaf0 con 0x7fdbe00039c0 2024-05-08T04:42:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe000c070 con 0x7fdbe0004810 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe000c270 con 0x7fdbe00051d0 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 0x7fdbe0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfb400640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe0004810 0x7fdbe0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdc00a00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00051d0 0x7fdbe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 -- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00051d0 msgr2=0x7fdbe000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00051d0 0x7fdbe000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 -- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe0004810 msgr2=0x7fdbe0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe0004810 0x7fdbe0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 -- 172.21.15.164:0/2755940457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0003530 con 0x7fdbe00039c0 2024-05-08T04:42:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdc00a00640 1 -- 172.21.15.164:0/2755940457 reap_dead start 2024-05-08T04:42:03.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 0x7fdbe0003de0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00a850 tx=0x7fdbec01d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a8304cab588328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00a4a0 con 0x7fdbe00039c0 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec0ad650 con 0x7fdbe00039c0 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.852+0000 7fdbfaa00640 1 -- 172.21.15.164:0/2755940457 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0ada80 con 0x7fdbe00039c0 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 msgr2=0x7fdbe0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 0x7fdbe0003de0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00a850 tx=0x7fdbec01d370 comp rx=0 tx=0).stop 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- 172.21.15.164:0/2755940457 shutdown_connections 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 --2- 172.21.15.164:0/2755940457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbe00039c0 0x7fdbe0003de0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- 172.21.15.164:0/2755940457 >> 172.21.15.164:0/2755940457 conn(0x7fdbfc0b34d0 msgr2=0x7fdbfc0b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- 172.21.15.164:0/2755940457 shutdown_connections 2024-05-08T04:42:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- 172.21.15.164:0/2755940457 wait complete. 2024-05-08T04:42:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 Processor -- start 2024-05-08T04:42:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- start start 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 0x7fdbfc1ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 0x7fdbfc1fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1603b0 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc160230 con 0x7fdbe00039c0 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc02a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1e9930 con 0x7fdbfc1fb1b0 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50584/0 (socket says 172.21.15.164:50584) 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 -- 172.21.15.164:0/3115142941 learned_addr learned my addr 172.21.15.164:0/3115142941 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc00a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 0x7fdbfc1fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 0x7fdbfc1ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 -- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 msgr2=0x7fdbfc1fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 0x7fdbfc1fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 -- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 msgr2=0x7fdbfc1ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 0x7fdbfc1ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfb400640 1 -- 172.21.15.164:0/3115142941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc1fbdc0 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdc00a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 0x7fdbfc1fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.856+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 0x7fdbfc1ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbfb400640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80119e0 tx=0x7fdbe8011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8019070 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8004240 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1fc0a0 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800f910 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc0b58c0 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc09fb30 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fdbe801c070 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 0x7fdbdc069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fdbe80860b0 con 0x7fdbfc06d1f0 2024-05-08T04:42:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.860+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 0x7fdbdc069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.864+0000 7fdbfbe00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 0x7fdbdc069a50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0ad000 tx=0x7fdbec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:03.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:03.864+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbe8056700 con 0x7fdbfc06d1f0 2024-05-08T04:42:04.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.024+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbfc0d13c0 con 0x7fdbfc06d1f0 2024-05-08T04:42:04.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.024+0000 7fdbf3e00640 1 -- 172.21.15.164:0/3115142941 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbe8056000 con 0x7fdbfc06d1f0 2024-05-08T04:42:04.031 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 msgr2=0x7fdbdc069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 0x7fdbdc069a50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0ad000 tx=0x7fdbec0a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 msgr2=0x7fdbfc06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80119e0 tx=0x7fdbe8011ea0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 shutdown_connections 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdbdc067590 0x7fdbdc069a50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbfc1fb1b0 0x7fdbfc1fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc06d1f0 0x7fdbfc06d670 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 --2- 172.21.15.164:0/3115142941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00039c0 0x7fdbfc1ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 >> 172.21.15.164:0/3115142941 conn(0x7fdbfc0b34d0 msgr2=0x7fdbfc15b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 shutdown_connections 2024-05-08T04:42:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.028+0000 7fdc02a00640 1 -- 172.21.15.164:0/3115142941 wait complete. 2024-05-08T04:42:04.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:42:04.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 Processor -- start 2024-05-08T04:42:04.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 -- start start 2024-05-08T04:42:04.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc69c15bb60 0x7fc69c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc69c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c0cdf60 con 0x7fc69c15f5b0 2024-05-08T04:42:04.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c15d000 con 0x7fc69c15bb60 2024-05-08T04:42:04.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a2600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0d01d0 con 0x7fc69c0cf850 2024-05-08T04:42:04.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc69c15f5b0 0x7fc69c15fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49802/0 (socket says 172.21.15.164:49802) 2024-05-08T04:42:04.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 -- 172.21.15.164:0/1627689306 learned_addr learned my addr 172.21.15.164:0/1627689306 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69be00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc69c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1200640 1 --2- 172.21.15.164:0/1627689306 >> v2:172.21.15.172:3300/0 conn(0x7fc69c15bb60 0x7fc69c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065778117 0 0) 0x7fc69c0cdf60 con 0x7fc69c15f5b0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc684003530 con 0x7fc69c15f5b0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262984875 0 0) 0x7fc684003530 con 0x7fc69c15f5b0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 msgr2=0x7fc69c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc69c0cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 >> v2:172.21.15.172:3300/0 conn(0x7fc69c15bb60 msgr2=0x7fc69c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 --2- 172.21.15.164:0/1627689306 >> v2:172.21.15.172:3300/0 conn(0x7fc69c15bb60 0x7fc69c15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0ce240 con 0x7fc69c15f5b0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc694001a00 con 0x7fc69c15f5b0 2024-05-08T04:42:04.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 >> v1:172.21.15.164:6789/0 conn(0x7fc69c15f5b0 legacy=0x7fc69c15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6840039c0 0x7fc684003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc684004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc684004d50 0x7fc6840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c15d000 con 0x7fc684004d50 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0d01d0 con 0x7fc6840039c0 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68400b9d0 con 0x7fc69c0cf850 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc684004d50 0x7fc6840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1200640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6840039c0 0x7fc684003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc69be00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc684004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 -- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 msgr2=0x7fc684004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c0cf850 0x7fc684004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 -- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6840039c0 msgr2=0x7fc684003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:04.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6840039c0 0x7fc684003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.184+0000 7fc6a1c00640 1 -- 172.21.15.164:0/1627689306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684003530 con 0x7fc684004d50 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc684004d50 0x7fc6840051d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a8a0 tx=0x7fc69401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2fa3293a44de1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400a4c0 con 0x7fc684004d50 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6940a9020 con 0x7fc684004d50 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc69b400640 1 -- 172.21.15.164:0/1627689306 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6940072d0 con 0x7fc684004d50 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 -- 172.21.15.164:0/1627689306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc684004d50 msgr2=0x7fc6840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 --2- 172.21.15.164:0/1627689306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc684004d50 0x7fc6840051d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a8a0 tx=0x7fc69401d3c0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a1c00640 1 -- 172.21.15.164:0/1627689306 reap_dead start 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 -- 172.21.15.164:0/1627689306 shutdown_connections 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 -- 172.21.15.164:0/1627689306 >> 172.21.15.164:0/1627689306 conn(0x7fc69c0b32e0 msgr2=0x7fc69c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:04.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 -- 172.21.15.164:0/1627689306 shutdown_connections 2024-05-08T04:42:04.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.188+0000 7fc6a2600640 1 -- 172.21.15.164:0/1627689306 wait complete. 2024-05-08T04:42:04.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 Processor -- start 2024-05-08T04:42:04.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- start start 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc684004d50 0x7fc69c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c09c0b0 0x7fc69c09c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0ceae0 con 0x7fc69c09df50 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0ce800 con 0x7fc69c09c0b0 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0cedc0 con 0x7fc684004d50 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50612/0 (socket says 172.21.15.164:50612) 2024-05-08T04:42:04.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 -- 172.21.15.164:0/2538071171 learned_addr learned my addr 172.21.15.164:0/2538071171 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:04.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1200640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc684004d50 0x7fc69c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc69be00640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c09c0b0 0x7fc69c09c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 -- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc684004d50 msgr2=0x7fc69c152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc684004d50 0x7fc69c152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 -- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c09c0b0 msgr2=0x7fc69c09c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c09c0b0 0x7fc69c09c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 -- 172.21.15.164:0/2538071171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c09ed40 con 0x7fc69c09df50 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a1c00640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc694007040 tx=0x7fc684005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400a4c0 con 0x7fc69c09df50 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6940074e0 con 0x7fc69c09df50 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694002020 con 0x7fc69c09df50 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c09f020 con 0x7fc69c09df50 2024-05-08T04:42:04.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.192+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c09f5e0 con 0x7fc69c09df50 2024-05-08T04:42:04.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c04ef30 con 0x7fc69c09df50 2024-05-08T04:42:04.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) v1 ==== 87288+0+0 (secure 0 0 0) 0x7fc6940078d0 con 0x7fc69c09df50 2024-05-08T04:42:04.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc698c00640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 0x7fc67c069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc69411f990 con 0x7fc69c09df50 2024-05-08T04:42:04.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc6a1200640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 0x7fc67c069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.196+0000 7fc6a1200640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 0x7fc67c069a80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc690009c60 tx=0x7fc690009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.200+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6940c2020 con 0x7fc69c09df50 2024-05-08T04:42:04.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.320+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fc6940ec050 con 0x7fc69c09df50 2024-05-08T04:42:04.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc69c0c8640 con 0x7fc69c09df50 2024-05-08T04:42:04.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc698c00640 1 -- 172.21.15.164:0/2538071171 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6940170d0 con 0x7fc69c09df50 2024-05-08T04:42:04.385 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 msgr2=0x7fc67c069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 0x7fc67c069a80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc690009c60 tx=0x7fc690009190 comp rx=0 tx=0).stop 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 msgr2=0x7fc69c09e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc694007040 tx=0x7fc684005840 comp rx=0 tx=0).stop 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 shutdown_connections 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc67c0675c0 0x7fc67c069a80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc69c09df50 0x7fc69c09e3d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c09c0b0 0x7fc69c09c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 --2- 172.21.15.164:0/2538071171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc684004d50 0x7fc69c152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 >> 172.21.15.164:0/2538071171 conn(0x7fc69c0b32e0 msgr2=0x7fc69c06f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:04.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 shutdown_connections 2024-05-08T04:42:04.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.380+0000 7fc6a2600640 1 -- 172.21.15.164:0/2538071171 wait complete. 2024-05-08T04:42:04.408 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:42:04.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:04.408 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:04.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:42:04.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:04.434 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:42:04.487 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:04.487 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:04.500 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:04.500 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:04.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:04.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.680+0000 7f266ea00640 1 Processor -- start 2024-05-08T04:42:04.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.680+0000 7f266ea00640 1 -- start start 2024-05-08T04:42:04.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26680cfda0 0x7f26680cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2668072640 con 0x7f26680b26c0 2024-05-08T04:42:04.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266806c770 con 0x7f26680cfda0 2024-05-08T04:42:04.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266806ca50 con 0x7f266815ab60 2024-05-08T04:42:04.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:55986/0 (socket says 172.21.15.164:55986) 2024-05-08T04:42:04.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 -- 172.21.15.164:0/977556654 learned_addr learned my addr 172.21.15.164:0/977556654 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:04.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f266ca00640 1 --2- 172.21.15.164:0/977556654 >> v2:172.21.15.172:3300/0 conn(0x7f26680cfda0 0x7f26680cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 -- 172.21.15.164:0/977556654 >> v2:172.21.15.172:3300/0 conn(0x7f26680cfda0 msgr2=0x7f26680cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:04.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 --2- 172.21.15.164:0/977556654 >> v2:172.21.15.172:3300/0 conn(0x7f26680cfda0 0x7f26680cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 -- 172.21.15.164:0/977556654 >> v1:172.21.15.172:6789/0 conn(0x7f26680b26c0 legacy=0x7f26680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 -- 172.21.15.164:0/977556654 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266806cd30 con 0x7f266815ab60 2024-05-08T04:42:04.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.684+0000 7f2667e00640 1 --2- 172.21.15.164:0/977556654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f265801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba57e160d5e0f25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2666a00640 1 -- 172.21.15.164:0/977556654 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580a5700 con 0x7f266815ab60 2024-05-08T04:42:04.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2666a00640 1 -- 172.21.15.164:0/977556654 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26580a58a0 con 0x7f266815ab60 2024-05-08T04:42:04.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2666a00640 1 -- 172.21.15.164:0/977556654 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580a5c70 con 0x7f266815ab60 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- 172.21.15.164:0/977556654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 msgr2=0x7f2668072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- 172.21.15.164:0/977556654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2658009e50 tx=0x7f265801ca30 comp rx=0 tx=0).stop 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- 172.21.15.164:0/977556654 shutdown_connections 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- 172.21.15.164:0/977556654 >> v2:172.21.15.172:3300/0 conn(0x7f26680cfda0 0x7f26680cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- 172.21.15.164:0/977556654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f266815ab60 0x7f2668072100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- 172.21.15.164:0/977556654 >> 172.21.15.164:0/977556654 conn(0x7f266806ac90 msgr2=0x7f266806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:04.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- 172.21.15.164:0/977556654 shutdown_connections 2024-05-08T04:42:04.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- 172.21.15.164:0/977556654 wait complete. 2024-05-08T04:42:04.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 Processor -- start 2024-05-08T04:42:04.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- start start 2024-05-08T04:42:04.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26680cfda0 0x7f26681554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266815ab60 0x7f266814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26680aeeb0 con 0x7f26680b26c0 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26680aed30 con 0x7f266815ab60 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f266ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2668167970 con 0x7f26680cfda0 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50622/0 (socket says 172.21.15.164:50622) 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2667e00640 1 -- 172.21.15.164:0/3766607496 learned_addr learned my addr 172.21.15.164:0/3766607496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.688+0000 7f2667400640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26680cfda0 0x7f26681554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f266ca00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266815ab60 0x7f266814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 -- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26680cfda0 msgr2=0x7f26681554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26680cfda0 0x7f26681554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 -- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266815ab60 msgr2=0x7f266814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266815ab60 0x7f266814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 -- 172.21.15.164:0/3766607496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266815b050 con 0x7f26680b26c0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667e00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f265801a660 tx=0x7f26580197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658009510 con 0x7f26680b26c0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26580b8470 con 0x7f26680b26c0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580b8860 con 0x7f26680b26c0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26681568c0 con 0x7f26680b26c0 2024-05-08T04:42:04.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266815b5f0 con 0x7f26680b26c0 2024-05-08T04:42:04.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2668069d60 con 0x7f26680b26c0 2024-05-08T04:42:04.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f26580c0020 con 0x7f26680b26c0 2024-05-08T04:42:04.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 0x7f2648069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:04.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f26581205c0 con 0x7f26680b26c0 2024-05-08T04:42:04.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.692+0000 7f2667400640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 0x7f2648069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:04.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.696+0000 7f2667400640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 0x7f2648069b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26681597c0 tx=0x7f2654009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:04.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.700+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26580c7020 con 0x7f26680b26c0 2024-05-08T04:42:04.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.864+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f266815b7c0 con 0x7f26680b26c0 2024-05-08T04:42:04.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.864+0000 7f265fe00640 1 -- 172.21.15.164:0/3766607496 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f2658016620 con 0x7f26680b26c0 2024-05-08T04:42:04.870 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:04.870 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":87,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:04.308390+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":11,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 msgr2=0x7f2648069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 0x7f2648069b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26681597c0 tx=0x7f2654009040 comp rx=0 tx=0).stop 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 msgr2=0x7f2668156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f265801a660 tx=0x7f26580197d0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 shutdown_connections 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f26480676b0 0x7f2648069b70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26680cfda0 0x7f26681554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266815ab60 0x7f266814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 --2- 172.21.15.164:0/3766607496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26680b26c0 0x7f2668156000 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:04.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 >> 172.21.15.164:0/3766607496 conn(0x7f266806ac90 msgr2=0x7f266815eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:04.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 shutdown_connections 2024-05-08T04:42:04.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:04.868+0000 7f266ea00640 1 -- 172.21.15.164:0/3766607496 wait complete. 2024-05-08T04:42:04.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:41.603797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:42.235153+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 11, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:04.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:05.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 Processor -- start 2024-05-08T04:42:05.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 -- start start 2024-05-08T04:42:05.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ee015ce10 0x7f3ee006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee00c6600 con 0x7f3ee006e390 2024-05-08T04:42:05.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee00c6040 con 0x7f3ee015ba20 2024-05-08T04:42:05.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ee6400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee015fbe0 con 0x7f3ee015ce10 2024-05-08T04:42:05.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3ee006e390 0x7f3ee00ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49816/0 (socket says 172.21.15.164:49816) 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edfe00640 1 -- 172.21.15.164:0/3668254462 learned_addr learned my addr 172.21.15.164:0/3668254462 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edea00640 1 --2- 172.21.15.164:0/3668254462 >> v2:172.21.15.172:3300/0 conn(0x7f3ee015ce10 0x7f3ee006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402420341 0 0) 0x7f3ee00c6600 con 0x7f3ee006e390 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec4003530 con 0x7f3ee006e390 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597906925 0 0) 0x7f3ee00c6040 con 0x7f3ee015ba20 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee00c6600 con 0x7f3ee015ba20 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edea00640 1 -- 172.21.15.164:0/3668254462 >> v1:172.21.15.172:6789/0 conn(0x7f3ee015ba20 legacy=0x7f3ee0160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edea00640 1 -- 172.21.15.164:0/3668254462 >> v1:172.21.15.164:6789/0 conn(0x7f3ee006e390 legacy=0x7f3ee00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.032+0000 7f3edea00640 1 -- 172.21.15.164:0/3668254462 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00c6320 con 0x7f3ee015ce10 2024-05-08T04:42:05.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3edea00640 1 --2- 172.21.15.164:0/3668254462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ce10 0x7f3ee006de50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097c0 tx=0x7f3ecc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e153c0137d833967 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0b0070 con 0x7f3ee015ce10 2024-05-08T04:42:05.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0ab600 con 0x7f3ee015ce10 2024-05-08T04:42:05.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ede000640 1 -- 172.21.15.164:0/3668254462 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0ab9b0 con 0x7f3ee015ce10 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 -- 172.21.15.164:0/3668254462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ce10 msgr2=0x7f3ee006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 --2- 172.21.15.164:0/3668254462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ce10 0x7f3ee006de50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097c0 tx=0x7f3ecc01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 -- 172.21.15.164:0/3668254462 shutdown_connections 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 --2- 172.21.15.164:0/3668254462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ce10 0x7f3ee006de50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 -- 172.21.15.164:0/3668254462 >> 172.21.15.164:0/3668254462 conn(0x7f3ee00cf770 msgr2=0x7f3ee00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 -- 172.21.15.164:0/3668254462 shutdown_connections 2024-05-08T04:42:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.036+0000 7f3ee6400640 1 -- 172.21.15.164:0/3668254462 wait complete. 2024-05-08T04:42:05.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 Processor -- start 2024-05-08T04:42:05.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- start start 2024-05-08T04:42:05.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ba20 0x7f3ee00718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 0x7f3ee0153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00ce000 con 0x7f3ee006e390 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00cde80 con 0x7f3ee015ba20 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0169b80 con 0x7f3ee015ce10 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50626/0 (socket says 172.21.15.164:50626) 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 -- 172.21.15.164:0/732763791 learned_addr learned my addr 172.21.15.164:0/732763791 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edea00640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ba20 0x7f3ee00718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edfe00640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 0x7f3ee0153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 -- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 msgr2=0x7f3ee0153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 0x7f3ee0153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 -- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ba20 msgr2=0x7f3ee00718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ba20 0x7f3ee00718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 -- 172.21.15.164:0/732763791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee01487c0 con 0x7f3ee006e390 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edfe00640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 0x7f3ee0153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3edf400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c4c0 tx=0x7f3ed000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00052d0 con 0x7f3ee006e390 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0005470 con 0x7f3ee006e390 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0012870 con 0x7f3ee006e390 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee0070830 con 0x7f3ee006e390 2024-05-08T04:42:05.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.040+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee0070df0 con 0x7f3ee006e390 2024-05-08T04:42:05.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee0072d30 con 0x7f3ee006e390 2024-05-08T04:42:05.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f3ed0012a10 con 0x7f3ee006e390 2024-05-08T04:42:05.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3ed7400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 0x7f3ec0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f3ed0085dd0 con 0x7f3ee006e390 2024-05-08T04:42:05.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3edea00640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 0x7f3ec0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.044+0000 7f3edea00640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 0x7f3ec0069b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0ad000 tx=0x7f3ecc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.048+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ed0055ec0 con 0x7f3ee006e390 2024-05-08T04:42:05.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ee00cea50 con 0x7f3ee006e390 2024-05-08T04:42:05.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ed7400640 1 -- 172.21.15.164:0/732763791 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ed00557c0 con 0x7f3ee006e390 2024-05-08T04:42:05.211 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 msgr2=0x7f3ec0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 0x7f3ec0069b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0ad000 tx=0x7f3ecc008040 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 msgr2=0x7f3ee0148280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.204+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c4c0 tx=0x7f3ed000c980 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 shutdown_connections 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3ec00676b0 0x7f3ec0069b70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee015ce10 0x7f3ee0153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee015ba20 0x7f3ee00718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 --2- 172.21.15.164:0/732763791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ee006e390 0x7f3ee0148280 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 >> 172.21.15.164:0/732763791 conn(0x7f3ee00cf770 msgr2=0x7f3ee006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 shutdown_connections 2024-05-08T04:42:05.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.208+0000 7f3ee6400640 1 -- 172.21.15.164:0/732763791 wait complete. 2024-05-08T04:42:05.231 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:05.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.364+0000 7fe7a9c00640 1 Processor -- start 2024-05-08T04:42:05.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.364+0000 7fe7a9c00640 1 -- start start 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe7a415dca0 0x7fe7a40b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a4161ad0 con 0x7fe7a40d1df0 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a4161db0 con 0x7fe7a40b0f00 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a4161880 con 0x7fe7a415dca0 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe7a40d1df0 0x7fe7a406c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49818/0 (socket says 172.21.15.164:49818) 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a9200640 1 -- 172.21.15.164:0/276645893 learned_addr learned my addr 172.21.15.164:0/276645893 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:05.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3e00640 1 --2- 172.21.15.164:0/276645893 >> v2:172.21.15.172:3300/0 conn(0x7fe7a415dca0 0x7fe7a40b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209648808 0 0) 0x7fe7a4161ad0 con 0x7fe7a40d1df0 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe788003530 con 0x7fe7a40d1df0 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3e00640 1 -- 172.21.15.164:0/276645893 >> v1:172.21.15.172:6789/0 conn(0x7fe7a40b0f00 legacy=0x7fe7a40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074104503 0 0) 0x7fe7a4161db0 con 0x7fe7a40b0f00 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3e00640 1 -- 172.21.15.164:0/276645893 >> v1:172.21.15.164:6789/0 conn(0x7fe7a40d1df0 legacy=0x7fe7a406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.368+0000 7fe7a3e00640 1 -- 172.21.15.164:0/276645893 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4162090 con 0x7fe7a415dca0 2024-05-08T04:42:05.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a3e00640 1 --2- 172.21.15.164:0/276645893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a415dca0 0x7fe7a40b0820 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0097f0 tx=0x7fe78c01c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654be4079da170d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0b0070 con 0x7fe7a415dca0 2024-05-08T04:42:05.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c0ab400 con 0x7fe7a415dca0 2024-05-08T04:42:05.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a3400640 1 -- 172.21.15.164:0/276645893 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c001a00 con 0x7fe7a415dca0 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- 172.21.15.164:0/276645893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a415dca0 msgr2=0x7fe7a40b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/276645893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a415dca0 0x7fe7a40b0820 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0097f0 tx=0x7fe78c01c320 comp rx=0 tx=0).stop 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- 172.21.15.164:0/276645893 shutdown_connections 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/276645893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a415dca0 0x7fe7a40b0820 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- 172.21.15.164:0/276645893 >> 172.21.15.164:0/276645893 conn(0x7fe7a40723e0 msgr2=0x7fe7a40727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- 172.21.15.164:0/276645893 shutdown_connections 2024-05-08T04:42:05.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- 172.21.15.164:0/276645893 wait complete. 2024-05-08T04:42:05.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 Processor -- start 2024-05-08T04:42:05.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- start start 2024-05-08T04:42:05.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a40b0f00 0x7fe7a4149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7a40d1df0 0x7fe7a414ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a406d790 con 0x7fe7a415dca0 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a406d610 con 0x7fe7a40b0f00 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.372+0000 7fe7a9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a406d910 con 0x7fe7a40d1df0 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50642/0 (socket says 172.21.15.164:50642) 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 -- 172.21.15.164:0/3346089770 learned_addr learned my addr 172.21.15.164:0/3346089770 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe79b400640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7a40d1df0 0x7fe7a414ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a3e00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a40b0f00 0x7fe7a4149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 -- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7a40d1df0 msgr2=0x7fe7a414ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7a40d1df0 0x7fe7a414ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 -- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a40b0f00 msgr2=0x7fe7a4149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a40b0f00 0x7fe7a4149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 -- 172.21.15.164:0/3346089770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a41388b0 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9200640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00cb30 tx=0x7fe79c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c018070 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c00a830 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00ac20 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a414ba20 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a414bfe0 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a404ef30 con 0x7fe7a415dca0 2024-05-08T04:42:05.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fe79c01b070 con 0x7fe7a415dca0 2024-05-08T04:42:05.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.376+0000 7fe7a0c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 0x7fe780069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.380+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fe79c085ac0 con 0x7fe7a415dca0 2024-05-08T04:42:05.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.380+0000 7fe7a3e00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 0x7fe780069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.380+0000 7fe7a3e00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 0x7fe780069b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0ad000 tx=0x7fe78c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.380+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe79c055560 con 0x7fe7a415dca0 2024-05-08T04:42:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7a414d470 con 0x7fe7a415dca0 2024-05-08T04:42:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a0c00640 1 -- 172.21.15.164:0/3346089770 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe79c059600 con 0x7fe7a415dca0 2024-05-08T04:42:05.544 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:05.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 msgr2=0x7fe780069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 0x7fe780069b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0ad000 tx=0x7fe78c0a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 msgr2=0x7fe7a4138370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00cb30 tx=0x7fe79c00a060 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 shutdown_connections 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7800676b0 0x7fe780069b70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7a40d1df0 0x7fe7a414ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7a415dca0 0x7fe7a4138370 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 --2- 172.21.15.164:0/3346089770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7a40b0f00 0x7fe7a4149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 >> 172.21.15.164:0/3346089770 conn(0x7fe7a40723e0 msgr2=0x7fe7a40cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 shutdown_connections 2024-05-08T04:42:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.540+0000 7fe7a9c00640 1 -- 172.21.15.164:0/3346089770 wait complete. 2024-05-08T04:42:05.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:05.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 Processor -- start 2024-05-08T04:42:05.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 -- start start 2024-05-08T04:42:05.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae580ae6e0 0x7fae580aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5815e870 con 0x7fae580ce780 2024-05-08T04:42:05.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5806bb80 con 0x7fae580ae6e0 2024-05-08T04:42:05.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae5e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5815eb50 con 0x7fae5815ab10 2024-05-08T04:42:05.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:56046/0 (socket says 172.21.15.164:56046) 2024-05-08T04:42:05.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 -- 172.21.15.164:0/468703548 learned_addr learned my addr 172.21.15.164:0/468703548 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:05.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae56a00640 1 --2- 172.21.15.164:0/468703548 >> v2:172.21.15.172:3300/0 conn(0x7fae580ae6e0 0x7fae580aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 -- 172.21.15.164:0/468703548 >> v2:172.21.15.172:3300/0 conn(0x7fae580ae6e0 msgr2=0x7fae580aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:05.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 --2- 172.21.15.164:0/468703548 >> v2:172.21.15.172:3300/0 conn(0x7fae580ae6e0 0x7fae580aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 -- 172.21.15.164:0/468703548 >> v1:172.21.15.172:6789/0 conn(0x7fae580ce780 legacy=0x7fae580c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.700+0000 7fae57400640 1 -- 172.21.15.164:0/468703548 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5815ee30 con 0x7fae5815ab10 2024-05-08T04:42:05.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae57400640 1 --2- 172.21.15.164:0/468703548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae48009870 tx=0x7fae4801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2002442be98a9b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae56000640 1 -- 172.21.15.164:0/468703548 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480aa600 con 0x7fae5815ab10 2024-05-08T04:42:05.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae56000640 1 -- 172.21.15.164:0/468703548 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae480af070 con 0x7fae5815ab10 2024-05-08T04:42:05.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae56000640 1 -- 172.21.15.164:0/468703548 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480aa9c0 con 0x7fae5815ab10 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 -- 172.21.15.164:0/468703548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 msgr2=0x7fae5806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 --2- 172.21.15.164:0/468703548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae48009870 tx=0x7fae4801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 -- 172.21.15.164:0/468703548 shutdown_connections 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 --2- 172.21.15.164:0/468703548 >> v2:172.21.15.172:3300/0 conn(0x7fae580ae6e0 0x7fae580aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 --2- 172.21.15.164:0/468703548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae5815ab10 0x7fae5806cce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 -- 172.21.15.164:0/468703548 >> 172.21.15.164:0/468703548 conn(0x7fae580cf770 msgr2=0x7fae580cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 -- 172.21.15.164:0/468703548 shutdown_connections 2024-05-08T04:42:05.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.704+0000 7fae5e800640 1 -- 172.21.15.164:0/468703548 wait complete. 2024-05-08T04:42:05.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 Processor -- start 2024-05-08T04:42:05.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- start start 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae580ae6e0 0x7fae580c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae5815ab10 0x7fae58153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cefa0 con 0x7fae580ce780 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cd670 con 0x7fae5815ab10 2024-05-08T04:42:05.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580c7a30 con 0x7fae580ae6e0 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50648/0 (socket says 172.21.15.164:50648) 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 -- 172.21.15.164:0/1765276043 learned_addr learned my addr 172.21.15.164:0/1765276043 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae57400640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae580ae6e0 0x7fae580c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae57e00640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae5815ab10 0x7fae58153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 -- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae580ae6e0 msgr2=0x7fae580c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae580ae6e0 0x7fae580c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 -- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae5815ab10 msgr2=0x7fae58153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae5815ab10 0x7fae58153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 -- 172.21.15.164:0/1765276043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580c6e20 con 0x7fae580ce780 2024-05-08T04:42:05.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae56a00640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e960 tx=0x7fae4400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44017070 con 0x7fae580ce780 2024-05-08T04:42:05.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4400ca00 con 0x7fae580ce780 2024-05-08T04:42:05.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4400cdb0 con 0x7fae580ce780 2024-05-08T04:42:05.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae580c7100 con 0x7fae580ce780 2024-05-08T04:42:05.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580c76c0 con 0x7fae580ce780 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.708+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae58072d30 con 0x7fae580ce780 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.712+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fae4401a070 con 0x7fae580ce780 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.712+0000 7fae4f400640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 0x7fae38069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.712+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fae440865d0 con 0x7fae580ce780 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.712+0000 7fae57400640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 0x7fae38069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:05.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.712+0000 7fae57400640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 0x7fae38069b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae480ae000 tx=0x7fae480a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:05.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.716+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae44055fe0 con 0x7fae580ce780 2024-05-08T04:42:05.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.868+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae5804f010 con 0x7fae580ce780 2024-05-08T04:42:05.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.868+0000 7fae4f400640 1 -- 172.21.15.164:0/1765276043 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae4405a080 con 0x7fae580ce780 2024-05-08T04:42:05.875 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 msgr2=0x7fae38069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 0x7fae38069b70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae480ae000 tx=0x7fae480a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 msgr2=0x7fae580c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e960 tx=0x7fae4400ee20 comp rx=0 tx=0).stop 2024-05-08T04:42:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 shutdown_connections 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fae380676b0 0x7fae38069b70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae5815ab10 0x7fae58153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae580ce780 0x7fae580c7ec0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 --2- 172.21.15.164:0/1765276043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae580ae6e0 0x7fae580c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 >> 172.21.15.164:0/1765276043 conn(0x7fae580cf770 msgr2=0x7fae580af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 shutdown_connections 2024-05-08T04:42:05.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:05.872+0000 7fae5e800640 1 -- 172.21.15.164:0/1765276043 wait complete. 2024-05-08T04:42:05.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:42:06.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.028+0000 7fae01600640 1 Processor -- start 2024-05-08T04:42:06.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.028+0000 7fae01600640 1 -- start start 2024-05-08T04:42:06.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae01600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc15aab0 0x7fadfc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae01600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fadfc0cc5e0 0x7fadfc0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae01600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc0ccc40 con 0x7fadfc0c5530 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae01600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc15ae90 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae01600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0c5ed0 con 0x7fadfc15aab0 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fadfb400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fadfc0c5530 0x7fadfc0cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49820/0 (socket says 172.21.15.164:49820) 2024-05-08T04:42:06.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fadfb400640 1 -- 172.21.15.164:0/3531968914 learned_addr learned my addr 172.21.15.164:0/3531968914 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae00c00640 1 --2- 172.21.15.164:0/3531968914 >> v2:172.21.15.172:3300/0 conn(0x7fadfc0cc5e0 0x7fadfc0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fadfbe00640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc15aab0 0x7fadfc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fadfaa00640 1 -- 172.21.15.164:0/3531968914 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966592674 0 0) 0x7fadfc0ccc40 con 0x7fadfc0c5530 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fadfaa00640 1 -- 172.21.15.164:0/3531968914 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade0003530 con 0x7fadfc0c5530 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae00c00640 1 -- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc15aab0 msgr2=0x7fadfc0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:06.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae00c00640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc15aab0 0x7fadfc0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae00c00640 1 -- 172.21.15.164:0/3531968914 >> v1:172.21.15.164:6789/0 conn(0x7fadfc0c5530 legacy=0x7fadfc0cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.032+0000 7fae00c00640 1 -- 172.21.15.164:0/3531968914 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0ce170 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae00c00640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc0cc5e0 0x7fadfc0c5990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097c0 tx=0x7fadf401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47291a719a6b08e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fadfaa00640 1 -- 172.21.15.164:0/3531968914 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40b0070 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fadfaa00640 1 -- 172.21.15.164:0/3531968914 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf40ab600 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fadfaa00640 1 -- 172.21.15.164:0/3531968914 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40ab9d0 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc0cc5e0 msgr2=0x7fadfc0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc0cc5e0 0x7fadfc0c5990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097c0 tx=0x7fadf401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- 172.21.15.164:0/3531968914 shutdown_connections 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc0cc5e0 0x7fadfc0c5990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- 172.21.15.164:0/3531968914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc15aab0 0x7fadfc0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- 172.21.15.164:0/3531968914 >> 172.21.15.164:0/3531968914 conn(0x7fadfc06b670 msgr2=0x7fadfc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- 172.21.15.164:0/3531968914 shutdown_connections 2024-05-08T04:42:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- 172.21.15.164:0/3531968914 wait complete. 2024-05-08T04:42:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 Processor -- start 2024-05-08T04:42:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- start start 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cc5e0 0x7fadfc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc15aab0 0x7fadfc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0724b0 con 0x7fadfc0c5530 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0713f0 con 0x7fadfc15aab0 2024-05-08T04:42:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.036+0000 7fae01600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0ce7c0 con 0x7fadfc0cc5e0 2024-05-08T04:42:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50664/0 (socket says 172.21.15.164:50664) 2024-05-08T04:42:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 -- 172.21.15.164:0/2530912003 learned_addr learned my addr 172.21.15.164:0/2530912003 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 -- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cc5e0 msgr2=0x7fadfc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cc5e0 0x7fadfc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 -- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc15aab0 msgr2=0x7fadfc0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc15aab0 0x7fadfc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 -- 172.21.15.164:0/2530912003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc15ca40 con 0x7fadfc0c5530 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadfbe00640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fadec00efc0 tx=0x7fadec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec018020 con 0x7fadfc0c5530 2024-05-08T04:42:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec014480 con 0x7fadfc0c5530 2024-05-08T04:42:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec014850 con 0x7fadfc0c5530 2024-05-08T04:42:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc15cd20 con 0x7fadfc0c5530 2024-05-08T04:42:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0b27c0 con 0x7fadfc0c5530 2024-05-08T04:42:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc04ef30 con 0x7fadfc0c5530 2024-05-08T04:42:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.040+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fadec01b020 con 0x7fadfc0c5530 2024-05-08T04:42:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.044+0000 7fadf3e00640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 0x7faddc069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.044+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fadec086c30 con 0x7fadfc0c5530 2024-05-08T04:42:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.044+0000 7fadfb400640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 0x7faddc069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.044+0000 7fadfb400640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 0x7faddc069ba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.048+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fadec022030 con 0x7fadfc0c5530 2024-05-08T04:42:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fadfc0cd860 con 0x7fadfc0c5530 2024-05-08T04:42:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fadf3e00640 1 -- 172.21.15.164:0/2530912003 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7fadec010040 con 0x7fadfc0c5530 2024-05-08T04:42:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:42:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 msgr2=0x7faddc069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 0x7faddc069ba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8007900 comp rx=0 tx=0).stop 2024-05-08T04:42:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 msgr2=0x7fadfc146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fadec00efc0 tx=0x7fadec00c490 comp rx=0 tx=0).stop 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 shutdown_connections 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faddc0676e0 0x7faddc069ba0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadfc0cc5e0 0x7fadfc0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadfc15aab0 0x7fadfc0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 --2- 172.21.15.164:0/2530912003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fadfc0c5530 0x7fadfc146030 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 >> 172.21.15.164:0/2530912003 conn(0x7fadfc06b670 msgr2=0x7fadfc0cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 shutdown_connections 2024-05-08T04:42:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.224+0000 7fae01600640 1 -- 172.21.15.164:0/2530912003 wait complete. 2024-05-08T04:42:06.251 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:42:06.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:06.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:06.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:42:06.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:06.282 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:42:06.335 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:06.335 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:06.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:06.344 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:06.344 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:06.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:06.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:06.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 Processor -- start 2024-05-08T04:42:06.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 -- start start 2024-05-08T04:42:06.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f05240b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240b42a0 con 0x7f05240d0780 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240cd4e0 con 0x7f05240ccfe0 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f052a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cd7c0 con 0x7f052415bc50 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f05240d0780 0x7f052415f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49832/0 (socket says 172.21.15.164:49832) 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523e00640 1 -- 172.21.15.164:0/1180263698 learned_addr learned my addr 172.21.15.164:0/1180263698 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523400640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f05240b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744702495 0 0) 0x7f05240b42a0 con 0x7f05240d0780 2024-05-08T04:42:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0508003530 con 0x7f05240d0780 2024-05-08T04:42:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894174880 0 0) 0x7f05240cd4e0 con 0x7f05240ccfe0 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05240b42a0 con 0x7f05240ccfe0 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1805361050 0 0) 0x7f0508003530 con 0x7f05240d0780 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 msgr2=0x7f05240b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f05240b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 >> v1:172.21.15.172:6789/0 conn(0x7f05240ccfe0 legacy=0x7f05240d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240cf590 con 0x7f05240d0780 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523400640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f05240b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f051c001a00 con 0x7f05240d0780 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 >> v1:172.21.15.164:6789/0 conn(0x7f05240d0780 legacy=0x7f052415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0508003b80 0x7f0508003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f0508004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cd7c0 con 0x7f0508004d50 2024-05-08T04:42:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050800bab0 con 0x7f0508003b80 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050800bcb0 con 0x7f052415bc50 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523e00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f0523400640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0508003b80 0x7f0508003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.520+0000 7f051aa00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f0508004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 -- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 msgr2=0x7f0508004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f052415bc50 0x7f0508004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 -- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0508003b80 msgr2=0x7f0508003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0508003b80 0x7f0508003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 -- 172.21.15.164:0/1180263698 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508003530 con 0x7f0508004d50 2024-05-08T04:42:06.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05080051d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f051c00a800 tx=0x7f051c01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6b670222cbc83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00a430 con 0x7f0508004d50 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0ad630 con 0x7f0508004d50 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0522a00640 1 -- 172.21.15.164:0/1180263698 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0ada20 con 0x7f0508004d50 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f052a600640 1 -- 172.21.15.164:0/1180263698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 msgr2=0x7f05080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f052a600640 1 --2- 172.21.15.164:0/1180263698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05080051d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f051c00a800 tx=0x7f051c01d300 comp rx=0 tx=0).stop 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f0523e00640 1 -- 172.21.15.164:0/1180263698 reap_dead start 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f052a600640 1 -- 172.21.15.164:0/1180263698 shutdown_connections 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.524+0000 7f052a600640 1 -- 172.21.15.164:0/1180263698 >> 172.21.15.164:0/1180263698 conn(0x7f05240b34f0 msgr2=0x7f05240b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- 172.21.15.164:0/1180263698 shutdown_connections 2024-05-08T04:42:06.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- 172.21.15.164:0/1180263698 wait complete. 2024-05-08T04:42:06.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 Processor -- start 2024-05-08T04:42:06.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- start start 2024-05-08T04:42:06.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 0x7f052415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05240c7bb0 0x7f05240c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052415c030 con 0x7f0508004d50 2024-05-08T04:42:06.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cfb50 con 0x7f052415e770 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052415feb0 con 0x7f05240c7bb0 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50680/0 (socket says 172.21.15.164:50680) 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 -- 172.21.15.164:0/3297623126 learned_addr learned my addr 172.21.15.164:0/3297623126 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f051aa00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 0x7f052415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523e00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05240c7bb0 0x7f05240c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 -- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05240c7bb0 msgr2=0x7f05240c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05240c7bb0 0x7f05240c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 -- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 msgr2=0x7f052415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 0x7f052415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 -- 172.21.15.164:0/3297623126 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524156a20 con 0x7f0508004d50 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f051aa00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 0x7f052415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:06.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f0523400640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f05140119d0 tx=0x7f0514011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400faf0 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051400fc90 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.528+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05240cb280 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514004030 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240ca950 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0524069d60 con 0x7f0508004d50 2024-05-08T04:42:06.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f0514012040 con 0x7f0508004d50 2024-05-08T04:42:06.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051be00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 0x7f0500069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f0514086550 con 0x7f0508004d50 2024-05-08T04:42:06.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051aa00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 0x7f0500069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.532+0000 7f051aa00640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 0x7f0500069ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f052415b650 tx=0x7f050c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.536+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0514019070 con 0x7f0508004d50 2024-05-08T04:42:06.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.724+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05240b7940 con 0x7f0508004d50 2024-05-08T04:42:06.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.724+0000 7f051be00640 1 -- 172.21.15.164:0/3297623126 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f0514056640 con 0x7f0508004d50 2024-05-08T04:42:06.731 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:06.731 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:42:06.733 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 53 2024-05-08T04:42:06.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 msgr2=0x7f0500069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 0x7f0500069ba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f052415b650 tx=0x7f050c009040 comp rx=0 tx=0).stop 2024-05-08T04:42:06.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 msgr2=0x7f05241564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f05140119d0 tx=0x7f0514011e90 comp rx=0 tx=0).stop 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 shutdown_connections 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05000676e0 0x7f0500069ba0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05240c7bb0 0x7f05240c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052415e770 0x7f052415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 --2- 172.21.15.164:0/3297623126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0508004d50 0x7f05241564e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 >> 172.21.15.164:0/3297623126 conn(0x7f05240b34f0 msgr2=0x7f05240c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:06.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 shutdown_connections 2024-05-08T04:42:06.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.728+0000 7f052a600640 1 -- 172.21.15.164:0/3297623126 wait complete. 2024-05-08T04:42:06.751 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:06.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 Processor -- start 2024-05-08T04:42:06.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 -- start start 2024-05-08T04:42:06.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa99015db30 0x7fa99006d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9900afc90 0x7fa99006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900b3660 con 0x7fa9900720a0 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9900b3920 con 0x7fa9900afc90 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99006ed40 con 0x7fa99015db30 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9900afc90 0x7fa99006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa99015db30 0x7fa99006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995200640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa9900720a0 0x7fa9900d0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:41598/0 (socket says 172.21.15.164:41598) 2024-05-08T04:42:06.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9900afc90 0x7fa99006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:45394/0 (socket says 172.21.15.164:45394) 2024-05-08T04:42:06.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa995200640 1 -- 172.21.15.164:0/4019956193 learned_addr learned my addr 172.21.15.164:0/4019956193 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:06.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 -- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa99015db30 msgr2=0x7fa99006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:06.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 --2- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa99015db30 0x7fa99006d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 -- 172.21.15.164:0/4019956193 >> v1:172.21.15.172:6789/0 conn(0x7fa9900720a0 legacy=0x7fa9900d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.884+0000 7fa98f400640 1 -- 172.21.15.164:0/4019956193 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99006f020 con 0x7fa9900afc90 2024-05-08T04:42:06.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa98f400640 1 --2- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99006ac90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa6aca6b08a2f0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa98ea00640 1 -- 172.21.15.164:0/4019956193 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0b0070 con 0x7fa9900afc90 2024-05-08T04:42:06.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa98ea00640 1 -- 172.21.15.164:0/4019956193 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c0ab600 con 0x7fa9900afc90 2024-05-08T04:42:06.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa98ea00640 1 -- 172.21.15.164:0/4019956193 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0ab9d0 con 0x7fa9900afc90 2024-05-08T04:42:06.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 -- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 msgr2=0x7fa99006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 --2- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99006ac90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c01c410 comp rx=0 tx=0).stop 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 -- 172.21.15.164:0/4019956193 shutdown_connections 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 --2- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99006ac90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 --2- 172.21.15.164:0/4019956193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa99015db30 0x7fa99006d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 -- 172.21.15.164:0/4019956193 >> 172.21.15.164:0/4019956193 conn(0x7fa990066fa0 msgr2=0x7fa99006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 -- 172.21.15.164:0/4019956193 shutdown_connections 2024-05-08T04:42:06.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.888+0000 7fa995c00640 1 -- 172.21.15.164:0/4019956193 wait complete. 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 Processor -- start 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- start start 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9900720a0 0x7fa99014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99016b080 con 0x7fa99015db30 2024-05-08T04:42:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99006b5e0 con 0x7fa9900afc90 2024-05-08T04:42:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99016b200 con 0x7fa9900720a0 2024-05-08T04:42:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50686/0 (socket says 172.21.15.164:50686) 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 -- 172.21.15.164:0/857410535 learned_addr learned my addr 172.21.15.164:0/857410535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa98f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa98fe00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9900720a0 0x7fa99014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 -- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9900720a0 msgr2=0x7fa99014cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9900720a0 0x7fa99014cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 -- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 msgr2=0x7fa99015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:06.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:06.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 -- 172.21.15.164:0/857410535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99014e3e0 con 0x7fa99015db30 2024-05-08T04:42:06.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa98f400640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:06.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995200640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa98800ca80 tx=0x7fa98800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988017070 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98800ac30 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988012600 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa990160e70 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.892+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99015ee20 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99004ef30 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fa98801a070 con 0x7fa99015db30 2024-05-08T04:42:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa987e00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 0x7fa970069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fa988085dc0 con 0x7fa99015db30 2024-05-08T04:42:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa98fe00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 0x7fa970069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.896+0000 7fa98fe00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 0x7fa970069b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa99006d760 tx=0x7fa980009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:06.900+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa99014e3e0 con 0x7fa99015db30 2024-05-08T04:42:07.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.056+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa990067820 con 0x7fa99015db30 2024-05-08T04:42:07.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.056+0000 7fa987e00640 1 -- 172.21.15.164:0/857410535 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16922 (secure 0 0 0) 0x7fa98800ae20 con 0x7fa99015db30 2024-05-08T04:42:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":87,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:04.308390+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":11,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 msgr2=0x7fa970069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 0x7fa970069b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa99006d760 tx=0x7fa980009190 comp rx=0 tx=0).stop 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 msgr2=0x7fa990160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa98800ca80 tx=0x7fa98800cf40 comp rx=0 tx=0).stop 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 shutdown_connections 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa9700676b0 0x7fa970069b70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99015db30 0x7fa990160790 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9900afc90 0x7fa99015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 --2- 172.21.15.164:0/857410535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9900720a0 0x7fa99014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 >> 172.21.15.164:0/857410535 conn(0x7fa990066fa0 msgr2=0x7fa9900b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 shutdown_connections 2024-05-08T04:42:07.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.060+0000 7fa995c00640 1 -- 172.21.15.164:0/857410535 wait complete. 2024-05-08T04:42:07.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:41:41.603797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:41:42.235153+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 11, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:07.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:07.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.216+0000 7f7c1c000640 1 Processor -- start 2024-05-08T04:42:07.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.216+0000 7f7c1c000640 1 -- start start 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 0x7f7c1406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c1406e390 0x7f7c140ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1c000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c140c6600 con 0x7f7c1415ba20 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1415fbe0 con 0x7f7c1406e390 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c140c6040 con 0x7f7c1415ce10 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c1406e390 0x7f7c140ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c18c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 0x7f7c1406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c1406e390 0x7f7c140ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:45416/0 (socket says 172.21.15.164:45416) 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 -- 172.21.15.164:0/3901824824 learned_addr learned my addr 172.21.15.164:0/3901824824 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:07.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c19600640 1 --1- 172.21.15.164:0/3901824824 >> v1:172.21.15.172:6789/0 conn(0x7f7c1415ba20 0x7f7c14160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:41602/0 (socket says 172.21.15.164:41602) 2024-05-08T04:42:07.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 -- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 msgr2=0x7f7c1406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:07.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 0x7f7c1406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 -- 172.21.15.164:0/3901824824 >> v1:172.21.15.172:6789/0 conn(0x7f7c1415ba20 legacy=0x7f7c14160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c1a000640 1 -- 172.21.15.164:0/3901824824 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c140c6320 con 0x7f7c1406e390 2024-05-08T04:42:07.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.220+0000 7f7c18c00640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 0x7f7c1406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:07.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1a000640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1406e390 0x7f7c140ae6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009e50 tx=0x7f7c0c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae63bd7572e591a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c13e00640 1 -- 172.21.15.164:0/3901824824 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0a5660 con 0x7f7c1406e390 2024-05-08T04:42:07.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c13e00640 1 -- 172.21.15.164:0/3901824824 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0a5800 con 0x7f7c1406e390 2024-05-08T04:42:07.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c13e00640 1 -- 172.21.15.164:0/3901824824 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0a5bd0 con 0x7f7c1406e390 2024-05-08T04:42:07.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1406e390 msgr2=0x7f7c140ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1406e390 0x7f7c140ae6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009e50 tx=0x7f7c0c01c9a0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- 172.21.15.164:0/3901824824 shutdown_connections 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ce10 0x7f7c1406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3901824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1406e390 0x7f7c140ae6e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- 172.21.15.164:0/3901824824 >> 172.21.15.164:0/3901824824 conn(0x7f7c140cf770 msgr2=0x7f7c140cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- 172.21.15.164:0/3901824824 shutdown_connections 2024-05-08T04:42:07.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- 172.21.15.164:0/3901824824 wait complete. 2024-05-08T04:42:07.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 Processor -- start 2024-05-08T04:42:07.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.224+0000 7f7c1c000640 1 -- start start 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 0x7f7c1414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 0x7f7c1414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c14168b70 con 0x7f7c1406e390 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c140ce090 con 0x7f7c1415ce10 2024-05-08T04:42:07.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c14168cf0 con 0x7f7c1415ba20 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50688/0 (socket says 172.21.15.164:50688) 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 -- 172.21.15.164:0/3669242756 learned_addr learned my addr 172.21.15.164:0/3669242756 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c18c00640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 0x7f7c1414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1a000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 0x7f7c1414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 -- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 msgr2=0x7f7c1414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 0x7f7c1414dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 -- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 msgr2=0x7f7c1414f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 0x7f7c1414f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 -- 172.21.15.164:0/3669242756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1414e1b0 con 0x7f7c1406e390 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c18c00640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 0x7f7c1414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1a000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 0x7f7c1414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:07.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c19600640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c420 tx=0x7f7c0800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080074e0 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c08007680 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c140c9140 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08004a30 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c140b2e50 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c140c9660 con 0x7f7c1406e390 2024-05-08T04:42:07.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.228+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f7c0801a020 con 0x7f7c1406e390 2024-05-08T04:42:07.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.232+0000 7f7c11600640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 0x7f7bf4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.232+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f7c08085a80 con 0x7f7c1406e390 2024-05-08T04:42:07.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.232+0000 7f7c18c00640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 0x7f7bf4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.232+0000 7f7c18c00640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 0x7f7bf4069b70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7c14070290 tx=0x7f7c00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.236+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c140c9660 con 0x7f7c1406e390 2024-05-08T04:42:07.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c14072d30 con 0x7f7c1406e390 2024-05-08T04:42:07.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c11600640 1 -- 172.21.15.164:0/3669242756 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c08055ae0 con 0x7f7c1406e390 2024-05-08T04:42:07.396 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 msgr2=0x7f7bf4069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 0x7f7bf4069b70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7c14070290 tx=0x7f7c00008040 comp rx=0 tx=0).stop 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 msgr2=0x7f7c1414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c420 tx=0x7f7c0800c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 shutdown_connections 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bf40676b0 0x7f7bf4069b70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7c1415ba20 0x7f7c1414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415ce10 0x7f7c1414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 --2- 172.21.15.164:0/3669242756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1406e390 0x7f7c1414e3c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 >> 172.21.15.164:0/3669242756 conn(0x7f7c140cf770 msgr2=0x7f7c1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 shutdown_connections 2024-05-08T04:42:07.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.392+0000 7f7c1c000640 1 -- 172.21.15.164:0/3669242756 wait complete. 2024-05-08T04:42:07.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:07.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.552+0000 7f656d600640 1 Processor -- start 2024-05-08T04:42:07.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.552+0000 7f656d600640 1 -- start start 2024-05-08T04:42:07.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f656d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f65680b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f656d600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656815e300 con 0x7f65680b4dc0 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f656d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656815e570 con 0x7f65680af6d0 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f656d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656815e850 con 0x7f65680adfa0 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f65680b4dc0 0x7f656806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49840/0 (socket says 172.21.15.164:49840) 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 -- 172.21.15.164:0/1849766968 learned_addr learned my addr 172.21.15.164:0/1849766968 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f655e000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f65680b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209342329 0 0) 0x7f656815e300 con 0x7f65680b4dc0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654c003530 con 0x7f65680b4dc0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805464787 0 0) 0x7f656815e570 con 0x7f65680af6d0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f656815e300 con 0x7f65680af6d0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3853653700 0 0) 0x7f654c003530 con 0x7f65680b4dc0 2024-05-08T04:42:07.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 msgr2=0x7f65680b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f65680b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 >> v1:172.21.15.172:6789/0 conn(0x7f65680af6d0 legacy=0x7f65680ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680b5250 con 0x7f65680b4dc0 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f6560001a00 con 0x7f65680b4dc0 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f655e000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f65680b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 >> v1:172.21.15.164:6789/0 conn(0x7f65680b4dc0 legacy=0x7f656806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f654c003b80 0x7f654c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f654c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f654c004d50 0x7f654c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656815e850 con 0x7f654c004d50 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c00bab0 con 0x7f654c003b80 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c00bcb0 con 0x7f65680adfa0 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f654c004d50 0x7f654c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f655e000640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f654c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6566a00640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f654c003b80 0x7f654c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 -- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 msgr2=0x7f654c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65680adfa0 0x7f654c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 -- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f654c003b80 msgr2=0x7f654c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f654c003b80 0x7f654c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.556+0000 7f6567400640 1 -- 172.21.15.164:0/1849766968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c003530 con 0x7f654c004d50 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f6567400640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f654c004d50 0x7f654c0051d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f656000a800 tx=0x7f656001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8b3f63b5b33f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000a430 con 0x7f654c004d50 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600ad630 con 0x7f654c004d50 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f6566000640 1 -- 172.21.15.164:0/1849766968 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600ada20 con 0x7f654c004d50 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- 172.21.15.164:0/1849766968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f654c004d50 msgr2=0x7f654c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 --2- 172.21.15.164:0/1849766968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f654c004d50 0x7f654c0051d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f656000a800 tx=0x7f656001d300 comp rx=0 tx=0).stop 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f6567400640 1 -- 172.21.15.164:0/1849766968 reap_dead start 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- 172.21.15.164:0/1849766968 shutdown_connections 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- 172.21.15.164:0/1849766968 >> 172.21.15.164:0/1849766968 conn(0x7f656806cca0 msgr2=0x7f656806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- 172.21.15.164:0/1849766968 shutdown_connections 2024-05-08T04:42:07.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- 172.21.15.164:0/1849766968 wait complete. 2024-05-08T04:42:07.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 Processor -- start 2024-05-08T04:42:07.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.560+0000 7f656d600640 1 -- start start 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 0x7f6568160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6568147de0 0x7f6568160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680b3800 con 0x7f6568147a00 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680b3680 con 0x7f656815af70 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680b2ff0 con 0x7f6568147de0 2024-05-08T04:42:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50696/0 (socket says 172.21.15.164:50696) 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 -- 172.21.15.164:0/2856494860 learned_addr learned my addr 172.21.15.164:0/2856494860 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 -- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6568147de0 msgr2=0x7f6568160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6568147de0 0x7f6568160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 -- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 msgr2=0x7f6568160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655e000640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 0x7f6568160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 0x7f6568160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 -- 172.21.15.164:0/2856494860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6568158c70 con 0x7f6568147a00 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655e000640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 0x7f6568160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f6566a00640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6558005630 tx=0x7f6558012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558010b20 con 0x7f6568147a00 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655801b070 con 0x7f6568147a00 2024-05-08T04:42:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6568158f50 con 0x7f6568147a00 2024-05-08T04:42:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558016480 con 0x7f6568147a00 2024-05-08T04:42:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6568159510 con 0x7f6568147a00 2024-05-08T04:42:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656806ddf0 con 0x7f6568147a00 2024-05-08T04:42:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.564+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f6558016620 con 0x7f6568147a00 2024-05-08T04:42:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.568+0000 7f655f400640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 0x7f6544069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.568+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f65580895f0 con 0x7f6568147a00 2024-05-08T04:42:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.568+0000 7f655e000640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 0x7f6544069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.568+0000 7f655e000640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 0x7f6544069b70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f65681f6ec0 tx=0x7f6550008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.572+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6558059aa0 con 0x7f6568147a00 2024-05-08T04:42:07.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.724+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f656804f010 con 0x7f6568147a00 2024-05-08T04:42:07.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.724+0000 7f655f400640 1 -- 172.21.15.164:0/2856494860 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65580593a0 con 0x7f6568147a00 2024-05-08T04:42:07.730 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 msgr2=0x7f6544069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 0x7f6544069b70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f65681f6ec0 tx=0x7f6550008040 comp rx=0 tx=0).stop 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 msgr2=0x7f656815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6558005630 tx=0x7f6558012d00 comp rx=0 tx=0).stop 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 shutdown_connections 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f65440676b0 0x7f6544069b70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6568147de0 0x7f6568160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f656815af70 0x7f6568160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 --2- 172.21.15.164:0/2856494860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6568147a00 0x7f656815aa30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 >> 172.21.15.164:0/2856494860 conn(0x7f656806cca0 msgr2=0x7f656806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 shutdown_connections 2024-05-08T04:42:07.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.728+0000 7f656d600640 1 -- 172.21.15.164:0/2856494860 wait complete. 2024-05-08T04:42:07.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:07.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 Processor -- start 2024-05-08T04:42:07.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 -- start start 2024-05-08T04:42:07.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f98e415a9c0 0x7f98e40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98e4072280 0x7f98e406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40b3510 con 0x7f98e415a5e0 2024-05-08T04:42:07.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40b37f0 con 0x7f98e4072280 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40b3ad0 con 0x7f98e415a9c0 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.888+0000 7f98e2800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98e4072280 0x7f98e406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98e4072280 0x7f98e406e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50698/0 (socket says 172.21.15.164:50698) 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f98e415a9c0 0x7f98e40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 -- 172.21.15.164:0/1206728577 learned_addr learned my addr 172.21.15.164:0/1206728577 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:07.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 -- 172.21.15.164:0/1206728577 >> v2:172.21.15.172:3300/0 conn(0x7f98e415a9c0 msgr2=0x7f98e40b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:07.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 --2- 172.21.15.164:0/1206728577 >> v2:172.21.15.172:3300/0 conn(0x7f98e415a9c0 0x7f98e40b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 -- 172.21.15.164:0/1206728577 >> v1:172.21.15.172:6789/0 conn(0x7f98e415a5e0 legacy=0x7f98e4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 -- 172.21.15.164:0/1206728577 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40b3db0 con 0x7f98e4072280 2024-05-08T04:42:07.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e2800640 1 --2- 172.21.15.164:0/1206728577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e4072280 0x7f98e406e060 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f98e4066c50 tx=0x7f98dc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f6efba5e0e17b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e0a00640 1 -- 172.21.15.164:0/1206728577 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0a5710 con 0x7f98e4072280 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e0a00640 1 -- 172.21.15.164:0/1206728577 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc0a58b0 con 0x7f98e4072280 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.892+0000 7f98e0a00640 1 -- 172.21.15.164:0/1206728577 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0a5c80 con 0x7f98e4072280 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- 172.21.15.164:0/1206728577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e4072280 msgr2=0x7f98e406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- 172.21.15.164:0/1206728577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e4072280 0x7f98e406e060 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f98e4066c50 tx=0x7f98dc01c960 comp rx=0 tx=0).stop 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- 172.21.15.164:0/1206728577 shutdown_connections 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- 172.21.15.164:0/1206728577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e4072280 0x7f98e406e060 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- 172.21.15.164:0/1206728577 >> v2:172.21.15.172:3300/0 conn(0x7f98e415a9c0 0x7f98e40b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- 172.21.15.164:0/1206728577 >> 172.21.15.164:0/1206728577 conn(0x7f98e40d03f0 msgr2=0x7f98e40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- 172.21.15.164:0/1206728577 shutdown_connections 2024-05-08T04:42:07.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- 172.21.15.164:0/1206728577 wait complete. 2024-05-08T04:42:07.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 Processor -- start 2024-05-08T04:42:07.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- start start 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4072280 0x7f98e415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 0x7f98e4158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40b5150 con 0x7f98e415a5e0 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40b4fd0 con 0x7f98e4072280 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40b52d0 con 0x7f98e415a9c0 2024-05-08T04:42:07.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50700/0 (socket says 172.21.15.164:50700) 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 -- 172.21.15.164:0/1438783425 learned_addr learned my addr 172.21.15.164:0/1438783425 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1e00640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4072280 0x7f98e415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e2800640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 0x7f98e4158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 -- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 msgr2=0x7f98e4158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 0x7f98e4158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 -- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4072280 msgr2=0x7f98e415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4072280 0x7f98e415add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:07.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e1400640 1 -- 172.21.15.164:0/1438783425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e41512d0 con 0x7f98e415a5e0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.896+0000 7f98e2800640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 0x7f98e4158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98e1400640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e870 tx=0x7f98d000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0017070 con 0x7f98e415a5e0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000c940 con 0x7f98e415a5e0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000ccf0 con 0x7f98e415a5e0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e414c590 con 0x7f98e415a5e0 2024-05-08T04:42:07.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e40cd210 con 0x7f98e415a5e0 2024-05-08T04:42:07.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e404ef30 con 0x7f98e415a5e0 2024-05-08T04:42:07.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f98d001a070 con 0x7f98e415a5e0 2024-05-08T04:42:07.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 0x7f98c4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:07.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f98d0085fd0 con 0x7f98e415a5e0 2024-05-08T04:42:07.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.900+0000 7f98e1e00640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 0x7f98c4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:07.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.904+0000 7f98e1e00640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 0x7f98c4069b70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f98e41656e0 tx=0x7f98d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:07.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:07.904+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98d0055ab0 con 0x7f98e415a5e0 2024-05-08T04:42:08.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.060+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98e414bf60 con 0x7f98e415a5e0 2024-05-08T04:42:08.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.060+0000 7f98da000640 1 -- 172.21.15.164:0/1438783425 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98d0059b50 con 0x7f98e415a5e0 2024-05-08T04:42:08.066 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 msgr2=0x7f98c4069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 0x7f98c4069b70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f98e41656e0 tx=0x7f98d4009040 comp rx=0 tx=0).stop 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 msgr2=0x7f98e414ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e870 tx=0x7f98d000ec70 comp rx=0 tx=0).stop 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 shutdown_connections 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98c40676b0 0x7f98c4069b70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98e415a9c0 0x7f98e4158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98e415a5e0 0x7f98e414ba20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 --2- 172.21.15.164:0/1438783425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4072280 0x7f98e415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 >> 172.21.15.164:0/1438783425 conn(0x7f98e40d03f0 msgr2=0x7f98e4070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 shutdown_connections 2024-05-08T04:42:08.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.064+0000 7f98e3200640 1 -- 172.21.15.164:0/1438783425 wait complete. 2024-05-08T04:42:08.087 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:08.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.216+0000 7fdad7200640 1 Processor -- start 2024-05-08T04:42:08.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.216+0000 7fdad7200640 1 -- start start 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdad015dd70 0x7fdad01619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 0x7fdad0161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad0162420 0x7fdad00cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad01628b0 con 0x7fdad0162420 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad006cc70 con 0x7fdad015dd70 2024-05-08T04:42:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad006cf50 con 0x7fdad006c7b0 2024-05-08T04:42:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad0162420 0x7fdad00cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad0162420 0x7fdad00cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50716/0 (socket says 172.21.15.164:50716) 2024-05-08T04:42:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 -- 172.21.15.164:0/4113149878 learned_addr learned my addr 172.21.15.164:0/4113149878 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdacf400640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 0x7fdad0161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdacfe00640 1 --2- 172.21.15.164:0/4113149878 >> v2:172.21.15.172:3300/0 conn(0x7fdad015dd70 0x7fdad01619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 -- 172.21.15.164:0/4113149878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 msgr2=0x7fdad0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 0x7fdad0161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 -- 172.21.15.164:0/4113149878 >> v2:172.21.15.172:3300/0 conn(0x7fdad015dd70 msgr2=0x7fdad01619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 --2- 172.21.15.164:0/4113149878 >> v2:172.21.15.172:3300/0 conn(0x7fdad015dd70 0x7fdad01619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdad5200640 1 -- 172.21.15.164:0/4113149878 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad00b1570 con 0x7fdad0162420 2024-05-08T04:42:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.220+0000 7fdacf400640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 0x7fdad0161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:08.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad5200640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad0162420 0x7fdad00cfc00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdad0071af0 tx=0x7fdac801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3825390cc6f4e1a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdacea00640 1 -- 172.21.15.164:0/4113149878 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80b5070 con 0x7fdad0162420 2024-05-08T04:42:08.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdacea00640 1 -- 172.21.15.164:0/4113149878 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac80a89d0 con 0x7fdad0162420 2024-05-08T04:42:08.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdacea00640 1 -- 172.21.15.164:0/4113149878 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80a8da0 con 0x7fdad0162420 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- 172.21.15.164:0/4113149878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad0162420 msgr2=0x7fdad00cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad0162420 0x7fdad00cfc00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdad0071af0 tx=0x7fdac801ed10 comp rx=0 tx=0).stop 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- 172.21.15.164:0/4113149878 shutdown_connections 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad0162420 0x7fdad00cfc00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 --2- 172.21.15.164:0/4113149878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad006c7b0 0x7fdad0161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 --2- 172.21.15.164:0/4113149878 >> v2:172.21.15.172:3300/0 conn(0x7fdad015dd70 0x7fdad01619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- 172.21.15.164:0/4113149878 >> 172.21.15.164:0/4113149878 conn(0x7fdad00b34f0 msgr2=0x7fdad00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- 172.21.15.164:0/4113149878 shutdown_connections 2024-05-08T04:42:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- 172.21.15.164:0/4113149878 wait complete. 2024-05-08T04:42:08.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 Processor -- start 2024-05-08T04:42:08.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.224+0000 7fdad7200640 1 -- start start 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 0x7fdad00b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0162420 0x7fdad00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad015e150 con 0x7fdad006c7b0 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad00b4230 con 0x7fdad0162420 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad015e2d0 con 0x7fdad015dd70 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50724/0 (socket says 172.21.15.164:50724) 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 -- 172.21.15.164:0/3201883235 learned_addr learned my addr 172.21.15.164:0/3201883235 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad5200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0162420 0x7fdad00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacf400640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 0x7fdad00b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 -- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 msgr2=0x7fdad00b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 0x7fdad00b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 -- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0162420 msgr2=0x7fdad00b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0162420 0x7fdad00b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 -- 172.21.15.164:0/3201883235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad015fd10 con 0x7fdad006c7b0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacf400640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 0x7fdad00b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdacfe00640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdac0007f20 tx=0x7fdac000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0014620 con 0x7fdad006c7b0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0019070 con 0x7fdad006c7b0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0014a00 con 0x7fdad006c7b0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad015fff0 con 0x7fdad006c7b0 2024-05-08T04:42:08.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad00b49f0 con 0x7fdad006c7b0 2024-05-08T04:42:08.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad0069d60 con 0x7fdad006c7b0 2024-05-08T04:42:08.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.228+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fdac0014ba0 con 0x7fdad006c7b0 2024-05-08T04:42:08.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.232+0000 7fdac7e00640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 0x7fdab0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.232+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fdac0086a70 con 0x7fdad006c7b0 2024-05-08T04:42:08.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.232+0000 7fdacf400640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 0x7fdab0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.232+0000 7fdacf400640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 0x7fdab0069b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdad00c8970 tx=0x7fdabc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.236+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdac0056b60 con 0x7fdad006c7b0 2024-05-08T04:42:08.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.420+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdad00b4010 con 0x7fdad006c7b0 2024-05-08T04:42:08.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdac7e00640 1 -- 172.21.15.164:0/3201883235 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fdac0056460 con 0x7fdad006c7b0 2024-05-08T04:42:08.428 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:08.428 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:42:08.428 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 53 2024-05-08T04:42:08.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 msgr2=0x7fdab0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 0x7fdab0069b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdad00c8970 tx=0x7fdabc008040 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 msgr2=0x7fdad00b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdac0007f20 tx=0x7fdac000e570 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 shutdown_connections 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdab00676b0 0x7fdab0069b70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdad015dd70 0x7fdad00b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0162420 0x7fdad00b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 --2- 172.21.15.164:0/3201883235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad006c7b0 0x7fdad00b0020 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 >> 172.21.15.164:0/3201883235 conn(0x7fdad00b34f0 msgr2=0x7fdad006f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 shutdown_connections 2024-05-08T04:42:08.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.424+0000 7fdad7200640 1 -- 172.21.15.164:0/3201883235 wait complete. 2024-05-08T04:42:08.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 Processor -- start 2024-05-08T04:42:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 -- start start 2024-05-08T04:42:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f993415ba70 0x7f9934160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f993415be50 0x7f99340cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340cd9f0 con 0x7f99340aeda0 2024-05-08T04:42:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993415fe70 con 0x7f993415be50 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f9938e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340cdcd0 con 0x7f993415ba70 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f992be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f993415be50 0x7f99340cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f992be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f993415be50 0x7f99340cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50732/0 (socket says 172.21.15.164:50732) 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.580+0000 7f992be00640 1 -- 172.21.15.164:0/2461649582 learned_addr learned my addr 172.21.15.164:0/2461649582 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9931e00640 1 --2- 172.21.15.164:0/2461649582 >> v2:172.21.15.172:3300/0 conn(0x7f993415ba70 0x7f9934160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f992be00640 1 -- 172.21.15.164:0/2461649582 >> v2:172.21.15.172:3300/0 conn(0x7f993415ba70 msgr2=0x7f9934160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f992be00640 1 --2- 172.21.15.164:0/2461649582 >> v2:172.21.15.172:3300/0 conn(0x7f993415ba70 0x7f9934160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f992be00640 1 -- 172.21.15.164:0/2461649582 >> v1:172.21.15.172:6789/0 conn(0x7f99340aeda0 legacy=0x7f99340b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f992be00640 1 -- 172.21.15.164:0/2461649582 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993415f820 con 0x7f993415be50 2024-05-08T04:42:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f992be00640 1 --2- 172.21.15.164:0/2461649582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415be50 0x7f99340cf690 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f993406bc70 tx=0x7f991c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd719e0286c1e1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9931400640 1 -- 172.21.15.164:0/2461649582 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b1070 con 0x7f993415be50 2024-05-08T04:42:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9931400640 1 -- 172.21.15.164:0/2461649582 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0ac620 con 0x7f993415be50 2024-05-08T04:42:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9931400640 1 -- 172.21.15.164:0/2461649582 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0ac9f0 con 0x7f993415be50 2024-05-08T04:42:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9938e00640 1 -- 172.21.15.164:0/2461649582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415be50 msgr2=0x7f99340cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.584+0000 7f9938e00640 1 --2- 172.21.15.164:0/2461649582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415be50 0x7f99340cf690 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f993406bc70 tx=0x7f991c01c4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- 172.21.15.164:0/2461649582 shutdown_connections 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 --2- 172.21.15.164:0/2461649582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415be50 0x7f99340cf690 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 --2- 172.21.15.164:0/2461649582 >> v2:172.21.15.172:3300/0 conn(0x7f993415ba70 0x7f9934160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- 172.21.15.164:0/2461649582 >> 172.21.15.164:0/2461649582 conn(0x7f993406ac90 msgr2=0x7f993406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- 172.21.15.164:0/2461649582 shutdown_connections 2024-05-08T04:42:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- 172.21.15.164:0/2461649582 wait complete. 2024-05-08T04:42:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 Processor -- start 2024-05-08T04:42:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- start start 2024-05-08T04:42:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 0x7f99340af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f993415be50 0x7f993406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9934168d40 con 0x7f993415ba70 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9934168bc0 con 0x7f99340aeda0 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9934168ec0 con 0x7f993415be50 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50748/0 (socket says 172.21.15.164:50748) 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 -- 172.21.15.164:0/2465079678 learned_addr learned my addr 172.21.15.164:0/2465079678 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9931e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 0x7f99340af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9932800640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f993415be50 0x7f993406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 -- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f993415be50 msgr2=0x7f993406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f993415be50 0x7f993406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 -- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 msgr2=0x7f99340af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 0x7f99340af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 -- 172.21.15.164:0/2465079678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993415a1e0 con 0x7f993415ba70 2024-05-08T04:42:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9931e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 0x7f99340af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992be00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f991c0af040 tx=0x7f991c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0bc070 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0b7430 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b1040 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.588+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993415a4c0 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993415aa80 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9934069d60 con 0x7f993415ba70 2024-05-08T04:42:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f991c0b63d0 con 0x7f993415ba70 2024-05-08T04:42:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f992a000640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 0x7f9910069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f991c11f6f0 con 0x7f993415ba70 2024-05-08T04:42:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f9931e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 0x7f9910069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.592+0000 7f9931e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 0x7f9910069b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9924007a70 tx=0x7f9924008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.596+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f991c0160d0 con 0x7f993415ba70 2024-05-08T04:42:08.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.752+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f993406e210 con 0x7f993415ba70 2024-05-08T04:42:08.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.752+0000 7f992a000640 1 -- 172.21.15.164:0/2465079678 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f991c0efba0 con 0x7f993415ba70 2024-05-08T04:42:08.759 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":87,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:04.308390+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":11,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:08.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 msgr2=0x7f9910069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 0x7f9910069b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9924007a70 tx=0x7f9924008040 comp rx=0 tx=0).stop 2024-05-08T04:42:08.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 msgr2=0x7f99341474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f991c0af040 tx=0x7f991c019250 comp rx=0 tx=0).stop 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 shutdown_connections 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f99100676b0 0x7f9910069b70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f993415be50 0x7f993406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993415ba70 0x7f99341474c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 --2- 172.21.15.164:0/2465079678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99340aeda0 0x7f99340af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 >> 172.21.15.164:0/2465079678 conn(0x7f993406ac90 msgr2=0x7f99340b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 shutdown_connections 2024-05-08T04:42:08.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.756+0000 7f9938e00640 1 -- 172.21.15.164:0/2465079678 wait complete. 2024-05-08T04:42:08.780 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:42:08.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 Processor -- start 2024-05-08T04:42:08.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 -- start start 2024-05-08T04:42:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ddc15ba70 0x7f9ddc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0cd9f0 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0cdcd0 con 0x7f9ddc15be50 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de4800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ddc15fe70 con 0x7f9ddc15ba70 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de2800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9ddc0cfda0 0x7f9ddc0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49846/0 (socket says 172.21.15.164:49846) 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de2800640 1 -- 172.21.15.164:0/2790101038 learned_addr learned my addr 172.21.15.164:0/2790101038 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de1e00640 1 --2- 172.21.15.164:0/2790101038 >> v2:172.21.15.172:3300/0 conn(0x7f9ddc15ba70 0x7f9ddc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960636467 0 0) 0x7f9ddc0cd9f0 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc4003530 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de1e00640 1 -- 172.21.15.164:0/2790101038 >> v1:172.21.15.172:6789/0 conn(0x7f9ddc15be50 legacy=0x7f9ddc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746702600 0 0) 0x7f9ddc0cdcd0 con 0x7f9ddc15be50 2024-05-08T04:42:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de1e00640 1 -- 172.21.15.164:0/2790101038 >> v1:172.21.15.164:6789/0 conn(0x7f9ddc0cfda0 legacy=0x7f9ddc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.912+0000 7f9de1e00640 1 -- 172.21.15.164:0/2790101038 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc15f820 con 0x7f9ddc15ba70 2024-05-08T04:42:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de1e00640 1 --2- 172.21.15.164:0/2790101038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc160300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d174a0b53a1b681 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00b2070 con 0x7f9ddc15ba70 2024-05-08T04:42:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd00ad680 con 0x7f9ddc15ba70 2024-05-08T04:42:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de0a00640 1 -- 172.21.15.164:0/2790101038 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00ada50 con 0x7f9ddc15ba70 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 -- 172.21.15.164:0/2790101038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 msgr2=0x7f9ddc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 --2- 172.21.15.164:0/2790101038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc160300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd001c470 comp rx=0 tx=0).stop 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 -- 172.21.15.164:0/2790101038 shutdown_connections 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 --2- 172.21.15.164:0/2790101038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc160300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 -- 172.21.15.164:0/2790101038 >> 172.21.15.164:0/2790101038 conn(0x7f9ddc06ac90 msgr2=0x7f9ddc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 -- 172.21.15.164:0/2790101038 shutdown_connections 2024-05-08T04:42:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.916+0000 7f9de4800640 1 -- 172.21.15.164:0/2790101038 wait complete. 2024-05-08T04:42:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 Processor -- start 2024-05-08T04:42:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- start start 2024-05-08T04:42:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ddc15be50 0x7f9ddc0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc169f60 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc169de0 con 0x7f9ddc15ba70 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc16a0e0 con 0x7f9ddc15be50 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50764/0 (socket says 172.21.15.164:50764) 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 -- 172.21.15.164:0/3598770452 learned_addr learned my addr 172.21.15.164:0/3598770452 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de2800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ddc15be50 0x7f9ddc0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1400640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 -- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ddc15be50 msgr2=0x7f9ddc0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ddc15be50 0x7f9ddc0cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 -- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 msgr2=0x7f9ddc152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 -- 172.21.15.164:0/3598770452 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0ccea0 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de1e00640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00bc020 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd00adb70 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00b2040 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc15e210 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.920+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc15ea70 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc067820 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f9dd00bf020 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9dd6000640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 0x7f9dc0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f9dd011ff70 con 0x7f9ddc0cfda0 2024-05-08T04:42:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9de1400640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 0x7f9dc0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.924+0000 7f9de1400640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 0x7f9dc0069b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc160170 tx=0x7f9dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:08.928+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dd00c6030 con 0x7f9ddc0cfda0 2024-05-08T04:42:09.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.116+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ddc069d60 con 0x7f9ddc0cfda0 2024-05-08T04:42:09.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.116+0000 7f9dd6000640 1 -- 172.21.15.164:0/3598770452 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9dd0016090 con 0x7f9ddc0cfda0 2024-05-08T04:42:09.123 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:09.123 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:42:09.123 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 53 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 msgr2=0x7f9dc0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 0x7f9dc0069b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc160170 tx=0x7f9dcc009040 comp rx=0 tx=0).stop 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 msgr2=0x7f9ddc14d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd001acb0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 shutdown_connections 2024-05-08T04:42:09.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9dc00676b0 0x7f9dc0069b70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ddc15be50 0x7f9ddc0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc15ba70 0x7f9ddc152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 --2- 172.21.15.164:0/3598770452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ddc0cfda0 0x7f9ddc14d170 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 >> 172.21.15.164:0/3598770452 conn(0x7f9ddc06ac90 msgr2=0x7f9ddc06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 shutdown_connections 2024-05-08T04:42:09.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.120+0000 7f9de4800640 1 -- 172.21.15.164:0/3598770452 wait complete. 2024-05-08T04:42:09.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:09.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 Processor -- start 2024-05-08T04:42:09.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 -- start start 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c0c0cfda0 0x7f1c0c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0c0cd9f0 con 0x7f1c0c15be50 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0c0cdcd0 con 0x7f1c0c15ba70 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c13800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c15fe70 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c11800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1c0c15be50 0x7f1c0c0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40158/0 (socket says 172.21.15.164:40158) 2024-05-08T04:42:09.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c11800640 1 -- 172.21.15.164:0/1130383992 learned_addr learned my addr 172.21.15.164:0/1130383992 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:09.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0be00640 1 --2- 172.21.15.164:0/1130383992 >> v2:172.21.15.172:3300/0 conn(0x7f1c0c0cfda0 0x7f1c0c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078587614 0 0) 0x7f1c0c0cd9f0 con 0x7f1c0c15be50 2024-05-08T04:42:09.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf4003530 con 0x7f1c0c15be50 2024-05-08T04:42:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0be00640 1 -- 172.21.15.164:0/1130383992 >> v1:172.21.15.172:6789/0 conn(0x7f1c0c15ba70 legacy=0x7f1c0c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446829505 0 0) 0x7f1c0c0cdcd0 con 0x7f1c0c15ba70 2024-05-08T04:42:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0be00640 1 -- 172.21.15.164:0/1130383992 >> v1:172.21.15.164:6789/0 conn(0x7f1c0c15be50 legacy=0x7f1c0c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.276+0000 7f1c0be00640 1 -- 172.21.15.164:0/1130383992 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c15f820 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c0be00640 1 --2- 172.21.15.164:0/1130383992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c0cfda0 0x7f1c0c160300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097c0 tx=0x7f1bf801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308d087f341e26cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80b2070 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80ad680 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c0b400640 1 -- 172.21.15.164:0/1130383992 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80ada50 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 -- 172.21.15.164:0/1130383992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c0cfda0 msgr2=0x7f1c0c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 --2- 172.21.15.164:0/1130383992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c0cfda0 0x7f1c0c160300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097c0 tx=0x7f1bf801c470 comp rx=0 tx=0).stop 2024-05-08T04:42:09.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 -- 172.21.15.164:0/1130383992 shutdown_connections 2024-05-08T04:42:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 --2- 172.21.15.164:0/1130383992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c0cfda0 0x7f1c0c160300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 -- 172.21.15.164:0/1130383992 >> 172.21.15.164:0/1130383992 conn(0x7f1c0c06ac90 msgr2=0x7f1c0c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 -- 172.21.15.164:0/1130383992 shutdown_connections 2024-05-08T04:42:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.280+0000 7f1c13800640 1 -- 172.21.15.164:0/1130383992 wait complete. 2024-05-08T04:42:09.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 Processor -- start 2024-05-08T04:42:09.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- start start 2024-05-08T04:42:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 0x7f1c0c13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c15be50 0x7f1c0c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c169f60 con 0x7f1c0c15ba70 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c169de0 con 0x7f1c0c15be50 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c16a0e0 con 0x7f1c0c0cfda0 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43104/0 (socket says 172.21.15.164:43104) 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 -- 172.21.15.164:0/1982588356 learned_addr learned my addr 172.21.15.164:0/1982588356 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c11800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c15be50 0x7f1c0c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c10e00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 0x7f1c0c13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 -- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 msgr2=0x7f1c0c13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 0x7f1c0c13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 -- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c15be50 msgr2=0x7f1c0c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c15be50 0x7f1c0c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 -- 172.21.15.164:0/1982588356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c14dc80 con 0x7f1c0c15ba70 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c10e00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 0x7f1c0c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:09.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c0be00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1bf801a0a0 tx=0x7f1bf801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80b2040 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80ad8f0 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c14df60 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80adcc0 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.284+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c14e520 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c14e850 con 0x7f1c0c15ba70 2024-05-08T04:42:09.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f1bf80bf030 con 0x7f1c0c15ba70 2024-05-08T04:42:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c08c00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 0x7f1bec069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f1bf811f540 con 0x7f1c0c15ba70 2024-05-08T04:42:09.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c10e00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 0x7f1bec069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.288+0000 7f1c10e00640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 0x7f1bec069ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000f70 tx=0x7f1c00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.292+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bf8016090 con 0x7f1c0c15ba70 2024-05-08T04:42:09.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.484+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c0c069d60 con 0x7f1c0c15ba70 2024-05-08T04:42:09.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.484+0000 7f1c08c00640 1 -- 172.21.15.164:0/1982588356 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1bf811de20 con 0x7f1c0c15ba70 2024-05-08T04:42:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:09.490 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:41:43.227654+0000","modified":"2024-05-08T04:41:46.260430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1815840371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1815840371},{"type":"v1","addr":"172.21.15.164:6837","nonce":1815840371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-08T04:42:09.491 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 53 2024-05-08T04:42:09.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 msgr2=0x7f1bec069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 0x7f1bec069ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000f70 tx=0x7f1c00009040 comp rx=0 tx=0).stop 2024-05-08T04:42:09.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 msgr2=0x7f1c0c0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1bf801a0a0 tx=0x7f1bf801acb0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 shutdown_connections 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1bec0676e0 0x7f1bec069ba0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0c15be50 0x7f1c0c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c15ba70 0x7f1c0c0b0260 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 --2- 172.21.15.164:0/1982588356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0c0cfda0 0x7f1c0c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 >> 172.21.15.164:0/1982588356 conn(0x7f1c0c06ac90 msgr2=0x7f1c0c14ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 shutdown_connections 2024-05-08T04:42:09.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.488+0000 7f1c13800640 1 -- 172.21.15.164:0/1982588356 wait complete. 2024-05-08T04:42:09.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:09.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 Processor -- start 2024-05-08T04:42:09.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 -- start start 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb0166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb00b42a0 con 0x7f8cb00ccfe0 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb00cd4e0 con 0x7f8cb015bc50 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cb5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00cd7c0 con 0x7f8cb00d0780 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cae000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8cb00ccfe0 0x7f8cb00d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40168/0 (socket says 172.21.15.164:40168) 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cae000640 1 -- 172.21.15.164:0/1067242303 learned_addr learned my addr 172.21.15.164:0/1067242303 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:09.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8caf400640 1 --2- 172.21.15.164:0/1067242303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb0166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213220568 0 0) 0x7f8cb00b42a0 con 0x7f8cb00ccfe0 2024-05-08T04:42:09.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c94003530 con 0x7f8cb00ccfe0 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744538894 0 0) 0x7f8cb00cd4e0 con 0x7f8cb015bc50 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb00b42a0 con 0x7f8cb015bc50 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1627249898 0 0) 0x7f8c94003530 con 0x7f8cb00ccfe0 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8caf400640 1 -- 172.21.15.164:0/1067242303 >> v1:172.21.15.172:6789/0 conn(0x7f8cb015bc50 legacy=0x7f8cb00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8caf400640 1 -- 172.21.15.164:0/1067242303 >> v1:172.21.15.164:6789/0 conn(0x7f8cb00ccfe0 legacy=0x7f8cb00d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.644+0000 7f8caf400640 1 -- 172.21.15.164:0/1067242303 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00cf590 con 0x7f8cb00d0780 2024-05-08T04:42:09.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8caf400640 1 --2- 172.21.15.164:0/1067242303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb0166ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009dc0 tx=0x7f8ca801c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929bf546af9a5bce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80b2070 con 0x7f8cb00d0780 2024-05-08T04:42:09.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80a5630 con 0x7f8cb00d0780 2024-05-08T04:42:09.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cad600640 1 -- 172.21.15.164:0/1067242303 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c001680 con 0x7f8cb00d0780 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1067242303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 msgr2=0x7f8cb0166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1067242303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb0166ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009dc0 tx=0x7f8ca801c7f0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1067242303 shutdown_connections 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1067242303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb0166ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1067242303 >> 172.21.15.164:0/1067242303 conn(0x7f8cb00b34f0 msgr2=0x7f8cb00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1067242303 shutdown_connections 2024-05-08T04:42:09.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.648+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1067242303 wait complete. 2024-05-08T04:42:09.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 Processor -- start 2024-05-08T04:42:09.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- start start 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cb00ccfe0 0x7f8cb00c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb00c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00b50a0 con 0x7f8cb015bc50 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00b4f20 con 0x7f8cb00ccfe0 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0169830 con 0x7f8cb00d0780 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43116/0 (socket says 172.21.15.164:43116) 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 -- 172.21.15.164:0/1198879200 learned_addr learned my addr 172.21.15.164:0/1198879200 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caea00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cb00ccfe0 0x7f8cb00c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cae000640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb00c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 -- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 msgr2=0x7f8cb00c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb00c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 -- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cb00ccfe0 msgr2=0x7f8cb00c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cb00ccfe0 0x7f8cb00c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 -- 172.21.15.164:0/1198879200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb006f700 con 0x7f8cb015bc50 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8caf400640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca801ccd0 tx=0x7f8ca8019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80b2050 con 0x7f8cb015bc50 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80be070 con 0x7f8cb015bc50 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb00c8290 con 0x7f8cb015bc50 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8009450 con 0x7f8cb015bc50 2024-05-08T04:42:09.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.652+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb015c700 con 0x7f8cb015bc50 2024-05-08T04:42:09.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb0069d60 con 0x7f8cb015bc50 2024-05-08T04:42:09.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f8ca80ada70 con 0x7f8cb015bc50 2024-05-08T04:42:09.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8ca6a00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 0x7f8c90069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f8ca8120c80 con 0x7f8cb015bc50 2024-05-08T04:42:09.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8caea00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 0x7f8c90069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.656+0000 7f8caea00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 0x7f8c90069b70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0153e00 tx=0x7f8ca0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.660+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ca80164a0 con 0x7f8cb015bc50 2024-05-08T04:42:09.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.816+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cb00b7940 con 0x7f8cb015bc50 2024-05-08T04:42:09.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.816+0000 7f8ca6a00640 1 -- 172.21.15.164:0/1198879200 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f8ca80f0b30 con 0x7f8cb015bc50 2024-05-08T04:42:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:09.823 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":87,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:04.308390+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:41:41.603797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-08T04:41:42.235153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":11,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:09.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 msgr2=0x7f8c90069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 0x7f8c90069b70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0153e00 tx=0x7f8ca0009040 comp rx=0 tx=0).stop 2024-05-08T04:42:09.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 msgr2=0x7f8cb00c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca801ccd0 tx=0x7f8ca8019590 comp rx=0 tx=0).stop 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 shutdown_connections 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c900676b0 0x7f8c90069b70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cb00d0780 0x7f8cb00c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cb015bc50 0x7f8cb00c7780 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 --2- 172.21.15.164:0/1198879200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cb00ccfe0 0x7f8cb00c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 >> 172.21.15.164:0/1198879200 conn(0x7f8cb00b34f0 msgr2=0x7f8cb00d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 shutdown_connections 2024-05-08T04:42:09.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.820+0000 7f8cb5c00640 1 -- 172.21.15.164:0/1198879200 wait complete. 2024-05-08T04:42:09.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:42:09.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 Processor -- start 2024-05-08T04:42:09.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 -- start start 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 0x7f9f180cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1806c770 con 0x7f9f1815ab60 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f180d0060 con 0x7f9f180cfbd0 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f1da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1806ca50 con 0x7f9f180b26c0 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f17400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 0x7f9f180cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f0e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f0e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:46618/0 (socket says 172.21.15.164:46618) 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f0e000640 1 -- 172.21.15.164:0/247568127 learned_addr learned my addr 172.21.15.164:0/247568127 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.976+0000 7f9f17400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 0x7f9f180cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:47128/0 (socket says 172.21.15.164:47128) 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f0e000640 1 -- 172.21.15.164:0/247568127 >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 msgr2=0x7f9f180cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f0e000640 1 --2- 172.21.15.164:0/247568127 >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 0x7f9f180cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f0e000640 1 -- 172.21.15.164:0/247568127 >> v1:172.21.15.172:6789/0 conn(0x7f9f1815ab60 legacy=0x7f9f18072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f0e000640 1 -- 172.21.15.164:0/247568127 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1806cd30 con 0x7f9f180b26c0 2024-05-08T04:42:09.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f0e000640 1 --2- 172.21.15.164:0/247568127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25237993637806e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f16000640 1 -- 172.21.15.164:0/247568127 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000b2070 con 0x7f9f180b26c0 2024-05-08T04:42:09.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f16000640 1 -- 172.21.15.164:0/247568127 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000a9070 con 0x7f9f180b26c0 2024-05-08T04:42:09.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.980+0000 7f9f16000640 1 -- 172.21.15.164:0/247568127 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000ad880 con 0x7f9f180b26c0 2024-05-08T04:42:09.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- 172.21.15.164:0/247568127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 msgr2=0x7f9f180cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- 172.21.15.164:0/247568127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0001c560 comp rx=0 tx=0).stop 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- 172.21.15.164:0/247568127 shutdown_connections 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- 172.21.15.164:0/247568127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f180cf690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- 172.21.15.164:0/247568127 >> v2:172.21.15.172:3300/0 conn(0x7f9f180cfbd0 0x7f9f180cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- 172.21.15.164:0/247568127 >> 172.21.15.164:0/247568127 conn(0x7f9f1806ac90 msgr2=0x7f9f1806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- 172.21.15.164:0/247568127 shutdown_connections 2024-05-08T04:42:09.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- 172.21.15.164:0/247568127 wait complete. 2024-05-08T04:42:09.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 Processor -- start 2024-05-08T04:42:09.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- start start 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f1813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 0x7f9f1813d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180aeeb0 con 0x7f9f1815ab60 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180aed30 con 0x7f9f180cfbd0 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f1da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18167970 con 0x7f9f180b26c0 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f17400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 0x7f9f1813d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f17400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43132/0 (socket says 172.21.15.164:43132) 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f17400640 1 -- 172.21.15.164:0/1741615005 learned_addr learned my addr 172.21.15.164:0/1741615005 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:09.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f16a00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f1813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 -- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 msgr2=0x7f9f1813ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f1813ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 -- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 msgr2=0x7f9f18151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 -- 172.21.15.164:0/1741615005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180ae8a0 con 0x7f9f180cfbd0 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f17400640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:09.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.984+0000 7f9f0e000640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 0x7f9f1813d410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ac000 tx=0x7f9f00019300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000b2040 con 0x7f9f180cfbd0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000add80 con 0x7f9f180cfbd0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000b6660 con 0x7f9f180cfbd0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1813dd70 con 0x7f9f180cfbd0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1813e330 con 0x7f9f180cfbd0 2024-05-08T04:42:09.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f18069d60 con 0x7f9f180cfbd0 2024-05-08T04:42:09.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f9f000ab070 con 0x7f9f180cfbd0 2024-05-08T04:42:09.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 0x7f9ef4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:09.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f9f0011e800 con 0x7f9f180cfbd0 2024-05-08T04:42:09.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.988+0000 7f9f16a00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 0x7f9ef4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:09.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.992+0000 7f9f16a00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 0x7f9ef4069b70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001e00 tx=0x7f9f08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:09.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:09.992+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f00016150 con 0x7f9f180cfbd0 2024-05-08T04:42:10.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.184+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9f1806b110 con 0x7f9f180cfbd0 2024-05-08T04:42:10.189 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:42:10.184+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:42:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f0f400640 1 -- 172.21.15.164:0/1741615005 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9f000ee620 con 0x7f9f180cfbd0 2024-05-08T04:42:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:42:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 msgr2=0x7f9ef4069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 0x7f9ef4069b70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001e00 tx=0x7f9f08009040 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 msgr2=0x7f9f1813d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 0x7f9f1813d410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ac000 tx=0x7f9f00019300 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 shutdown_connections 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9ef40676b0 0x7f9ef4069b70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1815ab60 0x7f9f18151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f180cfbd0 0x7f9f1813d410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 --2- 172.21.15.164:0/1741615005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f180b26c0 0x7f9f1813ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 >> 172.21.15.164:0/1741615005 conn(0x7f9f1806ac90 msgr2=0x7f9f18072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 shutdown_connections 2024-05-08T04:42:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.192+0000 7f9f1da00640 1 -- 172.21.15.164:0/1741615005 wait complete. 2024-05-08T04:42:10.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:42:10.261 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:42:10.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 Processor -- start 2024-05-08T04:42:10.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 -- start start 2024-05-08T04:42:10.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 0x7fe628070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62806ac90 0x7fe62806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6280720f0 con 0x7fe62806d8f0 2024-05-08T04:42:10.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62806e650 con 0x7fe62806ac90 2024-05-08T04:42:10.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe628072360 con 0x7fe62806e1f0 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62806ac90 0x7fe62806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62806ac90 0x7fe62806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:47160/0 (socket says 172.21.15.164:47160) 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe61e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 0x7fe628070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe627400640 1 -- 172.21.15.164:0/225497559 learned_addr learned my addr 172.21.15.164:0/225497559 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.356+0000 7fe626a00640 1 --1- 172.21.15.164:0/225497559 >> v1:172.21.15.172:6789/0 conn(0x7fe62806d8f0 0x7fe62815b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:39778/0 (socket says 172.21.15.164:39778) 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627400640 1 -- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 msgr2=0x7fe628070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627400640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 0x7fe628070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627400640 1 -- 172.21.15.164:0/225497559 >> v1:172.21.15.172:6789/0 conn(0x7fe62806d8f0 legacy=0x7fe62815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627400640 1 -- 172.21.15.164:0/225497559 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280b4610 con 0x7fe62806ac90 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe61e000640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 0x7fe628070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:10.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe626000640 1 -- 172.21.15.164:0/225497559 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975369839 0 0) 0x7fe6280720f0 con 0x7fe62806d8f0 2024-05-08T04:42:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627400640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806ac90 0x7fe62806b120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe62000d180 tx=0x7fe62001ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0da427e2f33e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe626000640 1 -- 172.21.15.164:0/225497559 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000b2d0 con 0x7fe62806ac90 2024-05-08T04:42:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe626000640 1 -- 172.21.15.164:0/225497559 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62000b470 con 0x7fe62806ac90 2024-05-08T04:42:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe626000640 1 -- 172.21.15.164:0/225497559 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a7c10 con 0x7fe62806ac90 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627e00640 1 -- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806ac90 msgr2=0x7fe62806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.360+0000 7fe627e00640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806ac90 0x7fe62806b120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe62000d180 tx=0x7fe62001ec10 comp rx=0 tx=0).stop 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- 172.21.15.164:0/225497559 shutdown_connections 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806e1f0 0x7fe628070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 --2- 172.21.15.164:0/225497559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806ac90 0x7fe62806b120 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- 172.21.15.164:0/225497559 >> 172.21.15.164:0/225497559 conn(0x7fe6280b1450 msgr2=0x7fe6280b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- 172.21.15.164:0/225497559 shutdown_connections 2024-05-08T04:42:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- 172.21.15.164:0/225497559 wait complete. 2024-05-08T04:42:10.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 Processor -- start 2024-05-08T04:42:10.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- start start 2024-05-08T04:42:10.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806d8f0 0x7fe628148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806e1f0 0x7fe628148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280cc1e0 con 0x7fe62806ac90 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280ce0c0 con 0x7fe62806e1f0 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280cc360 con 0x7fe62806d8f0 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43150/0 (socket says 172.21.15.164:43150) 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 -- 172.21.15.164:0/2504685974 learned_addr learned my addr 172.21.15.164:0/2504685974 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe627400640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806e1f0 0x7fe628148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe61e000640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806d8f0 0x7fe628148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 -- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806d8f0 msgr2=0x7fe628148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806d8f0 0x7fe628148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 -- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806e1f0 msgr2=0x7fe628148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:10.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806e1f0 0x7fe628148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:10.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 -- 172.21.15.164:0/2504685974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628149520 con 0x7fe62806ac90 2024-05-08T04:42:10.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe626a00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe6180011f0 tx=0x7fe61800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618018070 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6180074e0 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.364+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618004a10 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe628143dc0 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe628144380 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62815a8c0 con 0x7fe62806ac90 2024-05-08T04:42:10.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fe61801b070 con 0x7fe62806ac90 2024-05-08T04:42:10.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe61f400640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 0x7fe604069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:10.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5879+0+0 (secure 0 0 0) 0x7fe618085530 con 0x7fe62806ac90 2024-05-08T04:42:10.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe61e000640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 0x7fe604069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:10.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.368+0000 7fe61e000640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 0x7fe604069b70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe610009580 tx=0x7fe610009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:10.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.372+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe618054f90 con 0x7fe62806ac90 2024-05-08T04:42:10.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:10.560+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe628068160 con 0x7fe62806ac90 2024-05-08T04:42:11.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe61f400640 1 -- 172.21.15.164:0/2504685974 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe618059030 con 0x7fe62806ac90 2024-05-08T04:42:11.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 msgr2=0x7fe604069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 0x7fe604069b70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe610009580 tx=0x7fe610009190 comp rx=0 tx=0).stop 2024-05-08T04:42:11.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 msgr2=0x7fe6280cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe6180011f0 tx=0x7fe61800c9b0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 shutdown_connections 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe6040676b0 0x7fe604069b70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62806d8f0 0x7fe628148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.188+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62806e1f0 0x7fe628148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.192+0000 7fe627e00640 1 --2- 172.21.15.164:0/2504685974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62806ac90 0x7fe6280cc8e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.192+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 >> 172.21.15.164:0/2504685974 conn(0x7fe6280b1450 msgr2=0x7fe6280cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.192+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 shutdown_connections 2024-05-08T04:42:11.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.192+0000 7fe627e00640 1 -- 172.21.15.164:0/2504685974 wait complete. 2024-05-08T04:42:11.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:42:11.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 Processor -- start 2024-05-08T04:42:11.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 -- start start 2024-05-08T04:42:11.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98806ac90 0x7fd98806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98806da30 con 0x7fd98806e7b0 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98806e320 con 0x7fd98806ac90 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd990000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98806dcd0 con 0x7fd9880affe0 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98806ac90 0x7fd98806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd98806ac90 0x7fd98806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43160/0 (socket says 172.21.15.164:43160) 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 -- 172.21.15.164:0/3740004364 learned_addr learned my addr 172.21.15.164:0/3740004364 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:11.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98d600640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 -- 172.21.15.164:0/3740004364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 msgr2=0x7fd98815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:11.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 -- 172.21.15.164:0/3740004364 >> v1:172.21.15.172:6789/0 conn(0x7fd98806e7b0 legacy=0x7fd98806f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.400+0000 7fd98e000640 1 -- 172.21.15.164:0/3740004364 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98806dfb0 con 0x7fd98806ac90 2024-05-08T04:42:11.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd98e000640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806ac90 0x7fd98806b120 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd980009e50 tx=0x7fd98001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8947d6cb72da8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:11.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd987e00640 1 -- 172.21.15.164:0/3740004364 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a5700 con 0x7fd98806ac90 2024-05-08T04:42:11.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd987e00640 1 -- 172.21.15.164:0/3740004364 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800a58a0 con 0x7fd98806ac90 2024-05-08T04:42:11.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd987e00640 1 -- 172.21.15.164:0/3740004364 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a5c70 con 0x7fd98806ac90 2024-05-08T04:42:11.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd990000640 1 -- 172.21.15.164:0/3740004364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806ac90 msgr2=0x7fd98806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.404+0000 7fd990000640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806ac90 0x7fd98806b120 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd980009e50 tx=0x7fd98001ca30 comp rx=0 tx=0).stop 2024-05-08T04:42:11.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- 172.21.15.164:0/3740004364 shutdown_connections 2024-05-08T04:42:11.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806ac90 0x7fd98806b120 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 --2- 172.21.15.164:0/3740004364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- 172.21.15.164:0/3740004364 >> 172.21.15.164:0/3740004364 conn(0x7fd9880b2d00 msgr2=0x7fd9880d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:11.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- 172.21.15.164:0/3740004364 shutdown_connections 2024-05-08T04:42:11.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- 172.21.15.164:0/3740004364 wait complete. 2024-05-08T04:42:11.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 Processor -- start 2024-05-08T04:42:11.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- start start 2024-05-08T04:42:11.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98806ac90 0x7fd988136570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880cf570 con 0x7fd98806e7b0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880cf3f0 con 0x7fd98806ac90 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd990000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880cf6f0 con 0x7fd9880affe0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43162/0 (socket says 172.21.15.164:43162) 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 -- 172.21.15.164:0/1731267276 learned_addr learned my addr 172.21.15.164:0/1731267276 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98d600640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98806ac90 0x7fd988136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98e000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 -- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 msgr2=0x7fd98815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 -- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98806ac90 msgr2=0x7fd988136570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98806ac90 0x7fd988136570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 -- 172.21.15.164:0/1731267276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988136ab0 con 0x7fd98806e7b0 2024-05-08T04:42:11.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd98cc00640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c450 tx=0x7fd97400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.408+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740052d0 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974005470 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974007260 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9881357b0 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988135d70 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988072e80 con 0x7fd98806e7b0 2024-05-08T04:42:11.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fd974007400 con 0x7fd98806e7b0 2024-05-08T04:42:11.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd985600640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 0x7fd968069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:11.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5908+0+0 (secure 0 0 0) 0x7fd974085670 con 0x7fd98806e7b0 2024-05-08T04:42:11.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd98d600640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 0x7fd968069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:11.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.412+0000 7fd98d600640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 0x7fd968069ba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd98814e1b0 tx=0x7fd97c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:11.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.416+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9740550b0 con 0x7fd98806e7b0 2024-05-08T04:42:11.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:11.572+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd98804f010 con 0x7fd98806e7b0 2024-05-08T04:42:12.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.344+0000 7fd985600640 1 -- 172.21.15.164:0/1731267276 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd974059150 con 0x7fd98806e7b0 2024-05-08T04:42:12.349 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 msgr2=0x7fd968069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 0x7fd968069ba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd98814e1b0 tx=0x7fd97c009190 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 msgr2=0x7fd9880b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c450 tx=0x7fd97400c910 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 shutdown_connections 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd9680676e0 0x7fd968069ba0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9880affe0 0x7fd98815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd98806e7b0 0x7fd9880b17a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 --2- 172.21.15.164:0/1731267276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd98806ac90 0x7fd988136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 >> 172.21.15.164:0/1731267276 conn(0x7fd9880b2d00 msgr2=0x7fd988137070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:12.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 shutdown_connections 2024-05-08T04:42:12.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.348+0000 7fd990000640 1 -- 172.21.15.164:0/1731267276 wait complete. 2024-05-08T04:42:12.375 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:42:12.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 Processor -- start 2024-05-08T04:42:12.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 -- start start 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a74160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a7406de50 0x7f3a7406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a740c6600 con 0x7f3a740ae6e0 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7415fbe0 con 0x7f3a7406de50 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a7b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740c6040 con 0x7f3a7415ba20 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a7406de50 0x7f3a7406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a7406de50 0x7f3a7406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43178/0 (socket says 172.21.15.164:43178) 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 -- 172.21.15.164:0/1283117864 learned_addr learned my addr 172.21.15.164:0/1283117864 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:12.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a73e00640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a74160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 -- 172.21.15.164:0/1283117864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 msgr2=0x7f3a74160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a74160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 -- 172.21.15.164:0/1283117864 >> v1:172.21.15.172:6789/0 conn(0x7f3a740ae6e0 legacy=0x7f3a740cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.512+0000 7f3a6be00640 1 -- 172.21.15.164:0/1283117864 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740c6320 con 0x7f3a7406de50 2024-05-08T04:42:12.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a6be00640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a7406e290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3a7415fd60 tx=0x7f3a5c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13693b8875e26485 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:12.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a73400640 1 -- 172.21.15.164:0/1283117864 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0ab430 con 0x7f3a7406de50 2024-05-08T04:42:12.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a73400640 1 -- 172.21.15.164:0/1283117864 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c0b0070 con 0x7f3a7406de50 2024-05-08T04:42:12.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a73400640 1 -- 172.21.15.164:0/1283117864 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0ab7f0 con 0x7f3a7406de50 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 -- 172.21.15.164:0/1283117864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 msgr2=0x7f3a7406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a7406e290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3a7415fd60 tx=0x7f3a5c01c4d0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 -- 172.21.15.164:0/1283117864 shutdown_connections 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a7406e290 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 --2- 172.21.15.164:0/1283117864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a74160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 -- 172.21.15.164:0/1283117864 >> 172.21.15.164:0/1283117864 conn(0x7f3a740cf770 msgr2=0x7f3a740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 -- 172.21.15.164:0/1283117864 shutdown_connections 2024-05-08T04:42:12.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.516+0000 7f3a7b200640 1 -- 172.21.15.164:0/1283117864 wait complete. 2024-05-08T04:42:12.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 Processor -- start 2024-05-08T04:42:12.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- start start 2024-05-08T04:42:12.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 0x7f3a7414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a7414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740aeca0 con 0x7f3a7406de50 2024-05-08T04:42:12.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740aeb20 con 0x7f3a740ae6e0 2024-05-08T04:42:12.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740aee20 con 0x7f3a7415ba20 2024-05-08T04:42:12.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43192/0 (socket says 172.21.15.164:43192) 2024-05-08T04:42:12.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 -- 172.21.15.164:0/503817112 learned_addr learned my addr 172.21.15.164:0/503817112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:12.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a6be00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 0x7f3a7414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a79200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a7414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 -- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 msgr2=0x7f3a7414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a7414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 -- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 msgr2=0x7f3a7414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 0x7f3a7414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 -- 172.21.15.164:0/503817112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740711d0 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a6be00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 0x7f3a7414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a79200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a7414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a73e00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400d5f0 tx=0x7f3a6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640125f0 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64019070 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a74070410 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640129b0 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a740709d0 con 0x7f3a7406de50 2024-05-08T04:42:12.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.520+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a74072d30 con 0x7f3a7406de50 2024-05-08T04:42:12.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.524+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f3a64012b50 con 0x7f3a7406de50 2024-05-08T04:42:12.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.524+0000 7f3a70c00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 0x7f3a50069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:12.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.524+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5470+0+0 (secure 0 0 0) 0x7f3a640864c0 con 0x7f3a7406de50 2024-05-08T04:42:12.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.524+0000 7f3a6be00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 0x7f3a50069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:12.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.524+0000 7f3a6be00640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 0x7f3a50069ba0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0af000 tx=0x7f3a5c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:12.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.528+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a64056680 con 0x7f3a7406de50 2024-05-08T04:42:12.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:12.688+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a7404f010 con 0x7f3a7406de50 2024-05-08T04:42:13.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.344+0000 7f3a70c00640 1 -- 172.21.15.164:0/503817112 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3a64055f80 con 0x7f3a7406de50 2024-05-08T04:42:13.351 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:42:13.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.348+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 msgr2=0x7f3a50069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.348+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 0x7f3a50069ba0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0af000 tx=0x7f3a5c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:13.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.348+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 msgr2=0x7f3a74151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.348+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400d5f0 tx=0x7f3a6400dab0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 shutdown_connections 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a500676e0 0x7f3a50069ba0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a7415ba20 0x7f3a7414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a740ae6e0 0x7f3a7414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 --2- 172.21.15.164:0/503817112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a7406de50 0x7f3a74151040 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 >> 172.21.15.164:0/503817112 conn(0x7f3a740cf770 msgr2=0x7f3a7415e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 shutdown_connections 2024-05-08T04:42:13.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.352+0000 7f3a7b200640 1 -- 172.21.15.164:0/503817112 wait complete. 2024-05-08T04:42:13.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:42:13.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.516+0000 7f7e45200640 1 Processor -- start 2024-05-08T04:42:13.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.516+0000 7f7e45200640 1 -- start start 2024-05-08T04:42:13.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7e4015e000 0x7f7e400b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006bd60 0x7f7e4006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7e4006c700 0x7f7e400ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4006cb90 con 0x7f7e4006c700 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e400b2fb0 con 0x7f7e4015e000 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e45200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400d06f0 con 0x7f7e4006bd60 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7e4006c700 0x7f7e400ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7e4006c700 0x7f7e400ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43206/0 (socket says 172.21.15.164:43206) 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 -- 172.21.15.164:0/889090956 learned_addr learned my addr 172.21.15.164:0/889090956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3de00640 1 --2- 172.21.15.164:0/889090956 >> v2:172.21.15.172:3300/0 conn(0x7f7e4015e000 0x7f7e400b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e35400640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006bd60 0x7f7e4006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 -- 172.21.15.164:0/889090956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006bd60 msgr2=0x7f7e4006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006bd60 0x7f7e4006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 -- 172.21.15.164:0/889090956 >> v2:172.21.15.172:3300/0 conn(0x7f7e4015e000 msgr2=0x7f7e400b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 --2- 172.21.15.164:0/889090956 >> v2:172.21.15.172:3300/0 conn(0x7f7e4015e000 0x7f7e400b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.520+0000 7f7e3e800640 1 -- 172.21.15.164:0/889090956 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400d09d0 con 0x7f7e4006c700 2024-05-08T04:42:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e3e800640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006c700 0x7f7e400ce1a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7e40071340 tx=0x7f7e3801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6a3d7da830b50e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e3d400640 1 -- 172.21.15.164:0/889090956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a5550 con 0x7f7e4006c700 2024-05-08T04:42:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e3d400640 1 -- 172.21.15.164:0/889090956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e380a56f0 con 0x7f7e4006c700 2024-05-08T04:42:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e3d400640 1 -- 172.21.15.164:0/889090956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a5ac0 con 0x7f7e4006c700 2024-05-08T04:42:13.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- 172.21.15.164:0/889090956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006c700 msgr2=0x7f7e400ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006c700 0x7f7e400ce1a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7e40071340 tx=0x7f7e3801c8d0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- 172.21.15.164:0/889090956 shutdown_connections 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006c700 0x7f7e400ce1a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- 172.21.15.164:0/889090956 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006bd60 0x7f7e4006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- 172.21.15.164:0/889090956 >> v2:172.21.15.172:3300/0 conn(0x7f7e4015e000 0x7f7e400b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- 172.21.15.164:0/889090956 >> 172.21.15.164:0/889090956 conn(0x7f7e400b34f0 msgr2=0x7f7e400b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- 172.21.15.164:0/889090956 shutdown_connections 2024-05-08T04:42:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- 172.21.15.164:0/889090956 wait complete. 2024-05-08T04:42:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 Processor -- start 2024-05-08T04:42:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- start start 2024-05-08T04:42:13.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006c700 0x7f7e400b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e4015e000 0x7f7e400b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b1f50 con 0x7f7e4006bd60 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b1dd0 con 0x7f7e4015e000 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.524+0000 7f7e45200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b20d0 con 0x7f7e4006c700 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43210/0 (socket says 172.21.15.164:43210) 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 -- 172.21.15.164:0/3806260822 learned_addr learned my addr 172.21.15.164:0/3806260822 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e35400640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006c700 0x7f7e400b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3e800640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e4015e000 0x7f7e400b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 -- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006c700 msgr2=0x7f7e400b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006c700 0x7f7e400b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 -- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e4015e000 msgr2=0x7f7e400b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e4015e000 0x7f7e400b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 -- 172.21.15.164:0/3806260822 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400af1e0 con 0x7f7e4006bd60 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e3de00640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e870 tx=0x7f7e2800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28017070 con 0x7f7e4006bd60 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800c940 con 0x7f7e4006bd60 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800ccf0 con 0x7f7e4006bd60 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e400af4c0 con 0x7f7e4006bd60 2024-05-08T04:42:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e400afa80 con 0x7f7e4006bd60 2024-05-08T04:42:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4009fb50 con 0x7f7e4006bd60 2024-05-08T04:42:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.528+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f7e2801a070 con 0x7f7e4006bd60 2024-05-08T04:42:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.532+0000 7f7e36a00640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 0x7f7e1c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.532+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f7e28085cc0 con 0x7f7e4006bd60 2024-05-08T04:42:13.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.532+0000 7f7e35400640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 0x7f7e1c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.532+0000 7f7e35400640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 0x7f7e1c069b70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e4009fb20 tx=0x7f7e30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.532+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e28055ab0 con 0x7f7e4006bd60 2024-05-08T04:42:13.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.688+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7e4006ed80 con 0x7f7e4006bd60 2024-05-08T04:42:13.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.692+0000 7f7e36a00640 1 -- 172.21.15.164:0/3806260822 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7e28059b50 con 0x7f7e4006bd60 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 msgr2=0x7f7e1c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 0x7f7e1c069b70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e4009fb20 tx=0x7f7e30008040 comp rx=0 tx=0).stop 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 msgr2=0x7f7e401f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e870 tx=0x7f7e2800ec70 comp rx=0 tx=0).stop 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 shutdown_connections 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7e1c0676b0 0x7f7e1c069b70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e4006c700 0x7f7e400b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e4015e000 0x7f7e400b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 --2- 172.21.15.164:0/3806260822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4006bd60 0x7f7e401f9f40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 >> 172.21.15.164:0/3806260822 conn(0x7f7e400b34f0 msgr2=0x7f7e400cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:13.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 shutdown_connections 2024-05-08T04:42:13.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.696+0000 7f7e45200640 1 -- 172.21.15.164:0/3806260822 wait complete. 2024-05-08T04:42:13.719 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2024-05-08T04:42:13.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 Processor -- start 2024-05-08T04:42:13.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 -- start start 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 0x7f63f0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63f00b26c0 0x7f63f00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 0x7f63f00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f00d0060 con 0x7f63f00b26c0 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f0072640 con 0x7f63f00cfbd0 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.856+0000 7f63f5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f006c770 con 0x7f63f015ab60 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63f00b26c0 0x7f63f00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63f00b26c0 0x7f63f00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43220/0 (socket says 172.21.15.164:43220) 2024-05-08T04:42:13.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 -- 172.21.15.164:0/2216074882 learned_addr learned my addr 172.21.15.164:0/2216074882 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 -- 172.21.15.164:0/2216074882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 msgr2=0x7f63f0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ee800640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 0x7f63f0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 0x7f63f0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 -- 172.21.15.164:0/2216074882 >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 msgr2=0x7f63f00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ef200640 1 --2- 172.21.15.164:0/2216074882 >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 0x7f63f00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 --2- 172.21.15.164:0/2216074882 >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 0x7f63f00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 -- 172.21.15.164:0/2216074882 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f006ca50 con 0x7f63f00b26c0 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ee800640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 0x7f63f0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:13.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ef200640 1 --2- 172.21.15.164:0/2216074882 >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 0x7f63f00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:13.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ede00640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f00b26c0 0x7f63f00cf690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f63f006dd50 tx=0x7f63dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0825bc15aa59ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ed400640 1 -- 172.21.15.164:0/2216074882 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0b0070 con 0x7f63f00b26c0 2024-05-08T04:42:13.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ed400640 1 -- 172.21.15.164:0/2216074882 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0ab600 con 0x7f63f00b26c0 2024-05-08T04:42:13.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.860+0000 7f63ed400640 1 -- 172.21.15.164:0/2216074882 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0ab9d0 con 0x7f63f00b26c0 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- 172.21.15.164:0/2216074882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f00b26c0 msgr2=0x7f63f00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f00b26c0 0x7f63f00cf690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f63f006dd50 tx=0x7f63dc01c520 comp rx=0 tx=0).stop 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- 172.21.15.164:0/2216074882 shutdown_connections 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- 172.21.15.164:0/2216074882 >> v2:172.21.15.172:3300/0 conn(0x7f63f00cfbd0 0x7f63f00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f00b26c0 0x7f63f00cf690 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- 172.21.15.164:0/2216074882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f015ab60 0x7f63f0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- 172.21.15.164:0/2216074882 >> 172.21.15.164:0/2216074882 conn(0x7f63f006ac90 msgr2=0x7f63f006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- 172.21.15.164:0/2216074882 shutdown_connections 2024-05-08T04:42:13.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- 172.21.15.164:0/2216074882 wait complete. 2024-05-08T04:42:13.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 Processor -- start 2024-05-08T04:42:13.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- start start 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f00b26c0 0x7f63f014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 0x7f63f015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f006c550 con 0x7f63f015ab60 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f006c3d0 con 0x7f63f00b26c0 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63f5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00aed30 con 0x7f63f00cfbd0 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43222/0 (socket says 172.21.15.164:43222) 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 -- 172.21.15.164:0/1526258715 learned_addr learned my addr 172.21.15.164:0/1526258715 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ede00640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 0x7f63f015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ee800640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f00b26c0 0x7f63f014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 -- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 msgr2=0x7f63f015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 0x7f63f015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 -- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f00b26c0 msgr2=0x7f63f014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.864+0000 7f63ef200640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f00b26c0 0x7f63f014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63ef200640 1 -- 172.21.15.164:0/1526258715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f015b7a0 con 0x7f63f015ab60 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63ede00640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 0x7f63f015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63ef200640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c7f0 tx=0x7f63e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8015070 con 0x7f63f015ab60 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8007820 con 0x7f63f015ab60 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8007c10 con 0x7f63f015ab60 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f015ba80 con 0x7f63f015ab60 2024-05-08T04:42:13.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f015c040 con 0x7f63f015ab60 2024-05-08T04:42:13.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0069d60 con 0x7f63f015ab60 2024-05-08T04:42:13.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f63e8018070 con 0x7f63f015ab60 2024-05-08T04:42:13.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 0x7f63d0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:13.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f63e8085fe0 con 0x7f63f015ab60 2024-05-08T04:42:13.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.868+0000 7f63ee800640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 0x7f63d0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:13.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.872+0000 7f63ee800640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 0x7f63d0069b70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f63e000f500 tx=0x7f63e0007770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:13.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:13.872+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63e8055d80 con 0x7f63f015ab60 2024-05-08T04:42:14.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.088+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f63f014fbd0 con 0x7f63f015ab60 2024-05-08T04:42:14.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63e6a00640 1 -- 172.21.15.164:0/1526258715 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v239) v1 ==== 164+0+0 (secure 0 0 0) 0x7f63e8059e20 con 0x7f63f015ab60 2024-05-08T04:42:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 msgr2=0x7f63d0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 0x7f63d0069b70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f63e000f500 tx=0x7f63e0007770 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 msgr2=0x7f63f00adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c7f0 tx=0x7f63e800ccb0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 shutdown_connections 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f63d00676b0 0x7f63d0069b70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63f00cfbd0 0x7f63f015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63f015ab60 0x7f63f00adb20 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 --2- 172.21.15.164:0/1526258715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f00b26c0 0x7f63f014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 >> 172.21.15.164:0/1526258715 conn(0x7f63f006ac90 msgr2=0x7f63f0070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 shutdown_connections 2024-05-08T04:42:14.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.360+0000 7f63f5400640 1 -- 172.21.15.164:0/1526258715 wait complete. 2024-05-08T04:42:14.387 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2024-05-08T04:42:14.387 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot rm' command 2024-05-08T04:42:14.388 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:42:14.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:14.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 Processor -- start 2024-05-08T04:42:14.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 -- start start 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3b806b3a0 0x7fe3b815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b8072440 con 0x7fe3b8071fb0 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80cef60 con 0x7fe3b806b3a0 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80cf1b0 con 0x7fe3b815bb10 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:46774/0 (socket says 172.21.15.164:46774) 2024-05-08T04:42:14.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 -- 172.21.15.164:0/1435905094 learned_addr learned my addr 172.21.15.164:0/1435905094 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:14.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3be000640 1 --2- 172.21.15.164:0/1435905094 >> v2:172.21.15.172:3300/0 conn(0x7fe3b806b3a0 0x7fe3b815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 -- 172.21.15.164:0/1435905094 >> v2:172.21.15.172:3300/0 conn(0x7fe3b806b3a0 msgr2=0x7fe3b815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:14.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 --2- 172.21.15.164:0/1435905094 >> v2:172.21.15.172:3300/0 conn(0x7fe3b806b3a0 0x7fe3b815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 -- 172.21.15.164:0/1435905094 >> v1:172.21.15.172:6789/0 conn(0x7fe3b8071fb0 legacy=0x7fe3b806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.520+0000 7fe3bd600640 1 -- 172.21.15.164:0/1435905094 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80cf490 con 0x7fe3b815bb10 2024-05-08T04:42:14.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bd600640 1 --2- 172.21.15.164:0/1435905094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009e50 tx=0x7fe3ac01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32ab8ef06ff7d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3b7e00640 1 -- 172.21.15.164:0/1435905094 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0a5770 con 0x7fe3b815bb10 2024-05-08T04:42:14.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3b7e00640 1 -- 172.21.15.164:0/1435905094 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac0a5910 con 0x7fe3b815bb10 2024-05-08T04:42:14.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3b7e00640 1 -- 172.21.15.164:0/1435905094 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0a5ce0 con 0x7fe3b815bb10 2024-05-08T04:42:14.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 -- 172.21.15.164:0/1435905094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 msgr2=0x7fe3b80b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 --2- 172.21.15.164:0/1435905094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009e50 tx=0x7fe3ac01c9a0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 -- 172.21.15.164:0/1435905094 shutdown_connections 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 --2- 172.21.15.164:0/1435905094 >> v2:172.21.15.172:3300/0 conn(0x7fe3b806b3a0 0x7fe3b815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 --2- 172.21.15.164:0/1435905094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b815bb10 0x7fe3b80b3d20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 -- 172.21.15.164:0/1435905094 >> 172.21.15.164:0/1435905094 conn(0x7fe3b806bda0 msgr2=0x7fe3b806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 -- 172.21.15.164:0/1435905094 shutdown_connections 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.524+0000 7fe3bea00640 1 -- 172.21.15.164:0/1435905094 wait complete. 2024-05-08T04:42:14.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 Processor -- start 2024-05-08T04:42:14.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- start start 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3b815bb10 0x7fe3b8152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b80b1e50 0x7fe3b81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ce5e0 con 0x7fe3b806b3a0 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ce290 con 0x7fe3b815bb10 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80ce760 con 0x7fe3b80b1e50 2024-05-08T04:42:14.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43230/0 (socket says 172.21.15.164:43230) 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 -- 172.21.15.164:0/3612387112 learned_addr learned my addr 172.21.15.164:0/3612387112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bcc00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3b815bb10 0x7fe3b8152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3be000640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b80b1e50 0x7fe3b81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 -- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b80b1e50 msgr2=0x7fe3b81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b80b1e50 0x7fe3b81533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 -- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3b815bb10 msgr2=0x7fe3b8152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3b815bb10 0x7fe3b8152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 -- 172.21.15.164:0/3612387112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b81541a0 con 0x7fe3b806b3a0 2024-05-08T04:42:14.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bd600640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0af040 tx=0x7fe3ac019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0bc070 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac0b8870 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b81513f0 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0b8c60 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.528+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b8151690 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b804ef30 con 0x7fe3b806b3a0 2024-05-08T04:42:14.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fe3ac0b6410 con 0x7fe3b806b3a0 2024-05-08T04:42:14.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3b5600640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 0x7fe398069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fe3ac120360 con 0x7fe3b806b3a0 2024-05-08T04:42:14.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3bcc00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 0x7fe398069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.532+0000 7fe3bcc00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 0x7fe398069ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe3b81f9e60 tx=0x7fe3a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.536+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3ac016620 con 0x7fe3b806b3a0 2024-05-08T04:42:14.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.692+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b80cd700 con 0x7fe3b806b3a0 2024-05-08T04:42:14.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.692+0000 7fe3b5600640 1 -- 172.21.15.164:0/3612387112 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13217 (secure 0 0 0) 0x7fe3ac0f0660 con 0x7fe3b806b3a0 2024-05-08T04:42:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:14.699 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":91,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:13.350009+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/1815840371":"2024-05-09T04:42:10.189333+0000","172.21.15.164:6836/1815840371":"2024-05-09T04:42:10.189333+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 msgr2=0x7fe398069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 0x7fe398069ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe3b81f9e60 tx=0x7fe3a4009040 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 msgr2=0x7fe3b814e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0af040 tx=0x7fe3ac019740 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 shutdown_connections 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe3980676e0 0x7fe398069ba0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3b80b1e50 0x7fe3b81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3b815bb10 0x7fe3b8152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 --2- 172.21.15.164:0/3612387112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3b806b3a0 0x7fe3b814e8e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 >> 172.21.15.164:0/3612387112 conn(0x7fe3b806bda0 msgr2=0x7fe3b8071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:14.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 shutdown_connections 2024-05-08T04:42:14.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.696+0000 7fe3bea00640 1 -- 172.21.15.164:0/3612387112 wait complete. 2024-05-08T04:42:14.720 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:14.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:14.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.852+0000 7f059cc00640 1 Processor -- start 2024-05-08T04:42:14.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.852+0000 7f059cc00640 1 -- start start 2024-05-08T04:42:14.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f059cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05980cec40 0x7f059815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f059cc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059815c900 con 0x7f05980b1d40 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f059cc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059815cbe0 con 0x7f05980b1960 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f059cc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059815cec0 con 0x7f05980cec40 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05980cec40 0x7f059815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0597e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f05980b1d40 0x7f05980ccca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40184/0 (socket says 172.21.15.164:40184) 2024-05-08T04:42:14.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0597e00640 1 -- 172.21.15.164:0/1395920760 learned_addr learned my addr 172.21.15.164:0/1395920760 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:14.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596a00640 1 -- 172.21.15.164:0/1395920760 >> v1:172.21.15.172:6789/0 conn(0x7f05980b1960 legacy=0x7f0598071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596a00640 1 -- 172.21.15.164:0/1395920760 >> v1:172.21.15.164:6789/0 conn(0x7f05980b1d40 legacy=0x7f05980ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596a00640 1 -- 172.21.15.164:0/1395920760 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980712a0 con 0x7f05980cec40 2024-05-08T04:42:14.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596000640 1 -- 172.21.15.164:0/1395920760 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034508602 0 0) 0x7f059815c900 con 0x7f05980b1d40 2024-05-08T04:42:14.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.856+0000 7f0596a00640 1 --2- 172.21.15.164:0/1395920760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980cec40 0x7f059815bf90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f059815d040 tx=0x7f058401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e4abae41f4557d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0596000640 1 -- 172.21.15.164:0/1395920760 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840b0070 con 0x7f05980cec40 2024-05-08T04:42:14.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0596000640 1 -- 172.21.15.164:0/1395920760 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05840ab600 con 0x7f05980cec40 2024-05-08T04:42:14.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0596000640 1 -- 172.21.15.164:0/1395920760 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840ab9d0 con 0x7f05980cec40 2024-05-08T04:42:14.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- 172.21.15.164:0/1395920760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980cec40 msgr2=0x7f059815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 --2- 172.21.15.164:0/1395920760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980cec40 0x7f059815bf90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f059815d040 tx=0x7f058401c380 comp rx=0 tx=0).stop 2024-05-08T04:42:14.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- 172.21.15.164:0/1395920760 shutdown_connections 2024-05-08T04:42:14.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 --2- 172.21.15.164:0/1395920760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980cec40 0x7f059815bf90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- 172.21.15.164:0/1395920760 >> 172.21.15.164:0/1395920760 conn(0x7f059806ac90 msgr2=0x7f059806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:14.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- 172.21.15.164:0/1395920760 shutdown_connections 2024-05-08T04:42:14.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- 172.21.15.164:0/1395920760 wait complete. 2024-05-08T04:42:14.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 Processor -- start 2024-05-08T04:42:14.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- start start 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980b1d40 0x7f059814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 0x7f059814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598169b40 con 0x7f05980b1960 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059815c6e0 con 0x7f05980cec40 2024-05-08T04:42:14.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f059cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0598169cc0 con 0x7f05980b1d40 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0597400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0597400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43252/0 (socket says 172.21.15.164:43252) 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0597400640 1 -- 172.21.15.164:0/182329347 learned_addr learned my addr 172.21.15.164:0/182329347 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0596a00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980b1d40 0x7f059814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.860+0000 7f0597e00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 0x7f059814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 -- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980b1d40 msgr2=0x7f059814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980b1d40 0x7f059814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 -- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 msgr2=0x7f059814ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 0x7f059814ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 -- 172.21.15.164:0/182329347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598148a00 con 0x7f05980b1960 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597e00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 0x7f059814ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:14.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f0597400640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f058800c7a0 tx=0x7f058800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588003b80 con 0x7f05980b1960 2024-05-08T04:42:14.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588003d20 con 0x7f05980b1960 2024-05-08T04:42:14.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800aa90 con 0x7f05980b1960 2024-05-08T04:42:14.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0598148ce0 con 0x7f05980b1960 2024-05-08T04:42:14.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05981492a0 con 0x7f05980b1960 2024-05-08T04:42:14.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059804ef30 con 0x7f05980b1960 2024-05-08T04:42:14.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f058800ac30 con 0x7f05980b1960 2024-05-08T04:42:14.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 0x7f0578069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:14.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.864+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0588085dd0 con 0x7f05980b1960 2024-05-08T04:42:14.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.868+0000 7f0596a00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 0x7f0578069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:14.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.868+0000 7f0596a00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 0x7f0578069b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f05840ad000 tx=0x7f05840a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:14.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:14.868+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0588056140 con 0x7f05980b1960 2024-05-08T04:42:15.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.024+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0598160420 con 0x7f05980b1960 2024-05-08T04:42:15.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.024+0000 7f058f400640 1 -- 172.21.15.164:0/182329347 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0588055a40 con 0x7f05980b1960 2024-05-08T04:42:15.030 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 msgr2=0x7f0578069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 0x7f0578069b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f05840ad000 tx=0x7f05840a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 msgr2=0x7f059814d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f058800c7a0 tx=0x7f058800cc60 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 shutdown_connections 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05780676b0 0x7f0578069b70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980b1d40 0x7f059814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980cec40 0x7f059814ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 --2- 172.21.15.164:0/182329347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05980b1960 0x7f059814d4c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 >> 172.21.15.164:0/182329347 conn(0x7f059806ac90 msgr2=0x7f059806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:15.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 shutdown_connections 2024-05-08T04:42:15.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.028+0000 7f059cc00640 1 -- 172.21.15.164:0/182329347 wait complete. 2024-05-08T04:42:15.050 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm' 2024-05-08T04:42:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 Processor -- start 2024-05-08T04:42:15.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 -- start start 2024-05-08T04:42:15.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f15d00aee30 0x7f15d00cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15d00cef80 0x7f15d006c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d006e0e0 con 0x7f15d015b390 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00cf3d0 con 0x7f15d00aee30 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.180+0000 7f15d7a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d015e8f0 con 0x7f15d00cef80 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f15d00aee30 0x7f15d00cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f15d00aee30 0x7f15d00cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43258/0 (socket says 172.21.15.164:43258) 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 -- 172.21.15.164:0/1960917574 learned_addr learned my addr 172.21.15.164:0/1960917574 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15d5a00640 1 --2- 172.21.15.164:0/1960917574 >> v2:172.21.15.172:3300/0 conn(0x7f15d00cef80 0x7f15d006c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 -- 172.21.15.164:0/1960917574 >> v2:172.21.15.172:3300/0 conn(0x7f15d00cef80 msgr2=0x7f15d006c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 --2- 172.21.15.164:0/1960917574 >> v2:172.21.15.172:3300/0 conn(0x7f15d00cef80 0x7f15d006c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 -- 172.21.15.164:0/1960917574 >> v1:172.21.15.172:6789/0 conn(0x7f15d015b390 legacy=0x7f15d006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 -- 172.21.15.164:0/1960917574 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d015ed80 con 0x7f15d00aee30 2024-05-08T04:42:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cfe00640 1 --2- 172.21.15.164:0/1960917574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00aee30 0x7f15d00cea40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f15d00aeb60 tx=0x7f15bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16521c32782ce73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cf400640 1 -- 172.21.15.164:0/1960917574 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0b0070 con 0x7f15d00aee30 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cf400640 1 -- 172.21.15.164:0/1960917574 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0ab600 con 0x7f15d00aee30 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.184+0000 7f15cf400640 1 -- 172.21.15.164:0/1960917574 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0ab9d0 con 0x7f15d00aee30 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- 172.21.15.164:0/1960917574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00aee30 msgr2=0x7f15d00cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1960917574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00aee30 0x7f15d00cea40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f15d00aeb60 tx=0x7f15bc01c520 comp rx=0 tx=0).stop 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- 172.21.15.164:0/1960917574 shutdown_connections 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1960917574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00aee30 0x7f15d00cea40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1960917574 >> v2:172.21.15.172:3300/0 conn(0x7f15d00cef80 0x7f15d006c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- 172.21.15.164:0/1960917574 >> 172.21.15.164:0/1960917574 conn(0x7f15d006ac90 msgr2=0x7f15d006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:15.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- 172.21.15.164:0/1960917574 shutdown_connections 2024-05-08T04:42:15.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- 172.21.15.164:0/1960917574 wait complete. 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 Processor -- start 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- start start 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15d00aee30 0x7f15d01f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15d015b390 0x7f15d013e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d006c910 con 0x7f15d00cef80 2024-05-08T04:42:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d015b770 con 0x7f15d015b390 2024-05-08T04:42:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d006ca90 con 0x7f15d00aee30 2024-05-08T04:42:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43274/0 (socket says 172.21.15.164:43274) 2024-05-08T04:42:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 -- 172.21.15.164:0/1694283580 learned_addr learned my addr 172.21.15.164:0/1694283580 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d5000640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15d00aee30 0x7f15d01f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15d5a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15d015b390 0x7f15d013e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 -- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15d00aee30 msgr2=0x7f15d01f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15d00aee30 0x7f15d01f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 -- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15d015b390 msgr2=0x7f15d013e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15d015b390 0x7f15d013e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.188+0000 7f15cfe00640 1 -- 172.21.15.164:0/1694283580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d01f7880 con 0x7f15d00cef80 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15cfe00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f15bc019130 tx=0x7f15bc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0b0040 con 0x7f15d00cef80 2024-05-08T04:42:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0b4620 con 0x7f15d00cef80 2024-05-08T04:42:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d01547f0 con 0x7f15d00cef80 2024-05-08T04:42:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0b4a10 con 0x7f15d00cef80 2024-05-08T04:42:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d00b1920 con 0x7f15d00cef80 2024-05-08T04:42:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d0069d60 con 0x7f15d00cef80 2024-05-08T04:42:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f15bc0ab4c0 con 0x7f15d00cef80 2024-05-08T04:42:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 0x7f15b0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f15bc11f6a0 con 0x7f15d00cef80 2024-05-08T04:42:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.192+0000 7f15d5000640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 0x7f15b0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.196+0000 7f15d5000640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 0x7f15b0069b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15c4001810 tx=0x7f15c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.196+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15d01f7880 con 0x7f15d00cef80 2024-05-08T04:42:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:15.404+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm"]} v 0) v1 -- 0x7f15d015af40 con 0x7f15d00cef80 2024-05-08T04:42:16.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15ccc00640 1 -- 172.21.15.164:0/1694283580 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm"]}]=0 v241) v1 ==== 167+0+0 (secure 0 0 0) 0x7f15bc016110 con 0x7f15d00cef80 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 msgr2=0x7f15b0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 0x7f15b0069b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15c4001810 tx=0x7f15c4009040 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 msgr2=0x7f15d01542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f15bc019130 tx=0x7f15bc0192c0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 shutdown_connections 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f15b00676b0 0x7f15b0069b70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15d015b390 0x7f15d013e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15d00cef80 0x7f15d01542b0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 --2- 172.21.15.164:0/1694283580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15d00aee30 0x7f15d01f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 >> 172.21.15.164:0/1694283580 conn(0x7f15d006ac90 msgr2=0x7f15d006d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 shutdown_connections 2024-05-08T04:42:16.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.368+0000 7f15d7a00640 1 -- 172.21.15.164:0/1694283580 wait complete. 2024-05-08T04:42:16.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:42:16.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 Processor -- start 2024-05-08T04:42:16.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 -- start start 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff75015a200 0x7ff7500aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 0x7ff7500ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500caa90 con 0x7ff7500ca680 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75006bca0 con 0x7ff75015a200 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff756000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500af150 con 0x7ff75006dfa0 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff75015a200 0x7ff7500aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff75015a200 0x7ff7500aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43286/0 (socket says 172.21.15.164:43286) 2024-05-08T04:42:16.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 -- 172.21.15.164:0/335897517 learned_addr learned my addr 172.21.15.164:0/335897517 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74ea00640 1 --2- 172.21.15.164:0/335897517 >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 0x7ff7500ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 -- 172.21.15.164:0/335897517 >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 msgr2=0x7ff7500ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 --2- 172.21.15.164:0/335897517 >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 0x7ff7500ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 -- 172.21.15.164:0/335897517 >> v1:172.21.15.172:6789/0 conn(0x7ff7500ca680 legacy=0x7ff75015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74f400640 1 -- 172.21.15.164:0/335897517 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75006d150 con 0x7ff75015a200 2024-05-08T04:42:16.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.536+0000 7ff74ea00640 1 --2- 172.21.15.164:0/335897517 >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 0x7ff7500ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:16.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff74f400640 1 --2- 172.21.15.164:0/335897517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff7500aeac0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff7500c65f0 tx=0x7ff74001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73af6ad8348cf61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff74e000640 1 -- 172.21.15.164:0/335897517 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400ac020 con 0x7ff75015a200 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff74e000640 1 -- 172.21.15.164:0/335897517 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7400a6450 con 0x7ff75015a200 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff74e000640 1 -- 172.21.15.164:0/335897517 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400a6820 con 0x7ff75015a200 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 -- 172.21.15.164:0/335897517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 msgr2=0x7ff7500aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 --2- 172.21.15.164:0/335897517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff7500aeac0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff7500c65f0 tx=0x7ff74001c710 comp rx=0 tx=0).stop 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 -- 172.21.15.164:0/335897517 shutdown_connections 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 --2- 172.21.15.164:0/335897517 >> v2:172.21.15.172:3300/0 conn(0x7ff75006dfa0 0x7ff7500ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 --2- 172.21.15.164:0/335897517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff7500aeac0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 -- 172.21.15.164:0/335897517 >> 172.21.15.164:0/335897517 conn(0x7ff7500cf770 msgr2=0x7ff7500cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 -- 172.21.15.164:0/335897517 shutdown_connections 2024-05-08T04:42:16.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.540+0000 7ff756000640 1 -- 172.21.15.164:0/335897517 wait complete. 2024-05-08T04:42:16.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 Processor -- start 2024-05-08T04:42:16.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- start start 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff75006dfa0 0x7ff75014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7500ca680 0x7ff7500b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75015a760 con 0x7ff75015a200 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75015a5e0 con 0x7ff75006dfa0 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75015e540 con 0x7ff7500ca680 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff75006dfa0 0x7ff75014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43290/0 (socket says 172.21.15.164:43290) 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 -- 172.21.15.164:0/2150153892 learned_addr learned my addr 172.21.15.164:0/2150153892 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74ea00640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7500ca680 0x7ff7500b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 -- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7500ca680 msgr2=0x7ff7500b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7500ca680 0x7ff7500b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 -- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff75006dfa0 msgr2=0x7ff75014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff75006dfa0 0x7ff75014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 -- 172.21.15.164:0/2150153892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500c5de0 con 0x7ff75015a200 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff74fe00640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c3c0 tx=0x7ff74800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748011790 con 0x7ff75015a200 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748017070 con 0x7ff75015a200 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748011b70 con 0x7ff75015a200 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7500b2ca0 con 0x7ff75015a200 2024-05-08T04:42:16.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.544+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7500b3260 con 0x7ff75015a200 2024-05-08T04:42:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750072d30 con 0x7ff75015a200 2024-05-08T04:42:16.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7ff748011d10 con 0x7ff75015a200 2024-05-08T04:42:16.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff747400640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 0x7ff730069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:16.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7ff748085bb0 con 0x7ff75015a200 2024-05-08T04:42:16.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff74f400640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 0x7ff730069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:16.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.548+0000 7ff74f400640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 0x7ff730069b70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff7400ad000 tx=0x7ff7400b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:16.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.552+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7480558d0 con 0x7ff75015a200 2024-05-08T04:42:16.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.708+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff7500aee70 con 0x7ff75015a200 2024-05-08T04:42:16.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff747400640 1 -- 172.21.15.164:0/2150153892 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff748059970 con 0x7ff75015a200 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 msgr2=0x7ff730069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 0x7ff730069b70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff7400ad000 tx=0x7ff7400b3040 comp rx=0 tx=0).stop 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 msgr2=0x7ff750168120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c3c0 tx=0x7ff74800c880 comp rx=0 tx=0).stop 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 shutdown_connections 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff7300676b0 0x7ff730069b70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7500ca680 0x7ff7500b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff75015a200 0x7ff750168120 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 --2- 172.21.15.164:0/2150153892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff75006dfa0 0x7ff75014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:16.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 >> 172.21.15.164:0/2150153892 conn(0x7ff7500cf770 msgr2=0x7ff7500c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:16.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 shutdown_connections 2024-05-08T04:42:16.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:16.712+0000 7ff756000640 1 -- 172.21.15.164:0/2150153892 wait complete. 2024-05-08T04:42:16.738 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:42:16.738 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:16.834 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:16.886 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:42:16.886 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:16.886 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:16.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:16.895 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:16.895 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:16.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:16.905 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:16.905 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:16.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:16.948 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:16.948 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:16.958 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:16.958 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:42:16.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:17.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.124+0000 7f7857a00640 1 Processor -- start 2024-05-08T04:42:17.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.124+0000 7f7857a00640 1 -- start start 2024-05-08T04:42:17.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7857a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7850144200 0x7f7850161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7857a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7850161fe0 con 0x7f785015ea20 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7857a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78501622c0 con 0x7f78500cfd60 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7857a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7850162880 con 0x7f7850144200 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7850144200 0x7f7850161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7850144200 0x7f7850161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43298/0 (socket says 172.21.15.164:43298) 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 -- 172.21.15.164:0/3325088337 learned_addr learned my addr 172.21.15.164:0/3325088337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855a00640 1 --1- 172.21.15.164:0/3325088337 >> v1:172.21.15.164:6789/0 conn(0x7f785015ea20 0x7f78500ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40198/0 (socket says 172.21.15.164:40198) 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 -- 172.21.15.164:0/3325088337 >> v1:172.21.15.172:6789/0 conn(0x7f78500cfd60 legacy=0x7f78500cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 -- 172.21.15.164:0/3325088337 >> v1:172.21.15.164:6789/0 conn(0x7f785015ea20 legacy=0x7f78500ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 -- 172.21.15.164:0/3325088337 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78501625a0 con 0x7f7850144200 2024-05-08T04:42:17.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f7855000640 1 --2- 172.21.15.164:0/3325088337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f7850161950 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7840009870 tx=0x7f784001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d510ba05262da65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.128+0000 7f784fe00640 1 -- 172.21.15.164:0/3325088337 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400b0070 con 0x7f7850144200 2024-05-08T04:42:17.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f784fe00640 1 -- 172.21.15.164:0/3325088337 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400ab600 con 0x7f7850144200 2024-05-08T04:42:17.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f784fe00640 1 -- 172.21.15.164:0/3325088337 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ab9d0 con 0x7f7850144200 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- 172.21.15.164:0/3325088337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 msgr2=0x7f7850161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 --2- 172.21.15.164:0/3325088337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f7850161950 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7840009870 tx=0x7f784001c490 comp rx=0 tx=0).stop 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- 172.21.15.164:0/3325088337 shutdown_connections 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 --2- 172.21.15.164:0/3325088337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f7850161950 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- 172.21.15.164:0/3325088337 >> 172.21.15.164:0/3325088337 conn(0x7f785006ac90 msgr2=0x7f785006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- 172.21.15.164:0/3325088337 shutdown_connections 2024-05-08T04:42:17.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- 172.21.15.164:0/3325088337 wait complete. 2024-05-08T04:42:17.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 Processor -- start 2024-05-08T04:42:17.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- start start 2024-05-08T04:42:17.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 0x7f785013b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 0x7f78500b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500cc940 con 0x7f7850144200 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500cc7c0 con 0x7f785015ea20 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7857a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785016c510 con 0x7f78500cfd60 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43314/0 (socket says 172.21.15.164:43314) 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 -- 172.21.15.164:0/249423935 learned_addr learned my addr 172.21.15.164:0/249423935 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7855a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 0x7f78500b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7855000640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 0x7f785013b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 -- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 msgr2=0x7f785013b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 0x7f785013b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 -- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 msgr2=0x7f78500b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 0x7f78500b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7847e00640 1 -- 172.21.15.164:0/249423935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785013b960 con 0x7f7850144200 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7855000640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 0x7f785013b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.132+0000 7f7855a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 0x7f78500b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f7847e00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f783800b440 tx=0x7f783800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838015020 con 0x7f7850144200 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838011890 con 0x7f7850144200 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838011c80 con 0x7f7850144200 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78500b3430 con 0x7f7850144200 2024-05-08T04:42:17.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78500b39f0 con 0x7f7850144200 2024-05-08T04:42:17.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850069d60 con 0x7f7850144200 2024-05-08T04:42:17.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f7838018020 con 0x7f7850144200 2024-05-08T04:42:17.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 0x7f782c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f7838085940 con 0x7f7850144200 2024-05-08T04:42:17.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.136+0000 7f7855000640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 0x7f782c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.140+0000 7f7855000640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 0x7f782c069b70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f78400ad000 tx=0x7f78400a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.140+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f783801f030 con 0x7f7850144200 2024-05-08T04:42:17.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.328+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78500af110 con 0x7f7850144200 2024-05-08T04:42:17.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.328+0000 7f784d600640 1 -- 172.21.15.164:0/249423935 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f783800d040 con 0x7f7850144200 2024-05-08T04:42:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:17.334 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6682,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1863435525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1863435525},{"type":"v1","addr":"172.21.15.172:6834","nonce":1863435525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/166886895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":166886895},{"type":"v1","addr":"172.21.15.172:6835","nonce":166886895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2380295639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2380295639},{"type":"v1","addr":"172.21.15.164:6835","nonce":2380295639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7356,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2221807175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":2221807175},{"type":"v1","addr":"172.21.15.164:6837","nonce":2221807175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2024-05-08T04:42:17.335 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 55 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 msgr2=0x7f782c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 0x7f782c069b70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f78400ad000 tx=0x7f78400a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 msgr2=0x7f785013d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f783800b440 tx=0x7f783800b900 comp rx=0 tx=0).stop 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 shutdown_connections 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f782c0676b0 0x7f782c069b70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f785015ea20 0x7f78500b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7850144200 0x7f785013d8a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 --2- 172.21.15.164:0/249423935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500cfd60 0x7f785013b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 >> 172.21.15.164:0/249423935 conn(0x7f785006ac90 msgr2=0x7f78500b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:17.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 shutdown_connections 2024-05-08T04:42:17.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.332+0000 7f7857a00640 1 -- 172.21.15.164:0/249423935 wait complete. 2024-05-08T04:42:17.355 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:42:17.355 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:42:17.355 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:42:17.355 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:17.356 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:42:17.356 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:42:17.356 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:42:17.356 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:17.356 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:42:17.356 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:42:17.357 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:42:17.357 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:17.357 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:42:17.357 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:42:17.357 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:42:17.357 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:17.371 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:42:17.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:42:17.372 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:42:17.372 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:42:17.374 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:42:17.374 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:42:17.376 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:42:17.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:42:17.422 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:42:17.423 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:17.423 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:42:17.424 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:17.424 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:42:17.425 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:17.427 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:42:17.427 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:17.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:42:17.510 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:42:17.510 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:42:17.514 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:42:17.515 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:42:17.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 Processor -- start 2024-05-08T04:42:17.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 -- start start 2024-05-08T04:42:17.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 0x7f382c1620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f382c1625f0 0x7f382c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c06cc70 con 0x7f382c15dd70 2024-05-08T04:42:17.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c06cf50 con 0x7f382c1625f0 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3834000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0b1570 con 0x7f382c06c7b0 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f382c1625f0 0x7f382c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f382c1625f0 0x7f382c0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43356/0 (socket says 172.21.15.164:43356) 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 -- 172.21.15.164:0/1515316124 learned_addr learned my addr 172.21.15.164:0/1515316124 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3830c00640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 0x7f382c1620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 -- 172.21.15.164:0/1515316124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 msgr2=0x7f382c1620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 0x7f382c1620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 -- 172.21.15.164:0/1515316124 >> v1:172.21.15.172:6789/0 conn(0x7f382c15dd70 legacy=0x7f382c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3832000640 1 -- 172.21.15.164:0/1515316124 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c0b1850 con 0x7f382c1625f0 2024-05-08T04:42:17.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.568+0000 7f3830c00640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 0x7f382c1620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:17.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3832000640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c1625f0 0x7f382c0b1d70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3824009e50 tx=0x7f382401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e68ac36ba630247 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f382be00640 1 -- 172.21.15.164:0/1515316124 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240b4070 con 0x7f382c1625f0 2024-05-08T04:42:17.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f382be00640 1 -- 172.21.15.164:0/1515316124 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240af400 con 0x7f382c1625f0 2024-05-08T04:42:17.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f382be00640 1 -- 172.21.15.164:0/1515316124 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240af7d0 con 0x7f382c1625f0 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 -- 172.21.15.164:0/1515316124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c1625f0 msgr2=0x7f382c0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c1625f0 0x7f382c0b1d70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3824009e50 tx=0x7f382401f210 comp rx=0 tx=0).stop 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 -- 172.21.15.164:0/1515316124 shutdown_connections 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c1625f0 0x7f382c0b1d70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 --2- 172.21.15.164:0/1515316124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c06c7b0 0x7f382c1620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 -- 172.21.15.164:0/1515316124 >> 172.21.15.164:0/1515316124 conn(0x7f382c0b34f0 msgr2=0x7f382c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 -- 172.21.15.164:0/1515316124 shutdown_connections 2024-05-08T04:42:17.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.572+0000 7f3834000640 1 -- 172.21.15.164:0/1515316124 wait complete. 2024-05-08T04:42:17.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 Processor -- start 2024-05-08T04:42:17.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- start start 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c15dd70 0x7f382c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f382c1625f0 0x7f382c15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c15e2d0 con 0x7f382c06c7b0 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c15e150 con 0x7f382c1625f0 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0b22b0 con 0x7f382c15dd70 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43362/0 (socket says 172.21.15.164:43362) 2024-05-08T04:42:17.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 -- 172.21.15.164:0/2747661808 learned_addr learned my addr 172.21.15.164:0/2747661808 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3830c00640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c15dd70 0x7f382c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3832000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f382c1625f0 0x7f382c15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 -- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c15dd70 msgr2=0x7f382c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c15dd70 0x7f382c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 -- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f382c1625f0 msgr2=0x7f382c15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f382c1625f0 0x7f382c15e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 -- 172.21.15.164:0/2747661808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c1f4990 con 0x7f382c06c7b0 2024-05-08T04:42:17.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3831600640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f382000c420 tx=0x7f382000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200052d0 con 0x7f382c06c7b0 2024-05-08T04:42:17.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820005470 con 0x7f382c06c7b0 2024-05-08T04:42:17.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820007260 con 0x7f382c06c7b0 2024-05-08T04:42:17.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c1f4c70 con 0x7f382c06c7b0 2024-05-08T04:42:17.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c0b5b40 con 0x7f382c06c7b0 2024-05-08T04:42:17.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.576+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c069d60 con 0x7f382c06c7b0 2024-05-08T04:42:17.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.580+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f3820007400 con 0x7f382c06c7b0 2024-05-08T04:42:17.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.580+0000 7f3829600640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 0x7f380c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:17.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.580+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f38200853e0 con 0x7f382c06c7b0 2024-05-08T04:42:17.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.580+0000 7f3830c00640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 0x7f380c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:17.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.580+0000 7f3830c00640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 0x7f380c069b70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f382c0c7c00 tx=0x7f3818008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:17.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.584+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3820055100 con 0x7f382c06c7b0 2024-05-08T04:42:17.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:17.748+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f382c0b4010 con 0x7f382c06c7b0 2024-05-08T04:42:18.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.384+0000 7f3829600640 1 -- 172.21.15.164:0/2747661808 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f38200591a0 con 0x7f382c06c7b0 2024-05-08T04:42:18.390 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 msgr2=0x7f380c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 0x7f380c069b70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f382c0c7c00 tx=0x7f3818008040 comp rx=0 tx=0).stop 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 msgr2=0x7f382c156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f382000c420 tx=0x7f382000c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 shutdown_connections 2024-05-08T04:42:18.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f380c0676b0 0x7f380c069b70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f382c15dd70 0x7f382c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f382c1625f0 0x7f382c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 --2- 172.21.15.164:0/2747661808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f382c06c7b0 0x7f382c156f10 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 >> 172.21.15.164:0/2747661808 conn(0x7f382c0b34f0 msgr2=0x7f382c0b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 shutdown_connections 2024-05-08T04:42:18.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.388+0000 7f3834000640 1 -- 172.21.15.164:0/2747661808 wait complete. 2024-05-08T04:42:18.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:42:18.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 Processor -- start 2024-05-08T04:42:18.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 -- start start 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4c40cfda0 0x7fb4c40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c406c770 con 0x7fb4c40b26c0 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c406ca50 con 0x7fb4c415ab60 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4cbe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c406cd30 con 0x7fb4c40cfda0 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c9e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4c40cfda0 0x7fb4c40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c8a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb4c40b26c0 0x7fb4c40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40222/0 (socket says 172.21.15.164:40222) 2024-05-08T04:42:18.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c8a00640 1 -- 172.21.15.164:0/1105365769 learned_addr learned my addr 172.21.15.164:0/1105365769 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1105365769 >> v1:172.21.15.172:6789/0 conn(0x7fb4c415ab60 legacy=0x7fb4c4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1105365769 >> v1:172.21.15.164:6789/0 conn(0x7fb4c40b26c0 legacy=0x7fb4c40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1105365769 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c406d010 con 0x7fb4c40cfda0 2024-05-08T04:42:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.552+0000 7fb4c3e00640 1 -- 172.21.15.164:0/1105365769 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868226716 0 0) 0x7fb4c406c770 con 0x7fb4c40b26c0 2024-05-08T04:42:18.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4c9e00640 1 --2- 172.21.15.164:0/1105365769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c40cfda0 0x7fb4c40cd910 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009d90 tx=0x7fb4bc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b493161776d52fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4c3e00640 1 -- 172.21.15.164:0/1105365769 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0b2070 con 0x7fb4c40cfda0 2024-05-08T04:42:18.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4c3e00640 1 -- 172.21.15.164:0/1105365769 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc0a5830 con 0x7fb4c40cfda0 2024-05-08T04:42:18.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4c3e00640 1 -- 172.21.15.164:0/1105365769 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0a5c00 con 0x7fb4c40cfda0 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1105365769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c40cfda0 msgr2=0x7fb4c40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1105365769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c40cfda0 0x7fb4c40cd910 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009d90 tx=0x7fb4bc01c970 comp rx=0 tx=0).stop 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1105365769 shutdown_connections 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1105365769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c40cfda0 0x7fb4c40cd910 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1105365769 >> 172.21.15.164:0/1105365769 conn(0x7fb4c406ac90 msgr2=0x7fb4c406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1105365769 shutdown_connections 2024-05-08T04:42:18.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.556+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1105365769 wait complete. 2024-05-08T04:42:18.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 Processor -- start 2024-05-08T04:42:18.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- start start 2024-05-08T04:42:18.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 0x7fb4c41565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 0x7fb4c414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4168b00 con 0x7fb4c415ab60 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4168980 con 0x7fb4c40b26c0 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4168c80 con 0x7fb4c40cfda0 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43386/0 (socket says 172.21.15.164:43386) 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1014798663 learned_addr learned my addr 172.21.15.164:0/1014798663 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9400640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 0x7fb4c41565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c8a00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 0x7fb4c414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 msgr2=0x7fb4c414d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 0x7fb4c414d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 msgr2=0x7fb4c41565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 0x7fb4c41565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 -- 172.21.15.164:0/1014798663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c414d8d0 con 0x7fb4c415ab60 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9400640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 0x7fb4c41565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c8a00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 0x7fb4c414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c9e00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc01a5a0 tx=0x7fb4bc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0b2050 con 0x7fb4c415ab60 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc0be070 con 0x7fb4c415ab60 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c4153d30 con 0x7fb4c415ab60 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0ada50 con 0x7fb4c415ab60 2024-05-08T04:42:18.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.560+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c4155dd0 con 0x7fb4c415ab60 2024-05-08T04:42:18.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c4069d60 con 0x7fb4c415ab60 2024-05-08T04:42:18.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fb4bc0adbf0 con 0x7fb4c415ab60 2024-05-08T04:42:18.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4c1600640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 0x7fb4a4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fb4bc120d00 con 0x7fb4c415ab60 2024-05-08T04:42:18.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4c9400640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 0x7fb4a4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.564+0000 7fb4c9400640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 0x7fb4a4069b70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40d0660 tx=0x7fb4b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.568+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4bc016560 con 0x7fb4c415ab60 2024-05-08T04:42:18.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb4c406d9a0 con 0x7fb4c415ab60 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4c1600640 1 -- 172.21.15.164:0/1014798663 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb4bc0ab340 con 0x7fb4c415ab60 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:42:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:42:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:42:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:42:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:42:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:42:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:42:18.798 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:42:18.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 msgr2=0x7fb4a4069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 0x7fb4a4069b70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40d0660 tx=0x7fb4b8009040 comp rx=0 tx=0).stop 2024-05-08T04:42:18.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 msgr2=0x7fb4c40b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc01a5a0 tx=0x7fb4bc019710 comp rx=0 tx=0).stop 2024-05-08T04:42:18.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 shutdown_connections 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4a40676b0 0x7fb4a4069b70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4c40cfda0 0x7fb4c414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c415ab60 0x7fb4c40b45e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 --2- 172.21.15.164:0/1014798663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c40b26c0 0x7fb4c41565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 >> 172.21.15.164:0/1014798663 conn(0x7fb4c406ac90 msgr2=0x7fb4c406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 shutdown_connections 2024-05-08T04:42:18.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.780+0000 7fb4cbe00640 1 -- 172.21.15.164:0/1014798663 wait complete. 2024-05-08T04:42:18.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:18.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 Processor -- start 2024-05-08T04:42:18.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 -- start start 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a8161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a8161fe0 con 0x7f43a80cfd60 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a81622c0 con 0x7f43a815ea20 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f43acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8162880 con 0x7f43a8144200 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f439d400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f43a80cfd60 0x7f43a80cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:40236/0 (socket says 172.21.15.164:40236) 2024-05-08T04:42:18.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.936+0000 7f439d400640 1 -- 172.21.15.164:0/865083768 learned_addr learned my addr 172.21.15.164:0/865083768 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5e00640 1 --2- 172.21.15.164:0/865083768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a8161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75624320 0 0) 0x7f43a8161fe0 con 0x7f43a80cfd60 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f438c003530 con 0x7f43a80cfd60 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243447661 0 0) 0x7f43a81622c0 con 0x7f43a815ea20 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a8161fe0 con 0x7f43a815ea20 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5e00640 1 -- 172.21.15.164:0/865083768 >> v1:172.21.15.172:6789/0 conn(0x7f43a815ea20 legacy=0x7f43a80ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3295253065 0 0) 0x7f438c003530 con 0x7f43a80cfd60 2024-05-08T04:42:18.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5e00640 1 -- 172.21.15.164:0/865083768 >> v1:172.21.15.164:6789/0 conn(0x7f43a80cfd60 legacy=0x7f43a80cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5e00640 1 -- 172.21.15.164:0/865083768 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a81625a0 con 0x7f43a8144200 2024-05-08T04:42:18.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5e00640 1 --2- 172.21.15.164:0/865083768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a8161950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4398009dc0 tx=0x7f439801c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3004a1384266f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980ac070 con 0x7f43a8144200 2024-05-08T04:42:18.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43980a6470 con 0x7f43a8144200 2024-05-08T04:42:18.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.940+0000 7f43a5400640 1 -- 172.21.15.164:0/865083768 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390001680 con 0x7f43a8144200 2024-05-08T04:42:18.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- 172.21.15.164:0/865083768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 msgr2=0x7f43a8161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 --2- 172.21.15.164:0/865083768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a8161950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4398009dc0 tx=0x7f439801c820 comp rx=0 tx=0).stop 2024-05-08T04:42:18.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- 172.21.15.164:0/865083768 shutdown_connections 2024-05-08T04:42:18.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 --2- 172.21.15.164:0/865083768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a8161950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- 172.21.15.164:0/865083768 >> 172.21.15.164:0/865083768 conn(0x7f43a806ac90 msgr2=0x7f43a806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:18.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- 172.21.15.164:0/865083768 shutdown_connections 2024-05-08T04:42:18.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- 172.21.15.164:0/865083768 wait complete. 2024-05-08T04:42:18.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 Processor -- start 2024-05-08T04:42:18.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- start start 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a80cfd60 0x7f43a813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a81576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cc940 con 0x7f43a815ea20 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cc7c0 con 0x7f43a80cfd60 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a816c510 con 0x7f43a8144200 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43392/0 (socket says 172.21.15.164:43392) 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 -- 172.21.15.164:0/4200684444 learned_addr learned my addr 172.21.15.164:0/4200684444 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f439d400640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a81576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a5e00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a80cfd60 0x7f43a813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 -- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 msgr2=0x7f43a81576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a81576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 -- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a80cfd60 msgr2=0x7f43a813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a80cfd60 0x7f43a813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.944+0000 7f43a6800640 1 -- 172.21.15.164:0/4200684444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a815bdb0 con 0x7f43a815ea20 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f43a6800640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c410 tx=0x7f43a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0017070 con 0x7f43a815ea20 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a00052d0 con 0x7f43a815ea20 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a815c090 con 0x7f43a815ea20 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00054c0 con 0x7f43a815ea20 2024-05-08T04:42:18.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a8157be0 con 0x7f43a815ea20 2024-05-08T04:42:18.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a8069d60 con 0x7f43a815ea20 2024-05-08T04:42:18.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f43a001a070 con 0x7f43a815ea20 2024-05-08T04:42:18.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 0x7f4384069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:18.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f43a0085210 con 0x7f43a815ea20 2024-05-08T04:42:18.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.948+0000 7f43a5e00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 0x7f4384069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:18.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.952+0000 7f43a5e00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 0x7f4384069ba0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43980ae000 tx=0x7f43980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:18.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:18.952+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43a0054f10 con 0x7f43a815ea20 2024-05-08T04:42:19.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.108+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a80add30 con 0x7f43a815ea20 2024-05-08T04:42:19.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.108+0000 7f439ea00640 1 -- 172.21.15.164:0/4200684444 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+13086 (secure 0 0 0) 0x7f43a0083e20 con 0x7f43a815ea20 2024-05-08T04:42:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:19.115 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":95,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:18.401072+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 msgr2=0x7f4384069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 0x7f4384069ba0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43980ae000 tx=0x7f43980a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 msgr2=0x7f43a815abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c410 tx=0x7f43a000c8d0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 shutdown_connections 2024-05-08T04:42:19.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43840676e0 0x7f4384069ba0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43a8144200 0x7f43a81576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43a815ea20 0x7f43a815abc0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 --2- 172.21.15.164:0/4200684444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a80cfd60 0x7f43a813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 >> 172.21.15.164:0/4200684444 conn(0x7f43a806ac90 msgr2=0x7f43a80b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 shutdown_connections 2024-05-08T04:42:19.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.112+0000 7f43acc00640 1 -- 172.21.15.164:0/4200684444 wait complete. 2024-05-08T04:42:19.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:19.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:19.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 Processor -- start 2024-05-08T04:42:19.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 -- start start 2024-05-08T04:42:19.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f581c06cce0 0x7f581c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c15e870 con 0x7f581c0cecc0 2024-05-08T04:42:19.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c15eb50 con 0x7f581c15ab10 2024-05-08T04:42:19.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5821600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c06bb80 con 0x7f581c06cce0 2024-05-08T04:42:19.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f581c0cecc0 0x7f581c0ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51806/0 (socket says 172.21.15.164:51806) 2024-05-08T04:42:19.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581b400640 1 -- 172.21.15.164:0/2320476723 learned_addr learned my addr 172.21.15.164:0/2320476723 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:19.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082890107 0 0) 0x7f581c15e870 con 0x7f581c0cecc0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5800003530 con 0x7f581c0cecc0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581a000640 1 --2- 172.21.15.164:0/2320476723 >> v2:172.21.15.172:3300/0 conn(0x7f581c06cce0 0x7f581c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916156525 0 0) 0x7f5800003530 con 0x7f581c0cecc0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 >> v2:172.21.15.172:3300/0 conn(0x7f581c06cce0 msgr2=0x7f581c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 --2- 172.21.15.164:0/2320476723 >> v2:172.21.15.172:3300/0 conn(0x7f581c06cce0 0x7f581c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 >> v1:172.21.15.172:6789/0 conn(0x7f581c15ab10 legacy=0x7f581c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c15ee30 con 0x7f581c0cecc0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5814001680 con 0x7f581c0cecc0 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 >> v1:172.21.15.164:6789/0 conn(0x7f581c0cecc0 legacy=0x7f581c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 0x7f5800003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000051d0 0x7f580000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c06bb80 con 0x7f58000039c0 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580000c170 con 0x7f58000051d0 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580000e580 con 0x7f5800004810 2024-05-08T04:42:19.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 0x7f5800003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581a000640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581b400640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000051d0 0x7f580000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 -- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5800004810 msgr2=0x7f5800004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:19.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 -- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000051d0 msgr2=0x7f580000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:19.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000051d0 0x7f580000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581aa00640 1 -- 172.21.15.164:0/2320476723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800003530 con 0x7f58000039c0 2024-05-08T04:42:19.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.272+0000 7f581b400640 1 -- 172.21.15.164:0/2320476723 reap_dead start 2024-05-08T04:42:19.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f581aa00640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 0x7f5800003de0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f580c001e70 tx=0x7f580c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0741b5babca43b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0b0070 con 0x7f58000039c0 2024-05-08T04:42:19.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c0a7070 con 0x7f58000039c0 2024-05-08T04:42:19.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5819600640 1 -- 172.21.15.164:0/2320476723 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c001a90 con 0x7f58000039c0 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 -- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 msgr2=0x7f5800003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 0x7f5800003de0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f580c001e70 tx=0x7f580c01c760 comp rx=0 tx=0).stop 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 -- 172.21.15.164:0/2320476723 shutdown_connections 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 --2- 172.21.15.164:0/2320476723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58000039c0 0x7f5800003de0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 -- 172.21.15.164:0/2320476723 >> 172.21.15.164:0/2320476723 conn(0x7f581c0cf770 msgr2=0x7f581c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 -- 172.21.15.164:0/2320476723 shutdown_connections 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.276+0000 7f5821600640 1 -- 172.21.15.164:0/2320476723 wait complete. 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 Processor -- start 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- start start 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000039c0 0x7f581c071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f581c15cc30 0x7f581c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c54e0 con 0x7f581c153f40 2024-05-08T04:42:19.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c5360 con 0x7f58000039c0 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c5660 con 0x7f581c15cc30 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44706/0 (socket says 172.21.15.164:44706) 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 -- 172.21.15.164:0/3520268187 learned_addr learned my addr 172.21.15.164:0/3520268187 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581a000640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f581c15cc30 0x7f581c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581aa00640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000039c0 0x7f581c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 -- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f581c15cc30 msgr2=0x7f581c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f581c15cc30 0x7f581c15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 -- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000039c0 msgr2=0x7f581c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000039c0 0x7f581c071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 -- 172.21.15.164:0/3520268187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c15bd20 con 0x7f581c153f40 2024-05-08T04:42:19.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f581b400640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f581c0cf6d0 tx=0x7f581400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400c690 con 0x7f581c153f40 2024-05-08T04:42:19.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814018070 con 0x7f581c153f40 2024-05-08T04:42:19.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400ca90 con 0x7f581c153f40 2024-05-08T04:42:19.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c15c000 con 0x7f581c153f40 2024-05-08T04:42:19.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c15c5c0 con 0x7f581c153f40 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.280+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c072d30 con 0x7f581c153f40 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.284+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f581400cc30 con 0x7f581c153f40 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.284+0000 7f5812a00640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 0x7f57fc069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.284+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f58140859d0 con 0x7f581c153f40 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.284+0000 7f581aa00640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 0x7f57fc069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.284+0000 7f581aa00640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 0x7f57fc069b70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f580c0ad000 tx=0x7f580c001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.288+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5814055700 con 0x7f581c153f40 2024-05-08T04:42:19.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f581c04f010 con 0x7f581c153f40 2024-05-08T04:42:19.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5812a00640 1 -- 172.21.15.164:0/3520268187 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58140597a0 con 0x7f581c153f40 2024-05-08T04:42:19.452 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 msgr2=0x7f57fc069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 0x7f57fc069b70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f580c0ad000 tx=0x7f580c001520 comp rx=0 tx=0).stop 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 msgr2=0x7f581c1543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f581c0cf6d0 tx=0x7f581400eb90 comp rx=0 tx=0).stop 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 shutdown_connections 2024-05-08T04:42:19.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f57fc0676b0 0x7f57fc069b70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f581c15cc30 0x7f581c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581c153f40 0x7f581c1543a0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 --2- 172.21.15.164:0/3520268187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58000039c0 0x7f581c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 >> 172.21.15.164:0/3520268187 conn(0x7f581c0cf770 msgr2=0x7f581c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 shutdown_connections 2024-05-08T04:42:19.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.448+0000 7f5821600640 1 -- 172.21.15.164:0/3520268187 wait complete. 2024-05-08T04:42:19.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:42:19.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 Processor -- start 2024-05-08T04:42:19.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 -- start start 2024-05-08T04:42:19.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9e44161230 0x7f9e440d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440b8440 con 0x7f9e440ce640 2024-05-08T04:42:19.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440d1ec0 con 0x7f9e440d17a0 2024-05-08T04:42:19.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e49e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440d1c70 con 0x7f9e44161230 2024-05-08T04:42:19.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e43400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9e440ce640 0x7f9e440cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51814/0 (socket says 172.21.15.164:51814) 2024-05-08T04:42:19.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e43400640 1 -- 172.21.15.164:0/1808278481 learned_addr learned my addr 172.21.15.164:0/1808278481 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:19.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e42a00640 1 --2- 172.21.15.164:0/1808278481 >> v2:172.21.15.172:3300/0 conn(0x7f9e44161230 0x7f9e440d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690363514 0 0) 0x7f9e3c0014a0 con 0x7f9e440ce640 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.604+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e28003530 con 0x7f9e440ce640 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923534269 0 0) 0x7f9e440d1ec0 con 0x7f9e440d17a0 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e3c0014a0 con 0x7f9e440d17a0 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42a00640 1 -- 172.21.15.164:0/1808278481 >> v1:172.21.15.172:6789/0 conn(0x7f9e440d17a0 legacy=0x7f9e440d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42a00640 1 -- 172.21.15.164:0/1808278481 >> v1:172.21.15.164:6789/0 conn(0x7f9e440ce640 legacy=0x7f9e440cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42a00640 1 -- 172.21.15.164:0/1808278481 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440d2250 con 0x7f9e44161230 2024-05-08T04:42:19.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42a00640 1 --2- 172.21.15.164:0/1808278481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e44161230 0x7f9e440d0370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e340097c0 tx=0x7f9e3401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f091cfe8279f4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340b0070 con 0x7f9e44161230 2024-05-08T04:42:19.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e340ab600 con 0x7f9e44161230 2024-05-08T04:42:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e42000640 1 -- 172.21.15.164:0/1808278481 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340ab9d0 con 0x7f9e44161230 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 -- 172.21.15.164:0/1808278481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e44161230 msgr2=0x7f9e440d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 --2- 172.21.15.164:0/1808278481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e44161230 0x7f9e440d0370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e340097c0 tx=0x7f9e3401c410 comp rx=0 tx=0).stop 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 -- 172.21.15.164:0/1808278481 shutdown_connections 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 --2- 172.21.15.164:0/1808278481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e44161230 0x7f9e440d0370 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 -- 172.21.15.164:0/1808278481 >> 172.21.15.164:0/1808278481 conn(0x7f9e4409cea0 msgr2=0x7f9e440d3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 -- 172.21.15.164:0/1808278481 shutdown_connections 2024-05-08T04:42:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.608+0000 7f9e49e00640 1 -- 172.21.15.164:0/1808278481 wait complete. 2024-05-08T04:42:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 Processor -- start 2024-05-08T04:42:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 -- start start 2024-05-08T04:42:19.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 0x7f9e440caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e44161230 0x7f9e440cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44161790 con 0x7f9e440ce640 2024-05-08T04:42:19.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44161610 con 0x7f9e440d17a0 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4416baa0 con 0x7f9e44161230 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44712/0 (socket says 172.21.15.164:44712) 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 -- 172.21.15.164:0/420782279 learned_addr learned my addr 172.21.15.164:0/420782279 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e43400640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e44161230 0x7f9e440cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e3a000640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 0x7f9e440caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 -- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e44161230 msgr2=0x7f9e440cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e44161230 0x7f9e440cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 -- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 msgr2=0x7f9e440caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 0x7f9e440caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 -- 172.21.15.164:0/420782279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440c7a10 con 0x7f9e440ce640 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e3a000640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 0x7f9e440caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e42a00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9e340ae040 tx=0x7f9e3401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340b0050 con 0x7f9e440ce640 2024-05-08T04:42:19.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e340bc070 con 0x7f9e440ce640 2024-05-08T04:42:19.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440c7cf0 con 0x7f9e440ce640 2024-05-08T04:42:19.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.612+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34006e20 con 0x7f9e440ce640 2024-05-08T04:42:19.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e440c82b0 con 0x7f9e440ce640 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44069d60 con 0x7f9e440ce640 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f9e340b5870 con 0x7f9e440ce640 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e3be00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 0x7f9e20069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f9e341202c0 con 0x7f9e440ce640 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e3a000640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 0x7f9e20069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.616+0000 7f9e3a000640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 0x7f9e20069b70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9e4406b910 tx=0x7f9e2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.620+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e34016090 con 0x7f9e440ce640 2024-05-08T04:42:19.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.776+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9e440b99e0 con 0x7f9e440ce640 2024-05-08T04:42:19.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.776+0000 7f9e3be00640 1 -- 172.21.15.164:0/420782279 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9e340f0900 con 0x7f9e440ce640 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:19.783 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 msgr2=0x7f9e20069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 0x7f9e20069b70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9e4406b910 tx=0x7f9e2c009040 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 msgr2=0x7f9e44141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9e340ae040 tx=0x7f9e3401ac50 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 shutdown_connections 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e200676b0 0x7f9e20069b70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e44161230 0x7f9e440cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e440d17a0 0x7f9e440caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 --2- 172.21.15.164:0/420782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e440ce640 0x7f9e44141db0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 >> 172.21.15.164:0/420782279 conn(0x7f9e4409cea0 msgr2=0x7f9e440b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 shutdown_connections 2024-05-08T04:42:19.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.780+0000 7f9e49e00640 1 -- 172.21.15.164:0/420782279 wait complete. 2024-05-08T04:42:19.803 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:42:19.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:42:19.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 Processor -- start 2024-05-08T04:42:19.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 -- start start 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 0x7fe2480ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24815e870 con 0x7fe2480cecc0 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24815eb50 con 0x7fe24815ab10 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480cf130 con 0x7fe24806cce0 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24d400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe2480cecc0 0x7fe2480ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51828/0 (socket says 172.21.15.164:51828) 2024-05-08T04:42:19.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe24d400640 1 -- 172.21.15.164:0/4218394946 learned_addr learned my addr 172.21.15.164:0/4218394946 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247e00640 1 --2- 172.21.15.164:0/4218394946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 0x7fe2480ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875133958 0 0) 0x7fe24815e870 con 0x7fe2480cecc0 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c003530 con 0x7fe2480cecc0 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165624387 0 0) 0x7fe24815eb50 con 0x7fe24815ab10 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24815e870 con 0x7fe24815ab10 2024-05-08T04:42:19.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247e00640 1 -- 172.21.15.164:0/4218394946 >> v1:172.21.15.172:6789/0 conn(0x7fe24815ab10 legacy=0x7fe2480ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247e00640 1 -- 172.21.15.164:0/4218394946 >> v1:172.21.15.164:6789/0 conn(0x7fe2480cecc0 legacy=0x7fe2480ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.940+0000 7fe247e00640 1 -- 172.21.15.164:0/4218394946 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24815ee30 con 0x7fe24806cce0 2024-05-08T04:42:19.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe247e00640 1 --2- 172.21.15.164:0/4218394946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 0x7fe2480ce780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2340097f0 tx=0x7fe23401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f45d42105f878 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340b0070 con 0x7fe24806cce0 2024-05-08T04:42:19.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2340ab600 con 0x7fe24806cce0 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe247400640 1 -- 172.21.15.164:0/4218394946 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340ab9d0 con 0x7fe24806cce0 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 -- 172.21.15.164:0/4218394946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 msgr2=0x7fe2480ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 --2- 172.21.15.164:0/4218394946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 0x7fe2480ce780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2340097f0 tx=0x7fe23401c450 comp rx=0 tx=0).stop 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 -- 172.21.15.164:0/4218394946 shutdown_connections 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 --2- 172.21.15.164:0/4218394946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe24806cce0 0x7fe2480ce780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 -- 172.21.15.164:0/4218394946 >> 172.21.15.164:0/4218394946 conn(0x7fe2480cf770 msgr2=0x7fe2480cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:19.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 -- 172.21.15.164:0/4218394946 shutdown_connections 2024-05-08T04:42:19.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 -- 172.21.15.164:0/4218394946 wait complete. 2024-05-08T04:42:19.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.944+0000 7fe24f400640 1 Processor -- start 2024-05-08T04:42:19.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- start start 2024-05-08T04:42:19.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe24806cce0 0x7fe2480c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2480cecc0 0x7fe2480ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480aef70 con 0x7fe24815ab10 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480aedf0 con 0x7fe24806cce0 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480cd670 con 0x7fe2480cecc0 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44728/0 (socket says 172.21.15.164:44728) 2024-05-08T04:42:19.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 -- 172.21.15.164:0/587094582 learned_addr learned my addr 172.21.15.164:0/587094582 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24ca00640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe24806cce0 0x7fe2480c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe247e00640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2480cecc0 0x7fe2480ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 -- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2480cecc0 msgr2=0x7fe2480ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2480cecc0 0x7fe2480ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 -- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe24806cce0 msgr2=0x7fe2480c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:19.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe24806cce0 0x7fe2480c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 -- 172.21.15.164:0/587094582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2480b3300 con 0x7fe24815ab10 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24d400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe24000cb70 tx=0x7fe24000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240017070 con 0x7fe24815ab10 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24000ad60 con 0x7fe24815ab10 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240012640 con 0x7fe24815ab10 2024-05-08T04:42:19.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2480c95f0 con 0x7fe24815ab10 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe248158570 con 0x7fe24815ab10 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.948+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248072d30 con 0x7fe24815ab10 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.952+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fe24001a070 con 0x7fe24815ab10 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.952+0000 7fe244c00640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 0x7fe228069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.952+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fe240085ca0 con 0x7fe24815ab10 2024-05-08T04:42:19.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.952+0000 7fe24ca00640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 0x7fe228069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:19.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.952+0000 7fe24ca00640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 0x7fe228069b70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe238001720 tx=0x7fe238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:19.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:19.956+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe240055990 con 0x7fe24815ab10 2024-05-08T04:42:20.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.112+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe2480c4bf0 con 0x7fe24815ab10 2024-05-08T04:42:20.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.388+0000 7fe244c00640 1 -- 172.21.15.164:0/587094582 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe240059a30 con 0x7fe24815ab10 2024-05-08T04:42:20.395 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 msgr2=0x7fe228069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 0x7fe228069b70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe238001720 tx=0x7fe238009040 comp rx=0 tx=0).stop 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 msgr2=0x7fe2480ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe24000cb70 tx=0x7fe24000a430 comp rx=0 tx=0).stop 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 shutdown_connections 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2280676b0 0x7fe228069b70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2480cecc0 0x7fe2480ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe24815ab10 0x7fe2480ade00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 --2- 172.21.15.164:0/587094582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe24806cce0 0x7fe2480c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 >> 172.21.15.164:0/587094582 conn(0x7fe2480cf770 msgr2=0x7fe2480ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:20.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 shutdown_connections 2024-05-08T04:42:20.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.392+0000 7fe24f400640 1 -- 172.21.15.164:0/587094582 wait complete. 2024-05-08T04:42:20.423 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:42:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.560+0000 7f9ebe800640 1 Processor -- start 2024-05-08T04:42:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.560+0000 7f9ebe800640 1 -- start start 2024-05-08T04:42:20.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9ebe800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9eb80d0c20 0x7f9eb80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:20.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9ebe800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb80cfe60 con 0x7f9eb80cdb10 2024-05-08T04:42:20.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9ebe800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb80d0140 con 0x7f9eb80ce760 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9ebe800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80d0790 con 0x7f9eb80d0c20 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9eb80d0c20 0x7f9eb80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9eb80d0c20 0x7f9eb80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44730/0 (socket says 172.21.15.164:44730) 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb7400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9eb80cdb10 0x7f9eb806efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51830/0 (socket says 172.21.15.164:51830) 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 -- 172.21.15.164:0/1743563159 learned_addr learned my addr 172.21.15.164:0/1743563159 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 -- 172.21.15.164:0/1743563159 >> v1:172.21.15.172:6789/0 conn(0x7f9eb80ce760 legacy=0x7f9eb80cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 -- 172.21.15.164:0/1743563159 >> v1:172.21.15.164:6789/0 conn(0x7f9eb80cdb10 legacy=0x7f9eb806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.564+0000 7f9eb6a00640 1 -- 172.21.15.164:0/1743563159 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80d0420 con 0x7f9eb80d0c20 2024-05-08T04:42:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9eb6a00640 1 --2- 172.21.15.164:0/1743563159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb80ce220 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12eb88a7ff3f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9eb6000640 1 -- 172.21.15.164:0/1743563159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b1070 con 0x7f9eb80d0c20 2024-05-08T04:42:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9eb6000640 1 -- 172.21.15.164:0/1743563159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea40a8050 con 0x7f9eb80d0c20 2024-05-08T04:42:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9eb6000640 1 -- 172.21.15.164:0/1743563159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40ac860 con 0x7f9eb80d0c20 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- 172.21.15.164:0/1743563159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 msgr2=0x7f9eb80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1743563159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb80ce220 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009870 tx=0x7f9ea401c4d0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- 172.21.15.164:0/1743563159 shutdown_connections 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1743563159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb80ce220 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- 172.21.15.164:0/1743563159 >> 172.21.15.164:0/1743563159 conn(0x7f9eb80b34f0 msgr2=0x7f9eb80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- 172.21.15.164:0/1743563159 shutdown_connections 2024-05-08T04:42:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- 172.21.15.164:0/1743563159 wait complete. 2024-05-08T04:42:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 Processor -- start 2024-05-08T04:42:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.568+0000 7f9ebe800640 1 -- start start 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb80cdb10 0x7f9eb80b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb80ce760 0x7f9eb814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb816c060 con 0x7f9eb80d0c20 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb816bee0 con 0x7f9eb80ce760 2024-05-08T04:42:20.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb816c1e0 con 0x7f9eb80cdb10 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44744/0 (socket says 172.21.15.164:44744) 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 -- 172.21.15.164:0/1601074347 learned_addr learned my addr 172.21.15.164:0/1601074347 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7400640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb80cdb10 0x7f9eb80b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb6a00640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb80ce760 0x7f9eb814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 -- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb80cdb10 msgr2=0x7f9eb80b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb80cdb10 0x7f9eb80b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 -- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb80ce760 msgr2=0x7f9eb814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb80ce760 0x7f9eb814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 -- 172.21.15.164:0/1601074347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80c8a70 con 0x7f9eb80d0c20 2024-05-08T04:42:20.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eb7e00640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c3c0 tx=0x7f9eb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00052d0 con 0x7f9eb80d0c20 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0005470 con 0x7f9eb80d0c20 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80c8d30 con 0x7f9eb80d0c20 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0007240 con 0x7f9eb80d0c20 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb80c92f0 con 0x7f9eb80d0c20 2024-05-08T04:42:20.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb809fae0 con 0x7f9eb80d0c20 2024-05-08T04:42:20.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.572+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f9eb00073e0 con 0x7f9eb80d0c20 2024-05-08T04:42:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.576+0000 7f9eaf400640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 0x7f9e98069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.576+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5311+0+0 (secure 0 0 0) 0x7f9eb0085bd0 con 0x7f9eb80d0c20 2024-05-08T04:42:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.576+0000 7f9eb7400640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 0x7f9e98069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:20.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.576+0000 7f9eb7400640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 0x7f9e98069b70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80012f0 tx=0x7f9ea8001660 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:20.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.580+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9eb0055860 con 0x7f9eb80d0c20 2024-05-08T04:42:20.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:20.732+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9eb804f010 con 0x7f9eb80d0c20 2024-05-08T04:42:21.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.396+0000 7f9eaf400640 1 -- 172.21.15.164:0/1601074347 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9eb0059900 con 0x7f9eb80d0c20 2024-05-08T04:42:21.403 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:42:21.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.400+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 msgr2=0x7f9e98069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.400+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 0x7f9e98069b70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80012f0 tx=0x7f9ea8001660 comp rx=0 tx=0).stop 2024-05-08T04:42:21.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.400+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 msgr2=0x7f9eb81503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.400+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c3c0 tx=0x7f9eb000c880 comp rx=0 tx=0).stop 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 shutdown_connections 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e980676b0 0x7f9e98069b70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9eb80d0c20 0x7f9eb81503b0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb80ce760 0x7f9eb814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 --2- 172.21.15.164:0/1601074347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb80cdb10 0x7f9eb80b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 >> 172.21.15.164:0/1601074347 conn(0x7f9eb80b34f0 msgr2=0x7f9eb80b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 shutdown_connections 2024-05-08T04:42:21.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.404+0000 7f9ebe800640 1 -- 172.21.15.164:0/1601074347 wait complete. 2024-05-08T04:42:21.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:42:21.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 Processor -- start 2024-05-08T04:42:21.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 -- start start 2024-05-08T04:42:21.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 0x7f626806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:21.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62680c6600 con 0x7f626806e390 2024-05-08T04:42:21.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62680c6040 con 0x7f626815ba20 2024-05-08T04:42:21.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f626e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626815fbe0 con 0x7f626815ce10 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f6267e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f626806e390 0x7f62680ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51840/0 (socket says 172.21.15.164:51840) 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f6267e00640 1 -- 172.21.15.164:0/2043876459 learned_addr learned my addr 172.21.15.164:0/2043876459 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.584+0000 7f6266a00640 1 --2- 172.21.15.164:0/2043876459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 0x7f626806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180513267 0 0) 0x7f62680c6600 con 0x7f626806e390 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f624c003530 con 0x7f626806e390 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292739846 0 0) 0x7f62680c6040 con 0x7f626815ba20 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62680c6600 con 0x7f626815ba20 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266a00640 1 -- 172.21.15.164:0/2043876459 >> v1:172.21.15.172:6789/0 conn(0x7f626815ba20 legacy=0x7f6268160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266a00640 1 -- 172.21.15.164:0/2043876459 >> v1:172.21.15.164:6789/0 conn(0x7f626806e390 legacy=0x7f62680ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266a00640 1 -- 172.21.15.164:0/2043876459 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680c6320 con 0x7f626815ce10 2024-05-08T04:42:21.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266a00640 1 --2- 172.21.15.164:0/2043876459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 0x7f626806de50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62540097f0 tx=0x7f625401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89840020cc3b037e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540b0070 con 0x7f626815ce10 2024-05-08T04:42:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540ab600 con 0x7f626815ce10 2024-05-08T04:42:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.588+0000 7f6266000640 1 -- 172.21.15.164:0/2043876459 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540ab9b0 con 0x7f626815ce10 2024-05-08T04:42:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- 172.21.15.164:0/2043876459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 msgr2=0x7f626806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 --2- 172.21.15.164:0/2043876459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 0x7f626806de50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62540097f0 tx=0x7f625401c450 comp rx=0 tx=0).stop 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- 172.21.15.164:0/2043876459 shutdown_connections 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 --2- 172.21.15.164:0/2043876459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626815ce10 0x7f626806de50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- 172.21.15.164:0/2043876459 >> 172.21.15.164:0/2043876459 conn(0x7f62680cf770 msgr2=0x7f62680cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- 172.21.15.164:0/2043876459 shutdown_connections 2024-05-08T04:42:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- 172.21.15.164:0/2043876459 wait complete. 2024-05-08T04:42:21.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 Processor -- start 2024-05-08T04:42:21.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- start start 2024-05-08T04:42:21.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 0x7f626814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:21.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626815ce10 0x7f62680b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62680cde80 con 0x7f626815ba20 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62680af040 con 0x7f626815ce10 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f626e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680ce000 con 0x7f626806e390 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44750/0 (socket says 172.21.15.164:44750) 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 -- 172.21.15.164:0/3904829608 learned_addr learned my addr 172.21.15.164:0/3904829608 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6267400640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 0x7f626814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6267e00640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626815ce10 0x7f62680b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 -- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 msgr2=0x7f626814b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 0x7f626814b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 -- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626815ce10 msgr2=0x7f62680b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:21.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626815ce10 0x7f62680b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 -- 172.21.15.164:0/3904829608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626815acf0 con 0x7f626815ba20 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6267400640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 0x7f626814b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.592+0000 7f6266a00640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f62540097f0 tx=0x7f62540191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540b8020 con 0x7f626815ba20 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540abe20 con 0x7f626815ba20 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540b0040 con 0x7f626815ba20 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626815afd0 con 0x7f626815ba20 2024-05-08T04:42:21.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626814bef0 con 0x7f626815ba20 2024-05-08T04:42:21.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6268072d30 con 0x7f626815ba20 2024-05-08T04:42:21.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f62540bb020 con 0x7f626815ba20 2024-05-08T04:42:21.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 0x7f6248069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:21.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5693+0+0 (secure 0 0 0) 0x7f625411fe00 con 0x7f626815ba20 2024-05-08T04:42:21.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.596+0000 7f6267400640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 0x7f6248069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:21.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.600+0000 7f6267400640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 0x7f6248069b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6258001d30 tx=0x7f6258009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:21.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.600+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62540c2030 con 0x7f626815ba20 2024-05-08T04:42:21.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:21.788+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6268167950 con 0x7f626815ba20 2024-05-08T04:42:22.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.416+0000 7f625f400640 1 -- 172.21.15.164:0/3904829608 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f62540160d0 con 0x7f626815ba20 2024-05-08T04:42:22.424 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 14 and data pool 15 2024-05-08T04:42:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.424+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 msgr2=0x7f6248069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.424+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 0x7f6248069b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6258001d30 tx=0x7f6258009040 comp rx=0 tx=0).stop 2024-05-08T04:42:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.424+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 msgr2=0x7f62680c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.424+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f62540097f0 tx=0x7f62540191f0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 shutdown_connections 2024-05-08T04:42:22.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62480676b0 0x7f6248069b70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626815ce10 0x7f62680b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f626815ba20 0x7f62680c9dc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 --2- 172.21.15.164:0/3904829608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f626806e390 0x7f626814b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.428+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 >> 172.21.15.164:0/3904829608 conn(0x7f62680cf770 msgr2=0x7f626815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:22.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.432+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 shutdown_connections 2024-05-08T04:42:22.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.432+0000 7f626e200640 1 -- 172.21.15.164:0/3904829608 wait complete. 2024-05-08T04:42:22.462 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:22.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 Processor -- start 2024-05-08T04:42:22.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 -- start start 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f7908160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f790815be50 0x7f79080b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79080cd9f0 con 0x7f790815ba70 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790815fe70 con 0x7f790815be50 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f790de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79080cdcd0 con 0x7f79080cfda0 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7906a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f790815ba70 0x7f79080cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51856/0 (socket says 172.21.15.164:51856) 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7906a00640 1 -- 172.21.15.164:0/2303982785 learned_addr learned my addr 172.21.15.164:0/2303982785 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:22.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573734083 0 0) 0x7f79080cd9f0 con 0x7f790815ba70 2024-05-08T04:42:22.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7907400640 1 --2- 172.21.15.164:0/2303982785 >> v2:172.21.15.172:3300/0 conn(0x7f790815be50 0x7f79080b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ec003530 con 0x7f790815ba70 2024-05-08T04:42:22.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.612+0000 7f7906000640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f7908160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4104951212 0 0) 0x7f78ec003530 con 0x7f790815ba70 2024-05-08T04:42:22.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 msgr2=0x7f7908160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f7908160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 >> v2:172.21.15.172:3300/0 conn(0x7f790815be50 msgr2=0x7f79080b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 --2- 172.21.15.164:0/2303982785 >> v2:172.21.15.172:3300/0 conn(0x7f790815be50 0x7f79080b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790815f820 con 0x7f790815ba70 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f78f8001a00 con 0x7f790815ba70 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 >> v1:172.21.15.164:6789/0 conn(0x7f790815ba70 legacy=0x7f79080cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f78ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f78ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec004d50 0x7f78ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790815fe70 con 0x7f78ec0039c0 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79080cdcd0 con 0x7f78ec004d50 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec00b9d0 con 0x7f79080cfda0 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f78ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906000640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f78ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7907400640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec004d50 0x7f78ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 -- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 msgr2=0x7f78ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:22.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080cfda0 0x7f78ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 -- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec004d50 msgr2=0x7f78ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:22.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec004d50 0x7f78ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 -- 172.21.15.164:0/2303982785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec003530 con 0x7f78ec0039c0 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.616+0000 7f7906a00640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f78ec003de0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f78f800a880 tx=0x7f78f801d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a89b6b2a4557db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f800a4a0 con 0x7f78ec0039c0 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80ad650 con 0x7f78ec0039c0 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7905600640 1 -- 172.21.15.164:0/2303982785 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80ada40 con 0x7f78ec0039c0 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- 172.21.15.164:0/2303982785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 msgr2=0x7f78ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 --2- 172.21.15.164:0/2303982785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f78ec003de0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f78f800a880 tx=0x7f78f801d370 comp rx=0 tx=0).stop 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7907400640 1 -- 172.21.15.164:0/2303982785 reap_dead start 2024-05-08T04:42:22.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- 172.21.15.164:0/2303982785 shutdown_connections 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- 172.21.15.164:0/2303982785 >> 172.21.15.164:0/2303982785 conn(0x7f790806ac90 msgr2=0x7f790806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- 172.21.15.164:0/2303982785 shutdown_connections 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- 172.21.15.164:0/2303982785 wait complete. 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 Processor -- start 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- start start 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79081f83a0 0x7f790815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080b09b0 0x7f790814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908160460 con 0x7f78ec0039c0 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79081602e0 con 0x7f79081f83a0 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f790de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79081605e0 con 0x7f79080b09b0 2024-05-08T04:42:22.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7906a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7906a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44760/0 (socket says 172.21.15.164:44760) 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7906a00640 1 -- 172.21.15.164:0/2175960277 learned_addr learned my addr 172.21.15.164:0/2175960277 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7907400640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080b09b0 0x7f790814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.620+0000 7f7906000640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79081f83a0 0x7f790815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 -- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080b09b0 msgr2=0x7f790814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080b09b0 0x7f790814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 -- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79081f83a0 msgr2=0x7f790815e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79081f83a0 0x7f790815e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 -- 172.21.15.164:0/2175960277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790815f0d0 con 0x7f78ec0039c0 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906a00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f78f801cde0 tx=0x7f78ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80bc070 con 0x7f78ec0039c0 2024-05-08T04:42:22.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80017c0 con 0x7f78ec0039c0 2024-05-08T04:42:22.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790815edf0 con 0x7f78ec0039c0 2024-05-08T04:42:22.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80b2040 con 0x7f78ec0039c0 2024-05-08T04:42:22.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790815c670 con 0x7f78ec0039c0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7908067820 con 0x7f78ec0039c0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f78f80bf070 con 0x7f78ec0039c0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 0x7f78e8069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f78f8120670 con 0x7f78ec0039c0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.624+0000 7f7906000640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 0x7f78e8069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.628+0000 7f7906000640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 0x7f78e8069ba0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f790815dcd0 tx=0x7f78f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.628+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78f80c6020 con 0x7f78ec0039c0 2024-05-08T04:42:22.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.784+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7908069d60 con 0x7f78ec0039c0 2024-05-08T04:42:22.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.784+0000 7f78ff400640 1 -- 172.21.15.164:0/2175960277 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16815 (secure 0 0 0) 0x7f78f80170b0 con 0x7f78ec0039c0 2024-05-08T04:42:22.793 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:22.793 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":98,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:21.793511+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:22.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 msgr2=0x7f78e8069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 0x7f78e8069ba0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f790815dcd0 tx=0x7f78f4008040 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 msgr2=0x7f790814d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f78f801cde0 tx=0x7f78ec005840 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 shutdown_connections 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f78e80676e0 0x7f78e8069ba0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79080b09b0 0x7f790814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79081f83a0 0x7f790815e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 --2- 172.21.15.164:0/2175960277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78ec0039c0 0x7f790814d3b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 >> 172.21.15.164:0/2175960277 conn(0x7f790806ac90 msgr2=0x7f790815a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 shutdown_connections 2024-05-08T04:42:22.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.788+0000 7f790de00640 1 -- 172.21.15.164:0/2175960277 wait complete. 2024-05-08T04:42:22.813 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:22.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 Processor -- start 2024-05-08T04:42:22.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 -- start start 2024-05-08T04:42:22.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec8015ab60 0x7fec80072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800cfda0 0x7fec800cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec80072640 con 0x7fec800b26c0 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8006c770 con 0x7fec8015ab60 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec85e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8006ca50 con 0x7fec800cfda0 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec7ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec8015ab60 0x7fec80072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec7ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec8015ab60 0x7fec80072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44772/0 (socket says 172.21.15.164:44772) 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec7ea00640 1 -- 172.21.15.164:0/3018482389 learned_addr learned my addr 172.21.15.164:0/3018482389 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.952+0000 7fec7f400640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800cfda0 0x7fec800cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7ea00640 1 -- 172.21.15.164:0/3018482389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800cfda0 msgr2=0x7fec800cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7ea00640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800cfda0 0x7fec800cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7ea00640 1 -- 172.21.15.164:0/3018482389 >> v1:172.21.15.164:6789/0 conn(0x7fec800b26c0 legacy=0x7fec800cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7ea00640 1 -- 172.21.15.164:0/3018482389 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8006cd30 con 0x7fec8015ab60 2024-05-08T04:42:22.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7ea00640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec80072100 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fec70009870 tx=0x7fec7001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f9ce1e5ab2fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7e000640 1 -- 172.21.15.164:0/3018482389 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700b2070 con 0x7fec8015ab60 2024-05-08T04:42:22.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7e000640 1 -- 172.21.15.164:0/3018482389 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700a9070 con 0x7fec8015ab60 2024-05-08T04:42:22.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec7e000640 1 -- 172.21.15.164:0/3018482389 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700ad880 con 0x7fec8015ab60 2024-05-08T04:42:22.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 -- 172.21.15.164:0/3018482389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 msgr2=0x7fec80072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec80072100 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fec70009870 tx=0x7fec7001c560 comp rx=0 tx=0).stop 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 -- 172.21.15.164:0/3018482389 shutdown_connections 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800cfda0 0x7fec800cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 --2- 172.21.15.164:0/3018482389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec80072100 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 -- 172.21.15.164:0/3018482389 >> 172.21.15.164:0/3018482389 conn(0x7fec8006ac90 msgr2=0x7fec8006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 -- 172.21.15.164:0/3018482389 shutdown_connections 2024-05-08T04:42:22.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.956+0000 7fec85e00640 1 -- 172.21.15.164:0/3018482389 wait complete. 2024-05-08T04:42:22.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 Processor -- start 2024-05-08T04:42:22.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 -- start start 2024-05-08T04:42:22.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800b26c0 0x7fec800ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec800cfda0 0x7fec800adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800aeeb0 con 0x7fec8015ab60 2024-05-08T04:42:22.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800aed30 con 0x7fec800cfda0 2024-05-08T04:42:22.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec80167970 con 0x7fec800b26c0 2024-05-08T04:42:22.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44788/0 (socket says 172.21.15.164:44788) 2024-05-08T04:42:22.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 -- 172.21.15.164:0/2075570908 learned_addr learned my addr 172.21.15.164:0/2075570908 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:22.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7ea00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800b26c0 0x7fec800ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec76000640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec800cfda0 0x7fec800adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 -- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800b26c0 msgr2=0x7fec800ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800b26c0 0x7fec800ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 -- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec800cfda0 msgr2=0x7fec800adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec800cfda0 0x7fec800adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 -- 172.21.15.164:0/2075570908 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800ab0c0 con 0x7fec8015ab60 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec7f400640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fec78007e40 tx=0x7fec7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78014620 con 0x7fec8015ab60 2024-05-08T04:42:22.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec78019070 con 0x7fec8015ab60 2024-05-08T04:42:22.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800ab3a0 con 0x7fec8015ab60 2024-05-08T04:42:22.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.960+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78014a20 con 0x7fec8015ab60 2024-05-08T04:42:22.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec800ab960 con 0x7fec8015ab60 2024-05-08T04:42:22.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec80069d60 con 0x7fec8015ab60 2024-05-08T04:42:22.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fec78014bc0 con 0x7fec8015ab60 2024-05-08T04:42:22.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec77e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 0x7fec5c069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:22.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fec780869a0 con 0x7fec8015ab60 2024-05-08T04:42:22.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec7ea00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 0x7fec5c069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:22.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.964+0000 7fec7ea00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 0x7fec5c069ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fec700af000 tx=0x7fec70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:22.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:22.968+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec780563c0 con 0x7fec8015ab60 2024-05-08T04:42:23.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec800cf9d0 con 0x7fec8015ab60 2024-05-08T04:42:23.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec77e00640 1 -- 172.21.15.164:0/2075570908 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16815 (secure 0 0 0) 0x7fec7805a460 con 0x7fec8015ab60 2024-05-08T04:42:23.129 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:23.129 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":98,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:21.793511+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:23.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 msgr2=0x7fec5c069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 0x7fec5c069ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fec700af000 tx=0x7fec70008040 comp rx=0 tx=0).stop 2024-05-08T04:42:23.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 msgr2=0x7fec801618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fec78007e40 tx=0x7fec7800e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 shutdown_connections 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec5c0676e0 0x7fec5c069ba0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8015ab60 0x7fec801618b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec800cfda0 0x7fec800adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 --2- 172.21.15.164:0/2075570908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec800b26c0 0x7fec800ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 >> 172.21.15.164:0/2075570908 conn(0x7fec8006ac90 msgr2=0x7fec8006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 shutdown_connections 2024-05-08T04:42:23.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.124+0000 7fec85e00640 1 -- 172.21.15.164:0/2075570908 wait complete. 2024-05-08T04:42:23.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:42:23.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.284+0000 7fb477e00640 1 Processor -- start 2024-05-08T04:42:23.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.284+0000 7fb477e00640 1 -- start start 2024-05-08T04:42:23.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb4780b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb478161e90 0x7fb4780cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47806eed0 con 0x7fb4780b40a0 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47806f1b0 con 0x7fb478161e90 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806f490 con 0x7fb47815db10 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb478161e90 0x7fb4780cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb478161e90 0x7fb4780cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44794/0 (socket says 172.21.15.164:44794) 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 -- 172.21.15.164:0/1849379244 learned_addr learned my addr 172.21.15.164:0/1849379244 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb476a00640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb4780b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 -- 172.21.15.164:0/1849379244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 msgr2=0x7fb4780b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb4780b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 -- 172.21.15.164:0/1849379244 >> v1:172.21.15.172:6789/0 conn(0x7fb4780b40a0 legacy=0x7fb478161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 -- 172.21.15.164:0/1849379244 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47806be40 con 0x7fb478161e90 2024-05-08T04:42:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.288+0000 7fb477400640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb4780cdef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb470009e50 tx=0x7fb47001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2fcd41f9db8350 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb475600640 1 -- 172.21.15.164:0/1849379244 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700a5920 con 0x7fb478161e90 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb475600640 1 -- 172.21.15.164:0/1849379244 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4700a5ac0 con 0x7fb478161e90 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb475600640 1 -- 172.21.15.164:0/1849379244 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700ad3f0 con 0x7fb478161e90 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- 172.21.15.164:0/1849379244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 msgr2=0x7fb4780cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb4780cdef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb470009e50 tx=0x7fb47001ca30 comp rx=0 tx=0).stop 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- 172.21.15.164:0/1849379244 shutdown_connections 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb4780cdef0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- 172.21.15.164:0/1849379244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb4780b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- 172.21.15.164:0/1849379244 >> 172.21.15.164:0/1849379244 conn(0x7fb47809aa20 msgr2=0x7fb4780b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- 172.21.15.164:0/1849379244 shutdown_connections 2024-05-08T04:42:23.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- 172.21.15.164:0/1849379244 wait complete. 2024-05-08T04:42:23.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 Processor -- start 2024-05-08T04:42:23.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- start start 2024-05-08T04:42:23.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4780b40a0 0x7fb47815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb478159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806c360 con 0x7fb478161e90 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781625e0 con 0x7fb4780b40a0 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806c4e0 con 0x7fb47815db10 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44804/0 (socket says 172.21.15.164:44804) 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb477400640 1 -- 172.21.15.164:0/4224902960 learned_addr learned my addr 172.21.15.164:0/4224902960 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.292+0000 7fb476a00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4780b40a0 0x7fb47815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb476000640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb478159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 -- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 msgr2=0x7fb478159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb478159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 -- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4780b40a0 msgr2=0x7fb47815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:23.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4780b40a0 0x7fb47815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:23.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 -- 172.21.15.164:0/4224902960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780cd8a0 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477400640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb4700af040 tx=0x7fb4700197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700be020 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4700b8440 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780cdb80 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700a5920 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780b0f60 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47804ef30 con 0x7fb478161e90 2024-05-08T04:42:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.296+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fb4700b8830 con 0x7fb478161e90 2024-05-08T04:42:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.300+0000 7fb46ea00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 0x7fb458069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.300+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb470120e90 con 0x7fb478161e90 2024-05-08T04:42:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.300+0000 7fb476a00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 0x7fb458069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:23.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.300+0000 7fb476a00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 0x7fb458069b70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb468006aa0 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:23.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.304+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4700c8020 con 0x7fb478161e90 2024-05-08T04:42:23.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:23.500+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb47809afe0 con 0x7fb478161e90 2024-05-08T04:42:24.393 INFO:tasks.ceph.mds.c.smithi164.stderr:2024-05-08T04:42:24.392+0000 7f026fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:42:24.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.420+0000 7fb46ea00640 1 -- 172.21.15.164:0/4224902960 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb470016620 con 0x7fb478161e90 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 msgr2=0x7fb458069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 0x7fb458069b70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb468006aa0 tx=0x7fb468008040 comp rx=0 tx=0).stop 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 msgr2=0x7fb47815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb4700af040 tx=0x7fb4700197d0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 shutdown_connections 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb4580676b0 0x7fb458069b70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb47815db10 0x7fb478159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb478161e90 0x7fb47815a260 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 --2- 172.21.15.164:0/4224902960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4780b40a0 0x7fb47815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 >> 172.21.15.164:0/4224902960 conn(0x7fb47809aa20 msgr2=0x7fb4780ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:24.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 shutdown_connections 2024-05-08T04:42:24.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.424+0000 7fb477e00640 1 -- 172.21.15.164:0/4224902960 wait complete. 2024-05-08T04:42:24.455 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:42:24.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:42:24.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.592+0000 7f4f87000640 1 Processor -- start 2024-05-08T04:42:24.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.592+0000 7f4f87000640 1 -- start start 2024-05-08T04:42:24.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f87000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f87000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f800aedf0 0x7f4f8006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f87000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f800c6600 con 0x7f4f8006de50 2024-05-08T04:42:24.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f87000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8015fbe0 con 0x7f4f800aedf0 2024-05-08T04:42:24.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f87000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800c6040 con 0x7f4f8015ba20 2024-05-08T04:42:24.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f85000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f800aedf0 0x7f4f8006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:54208/0 (socket says 172.21.15.164:54208) 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 -- 172.21.15.164:0/2857995845 learned_addr learned my addr 172.21.15.164:0/2857995845 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f77400640 1 --1- 172.21.15.164:0/2857995845 >> v1:172.21.15.172:6789/0 conn(0x7f4f8006de50 0x7f4f800ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:33596/0 (socket says 172.21.15.164:33596) 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 -- 172.21.15.164:0/2857995845 >> v2:172.21.15.172:3300/0 conn(0x7f4f800aedf0 msgr2=0x7f4f8006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 --2- 172.21.15.164:0/2857995845 >> v2:172.21.15.172:3300/0 conn(0x7f4f800aedf0 0x7f4f8006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 -- 172.21.15.164:0/2857995845 >> v1:172.21.15.172:6789/0 conn(0x7f4f8006de50 legacy=0x7f4f800ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.596+0000 7f4f7fe00640 1 -- 172.21.15.164:0/2857995845 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800c6320 con 0x7f4f8015ba20 2024-05-08T04:42:24.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f7fe00640 1 --2- 172.21.15.164:0/2857995845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f7001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc328ddd855c721 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:24.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f7f400640 1 -- 172.21.15.164:0/2857995845 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700aa430 con 0x7f4f8015ba20 2024-05-08T04:42:24.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f7f400640 1 -- 172.21.15.164:0/2857995845 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f700af070 con 0x7f4f8015ba20 2024-05-08T04:42:24.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f7f400640 1 -- 172.21.15.164:0/2857995845 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700aa7f0 con 0x7f4f8015ba20 2024-05-08T04:42:24.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- 172.21.15.164:0/2857995845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 msgr2=0x7f4f80160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- 172.21.15.164:0/2857995845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f7001c530 comp rx=0 tx=0).stop 2024-05-08T04:42:24.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- 172.21.15.164:0/2857995845 shutdown_connections 2024-05-08T04:42:24.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- 172.21.15.164:0/2857995845 >> v2:172.21.15.172:3300/0 conn(0x7f4f800aedf0 0x7f4f8006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- 172.21.15.164:0/2857995845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f8015ba20 0x7f4f80160070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- 172.21.15.164:0/2857995845 >> 172.21.15.164:0/2857995845 conn(0x7f4f800cf770 msgr2=0x7f4f800cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:24.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- 172.21.15.164:0/2857995845 shutdown_connections 2024-05-08T04:42:24.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- 172.21.15.164:0/2857995845 wait complete. 2024-05-08T04:42:24.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 Processor -- start 2024-05-08T04:42:24.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- start start 2024-05-08T04:42:24.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f800aedf0 0x7f4f8014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800ce000 con 0x7f4f8015ba20 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800cde80 con 0x7f4f8006de50 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.600+0000 7f4f87000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80168b70 con 0x7f4f800aedf0 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f85000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f85000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44806/0 (socket says 172.21.15.164:44806) 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:52646/0 (socket says 172.21.15.164:52646) 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f85000640 1 -- 172.21.15.164:0/1899663237 learned_addr learned my addr 172.21.15.164:0/1899663237 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f77400640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f800aedf0 0x7f4f8014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 -- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f800aedf0 msgr2=0x7f4f8014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f800aedf0 0x7f4f8014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 -- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 msgr2=0x7f4f801fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:24.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 -- 172.21.15.164:0/1899663237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f801fb780 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f85000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7fe00640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4f700ab000 tx=0x7f4f700192d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700b4600 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f700b47a0 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f801f77f0 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700b4b90 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801f7db0 con 0x7f4f8006de50 2024-05-08T04:42:24.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80150810 con 0x7f4f8006de50 2024-05-08T04:42:24.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.604+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f4f700bc070 con 0x7f4f8006de50 2024-05-08T04:42:24.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.608+0000 7f4f7cc00640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 0x7f4f5c069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:24.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.608+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4f7011fae0 con 0x7f4f8006de50 2024-05-08T04:42:24.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.608+0000 7f4f77400640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 0x7f4f5c069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:24.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.608+0000 7f4f77400640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 0x7f4f5c069ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f8006b740 tx=0x7f4f68007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:24.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.612+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f80150810 con 0x7f4f8006de50 2024-05-08T04:42:24.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:24.800+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4f80072d30 con 0x7f4f8006de50 2024-05-08T04:42:25.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.432+0000 7f4f7cc00640 1 -- 172.21.15.164:0/1899663237 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4f70016150 con 0x7f4f8006de50 2024-05-08T04:42:25.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 msgr2=0x7f4f5c069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 0x7f4f5c069ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f8006b740 tx=0x7f4f68007e90 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 msgr2=0x7f4f80151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4f700ab000 tx=0x7f4f700192d0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 shutdown_connections 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4f5c0676e0 0x7f4f5c069ba0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f800aedf0 0x7f4f8014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8015ba20 0x7f4f801fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 --2- 172.21.15.164:0/1899663237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f8006de50 0x7f4f80151150 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 >> 172.21.15.164:0/1899663237 conn(0x7f4f800cf770 msgr2=0x7f4f8006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:25.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 shutdown_connections 2024-05-08T04:42:25.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.436+0000 7f4f87000640 1 -- 172.21.15.164:0/1899663237 wait complete. 2024-05-08T04:42:25.461 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:42:25.462 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:25.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.592+0000 7fec89200640 1 Processor -- start 2024-05-08T04:42:25.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.592+0000 7fec89200640 1 -- start start 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec89200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec840cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec89200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec840cfbd0 0x7fec840cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec89200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8406c770 con 0x7fec8415ab60 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec89200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840d0060 con 0x7fec840cfbd0 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec89200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8406ca50 con 0x7fec840b26c0 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec840cfbd0 0x7fec840cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec840cfbd0 0x7fec840cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44814/0 (socket says 172.21.15.164:44814) 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 -- 172.21.15.164:0/1270020874 learned_addr learned my addr 172.21.15.164:0/1270020874 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec81e00640 1 --1- 172.21.15.164:0/1270020874 >> v1:172.21.15.164:6789/0 conn(0x7fec8415ab60 0x7fec84072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51864/0 (socket says 172.21.15.164:51864) 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec81400640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec840cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 -- 172.21.15.164:0/1270020874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 msgr2=0x7fec840cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec840cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 -- 172.21.15.164:0/1270020874 >> v1:172.21.15.164:6789/0 conn(0x7fec8415ab60 legacy=0x7fec84072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.596+0000 7fec82800640 1 -- 172.21.15.164:0/1270020874 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8406cd30 con 0x7fec840cfbd0 2024-05-08T04:42:25.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec82800640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec840cfbd0 0x7fec840cd910 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a660ba1a97341303 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec80a00640 1 -- 172.21.15.164:0/1270020874 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b2070 con 0x7fec840cfbd0 2024-05-08T04:42:25.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec80a00640 1 -- 172.21.15.164:0/1270020874 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c0a9070 con 0x7fec840cfbd0 2024-05-08T04:42:25.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec80a00640 1 -- 172.21.15.164:0/1270020874 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0ad880 con 0x7fec840cfbd0 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- 172.21.15.164:0/1270020874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec840cfbd0 msgr2=0x7fec840cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec840cfbd0 0x7fec840cd910 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c01c560 comp rx=0 tx=0).stop 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- 172.21.15.164:0/1270020874 shutdown_connections 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec840cfbd0 0x7fec840cd910 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 --2- 172.21.15.164:0/1270020874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec840cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- 172.21.15.164:0/1270020874 >> 172.21.15.164:0/1270020874 conn(0x7fec8406ac90 msgr2=0x7fec8406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- 172.21.15.164:0/1270020874 shutdown_connections 2024-05-08T04:42:25.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- 172.21.15.164:0/1270020874 wait complete. 2024-05-08T04:42:25.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 Processor -- start 2024-05-08T04:42:25.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.600+0000 7fec89200640 1 -- start start 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec84151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec840cfbd0 0x7fec8414b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec840aeeb0 con 0x7fec8415ab60 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec840aed30 con 0x7fec840cfbd0 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec84167970 con 0x7fec840b26c0 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44824/0 (socket says 172.21.15.164:44824) 2024-05-08T04:42:25.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 -- 172.21.15.164:0/342643404 learned_addr learned my addr 172.21.15.164:0/342643404 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec81e00640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec84151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec81400640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec840cfbd0 0x7fec8414b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 -- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 msgr2=0x7fec84151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec84151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 -- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec840cfbd0 msgr2=0x7fec8414b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec840cfbd0 0x7fec8414b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 -- 172.21.15.164:0/342643404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec841f9110 con 0x7fec8415ab60 2024-05-08T04:42:25.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec81e00640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec84151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:25.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec82800640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0bb070 con 0x7fec8415ab60 2024-05-08T04:42:25.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c0add50 con 0x7fec8415ab60 2024-05-08T04:42:25.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b2040 con 0x7fec8415ab60 2024-05-08T04:42:25.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8414bd00 con 0x7fec8415ab60 2024-05-08T04:42:25.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec841f8870 con 0x7fec8415ab60 2024-05-08T04:42:25.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec84069d60 con 0x7fec8415ab60 2024-05-08T04:42:25.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.604+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fec7c0ab070 con 0x7fec8415ab60 2024-05-08T04:42:25.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.608+0000 7fec7a000640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 0x7fec64069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.608+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fec7c11ee30 con 0x7fec8415ab60 2024-05-08T04:42:25.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.608+0000 7fec81e00640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 0x7fec64069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.608+0000 7fec81e00640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 0x7fec64069ba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec74005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.612+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec7c016150 con 0x7fec8415ab60 2024-05-08T04:42:25.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.796+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec8406d620 con 0x7fec8415ab60 2024-05-08T04:42:25.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.796+0000 7fec7a000640 1 -- 172.21.15.164:0/342643404 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fec7c0ef230 con 0x7fec8415ab60 2024-05-08T04:42:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:25.802 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:25.803 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 62 2024-05-08T04:42:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 msgr2=0x7fec64069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 0x7fec64069ba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec74005b80 comp rx=0 tx=0).stop 2024-05-08T04:42:25.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 msgr2=0x7fec8414da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c019300 comp rx=0 tx=0).stop 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 shutdown_connections 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fec640676e0 0x7fec64069ba0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec8415ab60 0x7fec8414da20 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec840cfbd0 0x7fec8414b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 --2- 172.21.15.164:0/342643404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec840b26c0 0x7fec84151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 >> 172.21.15.164:0/342643404 conn(0x7fec8406ac90 msgr2=0x7fec840af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 shutdown_connections 2024-05-08T04:42:25.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.800+0000 7fec89200640 1 -- 172.21.15.164:0/342643404 wait complete. 2024-05-08T04:42:25.823 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:25.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 Processor -- start 2024-05-08T04:42:25.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 -- start start 2024-05-08T04:42:25.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb34815ba70 0x7fb348160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3480cfda0 0x7fb3480b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3480cd9f0 con 0x7fb34815be50 2024-05-08T04:42:25.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34815fe70 con 0x7fb34815ba70 2024-05-08T04:42:25.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.952+0000 7fb34d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480cdcd0 con 0x7fb3480cfda0 2024-05-08T04:42:25.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb34815ba70 0x7fb348160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb34815ba70 0x7fb348160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44832/0 (socket says 172.21.15.164:44832) 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 -- 172.21.15.164:0/3151022727 learned_addr learned my addr 172.21.15.164:0/3151022727 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb33d400640 1 --1- 172.21.15.164:0/3151022727 >> v1:172.21.15.164:6789/0 conn(0x7fb34815be50 0x7fb3480cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51872/0 (socket says 172.21.15.164:51872) 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb346800640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3480cfda0 0x7fb3480b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 -- 172.21.15.164:0/3151022727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3480cfda0 msgr2=0x7fb3480b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3480cfda0 0x7fb3480b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 -- 172.21.15.164:0/3151022727 >> v1:172.21.15.164:6789/0 conn(0x7fb34815be50 legacy=0x7fb3480cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 -- 172.21.15.164:0/3151022727 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34815f820 con 0x7fb34815ba70 2024-05-08T04:42:25.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345400640 1 -- 172.21.15.164:0/3151022727 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444756358 0 0) 0x7fb3480cd9f0 con 0x7fb34815be50 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345e00640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb34815ba70 0x7fb348160300 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e50 tx=0x7fb33801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37ef9613c54f7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345400640 1 -- 172.21.15.164:0/3151022727 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380a57e0 con 0x7fb34815ba70 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345400640 1 -- 172.21.15.164:0/3151022727 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3380a5980 con 0x7fb34815ba70 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.956+0000 7fb345400640 1 -- 172.21.15.164:0/3151022727 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380a5d50 con 0x7fb34815ba70 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- 172.21.15.164:0/3151022727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb34815ba70 msgr2=0x7fb348160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb34815ba70 0x7fb348160300 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e50 tx=0x7fb33801ca00 comp rx=0 tx=0).stop 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- 172.21.15.164:0/3151022727 shutdown_connections 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3480cfda0 0x7fb3480b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- 172.21.15.164:0/3151022727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb34815ba70 0x7fb348160300 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- 172.21.15.164:0/3151022727 >> 172.21.15.164:0/3151022727 conn(0x7fb34806ac90 msgr2=0x7fb34806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:25.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- 172.21.15.164:0/3151022727 shutdown_connections 2024-05-08T04:42:25.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- 172.21.15.164:0/3151022727 wait complete. 2024-05-08T04:42:25.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 Processor -- start 2024-05-08T04:42:25.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- start start 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 0x7fb3480714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 0x7fb3480ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348168d40 con 0x7fb3480cfda0 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348168bc0 con 0x7fb34815be50 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb34d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb348168ec0 con 0x7fb34815ba70 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44834/0 (socket says 172.21.15.164:44834) 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 -- 172.21.15.164:0/1790710697 learned_addr learned my addr 172.21.15.164:0/1790710697 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb33d400640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 0x7fb3480714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb346800640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 0x7fb3480ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 -- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 msgr2=0x7fb3480714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 0x7fb3480714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 -- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 msgr2=0x7fb3480ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 0x7fb3480ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 -- 172.21.15.164:0/1790710697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348071d30 con 0x7fb3480cfda0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb33d400640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 0x7fb3480714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb346800640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 0x7fb3480ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.960+0000 7fb345e00640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb33801a630 tx=0x7fb3380197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380bc020 con 0x7fb3480cfda0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3380b8670 con 0x7fb3480cfda0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380b8a60 con 0x7fb3480cfda0 2024-05-08T04:42:25.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb348072010 con 0x7fb3480cfda0 2024-05-08T04:42:25.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3480722f0 con 0x7fb3480cfda0 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb348069d60 con 0x7fb3480cfda0 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fb3380bf020 con 0x7fb3480cfda0 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 0x7fb324069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb3381210e0 con 0x7fb3480cfda0 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33d400640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 0x7fb324069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:25.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.964+0000 7fb33d400640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 0x7fb324069b70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb34815f750 tx=0x7fb330007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:25.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:25.968+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3380c6030 con 0x7fb3480cfda0 2024-05-08T04:42:26.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.124+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb34806e210 con 0x7fb3480cfda0 2024-05-08T04:42:26.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.124+0000 7fb33ea00640 1 -- 172.21.15.164:0/1790710697 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16802 (secure 0 0 0) 0x7fb338016620 con 0x7fb3480cfda0 2024-05-08T04:42:26.130 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:26.130 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":99,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:23.416521+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 msgr2=0x7fb324069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 0x7fb324069b70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb34815f750 tx=0x7fb330007cd0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 msgr2=0x7fb3481491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb33801a630 tx=0x7fb3380197a0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 shutdown_connections 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3240676b0 0x7fb324069b70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb34815ba70 0x7fb3480714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb34815be50 0x7fb3480ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 --2- 172.21.15.164:0/1790710697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3480cfda0 0x7fb3481491e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 >> 172.21.15.164:0/1790710697 conn(0x7fb34806ac90 msgr2=0x7fb34806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:26.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 shutdown_connections 2024-05-08T04:42:26.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.128+0000 7fb34d000640 1 -- 172.21.15.164:0/1790710697 wait complete. 2024-05-08T04:42:26.151 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:42:26.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:26.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.280+0000 7f7f77a00640 1 Processor -- start 2024-05-08T04:42:26.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 -- start start 2024-05-08T04:42:26.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 0x7f7f701619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 0x7f7f70161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f701628b0 con 0x7f7f70162420 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7006cc70 con 0x7f7f7006c7b0 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f77a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7006cf50 con 0x7f7f7015dd70 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7f70162420 0x7f7f700cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51882/0 (socket says 172.21.15.164:51882) 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75a00640 1 -- 172.21.15.164:0/534317930 learned_addr learned my addr 172.21.15.164:0/534317930 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f6fe00640 1 --2- 172.21.15.164:0/534317930 >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 0x7f7f70161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75000640 1 --2- 172.21.15.164:0/534317930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 0x7f7f701619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f6f400640 1 -- 172.21.15.164:0/534317930 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863975216 0 0) 0x7f7f680014a0 con 0x7f7f70162420 2024-05-08T04:42:26.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f6f400640 1 -- 172.21.15.164:0/534317930 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f58003530 con 0x7f7f70162420 2024-05-08T04:42:26.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75000640 1 -- 172.21.15.164:0/534317930 >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 msgr2=0x7f7f70161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:26.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75000640 1 --2- 172.21.15.164:0/534317930 >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 0x7f7f70161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75000640 1 -- 172.21.15.164:0/534317930 >> v1:172.21.15.164:6789/0 conn(0x7f7f70162420 legacy=0x7f7f700cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f75000640 1 -- 172.21.15.164:0/534317930 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700b1570 con 0x7f7f7015dd70 2024-05-08T04:42:26.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.284+0000 7f7f6fe00640 1 --2- 172.21.15.164:0/534317930 >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 0x7f7f70161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:26.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f75000640 1 --2- 172.21.15.164:0/534317930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 0x7f7f701619a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f640097c0 tx=0x7f7f6401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3564e4168fe22c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f6f400640 1 -- 172.21.15.164:0/534317930 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640b5070 con 0x7f7f7015dd70 2024-05-08T04:42:26.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f6f400640 1 -- 172.21.15.164:0/534317930 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f640a8990 con 0x7f7f7015dd70 2024-05-08T04:42:26.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f6f400640 1 -- 172.21.15.164:0/534317930 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640a8d40 con 0x7f7f7015dd70 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 -- 172.21.15.164:0/534317930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 msgr2=0x7f7f701619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 --2- 172.21.15.164:0/534317930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 0x7f7f701619a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f640097c0 tx=0x7f7f6401ec90 comp rx=0 tx=0).stop 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 -- 172.21.15.164:0/534317930 shutdown_connections 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 --2- 172.21.15.164:0/534317930 >> v2:172.21.15.172:3300/0 conn(0x7f7f7006c7b0 0x7f7f70161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 --2- 172.21.15.164:0/534317930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7015dd70 0x7f7f701619a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 -- 172.21.15.164:0/534317930 >> 172.21.15.164:0/534317930 conn(0x7f7f700b34f0 msgr2=0x7f7f700b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 -- 172.21.15.164:0/534317930 shutdown_connections 2024-05-08T04:42:26.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 -- 172.21.15.164:0/534317930 wait complete. 2024-05-08T04:42:26.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.288+0000 7f7f77a00640 1 Processor -- start 2024-05-08T04:42:26.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- start start 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7006c7b0 0x7f7f7015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 0x7f7f700b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7015e150 con 0x7f7f70162420 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700b4230 con 0x7f7f7015dd70 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7015e2d0 con 0x7f7f7006c7b0 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44848/0 (socket says 172.21.15.164:44848) 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 -- 172.21.15.164:0/2305879259 learned_addr learned my addr 172.21.15.164:0/2305879259 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75000640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7006c7b0 0x7f7f7015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f6fe00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 0x7f7f700b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 -- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7006c7b0 msgr2=0x7f7f7015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7006c7b0 0x7f7f7015f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 -- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 msgr2=0x7f7f700b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 0x7f7f700b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 -- 172.21.15.164:0/2305879259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7015fdf0 con 0x7f7f70162420 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f6fe00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 0x7f7f700b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:26.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f75a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800cb50 tx=0x7f7f6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68017070 con 0x7f7f70162420 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6800ace0 con 0x7f7f70162420 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68012620 con 0x7f7f70162420 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7015fb10 con 0x7f7f70162420 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700b60c0 con 0x7f7f70162420 2024-05-08T04:42:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.292+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70069d60 con 0x7f7f70162420 2024-05-08T04:42:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.296+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f7f6801a070 con 0x7f7f70162420 2024-05-08T04:42:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.296+0000 7f7f6cc00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 0x7f7f50069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.296+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7f68085f40 con 0x7f7f70162420 2024-05-08T04:42:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.296+0000 7f7f75000640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 0x7f7f50069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.296+0000 7f7f75000640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 0x7f7f50069b70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f640b2000 tx=0x7f7f640aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.300+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f701628b0 con 0x7f7f70162420 2024-05-08T04:42:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.452+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f700b4010 con 0x7f7f500676b0 2024-05-08T04:42:26.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.460+0000 7f7f6cc00640 1 -- 172.21.15.164:0/2305879259 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252013 (secure 0 0 0) 0x7f7f700b4010 con 0x7f7f500676b0 2024-05-08T04:42:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 msgr2=0x7f7f50069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 0x7f7f50069b70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f640b2000 tx=0x7f7f640aa070 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 msgr2=0x7f7f700b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800cb50 tx=0x7f7f6800a430 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 shutdown_connections 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7f500676b0 0x7f7f50069b70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70162420 0x7f7f700b0810 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f7015dd70 0x7f7f700b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 --2- 172.21.15.164:0/2305879259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f7006c7b0 0x7f7f7015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 >> 172.21.15.164:0/2305879259 conn(0x7f7f700b34f0 msgr2=0x7f7f7006f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 shutdown_connections 2024-05-08T04:42:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.464+0000 7f7f77a00640 1 -- 172.21.15.164:0/2305879259 wait complete. 2024-05-08T04:42:26.489 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-05-08T04:42:25.477975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260036,"kb_used_data":10564,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714684,"statfs":{"total":773094113280,"available":772827836416,"internally_reserved":0,"allocated":10817536,"data_stored":5334920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076239"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427565+0000","last_change":"2024-05-08T04:42:22.425873+0000","last_active":"2024-05-08T04:42:22.427565+0000","last_peered":"2024-05-08T04:42:22.427565+0000","last_clean":"2024-05-08T04:42:22.427565+0000","last_became_active":"2024-05-08T04:42:22.425626+0000","last_became_peered":"2024-05-08T04:42:22.425626+0000","last_unstale":"2024-05-08T04:42:22.427565+0000","last_undegraded":"2024-05-08T04:42:22.427565+0000","last_fullsized":"2024-05-08T04:42:22.427565+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:10.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433589+0000","last_change":"2024-05-08T04:42:22.430499+0000","last_active":"2024-05-08T04:42:22.433589+0000","last_peered":"2024-05-08T04:42:22.433589+0000","last_clean":"2024-05-08T04:42:22.433589+0000","last_became_active":"2024-05-08T04:42:22.429164+0000","last_became_peered":"2024-05-08T04:42:22.429164+0000","last_unstale":"2024-05-08T04:42:22.433589+0000","last_undegraded":"2024-05-08T04:42:22.433589+0000","last_fullsized":"2024-05-08T04:42:22.433589+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:45.657760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.426034+0000","last_change":"2024-05-08T04:42:22.423160+0000","last_active":"2024-05-08T04:42:22.426034+0000","last_peered":"2024-05-08T04:42:22.426034+0000","last_clean":"2024-05-08T04:42:22.426034+0000","last_became_active":"2024-05-08T04:42:22.422911+0000","last_became_peered":"2024-05-08T04:42:22.422911+0000","last_unstale":"2024-05-08T04:42:22.426034+0000","last_undegraded":"2024-05-08T04:42:22.426034+0000","last_fullsized":"2024-05-08T04:42:22.426034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:42.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.418799+0000","last_change":"2024-05-08T04:42:22.415933+0000","last_active":"2024-05-08T04:42:22.418799+0000","last_peered":"2024-05-08T04:42:22.418799+0000","last_clean":"2024-05-08T04:42:22.418799+0000","last_became_active":"2024-05-08T04:42:22.415632+0000","last_became_peered":"2024-05-08T04:42:22.415632+0000","last_unstale":"2024-05-08T04:42:22.418799+0000","last_undegraded":"2024-05-08T04:42:22.418799+0000","last_fullsized":"2024-05-08T04:42:22.418799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:51.006973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438616+0000","last_change":"2024-05-08T04:42:22.436450+0000","last_active":"2024-05-08T04:42:22.438616+0000","last_peered":"2024-05-08T04:42:22.438616+0000","last_clean":"2024-05-08T04:42:22.438616+0000","last_became_active":"2024-05-08T04:42:22.436091+0000","last_became_peered":"2024-05-08T04:42:22.436091+0000","last_unstale":"2024-05-08T04:42:22.438616+0000","last_undegraded":"2024-05-08T04:42:22.438616+0000","last_fullsized":"2024-05-08T04:42:22.438616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:42.212232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438430+0000","last_change":"2024-05-08T04:42:22.435033+0000","last_active":"2024-05-08T04:42:22.438430+0000","last_peered":"2024-05-08T04:42:22.438430+0000","last_clean":"2024-05-08T04:42:22.438430+0000","last_became_active":"2024-05-08T04:42:22.434761+0000","last_became_peered":"2024-05-08T04:42:22.434761+0000","last_unstale":"2024-05-08T04:42:22.438430+0000","last_undegraded":"2024-05-08T04:42:22.438430+0000","last_fullsized":"2024-05-08T04:42:22.438430+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:32.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436137+0000","last_change":"2024-05-08T04:42:22.428131+0000","last_active":"2024-05-08T04:42:22.436137+0000","last_peered":"2024-05-08T04:42:22.436137+0000","last_clean":"2024-05-08T04:42:22.436137+0000","last_became_active":"2024-05-08T04:42:22.427869+0000","last_became_peered":"2024-05-08T04:42:22.427869+0000","last_unstale":"2024-05-08T04:42:22.436137+0000","last_undegraded":"2024-05-08T04:42:22.436137+0000","last_fullsized":"2024-05-08T04:42:22.436137+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:09.853850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433491+0000","last_change":"2024-05-08T04:42:22.430791+0000","last_active":"2024-05-08T04:42:22.433491+0000","last_peered":"2024-05-08T04:42:22.433491+0000","last_clean":"2024-05-08T04:42:22.433491+0000","last_became_active":"2024-05-08T04:42:22.428601+0000","last_became_peered":"2024-05-08T04:42:22.428601+0000","last_unstale":"2024-05-08T04:42:22.433491+0000","last_undegraded":"2024-05-08T04:42:22.433491+0000","last_fullsized":"2024-05-08T04:42:22.433491+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:32.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432044+0000","last_change":"2024-05-08T04:42:22.422526+0000","last_active":"2024-05-08T04:42:22.432044+0000","last_peered":"2024-05-08T04:42:22.432044+0000","last_clean":"2024-05-08T04:42:22.432044+0000","last_became_active":"2024-05-08T04:42:22.422227+0000","last_became_peered":"2024-05-08T04:42:22.422227+0000","last_unstale":"2024-05-08T04:42:22.432044+0000","last_undegraded":"2024-05-08T04:42:22.432044+0000","last_fullsized":"2024-05-08T04:42:22.432044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:37.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436098+0000","last_change":"2024-05-08T04:42:22.430864+0000","last_active":"2024-05-08T04:42:22.436098+0000","last_peered":"2024-05-08T04:42:22.436098+0000","last_clean":"2024-05-08T04:42:22.436098+0000","last_became_active":"2024-05-08T04:42:22.430579+0000","last_became_peered":"2024-05-08T04:42:22.430579+0000","last_unstale":"2024-05-08T04:42:22.436098+0000","last_undegraded":"2024-05-08T04:42:22.436098+0000","last_fullsized":"2024-05-08T04:42:22.436098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:21.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438285+0000","last_change":"2024-05-08T04:42:22.434115+0000","last_active":"2024-05-08T04:42:22.438285+0000","last_peered":"2024-05-08T04:42:22.438285+0000","last_clean":"2024-05-08T04:42:22.438285+0000","last_became_active":"2024-05-08T04:42:22.433822+0000","last_became_peered":"2024-05-08T04:42:22.433822+0000","last_unstale":"2024-05-08T04:42:22.438285+0000","last_undegraded":"2024-05-08T04:42:22.438285+0000","last_fullsized":"2024-05-08T04:42:22.438285+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:26.304165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438150+0000","last_change":"2024-05-08T04:42:22.426212+0000","last_active":"2024-05-08T04:42:22.438150+0000","last_peered":"2024-05-08T04:42:22.438150+0000","last_clean":"2024-05-08T04:42:22.438150+0000","last_became_active":"2024-05-08T04:42:22.425897+0000","last_became_peered":"2024-05-08T04:42:22.425897+0000","last_unstale":"2024-05-08T04:42:22.438150+0000","last_undegraded":"2024-05-08T04:42:22.438150+0000","last_fullsized":"2024-05-08T04:42:22.438150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:52.962208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432999+0000","last_change":"2024-05-08T04:42:22.422883+0000","last_active":"2024-05-08T04:42:22.432999+0000","last_peered":"2024-05-08T04:42:22.432999+0000","last_clean":"2024-05-08T04:42:22.432999+0000","last_became_active":"2024-05-08T04:42:22.422617+0000","last_became_peered":"2024-05-08T04:42:22.422617+0000","last_unstale":"2024-05-08T04:42:22.432999+0000","last_undegraded":"2024-05-08T04:42:22.432999+0000","last_fullsized":"2024-05-08T04:42:22.432999+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:32.821695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.441224+0000","last_change":"2024-05-08T04:42:22.431221+0000","last_active":"2024-05-08T04:42:22.441224+0000","last_peered":"2024-05-08T04:42:22.441224+0000","last_clean":"2024-05-08T04:42:22.441224+0000","last_became_active":"2024-05-08T04:42:22.430972+0000","last_became_peered":"2024-05-08T04:42:22.430972+0000","last_unstale":"2024-05-08T04:42:22.441224+0000","last_undegraded":"2024-05-08T04:42:22.441224+0000","last_fullsized":"2024-05-08T04:42:22.441224+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:24.776238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435326+0000","last_change":"2024-05-08T04:42:22.421333+0000","last_active":"2024-05-08T04:42:22.435326+0000","last_peered":"2024-05-08T04:42:22.435326+0000","last_clean":"2024-05-08T04:42:22.435326+0000","last_became_active":"2024-05-08T04:42:22.421080+0000","last_became_peered":"2024-05-08T04:42:22.421080+0000","last_unstale":"2024-05-08T04:42:22.435326+0000","last_undegraded":"2024-05-08T04:42:22.435326+0000","last_fullsized":"2024-05-08T04:42:22.435326+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:15.314837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438033+0000","last_change":"2024-05-08T04:42:22.434004+0000","last_active":"2024-05-08T04:42:22.438033+0000","last_peered":"2024-05-08T04:42:22.438033+0000","last_clean":"2024-05-08T04:42:22.438033+0000","last_became_active":"2024-05-08T04:42:22.433692+0000","last_became_peered":"2024-05-08T04:42:22.433692+0000","last_unstale":"2024-05-08T04:42:22.438033+0000","last_undegraded":"2024-05-08T04:42:22.438033+0000","last_fullsized":"2024-05-08T04:42:22.438033+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:30.653406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432841+0000","last_change":"2024-05-08T04:42:22.426494+0000","last_active":"2024-05-08T04:42:22.432841+0000","last_peered":"2024-05-08T04:42:22.432841+0000","last_clean":"2024-05-08T04:42:22.432841+0000","last_became_active":"2024-05-08T04:42:22.426238+0000","last_became_peered":"2024-05-08T04:42:22.426238+0000","last_unstale":"2024-05-08T04:42:22.432841+0000","last_undegraded":"2024-05-08T04:42:22.432841+0000","last_fullsized":"2024-05-08T04:42:22.432841+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:30.653406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432934+0000","last_change":"2024-05-08T04:42:22.423958+0000","last_active":"2024-05-08T04:42:22.432934+0000","last_peered":"2024-05-08T04:42:22.432934+0000","last_clean":"2024-05-08T04:42:22.432934+0000","last_became_active":"2024-05-08T04:42:22.423720+0000","last_became_peered":"2024-05-08T04:42:22.423720+0000","last_unstale":"2024-05-08T04:42:22.432934+0000","last_undegraded":"2024-05-08T04:42:22.432934+0000","last_fullsized":"2024-05-08T04:42:22.432934+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:36.254755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439390+0000","last_change":"2024-05-08T04:42:22.425698+0000","last_active":"2024-05-08T04:42:22.439390+0000","last_peered":"2024-05-08T04:42:22.439390+0000","last_clean":"2024-05-08T04:42:22.439390+0000","last_became_active":"2024-05-08T04:42:22.425440+0000","last_became_peered":"2024-05-08T04:42:22.425440+0000","last_unstale":"2024-05-08T04:42:22.439390+0000","last_undegraded":"2024-05-08T04:42:22.439390+0000","last_fullsized":"2024-05-08T04:42:22.439390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:02.174432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.431963+0000","last_change":"2024-05-08T04:42:22.422749+0000","last_active":"2024-05-08T04:42:22.431963+0000","last_peered":"2024-05-08T04:42:22.431963+0000","last_clean":"2024-05-08T04:42:22.431963+0000","last_became_active":"2024-05-08T04:42:22.422384+0000","last_became_peered":"2024-05-08T04:42:22.422384+0000","last_unstale":"2024-05-08T04:42:22.431963+0000","last_undegraded":"2024-05-08T04:42:22.431963+0000","last_fullsized":"2024-05-08T04:42:22.431963+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:21.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439397+0000","last_change":"2024-05-08T04:42:22.426484+0000","last_active":"2024-05-08T04:42:22.439397+0000","last_peered":"2024-05-08T04:42:22.439397+0000","last_clean":"2024-05-08T04:42:22.439397+0000","last_became_active":"2024-05-08T04:42:22.426077+0000","last_became_peered":"2024-05-08T04:42:22.426077+0000","last_unstale":"2024-05-08T04:42:22.439397+0000","last_undegraded":"2024-05-08T04:42:22.439397+0000","last_fullsized":"2024-05-08T04:42:22.439397+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:26:26.692543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433659+0000","last_change":"2024-05-08T04:42:22.425948+0000","last_active":"2024-05-08T04:42:22.433659+0000","last_peered":"2024-05-08T04:42:22.433659+0000","last_clean":"2024-05-08T04:42:22.433659+0000","last_became_active":"2024-05-08T04:42:22.425703+0000","last_became_peered":"2024-05-08T04:42:22.425703+0000","last_unstale":"2024-05-08T04:42:22.433659+0000","last_undegraded":"2024-05-08T04:42:22.433659+0000","last_fullsized":"2024-05-08T04:42:22.433659+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:55.985348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432930+0000","last_change":"2024-05-08T04:42:22.424608+0000","last_active":"2024-05-08T04:42:22.432930+0000","last_peered":"2024-05-08T04:42:22.432930+0000","last_clean":"2024-05-08T04:42:22.432930+0000","last_became_active":"2024-05-08T04:42:22.424361+0000","last_became_peered":"2024-05-08T04:42:22.424361+0000","last_unstale":"2024-05-08T04:42:22.432930+0000","last_undegraded":"2024-05-08T04:42:22.432930+0000","last_fullsized":"2024-05-08T04:42:22.432930+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:45.127283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432172+0000","last_change":"2024-05-08T04:42:22.425848+0000","last_active":"2024-05-08T04:42:22.432172+0000","last_peered":"2024-05-08T04:42:22.432172+0000","last_clean":"2024-05-08T04:42:22.432172+0000","last_became_active":"2024-05-08T04:42:22.425420+0000","last_became_peered":"2024-05-08T04:42:22.425420+0000","last_unstale":"2024-05-08T04:42:22.432172+0000","last_undegraded":"2024-05-08T04:42:22.432172+0000","last_fullsized":"2024-05-08T04:42:22.432172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:02.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436490+0000","last_change":"2024-05-08T04:42:22.431988+0000","last_active":"2024-05-08T04:42:22.436490+0000","last_peered":"2024-05-08T04:42:22.436490+0000","last_clean":"2024-05-08T04:42:22.436490+0000","last_became_active":"2024-05-08T04:42:22.431689+0000","last_became_peered":"2024-05-08T04:42:22.431689+0000","last_unstale":"2024-05-08T04:42:22.436490+0000","last_undegraded":"2024-05-08T04:42:22.436490+0000","last_fullsized":"2024-05-08T04:42:22.436490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:04.398485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433567+0000","last_change":"2024-05-08T04:42:22.430501+0000","last_active":"2024-05-08T04:42:22.433567+0000","last_peered":"2024-05-08T04:42:22.433567+0000","last_clean":"2024-05-08T04:42:22.433567+0000","last_became_active":"2024-05-08T04:42:22.428921+0000","last_became_peered":"2024-05-08T04:42:22.428921+0000","last_unstale":"2024-05-08T04:42:22.433567+0000","last_undegraded":"2024-05-08T04:42:22.433567+0000","last_fullsized":"2024-05-08T04:42:22.433567+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:06.071977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442043+0000","last_change":"2024-05-08T04:42:22.440319+0000","last_active":"2024-05-08T04:42:22.442043+0000","last_peered":"2024-05-08T04:42:22.442043+0000","last_clean":"2024-05-08T04:42:22.442043+0000","last_became_active":"2024-05-08T04:42:22.439995+0000","last_became_peered":"2024-05-08T04:42:22.439995+0000","last_unstale":"2024-05-08T04:42:22.442043+0000","last_undegraded":"2024-05-08T04:42:22.442043+0000","last_fullsized":"2024-05-08T04:42:22.442043+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:45.657760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437444+0000","last_change":"2024-05-08T04:42:22.433207+0000","last_active":"2024-05-08T04:42:22.437444+0000","last_peered":"2024-05-08T04:42:22.437444+0000","last_clean":"2024-05-08T04:42:22.437444+0000","last_became_active":"2024-05-08T04:42:22.431605+0000","last_became_peered":"2024-05-08T04:42:22.431605+0000","last_unstale":"2024-05-08T04:42:22.437444+0000","last_undegraded":"2024-05-08T04:42:22.437444+0000","last_fullsized":"2024-05-08T04:42:22.437444+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:52.962208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440596+0000","last_change":"2024-05-08T04:42:22.433234+0000","last_active":"2024-05-08T04:42:22.440596+0000","last_peered":"2024-05-08T04:42:22.440596+0000","last_clean":"2024-05-08T04:42:22.440596+0000","last_became_active":"2024-05-08T04:42:22.430129+0000","last_became_peered":"2024-05-08T04:42:22.430129+0000","last_unstale":"2024-05-08T04:42:22.440596+0000","last_undegraded":"2024-05-08T04:42:22.440596+0000","last_fullsized":"2024-05-08T04:42:22.440596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:59.782392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439234+0000","last_change":"2024-05-08T04:42:22.426669+0000","last_active":"2024-05-08T04:42:22.439234+0000","last_peered":"2024-05-08T04:42:22.439234+0000","last_clean":"2024-05-08T04:42:22.439234+0000","last_became_active":"2024-05-08T04:42:22.426203+0000","last_became_peered":"2024-05-08T04:42:22.426203+0000","last_unstale":"2024-05-08T04:42:22.439234+0000","last_undegraded":"2024-05-08T04:42:22.439234+0000","last_fullsized":"2024-05-08T04:42:22.439234+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:47.369721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439542+0000","last_change":"2024-05-08T04:42:22.432677+0000","last_active":"2024-05-08T04:42:22.439542+0000","last_peered":"2024-05-08T04:42:22.439542+0000","last_clean":"2024-05-08T04:42:22.439542+0000","last_became_active":"2024-05-08T04:42:22.432416+0000","last_became_peered":"2024-05-08T04:42:22.432416+0000","last_unstale":"2024-05-08T04:42:22.439542+0000","last_undegraded":"2024-05-08T04:42:22.439542+0000","last_fullsized":"2024-05-08T04:42:22.439542+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:09.853850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444883+0000","last_change":"2024-05-08T04:42:22.433885+0000","last_active":"2024-05-08T04:42:22.444883+0000","last_peered":"2024-05-08T04:42:22.444883+0000","last_clean":"2024-05-08T04:42:22.444883+0000","last_became_active":"2024-05-08T04:42:22.433704+0000","last_became_peered":"2024-05-08T04:42:22.433704+0000","last_unstale":"2024-05-08T04:42:22.444883+0000","last_undegraded":"2024-05-08T04:42:22.444883+0000","last_fullsized":"2024-05-08T04:42:22.444883+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:33.880491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436647+0000","last_change":"2024-05-08T04:42:22.422838+0000","last_active":"2024-05-08T04:42:22.436647+0000","last_peered":"2024-05-08T04:42:22.436647+0000","last_clean":"2024-05-08T04:42:22.436647+0000","last_became_active":"2024-05-08T04:42:22.422501+0000","last_became_peered":"2024-05-08T04:42:22.422501+0000","last_unstale":"2024-05-08T04:42:22.436647+0000","last_undegraded":"2024-05-08T04:42:22.436647+0000","last_fullsized":"2024-05-08T04:42:22.436647+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:08.596734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444208+0000","last_change":"2024-05-08T04:42:21.422505+0000","last_active":"2024-05-08T04:42:22.444208+0000","last_peered":"2024-05-08T04:42:22.444208+0000","last_clean":"2024-05-08T04:42:22.444208+0000","last_became_active":"2024-05-08T04:42:21.422235+0000","last_became_peered":"2024-05-08T04:42:21.422235+0000","last_unstale":"2024-05-08T04:42:22.444208+0000","last_undegraded":"2024-05-08T04:42:22.444208+0000","last_fullsized":"2024-05-08T04:42:22.444208+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:52.111176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435493+0000","last_change":"2024-05-08T04:42:22.429472+0000","last_active":"2024-05-08T04:42:22.435493+0000","last_peered":"2024-05-08T04:42:22.435493+0000","last_clean":"2024-05-08T04:42:22.435493+0000","last_became_active":"2024-05-08T04:42:22.429176+0000","last_became_peered":"2024-05-08T04:42:22.429176+0000","last_unstale":"2024-05-08T04:42:22.435493+0000","last_undegraded":"2024-05-08T04:42:22.435493+0000","last_fullsized":"2024-05-08T04:42:22.435493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:03.712503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444438+0000","last_change":"2024-05-08T04:42:21.433044+0000","last_active":"2024-05-08T04:42:22.444438+0000","last_peered":"2024-05-08T04:42:22.444438+0000","last_clean":"2024-05-08T04:42:22.444438+0000","last_became_active":"2024-05-08T04:42:21.432656+0000","last_became_peered":"2024-05-08T04:42:21.432656+0000","last_unstale":"2024-05-08T04:42:22.444438+0000","last_undegraded":"2024-05-08T04:42:22.444438+0000","last_fullsized":"2024-05-08T04:42:22.444438+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.813583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442553+0000","last_change":"2024-05-08T04:42:22.435158+0000","last_active":"2024-05-08T04:42:22.442553+0000","last_peered":"2024-05-08T04:42:22.442553+0000","last_clean":"2024-05-08T04:42:22.442553+0000","last_became_active":"2024-05-08T04:42:22.434913+0000","last_became_peered":"2024-05-08T04:42:22.434913+0000","last_unstale":"2024-05-08T04:42:22.442553+0000","last_undegraded":"2024-05-08T04:42:22.442553+0000","last_fullsized":"2024-05-08T04:42:22.442553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:32.303328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.445221+0000","last_change":"2024-05-08T04:42:21.420132+0000","last_active":"2024-05-08T04:42:22.445221+0000","last_peered":"2024-05-08T04:42:22.445221+0000","last_clean":"2024-05-08T04:42:22.445221+0000","last_became_active":"2024-05-08T04:42:21.419850+0000","last_became_peered":"2024-05-08T04:42:21.419850+0000","last_unstale":"2024-05-08T04:42:22.445221+0000","last_undegraded":"2024-05-08T04:42:22.445221+0000","last_fullsized":"2024-05-08T04:42:22.445221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:14.306726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437646+0000","last_change":"2024-05-08T04:42:22.425388+0000","last_active":"2024-05-08T04:42:22.437646+0000","last_peered":"2024-05-08T04:42:22.437646+0000","last_clean":"2024-05-08T04:42:22.437646+0000","last_became_active":"2024-05-08T04:42:22.425071+0000","last_became_peered":"2024-05-08T04:42:22.425071+0000","last_unstale":"2024-05-08T04:42:22.437646+0000","last_undegraded":"2024-05-08T04:42:22.437646+0000","last_fullsized":"2024-05-08T04:42:22.437646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:04.398485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443598+0000","last_change":"2024-05-08T04:42:21.427287+0000","last_active":"2024-05-08T04:42:22.443598+0000","last_peered":"2024-05-08T04:42:22.443598+0000","last_clean":"2024-05-08T04:42:22.443598+0000","last_became_active":"2024-05-08T04:42:21.426944+0000","last_became_peered":"2024-05-08T04:42:21.426944+0000","last_unstale":"2024-05-08T04:42:22.443598+0000","last_undegraded":"2024-05-08T04:42:22.443598+0000","last_fullsized":"2024-05-08T04:42:22.443598+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:41.204120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432282+0000","last_change":"2024-05-08T04:42:22.427253+0000","last_active":"2024-05-08T04:42:22.432282+0000","last_peered":"2024-05-08T04:42:22.432282+0000","last_clean":"2024-05-08T04:42:22.432282+0000","last_became_active":"2024-05-08T04:42:22.422929+0000","last_became_peered":"2024-05-08T04:42:22.422929+0000","last_unstale":"2024-05-08T04:42:22.432282+0000","last_undegraded":"2024-05-08T04:42:22.432282+0000","last_fullsized":"2024-05-08T04:42:22.432282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:22.974009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443159+0000","last_change":"2024-05-08T04:42:21.430742+0000","last_active":"2024-05-08T04:42:22.443159+0000","last_peered":"2024-05-08T04:42:22.443159+0000","last_clean":"2024-05-08T04:42:22.443159+0000","last_became_active":"2024-05-08T04:42:21.430472+0000","last_became_peered":"2024-05-08T04:42:21.430472+0000","last_unstale":"2024-05-08T04:42:22.443159+0000","last_undegraded":"2024-05-08T04:42:22.443159+0000","last_fullsized":"2024-05-08T04:42:22.443159+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:02.459794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438211+0000","last_change":"2024-05-08T04:42:22.434417+0000","last_active":"2024-05-08T04:42:22.438211+0000","last_peered":"2024-05-08T04:42:22.438211+0000","last_clean":"2024-05-08T04:42:22.438211+0000","last_became_active":"2024-05-08T04:42:22.433458+0000","last_became_peered":"2024-05-08T04:42:22.433458+0000","last_unstale":"2024-05-08T04:42:22.438211+0000","last_undegraded":"2024-05-08T04:42:22.438211+0000","last_fullsized":"2024-05-08T04:42:22.438211+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:06.071977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442739+0000","last_change":"2024-05-08T04:42:21.422965+0000","last_active":"2024-05-08T04:42:22.442739+0000","last_peered":"2024-05-08T04:42:22.442739+0000","last_clean":"2024-05-08T04:42:22.442739+0000","last_became_active":"2024-05-08T04:42:21.422643+0000","last_became_peered":"2024-05-08T04:42:21.422643+0000","last_unstale":"2024-05-08T04:42:22.442739+0000","last_undegraded":"2024-05-08T04:42:22.442739+0000","last_fullsized":"2024-05-08T04:42:22.442739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:49.998862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432944+0000","last_change":"2024-05-08T04:42:22.421216+0000","last_active":"2024-05-08T04:42:22.432944+0000","last_peered":"2024-05-08T04:42:22.432944+0000","last_clean":"2024-05-08T04:42:22.432944+0000","last_became_active":"2024-05-08T04:42:22.420767+0000","last_became_peered":"2024-05-08T04:42:22.420767+0000","last_unstale":"2024-05-08T04:42:22.432944+0000","last_undegraded":"2024-05-08T04:42:22.432944+0000","last_fullsized":"2024-05-08T04:42:22.432944+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:53.119288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.420522+0000","last_change":"2024-05-08T04:42:21.428589+0000","last_active":"2024-05-08T04:42:22.420522+0000","last_peered":"2024-05-08T04:42:22.420522+0000","last_clean":"2024-05-08T04:42:22.420522+0000","last_became_active":"2024-05-08T04:42:21.428180+0000","last_became_peered":"2024-05-08T04:42:21.428180+0000","last_unstale":"2024-05-08T04:42:22.420522+0000","last_undegraded":"2024-05-08T04:42:22.420522+0000","last_fullsized":"2024-05-08T04:42:22.420522+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:51.319334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.430898+0000","last_change":"2024-05-08T04:42:22.420870+0000","last_active":"2024-05-08T04:42:22.430898+0000","last_peered":"2024-05-08T04:42:22.430898+0000","last_clean":"2024-05-08T04:42:22.430898+0000","last_became_active":"2024-05-08T04:42:22.420582+0000","last_became_peered":"2024-05-08T04:42:22.420582+0000","last_unstale":"2024-05-08T04:42:22.430898+0000","last_undegraded":"2024-05-08T04:42:22.430898+0000","last_fullsized":"2024-05-08T04:42:22.430898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:39.812589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.417735+0000","last_change":"2024-05-08T04:42:21.432527+0000","last_active":"2024-05-08T04:42:22.417735+0000","last_peered":"2024-05-08T04:42:22.417735+0000","last_clean":"2024-05-08T04:42:22.417735+0000","last_became_active":"2024-05-08T04:42:21.431998+0000","last_became_peered":"2024-05-08T04:42:21.431998+0000","last_unstale":"2024-05-08T04:42:22.417735+0000","last_undegraded":"2024-05-08T04:42:22.417735+0000","last_fullsized":"2024-05-08T04:42:22.417735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:21:14.358754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433608+0000","last_change":"2024-05-08T04:42:22.431367+0000","last_active":"2024-05-08T04:42:22.433608+0000","last_peered":"2024-05-08T04:42:22.433608+0000","last_clean":"2024-05-08T04:42:22.433608+0000","last_became_active":"2024-05-08T04:42:22.431097+0000","last_became_peered":"2024-05-08T04:42:22.431097+0000","last_unstale":"2024-05-08T04:42:22.433608+0000","last_undegraded":"2024-05-08T04:42:22.433608+0000","last_fullsized":"2024-05-08T04:42:22.433608+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:21.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.419069+0000","last_change":"2024-05-08T04:42:21.424570+0000","last_active":"2024-05-08T04:42:22.419069+0000","last_peered":"2024-05-08T04:42:22.419069+0000","last_clean":"2024-05-08T04:42:22.419069+0000","last_became_active":"2024-05-08T04:42:21.423459+0000","last_became_peered":"2024-05-08T04:42:21.423459+0000","last_unstale":"2024-05-08T04:42:22.419069+0000","last_undegraded":"2024-05-08T04:42:22.419069+0000","last_fullsized":"2024-05-08T04:42:22.419069+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:38.488564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432020+0000","last_change":"2024-05-08T04:42:22.421880+0000","last_active":"2024-05-08T04:42:22.432020+0000","last_peered":"2024-05-08T04:42:22.432020+0000","last_clean":"2024-05-08T04:42:22.432020+0000","last_became_active":"2024-05-08T04:42:22.416477+0000","last_became_peered":"2024-05-08T04:42:22.416477+0000","last_unstale":"2024-05-08T04:42:22.432020+0000","last_undegraded":"2024-05-08T04:42:22.432020+0000","last_fullsized":"2024-05-08T04:42:22.432020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:46.298060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.408912+0000","last_change":"2024-05-08T04:42:21.425895+0000","last_active":"2024-05-08T04:42:22.408912+0000","last_peered":"2024-05-08T04:42:22.408912+0000","last_clean":"2024-05-08T04:42:22.408912+0000","last_became_active":"2024-05-08T04:42:21.424720+0000","last_became_peered":"2024-05-08T04:42:21.424720+0000","last_unstale":"2024-05-08T04:42:22.408912+0000","last_undegraded":"2024-05-08T04:42:22.408912+0000","last_fullsized":"2024-05-08T04:42:22.408912+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:51.635883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438582+0000","last_change":"2024-05-08T04:42:22.435868+0000","last_active":"2024-05-08T04:42:22.438582+0000","last_peered":"2024-05-08T04:42:22.438582+0000","last_clean":"2024-05-08T04:42:22.438582+0000","last_became_active":"2024-05-08T04:42:22.434819+0000","last_became_peered":"2024-05-08T04:42:22.434819+0000","last_unstale":"2024-05-08T04:42:22.438582+0000","last_undegraded":"2024-05-08T04:42:22.438582+0000","last_fullsized":"2024-05-08T04:42:22.438582+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:38.522734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.445289+0000","last_change":"2024-05-08T04:42:21.429376+0000","last_active":"2024-05-08T04:42:22.445289+0000","last_peered":"2024-05-08T04:42:22.445289+0000","last_clean":"2024-05-08T04:42:22.445289+0000","last_became_active":"2024-05-08T04:42:21.429079+0000","last_became_peered":"2024-05-08T04:42:21.429079+0000","last_unstale":"2024-05-08T04:42:22.445289+0000","last_undegraded":"2024-05-08T04:42:22.445289+0000","last_fullsized":"2024-05-08T04:42:22.445289+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:02.704392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438266+0000","last_change":"2024-05-08T04:42:22.433979+0000","last_active":"2024-05-08T04:42:22.438266+0000","last_peered":"2024-05-08T04:42:22.438266+0000","last_clean":"2024-05-08T04:42:22.438266+0000","last_became_active":"2024-05-08T04:42:22.433401+0000","last_became_peered":"2024-05-08T04:42:22.433401+0000","last_unstale":"2024-05-08T04:42:22.438266+0000","last_undegraded":"2024-05-08T04:42:22.438266+0000","last_fullsized":"2024-05-08T04:42:22.438266+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:55.985348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.451002+0000","last_change":"2024-05-08T04:42:21.427495+0000","last_active":"2024-05-08T04:42:22.451002+0000","last_peered":"2024-05-08T04:42:22.451002+0000","last_clean":"2024-05-08T04:42:22.451002+0000","last_became_active":"2024-05-08T04:42:21.426599+0000","last_became_peered":"2024-05-08T04:42:21.426599+0000","last_unstale":"2024-05-08T04:42:22.451002+0000","last_undegraded":"2024-05-08T04:42:22.451002+0000","last_fullsized":"2024-05-08T04:42:22.451002+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:51.319334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439459+0000","last_change":"2024-05-08T04:42:22.432112+0000","last_active":"2024-05-08T04:42:22.439459+0000","last_peered":"2024-05-08T04:42:22.439459+0000","last_clean":"2024-05-08T04:42:22.439459+0000","last_became_active":"2024-05-08T04:42:22.431864+0000","last_became_peered":"2024-05-08T04:42:22.431864+0000","last_unstale":"2024-05-08T04:42:22.439459+0000","last_undegraded":"2024-05-08T04:42:22.439459+0000","last_fullsized":"2024-05-08T04:42:22.439459+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:02.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444443+0000","last_change":"2024-05-08T04:42:21.429978+0000","last_active":"2024-05-08T04:42:22.444443+0000","last_peered":"2024-05-08T04:42:22.444443+0000","last_clean":"2024-05-08T04:42:22.444443+0000","last_became_active":"2024-05-08T04:42:21.428710+0000","last_became_peered":"2024-05-08T04:42:21.428710+0000","last_unstale":"2024-05-08T04:42:22.444443+0000","last_undegraded":"2024-05-08T04:42:22.444443+0000","last_fullsized":"2024-05-08T04:42:22.444443+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:10.851787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427126+0000","last_change":"2024-05-08T04:42:22.425336+0000","last_active":"2024-05-08T04:42:22.427126+0000","last_peered":"2024-05-08T04:42:22.427126+0000","last_clean":"2024-05-08T04:42:22.427126+0000","last_became_active":"2024-05-08T04:42:22.421634+0000","last_became_peered":"2024-05-08T04:42:22.421634+0000","last_unstale":"2024-05-08T04:42:22.427126+0000","last_undegraded":"2024-05-08T04:42:22.427126+0000","last_fullsized":"2024-05-08T04:42:22.427126+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:55.520501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412689+0000","last_change":"2024-05-08T04:42:21.431370+0000","last_active":"2024-05-08T04:42:22.412689+0000","last_peered":"2024-05-08T04:42:22.412689+0000","last_clean":"2024-05-08T04:42:22.412689+0000","last_became_active":"2024-05-08T04:42:21.430832+0000","last_became_peered":"2024-05-08T04:42:21.430832+0000","last_unstale":"2024-05-08T04:42:22.412689+0000","last_undegraded":"2024-05-08T04:42:22.412689+0000","last_fullsized":"2024-05-08T04:42:22.412689+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:07.928333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.426873+0000","last_change":"2024-05-08T04:42:22.419377+0000","last_active":"2024-05-08T04:42:22.426873+0000","last_peered":"2024-05-08T04:42:22.426873+0000","last_clean":"2024-05-08T04:42:22.426873+0000","last_became_active":"2024-05-08T04:42:22.415628+0000","last_became_peered":"2024-05-08T04:42:22.415628+0000","last_unstale":"2024-05-08T04:42:22.426873+0000","last_undegraded":"2024-05-08T04:42:22.426873+0000","last_fullsized":"2024-05-08T04:42:22.426873+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:12.495736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.413928+0000","last_change":"2024-05-08T04:42:21.424072+0000","last_active":"2024-05-08T04:42:22.413928+0000","last_peered":"2024-05-08T04:42:22.413928+0000","last_clean":"2024-05-08T04:42:22.413928+0000","last_became_active":"2024-05-08T04:42:21.423101+0000","last_became_peered":"2024-05-08T04:42:21.423101+0000","last_unstale":"2024-05-08T04:42:22.413928+0000","last_undegraded":"2024-05-08T04:42:22.413928+0000","last_fullsized":"2024-05-08T04:42:22.413928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:35.699449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436785+0000","last_change":"2024-05-08T04:42:22.423948+0000","last_active":"2024-05-08T04:42:22.436785+0000","last_peered":"2024-05-08T04:42:22.436785+0000","last_clean":"2024-05-08T04:42:22.436785+0000","last_became_active":"2024-05-08T04:42:22.423689+0000","last_became_peered":"2024-05-08T04:42:22.423689+0000","last_unstale":"2024-05-08T04:42:22.436785+0000","last_undegraded":"2024-05-08T04:42:22.436785+0000","last_fullsized":"2024-05-08T04:42:22.436785+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:37.674122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.449422+0000","last_change":"2024-05-08T04:42:21.429462+0000","last_active":"2024-05-08T04:42:22.449422+0000","last_peered":"2024-05-08T04:42:22.449422+0000","last_clean":"2024-05-08T04:42:22.449422+0000","last_became_active":"2024-05-08T04:42:21.429171+0000","last_became_peered":"2024-05-08T04:42:21.429171+0000","last_unstale":"2024-05-08T04:42:22.449422+0000","last_undegraded":"2024-05-08T04:42:22.449422+0000","last_fullsized":"2024-05-08T04:42:22.449422+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:45.462102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437399+0000","last_change":"2024-05-08T04:42:22.435406+0000","last_active":"2024-05-08T04:42:22.437399+0000","last_peered":"2024-05-08T04:42:22.437399+0000","last_clean":"2024-05-08T04:42:22.437399+0000","last_became_active":"2024-05-08T04:42:22.434844+0000","last_became_peered":"2024-05-08T04:42:22.434844+0000","last_unstale":"2024-05-08T04:42:22.437399+0000","last_undegraded":"2024-05-08T04:42:22.437399+0000","last_fullsized":"2024-05-08T04:42:22.437399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:43.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.449482+0000","last_change":"2024-05-08T04:42:21.431049+0000","last_active":"2024-05-08T04:42:22.449482+0000","last_peered":"2024-05-08T04:42:22.449482+0000","last_clean":"2024-05-08T04:42:22.449482+0000","last_became_active":"2024-05-08T04:42:21.430688+0000","last_became_peered":"2024-05-08T04:42:21.430688+0000","last_unstale":"2024-05-08T04:42:22.449482+0000","last_undegraded":"2024-05-08T04:42:22.449482+0000","last_fullsized":"2024-05-08T04:42:22.449482+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:46.560969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.434662+0000","last_change":"2024-05-08T04:42:22.429405+0000","last_active":"2024-05-08T04:42:22.434662+0000","last_peered":"2024-05-08T04:42:22.434662+0000","last_clean":"2024-05-08T04:42:22.434662+0000","last_became_active":"2024-05-08T04:42:22.428503+0000","last_became_peered":"2024-05-08T04:42:22.428503+0000","last_unstale":"2024-05-08T04:42:22.434662+0000","last_undegraded":"2024-05-08T04:42:22.434662+0000","last_fullsized":"2024-05-08T04:42:22.434662+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:40.749846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440948+0000","last_change":"2024-05-08T04:42:21.422899+0000","last_active":"2024-05-08T04:42:22.440948+0000","last_peered":"2024-05-08T04:42:22.440948+0000","last_clean":"2024-05-08T04:42:22.440948+0000","last_became_active":"2024-05-08T04:42:21.421585+0000","last_became_peered":"2024-05-08T04:42:21.421585+0000","last_unstale":"2024-05-08T04:42:22.440948+0000","last_undegraded":"2024-05-08T04:42:22.440948+0000","last_fullsized":"2024-05-08T04:42:22.440948+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:38.804477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437836+0000","last_change":"2024-05-08T04:42:22.432067+0000","last_active":"2024-05-08T04:42:22.437836+0000","last_peered":"2024-05-08T04:42:22.437836+0000","last_clean":"2024-05-08T04:42:22.437836+0000","last_became_active":"2024-05-08T04:42:22.431529+0000","last_became_peered":"2024-05-08T04:42:22.431529+0000","last_unstale":"2024-05-08T04:42:22.437836+0000","last_undegraded":"2024-05-08T04:42:22.437836+0000","last_fullsized":"2024-05-08T04:42:22.437836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:45.127283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412964+0000","last_change":"2024-05-08T04:42:21.420430+0000","last_active":"2024-05-08T04:42:22.412964+0000","last_peered":"2024-05-08T04:42:22.412964+0000","last_clean":"2024-05-08T04:42:22.412964+0000","last_became_active":"2024-05-08T04:42:21.420165+0000","last_became_peered":"2024-05-08T04:42:21.420165+0000","last_unstale":"2024-05-08T04:42:22.412964+0000","last_undegraded":"2024-05-08T04:42:22.412964+0000","last_fullsized":"2024-05-08T04:42:22.412964+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:51.635883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436222+0000","last_change":"2024-05-08T04:42:22.419038+0000","last_active":"2024-05-08T04:42:22.436222+0000","last_peered":"2024-05-08T04:42:22.436222+0000","last_clean":"2024-05-08T04:42:22.436222+0000","last_became_active":"2024-05-08T04:42:22.418802+0000","last_became_peered":"2024-05-08T04:42:22.418802+0000","last_unstale":"2024-05-08T04:42:22.436222+0000","last_undegraded":"2024-05-08T04:42:22.436222+0000","last_fullsized":"2024-05-08T04:42:22.436222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:44:45.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.446558+0000","last_change":"2024-05-08T04:42:21.426859+0000","last_active":"2024-05-08T04:42:22.446558+0000","last_peered":"2024-05-08T04:42:22.446558+0000","last_clean":"2024-05-08T04:42:22.446558+0000","last_became_active":"2024-05-08T04:42:21.426539+0000","last_became_peered":"2024-05-08T04:42:21.426539+0000","last_unstale":"2024-05-08T04:42:22.446558+0000","last_undegraded":"2024-05-08T04:42:22.446558+0000","last_fullsized":"2024-05-08T04:42:22.446558+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:35.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439839+0000","last_change":"2024-05-08T04:42:22.428001+0000","last_active":"2024-05-08T04:42:22.439839+0000","last_peered":"2024-05-08T04:42:22.439839+0000","last_clean":"2024-05-08T04:42:22.439839+0000","last_became_active":"2024-05-08T04:42:22.427743+0000","last_became_peered":"2024-05-08T04:42:22.427743+0000","last_unstale":"2024-05-08T04:42:22.439839+0000","last_undegraded":"2024-05-08T04:42:22.439839+0000","last_fullsized":"2024-05-08T04:42:22.439839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:21.324641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.411137+0000","last_change":"2024-05-08T04:42:21.427576+0000","last_active":"2024-05-08T04:42:22.411137+0000","last_peered":"2024-05-08T04:42:22.411137+0000","last_clean":"2024-05-08T04:42:22.411137+0000","last_became_active":"2024-05-08T04:42:21.427318+0000","last_became_peered":"2024-05-08T04:42:21.427318+0000","last_unstale":"2024-05-08T04:42:22.411137+0000","last_undegraded":"2024-05-08T04:42:22.411137+0000","last_fullsized":"2024-05-08T04:42:22.411137+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:01.565043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.422980+0000","last_change":"2024-05-08T04:42:22.418512+0000","last_active":"2024-05-08T04:42:22.422980+0000","last_peered":"2024-05-08T04:42:22.422980+0000","last_clean":"2024-05-08T04:42:22.422980+0000","last_became_active":"2024-05-08T04:42:22.418221+0000","last_became_peered":"2024-05-08T04:42:22.418221+0000","last_unstale":"2024-05-08T04:42:22.422980+0000","last_undegraded":"2024-05-08T04:42:22.422980+0000","last_fullsized":"2024-05-08T04:42:22.422980+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:00:14.413385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.409879+0000","last_change":"2024-05-08T04:42:21.426559+0000","last_active":"2024-05-08T04:42:22.409879+0000","last_peered":"2024-05-08T04:42:22.409879+0000","last_clean":"2024-05-08T04:42:22.409879+0000","last_became_active":"2024-05-08T04:42:21.426238+0000","last_became_peered":"2024-05-08T04:42:21.426238+0000","last_unstale":"2024-05-08T04:42:22.409879+0000","last_undegraded":"2024-05-08T04:42:22.409879+0000","last_fullsized":"2024-05-08T04:42:22.409879+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:41.187717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.430467+0000","last_change":"2024-05-08T04:42:22.429090+0000","last_active":"2024-05-08T04:42:22.430467+0000","last_peered":"2024-05-08T04:42:22.430467+0000","last_clean":"2024-05-08T04:42:22.430467+0000","last_became_active":"2024-05-08T04:42:22.428735+0000","last_became_peered":"2024-05-08T04:42:22.428735+0000","last_unstale":"2024-05-08T04:42:22.430467+0000","last_undegraded":"2024-05-08T04:42:22.430467+0000","last_fullsized":"2024-05-08T04:42:22.430467+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:37.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414920+0000","last_change":"2024-05-08T04:42:21.430003+0000","last_active":"2024-05-08T04:42:22.414920+0000","last_peered":"2024-05-08T04:42:22.414920+0000","last_clean":"2024-05-08T04:42:22.414920+0000","last_became_active":"2024-05-08T04:42:21.429569+0000","last_became_peered":"2024-05-08T04:42:21.429569+0000","last_unstale":"2024-05-08T04:42:22.414920+0000","last_undegraded":"2024-05-08T04:42:22.414920+0000","last_fullsized":"2024-05-08T04:42:22.414920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:01.168478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433545+0000","last_change":"2024-05-08T04:42:22.429487+0000","last_active":"2024-05-08T04:42:22.433545+0000","last_peered":"2024-05-08T04:42:22.433545+0000","last_clean":"2024-05-08T04:42:22.433545+0000","last_became_active":"2024-05-08T04:42:22.428814+0000","last_became_peered":"2024-05-08T04:42:22.428814+0000","last_unstale":"2024-05-08T04:42:22.433545+0000","last_undegraded":"2024-05-08T04:42:22.433545+0000","last_fullsized":"2024-05-08T04:42:22.433545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:26.304165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443491+0000","last_change":"2024-05-08T04:42:21.422300+0000","last_active":"2024-05-08T04:42:22.443491+0000","last_peered":"2024-05-08T04:42:22.443491+0000","last_clean":"2024-05-08T04:42:22.443491+0000","last_became_active":"2024-05-08T04:42:21.421987+0000","last_became_peered":"2024-05-08T04:42:21.421987+0000","last_unstale":"2024-05-08T04:42:22.443491+0000","last_undegraded":"2024-05-08T04:42:22.443491+0000","last_fullsized":"2024-05-08T04:42:22.443491+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:38:30.474789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427458+0000","last_change":"2024-05-08T04:42:22.422115+0000","last_active":"2024-05-08T04:42:22.427458+0000","last_peered":"2024-05-08T04:42:22.427458+0000","last_clean":"2024-05-08T04:42:22.427458+0000","last_became_active":"2024-05-08T04:42:22.421852+0000","last_became_peered":"2024-05-08T04:42:22.421852+0000","last_unstale":"2024-05-08T04:42:22.427458+0000","last_undegraded":"2024-05-08T04:42:22.427458+0000","last_fullsized":"2024-05-08T04:42:22.427458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:36.788761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.416633+0000","last_change":"2024-05-08T04:42:21.427629+0000","last_active":"2024-05-08T04:42:22.416633+0000","last_peered":"2024-05-08T04:42:22.416633+0000","last_clean":"2024-05-08T04:42:22.416633+0000","last_became_active":"2024-05-08T04:42:21.425992+0000","last_became_peered":"2024-05-08T04:42:21.425992+0000","last_unstale":"2024-05-08T04:42:22.416633+0000","last_undegraded":"2024-05-08T04:42:22.416633+0000","last_fullsized":"2024-05-08T04:42:22.416633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:08.845738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435775+0000","last_change":"2024-05-08T04:42:22.429801+0000","last_active":"2024-05-08T04:42:22.435775+0000","last_peered":"2024-05-08T04:42:22.435775+0000","last_clean":"2024-05-08T04:42:22.435775+0000","last_became_active":"2024-05-08T04:42:22.429547+0000","last_became_peered":"2024-05-08T04:42:22.429547+0000","last_unstale":"2024-05-08T04:42:22.435775+0000","last_undegraded":"2024-05-08T04:42:22.435775+0000","last_fullsized":"2024-05-08T04:42:22.435775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:30.870407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414418+0000","last_change":"2024-05-08T04:42:21.416010+0000","last_active":"2024-05-08T04:42:22.414418+0000","last_peered":"2024-05-08T04:42:22.414418+0000","last_clean":"2024-05-08T04:42:22.414418+0000","last_became_active":"2024-05-08T04:42:21.415739+0000","last_became_peered":"2024-05-08T04:42:21.415739+0000","last_unstale":"2024-05-08T04:42:22.414418+0000","last_undegraded":"2024-05-08T04:42:22.414418+0000","last_fullsized":"2024-05-08T04:42:22.414418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.813583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439676+0000","last_change":"2024-05-08T04:42:22.437183+0000","last_active":"2024-05-08T04:42:22.439676+0000","last_peered":"2024-05-08T04:42:22.439676+0000","last_clean":"2024-05-08T04:42:22.439676+0000","last_became_active":"2024-05-08T04:42:22.436635+0000","last_became_peered":"2024-05-08T04:42:22.436635+0000","last_unstale":"2024-05-08T04:42:22.439676+0000","last_undegraded":"2024-05-08T04:42:22.439676+0000","last_fullsized":"2024-05-08T04:42:22.439676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:52.643995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.423714+0000","last_change":"2024-05-08T04:42:21.427630+0000","last_active":"2024-05-08T04:42:22.423714+0000","last_peered":"2024-05-08T04:42:22.423714+0000","last_clean":"2024-05-08T04:42:22.423714+0000","last_became_active":"2024-05-08T04:42:21.427353+0000","last_became_peered":"2024-05-08T04:42:21.427353+0000","last_unstale":"2024-05-08T04:42:22.423714+0000","last_undegraded":"2024-05-08T04:42:22.423714+0000","last_fullsized":"2024-05-08T04:42:22.423714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:45.289949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439022+0000","last_change":"2024-05-08T04:42:22.426475+0000","last_active":"2024-05-08T04:42:22.439022+0000","last_peered":"2024-05-08T04:42:22.439022+0000","last_clean":"2024-05-08T04:42:22.439022+0000","last_became_active":"2024-05-08T04:42:22.425853+0000","last_became_peered":"2024-05-08T04:42:22.425853+0000","last_unstale":"2024-05-08T04:42:22.439022+0000","last_undegraded":"2024-05-08T04:42:22.439022+0000","last_fullsized":"2024-05-08T04:42:22.439022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:59.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414751+0000","last_change":"2024-05-08T04:42:21.423787+0000","last_active":"2024-05-08T04:42:22.414751+0000","last_peered":"2024-05-08T04:42:22.414751+0000","last_clean":"2024-05-08T04:42:22.414751+0000","last_became_active":"2024-05-08T04:42:21.421769+0000","last_became_peered":"2024-05-08T04:42:21.421769+0000","last_unstale":"2024-05-08T04:42:22.414751+0000","last_undegraded":"2024-05-08T04:42:22.414751+0000","last_fullsized":"2024-05-08T04:42:22.414751+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:02.704392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438251+0000","last_change":"2024-05-08T04:42:22.430158+0000","last_active":"2024-05-08T04:42:22.438251+0000","last_peered":"2024-05-08T04:42:22.438251+0000","last_clean":"2024-05-08T04:42:22.438251+0000","last_became_active":"2024-05-08T04:42:22.429555+0000","last_became_peered":"2024-05-08T04:42:22.429555+0000","last_unstale":"2024-05-08T04:42:22.438251+0000","last_undegraded":"2024-05-08T04:42:22.438251+0000","last_fullsized":"2024-05-08T04:42:22.438251+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:43.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.421006+0000","last_change":"2024-05-08T04:42:21.432634+0000","last_active":"2024-05-08T04:42:22.421006+0000","last_peered":"2024-05-08T04:42:22.421006+0000","last_clean":"2024-05-08T04:42:22.421006+0000","last_became_active":"2024-05-08T04:42:21.432330+0000","last_became_peered":"2024-05-08T04:42:21.432330+0000","last_unstale":"2024-05-08T04:42:22.421006+0000","last_undegraded":"2024-05-08T04:42:22.421006+0000","last_fullsized":"2024-05-08T04:42:22.421006+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:11.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436794+0000","last_change":"2024-05-08T04:42:22.423878+0000","last_active":"2024-05-08T04:42:22.436794+0000","last_peered":"2024-05-08T04:42:22.436794+0000","last_clean":"2024-05-08T04:42:22.436794+0000","last_became_active":"2024-05-08T04:42:22.423598+0000","last_became_peered":"2024-05-08T04:42:22.423598+0000","last_unstale":"2024-05-08T04:42:22.436794+0000","last_undegraded":"2024-05-08T04:42:22.436794+0000","last_fullsized":"2024-05-08T04:42:22.436794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:21.158355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.408583+0000","last_change":"2024-05-08T04:42:21.431224+0000","last_active":"2024-05-08T04:42:22.408583+0000","last_peered":"2024-05-08T04:42:22.408583+0000","last_clean":"2024-05-08T04:42:22.408583+0000","last_became_active":"2024-05-08T04:42:21.430082+0000","last_became_peered":"2024-05-08T04:42:21.430082+0000","last_unstale":"2024-05-08T04:42:22.408583+0000","last_undegraded":"2024-05-08T04:42:22.408583+0000","last_fullsized":"2024-05-08T04:42:22.408583+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:41.204120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440387+0000","last_change":"2024-05-08T04:42:22.438057+0000","last_active":"2024-05-08T04:42:22.440387+0000","last_peered":"2024-05-08T04:42:22.440387+0000","last_clean":"2024-05-08T04:42:22.440387+0000","last_became_active":"2024-05-08T04:42:22.437775+0000","last_became_peered":"2024-05-08T04:42:22.437775+0000","last_unstale":"2024-05-08T04:42:22.440387+0000","last_undegraded":"2024-05-08T04:42:22.440387+0000","last_fullsized":"2024-05-08T04:42:22.440387+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:51.006973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412150+0000","last_change":"2024-05-08T04:42:21.419876+0000","last_active":"2024-05-08T04:42:22.412150+0000","last_peered":"2024-05-08T04:42:22.412150+0000","last_clean":"2024-05-08T04:42:22.412150+0000","last_became_active":"2024-05-08T04:42:21.419601+0000","last_became_peered":"2024-05-08T04:42:21.419601+0000","last_unstale":"2024-05-08T04:42:22.412150+0000","last_undegraded":"2024-05-08T04:42:22.412150+0000","last_fullsized":"2024-05-08T04:42:22.412150+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:46.114943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439894+0000","last_change":"2024-05-08T04:42:22.427411+0000","last_active":"2024-05-08T04:42:22.439894+0000","last_peered":"2024-05-08T04:42:22.439894+0000","last_clean":"2024-05-08T04:42:22.439894+0000","last_became_active":"2024-05-08T04:42:22.427112+0000","last_became_peered":"2024-05-08T04:42:22.427112+0000","last_unstale":"2024-05-08T04:42:22.439894+0000","last_undegraded":"2024-05-08T04:42:22.439894+0000","last_fullsized":"2024-05-08T04:42:22.439894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:19:28.215486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.446468+0000","last_change":"2024-05-08T04:42:21.431821+0000","last_active":"2024-05-08T04:42:22.446468+0000","last_peered":"2024-05-08T04:42:22.446468+0000","last_clean":"2024-05-08T04:42:22.446468+0000","last_became_active":"2024-05-08T04:42:21.431401+0000","last_became_peered":"2024-05-08T04:42:21.431401+0000","last_unstale":"2024-05-08T04:42:22.446468+0000","last_undegraded":"2024-05-08T04:42:22.446468+0000","last_fullsized":"2024-05-08T04:42:22.446468+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:35.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":317,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.419735+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:22.419735+0000","last_peered":"2024-05-08T04:42:22.419735+0000","last_clean":"2024-05-08T04:42:22.419735+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:22.419735+0000","last_undegraded":"2024-05-08T04:42:22.419735+0000","last_fullsized":"2024-05-08T04:42:22.419735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1107922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":524413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":524447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":1107922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:42:26.493 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:26.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.664+0000 7f6f34a00640 1 Processor -- start 2024-05-08T04:42:26.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.664+0000 7f6f34a00640 1 -- start start 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f34a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f300b26c0 0x7f6f300cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f34a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3006c770 con 0x7f6f300cfbd0 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f34a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3006ca50 con 0x7f6f3015ab60 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f34a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f300d0060 con 0x7f6f300b26c0 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f300b26c0 0x7f6f300cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f300b26c0 0x7f6f300cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44856/0 (socket says 172.21.15.164:44856) 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 -- 172.21.15.164:0/844726188 learned_addr learned my addr 172.21.15.164:0/844726188 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:26.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f2e800640 1 --1- 172.21.15.164:0/844726188 >> v1:172.21.15.164:6789/0 conn(0x7f6f300cfbd0 0x7f6f300cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51894/0 (socket says 172.21.15.164:51894) 2024-05-08T04:42:26.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 -- 172.21.15.164:0/844726188 >> v1:172.21.15.172:6789/0 conn(0x7f6f3015ab60 legacy=0x7f6f30072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 -- 172.21.15.164:0/844726188 >> v1:172.21.15.164:6789/0 conn(0x7f6f300cfbd0 legacy=0x7f6f300cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.668+0000 7f6f25400640 1 -- 172.21.15.164:0/844726188 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3006cd30 con 0x7f6f300b26c0 2024-05-08T04:42:26.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f25400640 1 --2- 172.21.15.164:0/844726188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f300b26c0 0x7f6f300cf690 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009870 tx=0x7f6f1801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8727942ad244e9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f2d400640 1 -- 172.21.15.164:0/844726188 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180b2070 con 0x7f6f300b26c0 2024-05-08T04:42:26.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f2d400640 1 -- 172.21.15.164:0/844726188 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180a9070 con 0x7f6f300b26c0 2024-05-08T04:42:26.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f2d400640 1 -- 172.21.15.164:0/844726188 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180ad880 con 0x7f6f300b26c0 2024-05-08T04:42:26.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- 172.21.15.164:0/844726188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f300b26c0 msgr2=0x7f6f300cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 --2- 172.21.15.164:0/844726188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f300b26c0 0x7f6f300cf690 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009870 tx=0x7f6f1801c560 comp rx=0 tx=0).stop 2024-05-08T04:42:26.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- 172.21.15.164:0/844726188 shutdown_connections 2024-05-08T04:42:26.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 --2- 172.21.15.164:0/844726188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f300b26c0 0x7f6f300cf690 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- 172.21.15.164:0/844726188 >> 172.21.15.164:0/844726188 conn(0x7f6f3006ac90 msgr2=0x7f6f3006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:26.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- 172.21.15.164:0/844726188 shutdown_connections 2024-05-08T04:42:26.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- 172.21.15.164:0/844726188 wait complete. 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 Processor -- start 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.672+0000 7f6f34a00640 1 -- start start 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 0x7f6f3014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300cfbd0 0x7f6f300b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300aeeb0 con 0x7f6f3015ab60 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300aed30 con 0x7f6f300b26c0 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30168980 con 0x7f6f300cfbd0 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44866/0 (socket says 172.21.15.164:44866) 2024-05-08T04:42:26.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 -- 172.21.15.164:0/1395594457 learned_addr learned my addr 172.21.15.164:0/1395594457 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f25400640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300cfbd0 0x7f6f300b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2de00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 0x7f6f3014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 -- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300cfbd0 msgr2=0x7f6f300b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300cfbd0 0x7f6f300b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 -- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 msgr2=0x7f6f3014c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 0x7f6f3014c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 -- 172.21.15.164:0/1395594457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300b3fb0 con 0x7f6f3015ab60 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2de00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 0x7f6f3014c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f2e800640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c7d0 tx=0x7f6f2800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28017070 con 0x7f6f3015ab60 2024-05-08T04:42:26.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2800a640 con 0x7f6f3015ab60 2024-05-08T04:42:26.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f300b3a00 con 0x7f6f3015ab60 2024-05-08T04:42:26.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800a9f0 con 0x7f6f3015ab60 2024-05-08T04:42:26.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3014bad0 con 0x7f6f3015ab60 2024-05-08T04:42:26.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f30069d60 con 0x7f6f3015ab60 2024-05-08T04:42:26.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.676+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f6f2801a070 con 0x7f6f3015ab60 2024-05-08T04:42:26.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.680+0000 7f6f26a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 0x7f6f0c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:26.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.680+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f6f280856a0 con 0x7f6f3015ab60 2024-05-08T04:42:26.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.680+0000 7f6f2de00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 0x7f6f0c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:26.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.680+0000 7f6f2de00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 0x7f6f0c069b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f20001670 tx=0x7f6f20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:26.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.684+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f28055200 con 0x7f6f3015ab60 2024-05-08T04:42:26.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.836+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f30156c00 con 0x7f6f0c0676b0 2024-05-08T04:42:26.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.840+0000 7f6f26a00640 1 -- 172.21.15.164:0/1395594457 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252013 (secure 0 0 0) 0x7f6f30156c00 con 0x7f6f0c0676b0 2024-05-08T04:42:26.850 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:26.851 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:26.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 msgr2=0x7f6f0c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 0x7f6f0c069b70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f20001670 tx=0x7f6f20009040 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 msgr2=0x7f6f3014a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c7d0 tx=0x7f6f2800cc90 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 shutdown_connections 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f0c0676b0 0x7f6f0c069b70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300cfbd0 0x7f6f300b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f3015ab60 0x7f6f3014a6b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 --2- 172.21.15.164:0/1395594457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f300b26c0 0x7f6f3014c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.844+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 >> 172.21.15.164:0/1395594457 conn(0x7f6f3006ac90 msgr2=0x7f6f30157550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.848+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 shutdown_connections 2024-05-08T04:42:26.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:26.848+0000 7f6f34a00640 1 -- 172.21.15.164:0/1395594457 wait complete. 2024-05-08T04:42:26.871 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-05-08T04:42:25.477975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260036,"kb_used_data":10564,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714684,"statfs":{"total":773094113280,"available":772827836416,"internally_reserved":0,"allocated":10817536,"data_stored":5334920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076239"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427565+0000","last_change":"2024-05-08T04:42:22.425873+0000","last_active":"2024-05-08T04:42:22.427565+0000","last_peered":"2024-05-08T04:42:22.427565+0000","last_clean":"2024-05-08T04:42:22.427565+0000","last_became_active":"2024-05-08T04:42:22.425626+0000","last_became_peered":"2024-05-08T04:42:22.425626+0000","last_unstale":"2024-05-08T04:42:22.427565+0000","last_undegraded":"2024-05-08T04:42:22.427565+0000","last_fullsized":"2024-05-08T04:42:22.427565+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:10.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433589+0000","last_change":"2024-05-08T04:42:22.430499+0000","last_active":"2024-05-08T04:42:22.433589+0000","last_peered":"2024-05-08T04:42:22.433589+0000","last_clean":"2024-05-08T04:42:22.433589+0000","last_became_active":"2024-05-08T04:42:22.429164+0000","last_became_peered":"2024-05-08T04:42:22.429164+0000","last_unstale":"2024-05-08T04:42:22.433589+0000","last_undegraded":"2024-05-08T04:42:22.433589+0000","last_fullsized":"2024-05-08T04:42:22.433589+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:45.657760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.426034+0000","last_change":"2024-05-08T04:42:22.423160+0000","last_active":"2024-05-08T04:42:22.426034+0000","last_peered":"2024-05-08T04:42:22.426034+0000","last_clean":"2024-05-08T04:42:22.426034+0000","last_became_active":"2024-05-08T04:42:22.422911+0000","last_became_peered":"2024-05-08T04:42:22.422911+0000","last_unstale":"2024-05-08T04:42:22.426034+0000","last_undegraded":"2024-05-08T04:42:22.426034+0000","last_fullsized":"2024-05-08T04:42:22.426034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:42.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.418799+0000","last_change":"2024-05-08T04:42:22.415933+0000","last_active":"2024-05-08T04:42:22.418799+0000","last_peered":"2024-05-08T04:42:22.418799+0000","last_clean":"2024-05-08T04:42:22.418799+0000","last_became_active":"2024-05-08T04:42:22.415632+0000","last_became_peered":"2024-05-08T04:42:22.415632+0000","last_unstale":"2024-05-08T04:42:22.418799+0000","last_undegraded":"2024-05-08T04:42:22.418799+0000","last_fullsized":"2024-05-08T04:42:22.418799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:51.006973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438616+0000","last_change":"2024-05-08T04:42:22.436450+0000","last_active":"2024-05-08T04:42:22.438616+0000","last_peered":"2024-05-08T04:42:22.438616+0000","last_clean":"2024-05-08T04:42:22.438616+0000","last_became_active":"2024-05-08T04:42:22.436091+0000","last_became_peered":"2024-05-08T04:42:22.436091+0000","last_unstale":"2024-05-08T04:42:22.438616+0000","last_undegraded":"2024-05-08T04:42:22.438616+0000","last_fullsized":"2024-05-08T04:42:22.438616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:42.212232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438430+0000","last_change":"2024-05-08T04:42:22.435033+0000","last_active":"2024-05-08T04:42:22.438430+0000","last_peered":"2024-05-08T04:42:22.438430+0000","last_clean":"2024-05-08T04:42:22.438430+0000","last_became_active":"2024-05-08T04:42:22.434761+0000","last_became_peered":"2024-05-08T04:42:22.434761+0000","last_unstale":"2024-05-08T04:42:22.438430+0000","last_undegraded":"2024-05-08T04:42:22.438430+0000","last_fullsized":"2024-05-08T04:42:22.438430+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:32.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436137+0000","last_change":"2024-05-08T04:42:22.428131+0000","last_active":"2024-05-08T04:42:22.436137+0000","last_peered":"2024-05-08T04:42:22.436137+0000","last_clean":"2024-05-08T04:42:22.436137+0000","last_became_active":"2024-05-08T04:42:22.427869+0000","last_became_peered":"2024-05-08T04:42:22.427869+0000","last_unstale":"2024-05-08T04:42:22.436137+0000","last_undegraded":"2024-05-08T04:42:22.436137+0000","last_fullsized":"2024-05-08T04:42:22.436137+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:09.853850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433491+0000","last_change":"2024-05-08T04:42:22.430791+0000","last_active":"2024-05-08T04:42:22.433491+0000","last_peered":"2024-05-08T04:42:22.433491+0000","last_clean":"2024-05-08T04:42:22.433491+0000","last_became_active":"2024-05-08T04:42:22.428601+0000","last_became_peered":"2024-05-08T04:42:22.428601+0000","last_unstale":"2024-05-08T04:42:22.433491+0000","last_undegraded":"2024-05-08T04:42:22.433491+0000","last_fullsized":"2024-05-08T04:42:22.433491+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:32.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432044+0000","last_change":"2024-05-08T04:42:22.422526+0000","last_active":"2024-05-08T04:42:22.432044+0000","last_peered":"2024-05-08T04:42:22.432044+0000","last_clean":"2024-05-08T04:42:22.432044+0000","last_became_active":"2024-05-08T04:42:22.422227+0000","last_became_peered":"2024-05-08T04:42:22.422227+0000","last_unstale":"2024-05-08T04:42:22.432044+0000","last_undegraded":"2024-05-08T04:42:22.432044+0000","last_fullsized":"2024-05-08T04:42:22.432044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:37.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436098+0000","last_change":"2024-05-08T04:42:22.430864+0000","last_active":"2024-05-08T04:42:22.436098+0000","last_peered":"2024-05-08T04:42:22.436098+0000","last_clean":"2024-05-08T04:42:22.436098+0000","last_became_active":"2024-05-08T04:42:22.430579+0000","last_became_peered":"2024-05-08T04:42:22.430579+0000","last_unstale":"2024-05-08T04:42:22.436098+0000","last_undegraded":"2024-05-08T04:42:22.436098+0000","last_fullsized":"2024-05-08T04:42:22.436098+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:21.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438285+0000","last_change":"2024-05-08T04:42:22.434115+0000","last_active":"2024-05-08T04:42:22.438285+0000","last_peered":"2024-05-08T04:42:22.438285+0000","last_clean":"2024-05-08T04:42:22.438285+0000","last_became_active":"2024-05-08T04:42:22.433822+0000","last_became_peered":"2024-05-08T04:42:22.433822+0000","last_unstale":"2024-05-08T04:42:22.438285+0000","last_undegraded":"2024-05-08T04:42:22.438285+0000","last_fullsized":"2024-05-08T04:42:22.438285+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:26.304165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438150+0000","last_change":"2024-05-08T04:42:22.426212+0000","last_active":"2024-05-08T04:42:22.438150+0000","last_peered":"2024-05-08T04:42:22.438150+0000","last_clean":"2024-05-08T04:42:22.438150+0000","last_became_active":"2024-05-08T04:42:22.425897+0000","last_became_peered":"2024-05-08T04:42:22.425897+0000","last_unstale":"2024-05-08T04:42:22.438150+0000","last_undegraded":"2024-05-08T04:42:22.438150+0000","last_fullsized":"2024-05-08T04:42:22.438150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:52.962208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432999+0000","last_change":"2024-05-08T04:42:22.422883+0000","last_active":"2024-05-08T04:42:22.432999+0000","last_peered":"2024-05-08T04:42:22.432999+0000","last_clean":"2024-05-08T04:42:22.432999+0000","last_became_active":"2024-05-08T04:42:22.422617+0000","last_became_peered":"2024-05-08T04:42:22.422617+0000","last_unstale":"2024-05-08T04:42:22.432999+0000","last_undegraded":"2024-05-08T04:42:22.432999+0000","last_fullsized":"2024-05-08T04:42:22.432999+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:32.821695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.441224+0000","last_change":"2024-05-08T04:42:22.431221+0000","last_active":"2024-05-08T04:42:22.441224+0000","last_peered":"2024-05-08T04:42:22.441224+0000","last_clean":"2024-05-08T04:42:22.441224+0000","last_became_active":"2024-05-08T04:42:22.430972+0000","last_became_peered":"2024-05-08T04:42:22.430972+0000","last_unstale":"2024-05-08T04:42:22.441224+0000","last_undegraded":"2024-05-08T04:42:22.441224+0000","last_fullsized":"2024-05-08T04:42:22.441224+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:24.776238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435326+0000","last_change":"2024-05-08T04:42:22.421333+0000","last_active":"2024-05-08T04:42:22.435326+0000","last_peered":"2024-05-08T04:42:22.435326+0000","last_clean":"2024-05-08T04:42:22.435326+0000","last_became_active":"2024-05-08T04:42:22.421080+0000","last_became_peered":"2024-05-08T04:42:22.421080+0000","last_unstale":"2024-05-08T04:42:22.435326+0000","last_undegraded":"2024-05-08T04:42:22.435326+0000","last_fullsized":"2024-05-08T04:42:22.435326+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:15.314837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438033+0000","last_change":"2024-05-08T04:42:22.434004+0000","last_active":"2024-05-08T04:42:22.438033+0000","last_peered":"2024-05-08T04:42:22.438033+0000","last_clean":"2024-05-08T04:42:22.438033+0000","last_became_active":"2024-05-08T04:42:22.433692+0000","last_became_peered":"2024-05-08T04:42:22.433692+0000","last_unstale":"2024-05-08T04:42:22.438033+0000","last_undegraded":"2024-05-08T04:42:22.438033+0000","last_fullsized":"2024-05-08T04:42:22.438033+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:30.653406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432841+0000","last_change":"2024-05-08T04:42:22.426494+0000","last_active":"2024-05-08T04:42:22.432841+0000","last_peered":"2024-05-08T04:42:22.432841+0000","last_clean":"2024-05-08T04:42:22.432841+0000","last_became_active":"2024-05-08T04:42:22.426238+0000","last_became_peered":"2024-05-08T04:42:22.426238+0000","last_unstale":"2024-05-08T04:42:22.432841+0000","last_undegraded":"2024-05-08T04:42:22.432841+0000","last_fullsized":"2024-05-08T04:42:22.432841+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:30.653406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432934+0000","last_change":"2024-05-08T04:42:22.423958+0000","last_active":"2024-05-08T04:42:22.432934+0000","last_peered":"2024-05-08T04:42:22.432934+0000","last_clean":"2024-05-08T04:42:22.432934+0000","last_became_active":"2024-05-08T04:42:22.423720+0000","last_became_peered":"2024-05-08T04:42:22.423720+0000","last_unstale":"2024-05-08T04:42:22.432934+0000","last_undegraded":"2024-05-08T04:42:22.432934+0000","last_fullsized":"2024-05-08T04:42:22.432934+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:36.254755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439390+0000","last_change":"2024-05-08T04:42:22.425698+0000","last_active":"2024-05-08T04:42:22.439390+0000","last_peered":"2024-05-08T04:42:22.439390+0000","last_clean":"2024-05-08T04:42:22.439390+0000","last_became_active":"2024-05-08T04:42:22.425440+0000","last_became_peered":"2024-05-08T04:42:22.425440+0000","last_unstale":"2024-05-08T04:42:22.439390+0000","last_undegraded":"2024-05-08T04:42:22.439390+0000","last_fullsized":"2024-05-08T04:42:22.439390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:02.174432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.431963+0000","last_change":"2024-05-08T04:42:22.422749+0000","last_active":"2024-05-08T04:42:22.431963+0000","last_peered":"2024-05-08T04:42:22.431963+0000","last_clean":"2024-05-08T04:42:22.431963+0000","last_became_active":"2024-05-08T04:42:22.422384+0000","last_became_peered":"2024-05-08T04:42:22.422384+0000","last_unstale":"2024-05-08T04:42:22.431963+0000","last_undegraded":"2024-05-08T04:42:22.431963+0000","last_fullsized":"2024-05-08T04:42:22.431963+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:21.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439397+0000","last_change":"2024-05-08T04:42:22.426484+0000","last_active":"2024-05-08T04:42:22.439397+0000","last_peered":"2024-05-08T04:42:22.439397+0000","last_clean":"2024-05-08T04:42:22.439397+0000","last_became_active":"2024-05-08T04:42:22.426077+0000","last_became_peered":"2024-05-08T04:42:22.426077+0000","last_unstale":"2024-05-08T04:42:22.439397+0000","last_undegraded":"2024-05-08T04:42:22.439397+0000","last_fullsized":"2024-05-08T04:42:22.439397+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:26:26.692543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433659+0000","last_change":"2024-05-08T04:42:22.425948+0000","last_active":"2024-05-08T04:42:22.433659+0000","last_peered":"2024-05-08T04:42:22.433659+0000","last_clean":"2024-05-08T04:42:22.433659+0000","last_became_active":"2024-05-08T04:42:22.425703+0000","last_became_peered":"2024-05-08T04:42:22.425703+0000","last_unstale":"2024-05-08T04:42:22.433659+0000","last_undegraded":"2024-05-08T04:42:22.433659+0000","last_fullsized":"2024-05-08T04:42:22.433659+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:55.985348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432930+0000","last_change":"2024-05-08T04:42:22.424608+0000","last_active":"2024-05-08T04:42:22.432930+0000","last_peered":"2024-05-08T04:42:22.432930+0000","last_clean":"2024-05-08T04:42:22.432930+0000","last_became_active":"2024-05-08T04:42:22.424361+0000","last_became_peered":"2024-05-08T04:42:22.424361+0000","last_unstale":"2024-05-08T04:42:22.432930+0000","last_undegraded":"2024-05-08T04:42:22.432930+0000","last_fullsized":"2024-05-08T04:42:22.432930+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:45.127283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432172+0000","last_change":"2024-05-08T04:42:22.425848+0000","last_active":"2024-05-08T04:42:22.432172+0000","last_peered":"2024-05-08T04:42:22.432172+0000","last_clean":"2024-05-08T04:42:22.432172+0000","last_became_active":"2024-05-08T04:42:22.425420+0000","last_became_peered":"2024-05-08T04:42:22.425420+0000","last_unstale":"2024-05-08T04:42:22.432172+0000","last_undegraded":"2024-05-08T04:42:22.432172+0000","last_fullsized":"2024-05-08T04:42:22.432172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:02.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436490+0000","last_change":"2024-05-08T04:42:22.431988+0000","last_active":"2024-05-08T04:42:22.436490+0000","last_peered":"2024-05-08T04:42:22.436490+0000","last_clean":"2024-05-08T04:42:22.436490+0000","last_became_active":"2024-05-08T04:42:22.431689+0000","last_became_peered":"2024-05-08T04:42:22.431689+0000","last_unstale":"2024-05-08T04:42:22.436490+0000","last_undegraded":"2024-05-08T04:42:22.436490+0000","last_fullsized":"2024-05-08T04:42:22.436490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:04.398485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433567+0000","last_change":"2024-05-08T04:42:22.430501+0000","last_active":"2024-05-08T04:42:22.433567+0000","last_peered":"2024-05-08T04:42:22.433567+0000","last_clean":"2024-05-08T04:42:22.433567+0000","last_became_active":"2024-05-08T04:42:22.428921+0000","last_became_peered":"2024-05-08T04:42:22.428921+0000","last_unstale":"2024-05-08T04:42:22.433567+0000","last_undegraded":"2024-05-08T04:42:22.433567+0000","last_fullsized":"2024-05-08T04:42:22.433567+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:06.071977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442043+0000","last_change":"2024-05-08T04:42:22.440319+0000","last_active":"2024-05-08T04:42:22.442043+0000","last_peered":"2024-05-08T04:42:22.442043+0000","last_clean":"2024-05-08T04:42:22.442043+0000","last_became_active":"2024-05-08T04:42:22.439995+0000","last_became_peered":"2024-05-08T04:42:22.439995+0000","last_unstale":"2024-05-08T04:42:22.442043+0000","last_undegraded":"2024-05-08T04:42:22.442043+0000","last_fullsized":"2024-05-08T04:42:22.442043+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:47:45.657760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437444+0000","last_change":"2024-05-08T04:42:22.433207+0000","last_active":"2024-05-08T04:42:22.437444+0000","last_peered":"2024-05-08T04:42:22.437444+0000","last_clean":"2024-05-08T04:42:22.437444+0000","last_became_active":"2024-05-08T04:42:22.431605+0000","last_became_peered":"2024-05-08T04:42:22.431605+0000","last_unstale":"2024-05-08T04:42:22.437444+0000","last_undegraded":"2024-05-08T04:42:22.437444+0000","last_fullsized":"2024-05-08T04:42:22.437444+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:52.962208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440596+0000","last_change":"2024-05-08T04:42:22.433234+0000","last_active":"2024-05-08T04:42:22.440596+0000","last_peered":"2024-05-08T04:42:22.440596+0000","last_clean":"2024-05-08T04:42:22.440596+0000","last_became_active":"2024-05-08T04:42:22.430129+0000","last_became_peered":"2024-05-08T04:42:22.430129+0000","last_unstale":"2024-05-08T04:42:22.440596+0000","last_undegraded":"2024-05-08T04:42:22.440596+0000","last_fullsized":"2024-05-08T04:42:22.440596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:59.782392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439234+0000","last_change":"2024-05-08T04:42:22.426669+0000","last_active":"2024-05-08T04:42:22.439234+0000","last_peered":"2024-05-08T04:42:22.439234+0000","last_clean":"2024-05-08T04:42:22.439234+0000","last_became_active":"2024-05-08T04:42:22.426203+0000","last_became_peered":"2024-05-08T04:42:22.426203+0000","last_unstale":"2024-05-08T04:42:22.439234+0000","last_undegraded":"2024-05-08T04:42:22.439234+0000","last_fullsized":"2024-05-08T04:42:22.439234+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:47.369721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439542+0000","last_change":"2024-05-08T04:42:22.432677+0000","last_active":"2024-05-08T04:42:22.439542+0000","last_peered":"2024-05-08T04:42:22.439542+0000","last_clean":"2024-05-08T04:42:22.439542+0000","last_became_active":"2024-05-08T04:42:22.432416+0000","last_became_peered":"2024-05-08T04:42:22.432416+0000","last_unstale":"2024-05-08T04:42:22.439542+0000","last_undegraded":"2024-05-08T04:42:22.439542+0000","last_fullsized":"2024-05-08T04:42:22.439542+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:09.853850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444883+0000","last_change":"2024-05-08T04:42:22.433885+0000","last_active":"2024-05-08T04:42:22.444883+0000","last_peered":"2024-05-08T04:42:22.444883+0000","last_clean":"2024-05-08T04:42:22.444883+0000","last_became_active":"2024-05-08T04:42:22.433704+0000","last_became_peered":"2024-05-08T04:42:22.433704+0000","last_unstale":"2024-05-08T04:42:22.444883+0000","last_undegraded":"2024-05-08T04:42:22.444883+0000","last_fullsized":"2024-05-08T04:42:22.444883+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:33.880491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436647+0000","last_change":"2024-05-08T04:42:22.422838+0000","last_active":"2024-05-08T04:42:22.436647+0000","last_peered":"2024-05-08T04:42:22.436647+0000","last_clean":"2024-05-08T04:42:22.436647+0000","last_became_active":"2024-05-08T04:42:22.422501+0000","last_became_peered":"2024-05-08T04:42:22.422501+0000","last_unstale":"2024-05-08T04:42:22.436647+0000","last_undegraded":"2024-05-08T04:42:22.436647+0000","last_fullsized":"2024-05-08T04:42:22.436647+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:08.596734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444208+0000","last_change":"2024-05-08T04:42:21.422505+0000","last_active":"2024-05-08T04:42:22.444208+0000","last_peered":"2024-05-08T04:42:22.444208+0000","last_clean":"2024-05-08T04:42:22.444208+0000","last_became_active":"2024-05-08T04:42:21.422235+0000","last_became_peered":"2024-05-08T04:42:21.422235+0000","last_unstale":"2024-05-08T04:42:22.444208+0000","last_undegraded":"2024-05-08T04:42:22.444208+0000","last_fullsized":"2024-05-08T04:42:22.444208+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:52.111176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435493+0000","last_change":"2024-05-08T04:42:22.429472+0000","last_active":"2024-05-08T04:42:22.435493+0000","last_peered":"2024-05-08T04:42:22.435493+0000","last_clean":"2024-05-08T04:42:22.435493+0000","last_became_active":"2024-05-08T04:42:22.429176+0000","last_became_peered":"2024-05-08T04:42:22.429176+0000","last_unstale":"2024-05-08T04:42:22.435493+0000","last_undegraded":"2024-05-08T04:42:22.435493+0000","last_fullsized":"2024-05-08T04:42:22.435493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:03.712503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444438+0000","last_change":"2024-05-08T04:42:21.433044+0000","last_active":"2024-05-08T04:42:22.444438+0000","last_peered":"2024-05-08T04:42:22.444438+0000","last_clean":"2024-05-08T04:42:22.444438+0000","last_became_active":"2024-05-08T04:42:21.432656+0000","last_became_peered":"2024-05-08T04:42:21.432656+0000","last_unstale":"2024-05-08T04:42:22.444438+0000","last_undegraded":"2024-05-08T04:42:22.444438+0000","last_fullsized":"2024-05-08T04:42:22.444438+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.813583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442553+0000","last_change":"2024-05-08T04:42:22.435158+0000","last_active":"2024-05-08T04:42:22.442553+0000","last_peered":"2024-05-08T04:42:22.442553+0000","last_clean":"2024-05-08T04:42:22.442553+0000","last_became_active":"2024-05-08T04:42:22.434913+0000","last_became_peered":"2024-05-08T04:42:22.434913+0000","last_unstale":"2024-05-08T04:42:22.442553+0000","last_undegraded":"2024-05-08T04:42:22.442553+0000","last_fullsized":"2024-05-08T04:42:22.442553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:32.303328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.445221+0000","last_change":"2024-05-08T04:42:21.420132+0000","last_active":"2024-05-08T04:42:22.445221+0000","last_peered":"2024-05-08T04:42:22.445221+0000","last_clean":"2024-05-08T04:42:22.445221+0000","last_became_active":"2024-05-08T04:42:21.419850+0000","last_became_peered":"2024-05-08T04:42:21.419850+0000","last_unstale":"2024-05-08T04:42:22.445221+0000","last_undegraded":"2024-05-08T04:42:22.445221+0000","last_fullsized":"2024-05-08T04:42:22.445221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:14.306726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437646+0000","last_change":"2024-05-08T04:42:22.425388+0000","last_active":"2024-05-08T04:42:22.437646+0000","last_peered":"2024-05-08T04:42:22.437646+0000","last_clean":"2024-05-08T04:42:22.437646+0000","last_became_active":"2024-05-08T04:42:22.425071+0000","last_became_peered":"2024-05-08T04:42:22.425071+0000","last_unstale":"2024-05-08T04:42:22.437646+0000","last_undegraded":"2024-05-08T04:42:22.437646+0000","last_fullsized":"2024-05-08T04:42:22.437646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:04.398485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443598+0000","last_change":"2024-05-08T04:42:21.427287+0000","last_active":"2024-05-08T04:42:22.443598+0000","last_peered":"2024-05-08T04:42:22.443598+0000","last_clean":"2024-05-08T04:42:22.443598+0000","last_became_active":"2024-05-08T04:42:21.426944+0000","last_became_peered":"2024-05-08T04:42:21.426944+0000","last_unstale":"2024-05-08T04:42:22.443598+0000","last_undegraded":"2024-05-08T04:42:22.443598+0000","last_fullsized":"2024-05-08T04:42:22.443598+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:41.204120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432282+0000","last_change":"2024-05-08T04:42:22.427253+0000","last_active":"2024-05-08T04:42:22.432282+0000","last_peered":"2024-05-08T04:42:22.432282+0000","last_clean":"2024-05-08T04:42:22.432282+0000","last_became_active":"2024-05-08T04:42:22.422929+0000","last_became_peered":"2024-05-08T04:42:22.422929+0000","last_unstale":"2024-05-08T04:42:22.432282+0000","last_undegraded":"2024-05-08T04:42:22.432282+0000","last_fullsized":"2024-05-08T04:42:22.432282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:22.974009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443159+0000","last_change":"2024-05-08T04:42:21.430742+0000","last_active":"2024-05-08T04:42:22.443159+0000","last_peered":"2024-05-08T04:42:22.443159+0000","last_clean":"2024-05-08T04:42:22.443159+0000","last_became_active":"2024-05-08T04:42:21.430472+0000","last_became_peered":"2024-05-08T04:42:21.430472+0000","last_unstale":"2024-05-08T04:42:22.443159+0000","last_undegraded":"2024-05-08T04:42:22.443159+0000","last_fullsized":"2024-05-08T04:42:22.443159+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:02.459794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438211+0000","last_change":"2024-05-08T04:42:22.434417+0000","last_active":"2024-05-08T04:42:22.438211+0000","last_peered":"2024-05-08T04:42:22.438211+0000","last_clean":"2024-05-08T04:42:22.438211+0000","last_became_active":"2024-05-08T04:42:22.433458+0000","last_became_peered":"2024-05-08T04:42:22.433458+0000","last_unstale":"2024-05-08T04:42:22.438211+0000","last_undegraded":"2024-05-08T04:42:22.438211+0000","last_fullsized":"2024-05-08T04:42:22.438211+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:06.071977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.442739+0000","last_change":"2024-05-08T04:42:21.422965+0000","last_active":"2024-05-08T04:42:22.442739+0000","last_peered":"2024-05-08T04:42:22.442739+0000","last_clean":"2024-05-08T04:42:22.442739+0000","last_became_active":"2024-05-08T04:42:21.422643+0000","last_became_peered":"2024-05-08T04:42:21.422643+0000","last_unstale":"2024-05-08T04:42:22.442739+0000","last_undegraded":"2024-05-08T04:42:22.442739+0000","last_fullsized":"2024-05-08T04:42:22.442739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:49.998862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432944+0000","last_change":"2024-05-08T04:42:22.421216+0000","last_active":"2024-05-08T04:42:22.432944+0000","last_peered":"2024-05-08T04:42:22.432944+0000","last_clean":"2024-05-08T04:42:22.432944+0000","last_became_active":"2024-05-08T04:42:22.420767+0000","last_became_peered":"2024-05-08T04:42:22.420767+0000","last_unstale":"2024-05-08T04:42:22.432944+0000","last_undegraded":"2024-05-08T04:42:22.432944+0000","last_fullsized":"2024-05-08T04:42:22.432944+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:53.119288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.420522+0000","last_change":"2024-05-08T04:42:21.428589+0000","last_active":"2024-05-08T04:42:22.420522+0000","last_peered":"2024-05-08T04:42:22.420522+0000","last_clean":"2024-05-08T04:42:22.420522+0000","last_became_active":"2024-05-08T04:42:21.428180+0000","last_became_peered":"2024-05-08T04:42:21.428180+0000","last_unstale":"2024-05-08T04:42:22.420522+0000","last_undegraded":"2024-05-08T04:42:22.420522+0000","last_fullsized":"2024-05-08T04:42:22.420522+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:51.319334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.430898+0000","last_change":"2024-05-08T04:42:22.420870+0000","last_active":"2024-05-08T04:42:22.430898+0000","last_peered":"2024-05-08T04:42:22.430898+0000","last_clean":"2024-05-08T04:42:22.430898+0000","last_became_active":"2024-05-08T04:42:22.420582+0000","last_became_peered":"2024-05-08T04:42:22.420582+0000","last_unstale":"2024-05-08T04:42:22.430898+0000","last_undegraded":"2024-05-08T04:42:22.430898+0000","last_fullsized":"2024-05-08T04:42:22.430898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:39.812589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.417735+0000","last_change":"2024-05-08T04:42:21.432527+0000","last_active":"2024-05-08T04:42:22.417735+0000","last_peered":"2024-05-08T04:42:22.417735+0000","last_clean":"2024-05-08T04:42:22.417735+0000","last_became_active":"2024-05-08T04:42:21.431998+0000","last_became_peered":"2024-05-08T04:42:21.431998+0000","last_unstale":"2024-05-08T04:42:22.417735+0000","last_undegraded":"2024-05-08T04:42:22.417735+0000","last_fullsized":"2024-05-08T04:42:22.417735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:21:14.358754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433608+0000","last_change":"2024-05-08T04:42:22.431367+0000","last_active":"2024-05-08T04:42:22.433608+0000","last_peered":"2024-05-08T04:42:22.433608+0000","last_clean":"2024-05-08T04:42:22.433608+0000","last_became_active":"2024-05-08T04:42:22.431097+0000","last_became_peered":"2024-05-08T04:42:22.431097+0000","last_unstale":"2024-05-08T04:42:22.433608+0000","last_undegraded":"2024-05-08T04:42:22.433608+0000","last_fullsized":"2024-05-08T04:42:22.433608+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:21.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.419069+0000","last_change":"2024-05-08T04:42:21.424570+0000","last_active":"2024-05-08T04:42:22.419069+0000","last_peered":"2024-05-08T04:42:22.419069+0000","last_clean":"2024-05-08T04:42:22.419069+0000","last_became_active":"2024-05-08T04:42:21.423459+0000","last_became_peered":"2024-05-08T04:42:21.423459+0000","last_unstale":"2024-05-08T04:42:22.419069+0000","last_undegraded":"2024-05-08T04:42:22.419069+0000","last_fullsized":"2024-05-08T04:42:22.419069+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:38.488564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.432020+0000","last_change":"2024-05-08T04:42:22.421880+0000","last_active":"2024-05-08T04:42:22.432020+0000","last_peered":"2024-05-08T04:42:22.432020+0000","last_clean":"2024-05-08T04:42:22.432020+0000","last_became_active":"2024-05-08T04:42:22.416477+0000","last_became_peered":"2024-05-08T04:42:22.416477+0000","last_unstale":"2024-05-08T04:42:22.432020+0000","last_undegraded":"2024-05-08T04:42:22.432020+0000","last_fullsized":"2024-05-08T04:42:22.432020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:46.298060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.408912+0000","last_change":"2024-05-08T04:42:21.425895+0000","last_active":"2024-05-08T04:42:22.408912+0000","last_peered":"2024-05-08T04:42:22.408912+0000","last_clean":"2024-05-08T04:42:22.408912+0000","last_became_active":"2024-05-08T04:42:21.424720+0000","last_became_peered":"2024-05-08T04:42:21.424720+0000","last_unstale":"2024-05-08T04:42:22.408912+0000","last_undegraded":"2024-05-08T04:42:22.408912+0000","last_fullsized":"2024-05-08T04:42:22.408912+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:51.635883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438582+0000","last_change":"2024-05-08T04:42:22.435868+0000","last_active":"2024-05-08T04:42:22.438582+0000","last_peered":"2024-05-08T04:42:22.438582+0000","last_clean":"2024-05-08T04:42:22.438582+0000","last_became_active":"2024-05-08T04:42:22.434819+0000","last_became_peered":"2024-05-08T04:42:22.434819+0000","last_unstale":"2024-05-08T04:42:22.438582+0000","last_undegraded":"2024-05-08T04:42:22.438582+0000","last_fullsized":"2024-05-08T04:42:22.438582+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:38.522734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.445289+0000","last_change":"2024-05-08T04:42:21.429376+0000","last_active":"2024-05-08T04:42:22.445289+0000","last_peered":"2024-05-08T04:42:22.445289+0000","last_clean":"2024-05-08T04:42:22.445289+0000","last_became_active":"2024-05-08T04:42:21.429079+0000","last_became_peered":"2024-05-08T04:42:21.429079+0000","last_unstale":"2024-05-08T04:42:22.445289+0000","last_undegraded":"2024-05-08T04:42:22.445289+0000","last_fullsized":"2024-05-08T04:42:22.445289+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:02.704392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438266+0000","last_change":"2024-05-08T04:42:22.433979+0000","last_active":"2024-05-08T04:42:22.438266+0000","last_peered":"2024-05-08T04:42:22.438266+0000","last_clean":"2024-05-08T04:42:22.438266+0000","last_became_active":"2024-05-08T04:42:22.433401+0000","last_became_peered":"2024-05-08T04:42:22.433401+0000","last_unstale":"2024-05-08T04:42:22.438266+0000","last_undegraded":"2024-05-08T04:42:22.438266+0000","last_fullsized":"2024-05-08T04:42:22.438266+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:55.985348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.451002+0000","last_change":"2024-05-08T04:42:21.427495+0000","last_active":"2024-05-08T04:42:22.451002+0000","last_peered":"2024-05-08T04:42:22.451002+0000","last_clean":"2024-05-08T04:42:22.451002+0000","last_became_active":"2024-05-08T04:42:21.426599+0000","last_became_peered":"2024-05-08T04:42:21.426599+0000","last_unstale":"2024-05-08T04:42:22.451002+0000","last_undegraded":"2024-05-08T04:42:22.451002+0000","last_fullsized":"2024-05-08T04:42:22.451002+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:51.319334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439459+0000","last_change":"2024-05-08T04:42:22.432112+0000","last_active":"2024-05-08T04:42:22.439459+0000","last_peered":"2024-05-08T04:42:22.439459+0000","last_clean":"2024-05-08T04:42:22.439459+0000","last_became_active":"2024-05-08T04:42:22.431864+0000","last_became_peered":"2024-05-08T04:42:22.431864+0000","last_unstale":"2024-05-08T04:42:22.439459+0000","last_undegraded":"2024-05-08T04:42:22.439459+0000","last_fullsized":"2024-05-08T04:42:22.439459+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:02.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.444443+0000","last_change":"2024-05-08T04:42:21.429978+0000","last_active":"2024-05-08T04:42:22.444443+0000","last_peered":"2024-05-08T04:42:22.444443+0000","last_clean":"2024-05-08T04:42:22.444443+0000","last_became_active":"2024-05-08T04:42:21.428710+0000","last_became_peered":"2024-05-08T04:42:21.428710+0000","last_unstale":"2024-05-08T04:42:22.444443+0000","last_undegraded":"2024-05-08T04:42:22.444443+0000","last_fullsized":"2024-05-08T04:42:22.444443+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:10.851787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427126+0000","last_change":"2024-05-08T04:42:22.425336+0000","last_active":"2024-05-08T04:42:22.427126+0000","last_peered":"2024-05-08T04:42:22.427126+0000","last_clean":"2024-05-08T04:42:22.427126+0000","last_became_active":"2024-05-08T04:42:22.421634+0000","last_became_peered":"2024-05-08T04:42:22.421634+0000","last_unstale":"2024-05-08T04:42:22.427126+0000","last_undegraded":"2024-05-08T04:42:22.427126+0000","last_fullsized":"2024-05-08T04:42:22.427126+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:55.520501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412689+0000","last_change":"2024-05-08T04:42:21.431370+0000","last_active":"2024-05-08T04:42:22.412689+0000","last_peered":"2024-05-08T04:42:22.412689+0000","last_clean":"2024-05-08T04:42:22.412689+0000","last_became_active":"2024-05-08T04:42:21.430832+0000","last_became_peered":"2024-05-08T04:42:21.430832+0000","last_unstale":"2024-05-08T04:42:22.412689+0000","last_undegraded":"2024-05-08T04:42:22.412689+0000","last_fullsized":"2024-05-08T04:42:22.412689+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:07.928333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.426873+0000","last_change":"2024-05-08T04:42:22.419377+0000","last_active":"2024-05-08T04:42:22.426873+0000","last_peered":"2024-05-08T04:42:22.426873+0000","last_clean":"2024-05-08T04:42:22.426873+0000","last_became_active":"2024-05-08T04:42:22.415628+0000","last_became_peered":"2024-05-08T04:42:22.415628+0000","last_unstale":"2024-05-08T04:42:22.426873+0000","last_undegraded":"2024-05-08T04:42:22.426873+0000","last_fullsized":"2024-05-08T04:42:22.426873+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:12.495736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.413928+0000","last_change":"2024-05-08T04:42:21.424072+0000","last_active":"2024-05-08T04:42:22.413928+0000","last_peered":"2024-05-08T04:42:22.413928+0000","last_clean":"2024-05-08T04:42:22.413928+0000","last_became_active":"2024-05-08T04:42:21.423101+0000","last_became_peered":"2024-05-08T04:42:21.423101+0000","last_unstale":"2024-05-08T04:42:22.413928+0000","last_undegraded":"2024-05-08T04:42:22.413928+0000","last_fullsized":"2024-05-08T04:42:22.413928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:35.699449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436785+0000","last_change":"2024-05-08T04:42:22.423948+0000","last_active":"2024-05-08T04:42:22.436785+0000","last_peered":"2024-05-08T04:42:22.436785+0000","last_clean":"2024-05-08T04:42:22.436785+0000","last_became_active":"2024-05-08T04:42:22.423689+0000","last_became_peered":"2024-05-08T04:42:22.423689+0000","last_unstale":"2024-05-08T04:42:22.436785+0000","last_undegraded":"2024-05-08T04:42:22.436785+0000","last_fullsized":"2024-05-08T04:42:22.436785+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:37.674122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.449422+0000","last_change":"2024-05-08T04:42:21.429462+0000","last_active":"2024-05-08T04:42:22.449422+0000","last_peered":"2024-05-08T04:42:22.449422+0000","last_clean":"2024-05-08T04:42:22.449422+0000","last_became_active":"2024-05-08T04:42:21.429171+0000","last_became_peered":"2024-05-08T04:42:21.429171+0000","last_unstale":"2024-05-08T04:42:22.449422+0000","last_undegraded":"2024-05-08T04:42:22.449422+0000","last_fullsized":"2024-05-08T04:42:22.449422+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:45.462102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437399+0000","last_change":"2024-05-08T04:42:22.435406+0000","last_active":"2024-05-08T04:42:22.437399+0000","last_peered":"2024-05-08T04:42:22.437399+0000","last_clean":"2024-05-08T04:42:22.437399+0000","last_became_active":"2024-05-08T04:42:22.434844+0000","last_became_peered":"2024-05-08T04:42:22.434844+0000","last_unstale":"2024-05-08T04:42:22.437399+0000","last_undegraded":"2024-05-08T04:42:22.437399+0000","last_fullsized":"2024-05-08T04:42:22.437399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:43.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.449482+0000","last_change":"2024-05-08T04:42:21.431049+0000","last_active":"2024-05-08T04:42:22.449482+0000","last_peered":"2024-05-08T04:42:22.449482+0000","last_clean":"2024-05-08T04:42:22.449482+0000","last_became_active":"2024-05-08T04:42:21.430688+0000","last_became_peered":"2024-05-08T04:42:21.430688+0000","last_unstale":"2024-05-08T04:42:22.449482+0000","last_undegraded":"2024-05-08T04:42:22.449482+0000","last_fullsized":"2024-05-08T04:42:22.449482+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:46.560969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.434662+0000","last_change":"2024-05-08T04:42:22.429405+0000","last_active":"2024-05-08T04:42:22.434662+0000","last_peered":"2024-05-08T04:42:22.434662+0000","last_clean":"2024-05-08T04:42:22.434662+0000","last_became_active":"2024-05-08T04:42:22.428503+0000","last_became_peered":"2024-05-08T04:42:22.428503+0000","last_unstale":"2024-05-08T04:42:22.434662+0000","last_undegraded":"2024-05-08T04:42:22.434662+0000","last_fullsized":"2024-05-08T04:42:22.434662+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:40.749846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440948+0000","last_change":"2024-05-08T04:42:21.422899+0000","last_active":"2024-05-08T04:42:22.440948+0000","last_peered":"2024-05-08T04:42:22.440948+0000","last_clean":"2024-05-08T04:42:22.440948+0000","last_became_active":"2024-05-08T04:42:21.421585+0000","last_became_peered":"2024-05-08T04:42:21.421585+0000","last_unstale":"2024-05-08T04:42:22.440948+0000","last_undegraded":"2024-05-08T04:42:22.440948+0000","last_fullsized":"2024-05-08T04:42:22.440948+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:38.804477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.437836+0000","last_change":"2024-05-08T04:42:22.432067+0000","last_active":"2024-05-08T04:42:22.437836+0000","last_peered":"2024-05-08T04:42:22.437836+0000","last_clean":"2024-05-08T04:42:22.437836+0000","last_became_active":"2024-05-08T04:42:22.431529+0000","last_became_peered":"2024-05-08T04:42:22.431529+0000","last_unstale":"2024-05-08T04:42:22.437836+0000","last_undegraded":"2024-05-08T04:42:22.437836+0000","last_fullsized":"2024-05-08T04:42:22.437836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:45.127283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412964+0000","last_change":"2024-05-08T04:42:21.420430+0000","last_active":"2024-05-08T04:42:22.412964+0000","last_peered":"2024-05-08T04:42:22.412964+0000","last_clean":"2024-05-08T04:42:22.412964+0000","last_became_active":"2024-05-08T04:42:21.420165+0000","last_became_peered":"2024-05-08T04:42:21.420165+0000","last_unstale":"2024-05-08T04:42:22.412964+0000","last_undegraded":"2024-05-08T04:42:22.412964+0000","last_fullsized":"2024-05-08T04:42:22.412964+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:51.635883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436222+0000","last_change":"2024-05-08T04:42:22.419038+0000","last_active":"2024-05-08T04:42:22.436222+0000","last_peered":"2024-05-08T04:42:22.436222+0000","last_clean":"2024-05-08T04:42:22.436222+0000","last_became_active":"2024-05-08T04:42:22.418802+0000","last_became_peered":"2024-05-08T04:42:22.418802+0000","last_unstale":"2024-05-08T04:42:22.436222+0000","last_undegraded":"2024-05-08T04:42:22.436222+0000","last_fullsized":"2024-05-08T04:42:22.436222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:44:45.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.446558+0000","last_change":"2024-05-08T04:42:21.426859+0000","last_active":"2024-05-08T04:42:22.446558+0000","last_peered":"2024-05-08T04:42:22.446558+0000","last_clean":"2024-05-08T04:42:22.446558+0000","last_became_active":"2024-05-08T04:42:21.426539+0000","last_became_peered":"2024-05-08T04:42:21.426539+0000","last_unstale":"2024-05-08T04:42:22.446558+0000","last_undegraded":"2024-05-08T04:42:22.446558+0000","last_fullsized":"2024-05-08T04:42:22.446558+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:35.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439839+0000","last_change":"2024-05-08T04:42:22.428001+0000","last_active":"2024-05-08T04:42:22.439839+0000","last_peered":"2024-05-08T04:42:22.439839+0000","last_clean":"2024-05-08T04:42:22.439839+0000","last_became_active":"2024-05-08T04:42:22.427743+0000","last_became_peered":"2024-05-08T04:42:22.427743+0000","last_unstale":"2024-05-08T04:42:22.439839+0000","last_undegraded":"2024-05-08T04:42:22.439839+0000","last_fullsized":"2024-05-08T04:42:22.439839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:21.324641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.411137+0000","last_change":"2024-05-08T04:42:21.427576+0000","last_active":"2024-05-08T04:42:22.411137+0000","last_peered":"2024-05-08T04:42:22.411137+0000","last_clean":"2024-05-08T04:42:22.411137+0000","last_became_active":"2024-05-08T04:42:21.427318+0000","last_became_peered":"2024-05-08T04:42:21.427318+0000","last_unstale":"2024-05-08T04:42:22.411137+0000","last_undegraded":"2024-05-08T04:42:22.411137+0000","last_fullsized":"2024-05-08T04:42:22.411137+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:01.565043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.422980+0000","last_change":"2024-05-08T04:42:22.418512+0000","last_active":"2024-05-08T04:42:22.422980+0000","last_peered":"2024-05-08T04:42:22.422980+0000","last_clean":"2024-05-08T04:42:22.422980+0000","last_became_active":"2024-05-08T04:42:22.418221+0000","last_became_peered":"2024-05-08T04:42:22.418221+0000","last_unstale":"2024-05-08T04:42:22.422980+0000","last_undegraded":"2024-05-08T04:42:22.422980+0000","last_fullsized":"2024-05-08T04:42:22.422980+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:00:14.413385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.409879+0000","last_change":"2024-05-08T04:42:21.426559+0000","last_active":"2024-05-08T04:42:22.409879+0000","last_peered":"2024-05-08T04:42:22.409879+0000","last_clean":"2024-05-08T04:42:22.409879+0000","last_became_active":"2024-05-08T04:42:21.426238+0000","last_became_peered":"2024-05-08T04:42:21.426238+0000","last_unstale":"2024-05-08T04:42:22.409879+0000","last_undegraded":"2024-05-08T04:42:22.409879+0000","last_fullsized":"2024-05-08T04:42:22.409879+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:41.187717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.430467+0000","last_change":"2024-05-08T04:42:22.429090+0000","last_active":"2024-05-08T04:42:22.430467+0000","last_peered":"2024-05-08T04:42:22.430467+0000","last_clean":"2024-05-08T04:42:22.430467+0000","last_became_active":"2024-05-08T04:42:22.428735+0000","last_became_peered":"2024-05-08T04:42:22.428735+0000","last_unstale":"2024-05-08T04:42:22.430467+0000","last_undegraded":"2024-05-08T04:42:22.430467+0000","last_fullsized":"2024-05-08T04:42:22.430467+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:37.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414920+0000","last_change":"2024-05-08T04:42:21.430003+0000","last_active":"2024-05-08T04:42:22.414920+0000","last_peered":"2024-05-08T04:42:22.414920+0000","last_clean":"2024-05-08T04:42:22.414920+0000","last_became_active":"2024-05-08T04:42:21.429569+0000","last_became_peered":"2024-05-08T04:42:21.429569+0000","last_unstale":"2024-05-08T04:42:22.414920+0000","last_undegraded":"2024-05-08T04:42:22.414920+0000","last_fullsized":"2024-05-08T04:42:22.414920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:01.168478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.433545+0000","last_change":"2024-05-08T04:42:22.429487+0000","last_active":"2024-05-08T04:42:22.433545+0000","last_peered":"2024-05-08T04:42:22.433545+0000","last_clean":"2024-05-08T04:42:22.433545+0000","last_became_active":"2024-05-08T04:42:22.428814+0000","last_became_peered":"2024-05-08T04:42:22.428814+0000","last_unstale":"2024-05-08T04:42:22.433545+0000","last_undegraded":"2024-05-08T04:42:22.433545+0000","last_fullsized":"2024-05-08T04:42:22.433545+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:26.304165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.443491+0000","last_change":"2024-05-08T04:42:21.422300+0000","last_active":"2024-05-08T04:42:22.443491+0000","last_peered":"2024-05-08T04:42:22.443491+0000","last_clean":"2024-05-08T04:42:22.443491+0000","last_became_active":"2024-05-08T04:42:21.421987+0000","last_became_peered":"2024-05-08T04:42:21.421987+0000","last_unstale":"2024-05-08T04:42:22.443491+0000","last_undegraded":"2024-05-08T04:42:22.443491+0000","last_fullsized":"2024-05-08T04:42:22.443491+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:38:30.474789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.427458+0000","last_change":"2024-05-08T04:42:22.422115+0000","last_active":"2024-05-08T04:42:22.427458+0000","last_peered":"2024-05-08T04:42:22.427458+0000","last_clean":"2024-05-08T04:42:22.427458+0000","last_became_active":"2024-05-08T04:42:22.421852+0000","last_became_peered":"2024-05-08T04:42:22.421852+0000","last_unstale":"2024-05-08T04:42:22.427458+0000","last_undegraded":"2024-05-08T04:42:22.427458+0000","last_fullsized":"2024-05-08T04:42:22.427458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:36.788761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.416633+0000","last_change":"2024-05-08T04:42:21.427629+0000","last_active":"2024-05-08T04:42:22.416633+0000","last_peered":"2024-05-08T04:42:22.416633+0000","last_clean":"2024-05-08T04:42:22.416633+0000","last_became_active":"2024-05-08T04:42:21.425992+0000","last_became_peered":"2024-05-08T04:42:21.425992+0000","last_unstale":"2024-05-08T04:42:22.416633+0000","last_undegraded":"2024-05-08T04:42:22.416633+0000","last_fullsized":"2024-05-08T04:42:22.416633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:08.845738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.435775+0000","last_change":"2024-05-08T04:42:22.429801+0000","last_active":"2024-05-08T04:42:22.435775+0000","last_peered":"2024-05-08T04:42:22.435775+0000","last_clean":"2024-05-08T04:42:22.435775+0000","last_became_active":"2024-05-08T04:42:22.429547+0000","last_became_peered":"2024-05-08T04:42:22.429547+0000","last_unstale":"2024-05-08T04:42:22.435775+0000","last_undegraded":"2024-05-08T04:42:22.435775+0000","last_fullsized":"2024-05-08T04:42:22.435775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:30.870407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414418+0000","last_change":"2024-05-08T04:42:21.416010+0000","last_active":"2024-05-08T04:42:22.414418+0000","last_peered":"2024-05-08T04:42:22.414418+0000","last_clean":"2024-05-08T04:42:22.414418+0000","last_became_active":"2024-05-08T04:42:21.415739+0000","last_became_peered":"2024-05-08T04:42:21.415739+0000","last_unstale":"2024-05-08T04:42:22.414418+0000","last_undegraded":"2024-05-08T04:42:22.414418+0000","last_fullsized":"2024-05-08T04:42:22.414418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:31.813583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439676+0000","last_change":"2024-05-08T04:42:22.437183+0000","last_active":"2024-05-08T04:42:22.439676+0000","last_peered":"2024-05-08T04:42:22.439676+0000","last_clean":"2024-05-08T04:42:22.439676+0000","last_became_active":"2024-05-08T04:42:22.436635+0000","last_became_peered":"2024-05-08T04:42:22.436635+0000","last_unstale":"2024-05-08T04:42:22.439676+0000","last_undegraded":"2024-05-08T04:42:22.439676+0000","last_fullsized":"2024-05-08T04:42:22.439676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:52.643995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.423714+0000","last_change":"2024-05-08T04:42:21.427630+0000","last_active":"2024-05-08T04:42:22.423714+0000","last_peered":"2024-05-08T04:42:22.423714+0000","last_clean":"2024-05-08T04:42:22.423714+0000","last_became_active":"2024-05-08T04:42:21.427353+0000","last_became_peered":"2024-05-08T04:42:21.427353+0000","last_unstale":"2024-05-08T04:42:22.423714+0000","last_undegraded":"2024-05-08T04:42:22.423714+0000","last_fullsized":"2024-05-08T04:42:22.423714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:45.289949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439022+0000","last_change":"2024-05-08T04:42:22.426475+0000","last_active":"2024-05-08T04:42:22.439022+0000","last_peered":"2024-05-08T04:42:22.439022+0000","last_clean":"2024-05-08T04:42:22.439022+0000","last_became_active":"2024-05-08T04:42:22.425853+0000","last_became_peered":"2024-05-08T04:42:22.425853+0000","last_unstale":"2024-05-08T04:42:22.439022+0000","last_undegraded":"2024-05-08T04:42:22.439022+0000","last_fullsized":"2024-05-08T04:42:22.439022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:59.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.414751+0000","last_change":"2024-05-08T04:42:21.423787+0000","last_active":"2024-05-08T04:42:22.414751+0000","last_peered":"2024-05-08T04:42:22.414751+0000","last_clean":"2024-05-08T04:42:22.414751+0000","last_became_active":"2024-05-08T04:42:21.421769+0000","last_became_peered":"2024-05-08T04:42:21.421769+0000","last_unstale":"2024-05-08T04:42:22.414751+0000","last_undegraded":"2024-05-08T04:42:22.414751+0000","last_fullsized":"2024-05-08T04:42:22.414751+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:02.704392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.438251+0000","last_change":"2024-05-08T04:42:22.430158+0000","last_active":"2024-05-08T04:42:22.438251+0000","last_peered":"2024-05-08T04:42:22.438251+0000","last_clean":"2024-05-08T04:42:22.438251+0000","last_became_active":"2024-05-08T04:42:22.429555+0000","last_became_peered":"2024-05-08T04:42:22.429555+0000","last_unstale":"2024-05-08T04:42:22.438251+0000","last_undegraded":"2024-05-08T04:42:22.438251+0000","last_fullsized":"2024-05-08T04:42:22.438251+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:43.169839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.421006+0000","last_change":"2024-05-08T04:42:21.432634+0000","last_active":"2024-05-08T04:42:22.421006+0000","last_peered":"2024-05-08T04:42:22.421006+0000","last_clean":"2024-05-08T04:42:22.421006+0000","last_became_active":"2024-05-08T04:42:21.432330+0000","last_became_peered":"2024-05-08T04:42:21.432330+0000","last_unstale":"2024-05-08T04:42:22.421006+0000","last_undegraded":"2024-05-08T04:42:22.421006+0000","last_fullsized":"2024-05-08T04:42:22.421006+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:11.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.436794+0000","last_change":"2024-05-08T04:42:22.423878+0000","last_active":"2024-05-08T04:42:22.436794+0000","last_peered":"2024-05-08T04:42:22.436794+0000","last_clean":"2024-05-08T04:42:22.436794+0000","last_became_active":"2024-05-08T04:42:22.423598+0000","last_became_peered":"2024-05-08T04:42:22.423598+0000","last_unstale":"2024-05-08T04:42:22.436794+0000","last_undegraded":"2024-05-08T04:42:22.436794+0000","last_fullsized":"2024-05-08T04:42:22.436794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:21.158355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.408583+0000","last_change":"2024-05-08T04:42:21.431224+0000","last_active":"2024-05-08T04:42:22.408583+0000","last_peered":"2024-05-08T04:42:22.408583+0000","last_clean":"2024-05-08T04:42:22.408583+0000","last_became_active":"2024-05-08T04:42:21.430082+0000","last_became_peered":"2024-05-08T04:42:21.430082+0000","last_unstale":"2024-05-08T04:42:22.408583+0000","last_undegraded":"2024-05-08T04:42:22.408583+0000","last_fullsized":"2024-05-08T04:42:22.408583+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:41.204120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.440387+0000","last_change":"2024-05-08T04:42:22.438057+0000","last_active":"2024-05-08T04:42:22.440387+0000","last_peered":"2024-05-08T04:42:22.440387+0000","last_clean":"2024-05-08T04:42:22.440387+0000","last_became_active":"2024-05-08T04:42:22.437775+0000","last_became_peered":"2024-05-08T04:42:22.437775+0000","last_unstale":"2024-05-08T04:42:22.440387+0000","last_undegraded":"2024-05-08T04:42:22.440387+0000","last_fullsized":"2024-05-08T04:42:22.440387+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:51.006973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.412150+0000","last_change":"2024-05-08T04:42:21.419876+0000","last_active":"2024-05-08T04:42:22.412150+0000","last_peered":"2024-05-08T04:42:22.412150+0000","last_clean":"2024-05-08T04:42:22.412150+0000","last_became_active":"2024-05-08T04:42:21.419601+0000","last_became_peered":"2024-05-08T04:42:21.419601+0000","last_unstale":"2024-05-08T04:42:22.412150+0000","last_undegraded":"2024-05-08T04:42:22.412150+0000","last_fullsized":"2024-05-08T04:42:22.412150+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:46.114943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.439894+0000","last_change":"2024-05-08T04:42:22.427411+0000","last_active":"2024-05-08T04:42:22.439894+0000","last_peered":"2024-05-08T04:42:22.439894+0000","last_clean":"2024-05-08T04:42:22.439894+0000","last_became_active":"2024-05-08T04:42:22.427112+0000","last_became_peered":"2024-05-08T04:42:22.427112+0000","last_unstale":"2024-05-08T04:42:22.439894+0000","last_undegraded":"2024-05-08T04:42:22.439894+0000","last_fullsized":"2024-05-08T04:42:22.439894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:21.396251+0000","last_clean_scrub_stamp":"2024-05-08T04:42:21.396251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:19:28.215486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.446468+0000","last_change":"2024-05-08T04:42:21.431821+0000","last_active":"2024-05-08T04:42:22.446468+0000","last_peered":"2024-05-08T04:42:22.446468+0000","last_clean":"2024-05-08T04:42:22.446468+0000","last_became_active":"2024-05-08T04:42:21.431401+0000","last_became_peered":"2024-05-08T04:42:21.431401+0000","last_unstale":"2024-05-08T04:42:22.446468+0000","last_undegraded":"2024-05-08T04:42:22.446468+0000","last_fullsized":"2024-05-08T04:42:22.446468+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:20.388140+0000","last_clean_scrub_stamp":"2024-05-08T04:42:20.388140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:35.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":317,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-08T04:42:22.419735+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:22.419735+0000","last_peered":"2024-05-08T04:42:22.419735+0000","last_clean":"2024-05-08T04:42:22.419735+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:22.419735+0000","last_undegraded":"2024-05-08T04:42:22.419735+0000","last_fullsized":"2024-05-08T04:42:22.419735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1107922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":524413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":524447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1200128,"data_stored":517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":1107922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:42:26.876 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:42:26.876 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:42:27.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 Processor -- start 2024-05-08T04:42:27.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 -- start start 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd6f015ba20 0x7fd6f0160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6f006de50 0x7fd6f006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00c6600 con 0x7fd6f00ae6e0 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f015fbe0 con 0x7fd6f015ba20 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f7c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f00c6040 con 0x7fd6f006de50 2024-05-08T04:42:27.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd6f015ba20 0x7fd6f0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd6f00ae6e0 0x7fd6f00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51900/0 (socket says 172.21.15.164:51900) 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5c00640 1 -- 172.21.15.164:0/343496282 learned_addr learned my addr 172.21.15.164:0/343496282 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6efe00640 1 --2- 172.21.15.164:0/343496282 >> v2:172.21.15.172:3300/0 conn(0x7fd6f006de50 0x7fd6f006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5200640 1 -- 172.21.15.164:0/343496282 >> v2:172.21.15.172:3300/0 conn(0x7fd6f006de50 msgr2=0x7fd6f006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5200640 1 --2- 172.21.15.164:0/343496282 >> v2:172.21.15.172:3300/0 conn(0x7fd6f006de50 0x7fd6f006e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5200640 1 -- 172.21.15.164:0/343496282 >> v1:172.21.15.164:6789/0 conn(0x7fd6f00ae6e0 legacy=0x7fd6f00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6f5200640 1 -- 172.21.15.164:0/343496282 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00c6320 con 0x7fd6f015ba20 2024-05-08T04:42:27.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.052+0000 7fd6ef400640 1 -- 172.21.15.164:0/343496282 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940689409 0 0) 0x7fd6f00c6600 con 0x7fd6f00ae6e0 2024-05-08T04:42:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f5200640 1 --2- 172.21.15.164:0/343496282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f015ba20 0x7fd6f0160070 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b5980 tx=0x7fd6e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fcf194d6bf87d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6ef400640 1 -- 172.21.15.164:0/343496282 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ab5f0 con 0x7fd6f015ba20 2024-05-08T04:42:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6ef400640 1 -- 172.21.15.164:0/343496282 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00b0070 con 0x7fd6f015ba20 2024-05-08T04:42:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6ef400640 1 -- 172.21.15.164:0/343496282 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ab9b0 con 0x7fd6f015ba20 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 -- 172.21.15.164:0/343496282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f015ba20 msgr2=0x7fd6f0160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/343496282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f015ba20 0x7fd6f0160070 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b5980 tx=0x7fd6e001c450 comp rx=0 tx=0).stop 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 -- 172.21.15.164:0/343496282 shutdown_connections 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/343496282 >> v2:172.21.15.172:3300/0 conn(0x7fd6f006de50 0x7fd6f006e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/343496282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f015ba20 0x7fd6f0160070 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 -- 172.21.15.164:0/343496282 >> 172.21.15.164:0/343496282 conn(0x7fd6f00cf770 msgr2=0x7fd6f00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:27.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 -- 172.21.15.164:0/343496282 shutdown_connections 2024-05-08T04:42:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.056+0000 7fd6f7c00640 1 -- 172.21.15.164:0/343496282 wait complete. 2024-05-08T04:42:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 Processor -- start 2024-05-08T04:42:27.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- start start 2024-05-08T04:42:27.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f00ae6e0 0x7fd6f01438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 0x7fd6f00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00aec60 con 0x7fd6f006de50 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00aeae0 con 0x7fd6f015ba20 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00aede0 con 0x7fd6f00ae6e0 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44882/0 (socket says 172.21.15.164:44882) 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 -- 172.21.15.164:0/2203077728 learned_addr learned my addr 172.21.15.164:0/2203077728 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6efe00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f00ae6e0 0x7fd6f01438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 0x7fd6f00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 -- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f00ae6e0 msgr2=0x7fd6f01438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f00ae6e0 0x7fd6f01438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 -- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 msgr2=0x7fd6f00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 0x7fd6f00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 -- 172.21.15.164:0/2203077728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f014ef20 con 0x7fd6f006de50 2024-05-08T04:42:27.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 0x7fd6f00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:27.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f5200640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0019060 tx=0x7fd6e001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00abe60 con 0x7fd6f006de50 2024-05-08T04:42:27.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00bf610 con 0x7fd6f006de50 2024-05-08T04:42:27.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00bf9e0 con 0x7fd6f006de50 2024-05-08T04:42:27.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00c8fd0 con 0x7fd6f006de50 2024-05-08T04:42:27.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f00c9590 con 0x7fd6f006de50 2024-05-08T04:42:27.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.060+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0072d30 con 0x7fd6f006de50 2024-05-08T04:42:27.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.064+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fd6e00bd020 con 0x7fd6f006de50 2024-05-08T04:42:27.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.064+0000 7fd6ecc00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 0x7fd6d0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.064+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd6e011fb60 con 0x7fd6f006de50 2024-05-08T04:42:27.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.064+0000 7fd6efe00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 0x7fd6d0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.064+0000 7fd6efe00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 0x7fd6d0069b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00c8c40 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.068+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6e00c6070 con 0x7fd6f006de50 2024-05-08T04:42:27.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.276+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6f00cea50 con 0x7fd6f006de50 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6ecc00640 1 -- 172.21.15.164:0/2203077728 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd6e0016070 con 0x7fd6f006de50 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:42:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:42:27.285 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:42:27.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 msgr2=0x7fd6d0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 0x7fd6d0069b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00c8c40 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 msgr2=0x7fd6f014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0019060 tx=0x7fd6e001ac90 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 shutdown_connections 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd6d00676b0 0x7fd6d0069b70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f00ae6e0 0x7fd6f01438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6f015ba20 0x7fd6f00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 --2- 172.21.15.164:0/2203077728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6f006de50 0x7fd6f014e9e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 >> 172.21.15.164:0/2203077728 conn(0x7fd6f00cf770 msgr2=0x7fd6f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:27.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 shutdown_connections 2024-05-08T04:42:27.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.280+0000 7fd6f7c00640 1 -- 172.21.15.164:0/2203077728 wait complete. 2024-05-08T04:42:27.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:27.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.436+0000 7fa055e00640 1 Processor -- start 2024-05-08T04:42:27.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.436+0000 7fa055e00640 1 -- start start 2024-05-08T04:42:27.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa055e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 0x7fa0500b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa055e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0500d0780 0x7fa05015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa055e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0500b42a0 con 0x7fa0500ccfe0 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa055e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0500cd4e0 con 0x7fa0500d0780 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa055e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500cd7c0 con 0x7fa05015bc50 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa0500ccfe0 0x7fa0500d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:51912/0 (socket says 172.21.15.164:51912) 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04e000640 1 -- 172.21.15.164:0/706559339 learned_addr learned my addr 172.21.15.164:0/706559339 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04ea00640 1 --2- 172.21.15.164:0/706559339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 0x7fa0500b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04f400640 1 --2- 172.21.15.164:0/706559339 >> v2:172.21.15.172:3300/0 conn(0x7fa0500d0780 0x7fa05015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04d600640 1 -- 172.21.15.164:0/706559339 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113636152 0 0) 0x7fa0500b42a0 con 0x7fa0500ccfe0 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04d600640 1 -- 172.21.15.164:0/706559339 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa034003530 con 0x7fa0500ccfe0 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04ea00640 1 -- 172.21.15.164:0/706559339 >> v2:172.21.15.172:3300/0 conn(0x7fa0500d0780 msgr2=0x7fa05015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04ea00640 1 --2- 172.21.15.164:0/706559339 >> v2:172.21.15.172:3300/0 conn(0x7fa0500d0780 0x7fa05015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04ea00640 1 -- 172.21.15.164:0/706559339 >> v1:172.21.15.164:6789/0 conn(0x7fa0500ccfe0 legacy=0x7fa0500d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.440+0000 7fa04ea00640 1 -- 172.21.15.164:0/706559339 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500cf590 con 0x7fa05015bc50 2024-05-08T04:42:27.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa04ea00640 1 --2- 172.21.15.164:0/706559339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 0x7fa0500b4810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0400097c0 tx=0x7fa04001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef522baa3a48c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa04d600640 1 -- 172.21.15.164:0/706559339 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400b1070 con 0x7fa05015bc50 2024-05-08T04:42:27.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa04d600640 1 -- 172.21.15.164:0/706559339 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0400a8050 con 0x7fa05015bc50 2024-05-08T04:42:27.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa04d600640 1 -- 172.21.15.164:0/706559339 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400ac660 con 0x7fa05015bc50 2024-05-08T04:42:27.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- 172.21.15.164:0/706559339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 msgr2=0x7fa0500b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- 172.21.15.164:0/706559339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 0x7fa0500b4810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0400097c0 tx=0x7fa04001c3e0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- 172.21.15.164:0/706559339 shutdown_connections 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- 172.21.15.164:0/706559339 >> v2:172.21.15.172:3300/0 conn(0x7fa0500d0780 0x7fa05015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- 172.21.15.164:0/706559339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa05015bc50 0x7fa0500b4810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- 172.21.15.164:0/706559339 >> 172.21.15.164:0/706559339 conn(0x7fa0500b34f0 msgr2=0x7fa0500b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- 172.21.15.164:0/706559339 shutdown_connections 2024-05-08T04:42:27.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- 172.21.15.164:0/706559339 wait complete. 2024-05-08T04:42:27.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 Processor -- start 2024-05-08T04:42:27.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- start start 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0500ccfe0 0x7fa050153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0500d0780 0x7fa0500c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050166c50 con 0x7fa05015bc50 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050166ad0 con 0x7fa0500ccfe0 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.444+0000 7fa055e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa050166dd0 con 0x7fa0500d0780 2024-05-08T04:42:27.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44894/0 (socket says 172.21.15.164:44894) 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 -- 172.21.15.164:0/1391691887 learned_addr learned my addr 172.21.15.164:0/1391691887 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04ea00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0500ccfe0 0x7fa050153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04e000640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0500d0780 0x7fa0500c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 -- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0500d0780 msgr2=0x7fa0500c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0500d0780 0x7fa0500c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 -- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0500ccfe0 msgr2=0x7fa050153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0500ccfe0 0x7fa050153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 -- 172.21.15.164:0/1391691887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500ca130 con 0x7fa05015bc50 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa04f400640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa048012420 tx=0x7fa0480128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800d4e0 con 0x7fa05015bc50 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04801d070 con 0x7fa05015bc50 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0500ca410 con 0x7fa05015bc50 2024-05-08T04:42:27.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800b980 con 0x7fa05015bc50 2024-05-08T04:42:27.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0500ca9d0 con 0x7fa05015bc50 2024-05-08T04:42:27.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050069d60 con 0x7fa05015bc50 2024-05-08T04:42:27.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fa048016050 con 0x7fa05015bc50 2024-05-08T04:42:27.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 0x7fa030069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:27.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.448+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa04808a8b0 con 0x7fa05015bc50 2024-05-08T04:42:27.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.452+0000 7fa04ea00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 0x7fa030069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:27.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.452+0000 7fa04ea00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 0x7fa030069b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0400ae000 tx=0x7fa0400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:27.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.456+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa048014040 con 0x7fa05015bc50 2024-05-08T04:42:27.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.640+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0500b7940 con 0x7fa05015bc50 2024-05-08T04:42:27.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.640+0000 7fa047400640 1 -- 172.21.15.164:0/1391691887 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fa04805ac30 con 0x7fa05015bc50 2024-05-08T04:42:27.646 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:27.646 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:27.646 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 62 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 msgr2=0x7fa030069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 0x7fa030069b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0400ae000 tx=0x7fa0400a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 msgr2=0x7fa05015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa048012420 tx=0x7fa0480128e0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 shutdown_connections 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa0300676b0 0x7fa030069b70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0500d0780 0x7fa0500c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa05015bc50 0x7fa05015df10 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 --2- 172.21.15.164:0/1391691887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0500ccfe0 0x7fa050153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:27.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 >> 172.21.15.164:0/1391691887 conn(0x7fa0500b34f0 msgr2=0x7fa05015acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:27.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 shutdown_connections 2024-05-08T04:42:27.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:27.644+0000 7fa055e00640 1 -- 172.21.15.164:0/1391691887 wait complete. 2024-05-08T04:42:28.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:42:21.793962+0000', 'modified': '2024-05-08T04:42:24.805370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7443': {'gid': 7443, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/4096204626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6835', 'nonce': 4096204626}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 4096204626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:42:28.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:42:28.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T04:42:28.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 Processor -- start 2024-05-08T04:42:28.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 -- start start 2024-05-08T04:42:28.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 0x7f8a840cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8a840cfbd0 0x7f8a840cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8406c770 con 0x7f8a8415ab60 2024-05-08T04:42:28.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a840d0060 con 0x7f8a840cfbd0 2024-05-08T04:42:28.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a88c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8406ca50 con 0x7f8a840b26c0 2024-05-08T04:42:28.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a81e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8a8415ab60 0x7f8a84072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:33612/0 (socket says 172.21.15.164:33612) 2024-05-08T04:42:28.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a81400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 0x7f8a840cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a81e00640 1 -- 172.21.15.164:0/4131660745 learned_addr learned my addr 172.21.15.164:0/4131660745 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:28.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a82800640 1 --2- 172.21.15.164:0/4131660745 >> v2:172.21.15.172:3300/0 conn(0x7f8a840cfbd0 0x7f8a840cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a80a00640 1 -- 172.21.15.164:0/4131660745 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310984901 0 0) 0x7f8a8406c770 con 0x7f8a8415ab60 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a80a00640 1 -- 172.21.15.164:0/4131660745 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a68003530 con 0x7f8a8415ab60 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a82800640 1 -- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 msgr2=0x7f8a840cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a82800640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 0x7f8a840cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a82800640 1 -- 172.21.15.164:0/4131660745 >> v1:172.21.15.172:6789/0 conn(0x7f8a8415ab60 legacy=0x7f8a84072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a82800640 1 -- 172.21.15.164:0/4131660745 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8406cd30 con 0x7f8a840cfbd0 2024-05-08T04:42:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.204+0000 7f8a81400640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 0x7f8a840cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:28.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a82800640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840cfbd0 0x7f8a840cd910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009d90 tx=0x7f8a7c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33301304e5a1c640 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a80a00640 1 -- 172.21.15.164:0/4131660745 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0b2070 con 0x7f8a840cfbd0 2024-05-08T04:42:28.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a80a00640 1 -- 172.21.15.164:0/4131660745 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c0a5830 con 0x7f8a840cfbd0 2024-05-08T04:42:28.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a80a00640 1 -- 172.21.15.164:0/4131660745 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0a5be0 con 0x7f8a840cfbd0 2024-05-08T04:42:28.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 -- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840cfbd0 msgr2=0x7f8a840cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840cfbd0 0x7f8a840cd910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009d90 tx=0x7f8a7c01c970 comp rx=0 tx=0).stop 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 -- 172.21.15.164:0/4131660745 shutdown_connections 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a840b26c0 0x7f8a840cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 --2- 172.21.15.164:0/4131660745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840cfbd0 0x7f8a840cd910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 -- 172.21.15.164:0/4131660745 >> 172.21.15.164:0/4131660745 conn(0x7f8a8406ac90 msgr2=0x7f8a8406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 -- 172.21.15.164:0/4131660745 shutdown_connections 2024-05-08T04:42:28.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.208+0000 7f8a88c00640 1 -- 172.21.15.164:0/4131660745 wait complete. 2024-05-08T04:42:28.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 Processor -- start 2024-05-08T04:42:28.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- start start 2024-05-08T04:42:28.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 0x7f8a84159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a8415ab60 0x7f8a8414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840aeeb0 con 0x7f8a840cfbd0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840aed30 con 0x7f8a840b26c0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84167970 con 0x7f8a8415ab60 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44908/0 (socket says 172.21.15.164:44908) 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 -- 172.21.15.164:0/2393665751 learned_addr learned my addr 172.21.15.164:0/2393665751 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a82800640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a8415ab60 0x7f8a8414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 0x7f8a84159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 -- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a8415ab60 msgr2=0x7f8a8414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a8415ab60 0x7f8a8414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 -- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 msgr2=0x7f8a84159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 0x7f8a84159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 -- 172.21.15.164:0/2393665751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8415c2b0 con 0x7f8a840cfbd0 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 0x7f8a84159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:28.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a81400640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ebe0 tx=0x7f8a7000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a7a000640 1 -- 172.21.15.164:0/2393665751 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70017070 con 0x7f8a840cfbd0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a7a000640 1 -- 172.21.15.164:0/2393665751 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7000cc30 con 0x7f8a840cfbd0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8415c590 con 0x7f8a840cfbd0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a7a000640 1 -- 172.21.15.164:0/2393665751 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70013470 con 0x7f8a840cfbd0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.212+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8415ce30 con 0x7f8a840cfbd0 2024-05-08T04:42:28.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a7a000640 1 -- 172.21.15.164:0/2393665751 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f8a7001a070 con 0x7f8a840cfbd0 2024-05-08T04:42:28.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a7a000640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 0x7f8a64069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a7a000640 1 -- 172.21.15.164:0/2393665751 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8a70086460 con 0x7f8a840cfbd0 2024-05-08T04:42:28.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 Processor -- start 2024-05-08T04:42:28.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 0x7f8a64069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 -- start start 2024-05-08T04:42:28.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 0x7f8a64069b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005890 tx=0x7f8a74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2101db240 0x55d2101909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210190f30 0x55d210191310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2100785f0 con 0x55d2101da920 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d20ffdfda0 con 0x55d2101db240 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d20ff68150 con 0x55d210190f30 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44914/0 (socket says 172.21.15.164:44914) 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81e00640 1 -- 172.21.15.164:0/2236379954 learned_addr learned my addr 172.21.15.164:0/2236379954 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a81400640 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2101db240 0x55d2101909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.216+0000 7f8a82800640 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210190f30 0x55d210191310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 -- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210190f30 msgr2=0x55d210191310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210190f30 0x55d210191310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 -- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2101db240 msgr2=0x55d2101909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2101db240 0x55d2101909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 -- 172.21.15.164:0/2236379954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d20ff81ca0 con 0x55d2101da920 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8a740015b0 tx=0x7f8a74011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650312cdc59ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a6ea00640 1 -- 172.21.15.164:0/2236379954 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74005e60 con 0x55d2101da920 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a6ea00640 1 -- 172.21.15.164:0/2236379954 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400e7b0 con 0x55d2101da920 2024-05-08T04:42:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a6ea00640 1 -- 172.21.15.164:0/2236379954 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400eb60 con 0x55d2101da920 2024-05-08T04:42:28.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- 172.21.15.164:0/2236379954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 msgr2=0x55d2101dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8a740015b0 tx=0x7f8a74011000 comp rx=0 tx=0).stop 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- 172.21.15.164:0/2236379954 shutdown_connections 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210190f30 0x55d210191310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2101db240 0x55d2101909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- 172.21.15.164:0/2236379954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101dad00 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- 172.21.15.164:0/2236379954 >> 172.21.15.164:0/2236379954 conn(0x55d2101972a0 msgr2=0x55d21018f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- 172.21.15.164:0/2236379954 shutdown_connections 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- 172.21.15.164:0/2236379954 wait complete. 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- start start 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 Processor -- start 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d210194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d20ff9a990 con 0x55d2101da920 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d20ffdbe40 con 0x55d210190f30 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d20ff6b900 con 0x55d2101db240 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a8c004000 10 client.0 _ll_register_callbacks cb 0x55d2100931c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44924/0 (socket says 172.21.15.164:44924) 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 -- 172.21.15.164:0/3013878059 learned_addr learned my addr 172.21.15.164:0/3013878059 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a82800640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d210194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 -- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 msgr2=0x55d210194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d210194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 -- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 msgr2=0x55d2101e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81400640 1 -- 172.21.15.164:0/3013878059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d20ff98630 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.220+0000 7f8a81e00640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a81400640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004780 tx=0x7f8a70055930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa7e2548f7689a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7006d070 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8a70055360 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a7400e7b0 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a70021040 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 10 client.7584 fetch_fsmap learned FSMap version 62 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d20ff683a0 con 0x55d2101da920 2024-05-08T04:42:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400e7b0 con 0x55d2101da920 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a70051c60 con 0x55d2101da920 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 10 client.7584 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 10 client.7584 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 20 client.7584 populate_metadata read hostname 'smithi164' 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210196160 0x55d2101ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 4 client.7584 mds_command: new command op to 7443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 --> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d20ff7ba50 con 0x55d210196160 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a81e00640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210196160 0x55d2101ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a81e00640 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210196160 0x55d2101ebce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691b25b7462a957 server_cookie=fffbe50be509322e in_seq=0 out_seq=0 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 10 client.7584 ms_handle_connect on v2:172.21.15.164:6835/4096204626 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 1 -- 172.21.15.164:0/3013878059 <== mds.0 v2:172.21.15.164:6835/4096204626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d20ff7ba50 con 0x55d210196160 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 10 client.7584 handle_command_reply: tid=0 2024-05-08T04:42:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 client.7584 shutdown 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 msgr2=0x55d2101e21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004780 tx=0x7f8a70055930 comp rx=0 tx=0).stop 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 shutdown_connections 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210196160 0x55d2101ebce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d210194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d2101e21c0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a63400640 0 client.7584 ms_handle_reset on v2:172.21.15.164:6835/4096204626 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 --2- 172.21.15.164:0/3013878059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 >> 172.21.15.164:0/3013878059 conn(0x55d2101972a0 msgr2=0x55d21018f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 shutdown_connections 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 1 -- 172.21.15.164:0/3013878059 wait complete. 2024-05-08T04:42:28.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.224+0000 7f8a8c004000 20 client.7584 trim_cache size 0 max 16384 2024-05-08T04:42:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 Processor -- start 2024-05-08T04:42:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 -- start start 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d21023a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d21023b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a841f5cf0 con 0x55d2101da920 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84158e60 con 0x55d210190f30 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a841f56f0 con 0x55d2101db240 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44926/0 (socket says 172.21.15.164:44926) 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 -- 172.21.15.164:0/1107194942 learned_addr learned my addr 172.21.15.164:0/1107194942 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 -- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 msgr2=0x55d21023b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d21023b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81e00640 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d21023a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 -- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 msgr2=0x55d21023a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d21023a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 -- 172.21.15.164:0/1107194942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2101db070 con 0x55d2101da920 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81e00640 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d21023a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.240+0000 7f8a81400640 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8a70083e70 tx=0x7f8a7005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d7fbc76049ad81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a6e000640 1 -- 172.21.15.164:0/1107194942 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7006d040 con 0x55d2101da920 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a6e000640 1 -- 172.21.15.164:0/1107194942 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7005c070 con 0x55d2101da920 2024-05-08T04:42:28.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a6e000640 1 -- 172.21.15.164:0/1107194942 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70083950 con 0x55d2101da920 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/1107194942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 msgr2=0x55d21023aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8a70083e70 tx=0x7f8a7005c000 comp rx=0 tx=0).stop 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/1107194942 shutdown_connections 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d2101db240 0x55d21023b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101da920 0x55d21023aae0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- 172.21.15.164:0/1107194942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d21023a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/1107194942 >> 172.21.15.164:0/1107194942 conn(0x55d2101972a0 msgr2=0x55d210194220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/1107194942 shutdown_connections 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/1107194942 wait complete. 2024-05-08T04:42:28.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- start start 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 Processor -- start 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 0x55d210240210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d20ffe43d0 con 0x55d2101db240 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21009bd50 con 0x55d210190f30 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d20ff7a980 con 0x55d210196e00 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 10 client.0 _ll_register_callbacks cb 0x7f8a74001dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44934/0 (socket says 172.21.15.164:44934) 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 -- 172.21.15.164:0/176465675 learned_addr learned my addr 172.21.15.164:0/176465675 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a82800640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 0x55d210240210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81e00640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 -- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 msgr2=0x55d210240210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 0x55d210240210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 -- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 msgr2=0x55d2101e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 -- 172.21.15.164:0/176465675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d210061600 con 0x55d2101db240 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a82800640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 0x55d210240210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81e00640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a81400640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004780 tx=0x7f8a7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4619d93ffda5db01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a63400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7006d040 con 0x55d2101db240 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8a7007fb40 con 0x55d2101db240 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a7005c070 con 0x55d2101db240 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a70055a50 con 0x55d2101db240 2024-05-08T04:42:28.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 10 client.7590 fetch_fsmap learned FSMap version 62 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.244+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d20ff749b0 con 0x55d2101db240 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7005c070 con 0x55d2101db240 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a7007fd40 con 0x55d2101db240 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 10 client.7590 fetch_fsmap finished waiting for FSMap version 62 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 10 client.7590 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 20 client.7590 populate_metadata read hostname 'smithi164' 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210240750 0x55d2101f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 4 client.7590 mds_command: new command op to 7443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 --> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d20ffed2e0 con 0x55d210240750 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a81e00640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210240750 0x55d2101f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a81e00640 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210240750 0x55d2101f9b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a10b2eb5462f96f server_cookie=b5bad71b241849c in_seq=0 out_seq=0 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 10 client.7590 ms_handle_connect on v2:172.21.15.164:6835/4096204626 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 1 -- 172.21.15.164:0/176465675 <== mds.0 v2:172.21.15.164:6835/4096204626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d20ffed2e0 con 0x55d210240750 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 10 client.7590 handle_command_reply: tid=0 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 client.7590 shutdown 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 msgr2=0x55d21023fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004780 tx=0x7f8a7000ef60 comp rx=0 tx=0).stop 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 shutdown_connections 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:6835/4096204626,v1:172.21.15.164:6837/4096204626] conn(0x55d210240750 0x55d2101f9b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d210196e00 0x55d210240210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2101db240 0x55d21023fcd0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 --2- 172.21.15.164:0/176465675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d210190f30 0x55d2101e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a63400640 0 client.7590 ms_handle_reset on v2:172.21.15.164:6835/4096204626 2024-05-08T04:42:28.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 >> 172.21.15.164:0/176465675 conn(0x55d2101972a0 msgr2=0x55d2101e2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 shutdown_connections 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 1 -- 172.21.15.164:0/176465675 wait complete. 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a8c004000 20 client.7590 trim_cache size 0 max 16384 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 7443, 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:42:28.258 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 5.8319553229999999, 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 62, 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 99, 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 99, 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 10.739012312 2024-05-08T04:42:28.259 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 msgr2=0x7f8a64069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 0x7f8a64069b70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005890 tx=0x7f8a74009040 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 msgr2=0x7f8a8415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ebe0 tx=0x7f8a7000c520 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 shutdown_connections 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8a640676b0 0x7f8a64069b70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a8415ab60 0x7f8a8414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a840cfbd0 0x7f8a8415ba60 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 --2- 172.21.15.164:0/2393665751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a840b26c0 0x7f8a84159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 >> 172.21.15.164:0/2393665751 conn(0x7f8a8406ac90 msgr2=0x7f8a8415ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 shutdown_connections 2024-05-08T04:42:28.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.248+0000 7f8a88c00640 1 -- 172.21.15.164:0/2393665751 wait complete. 2024-05-08T04:42:28.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:42:28.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 Processor -- start 2024-05-08T04:42:28.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 -- start start 2024-05-08T04:42:28.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28dc15ab60 0x7f28dc072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f28dc0cfda0 0x7f28dc0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc072640 con 0x7f28dc0b26c0 2024-05-08T04:42:28.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc06c770 con 0x7f28dc15ab60 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e4000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc06ca50 con 0x7f28dc0cfda0 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28dc15ab60 0x7f28dc072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28dc15ab60 0x7f28dc072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44936/0 (socket says 172.21.15.164:44936) 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 -- 172.21.15.164:0/881388251 learned_addr learned my addr 172.21.15.164:0/881388251 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e2000640 1 --2- 172.21.15.164:0/881388251 >> v2:172.21.15.172:3300/0 conn(0x7f28dc0cfda0 0x7f28dc0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 -- 172.21.15.164:0/881388251 >> v2:172.21.15.172:3300/0 conn(0x7f28dc0cfda0 msgr2=0x7f28dc0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 --2- 172.21.15.164:0/881388251 >> v2:172.21.15.172:3300/0 conn(0x7f28dc0cfda0 0x7f28dc0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 -- 172.21.15.164:0/881388251 >> v1:172.21.15.172:6789/0 conn(0x7f28dc0b26c0 legacy=0x7f28dc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.404+0000 7f28e1600640 1 -- 172.21.15.164:0/881388251 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc06cd30 con 0x7f28dc15ab60 2024-05-08T04:42:28.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e1600640 1 --2- 172.21.15.164:0/881388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc072100 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0b2b20 tx=0x7f28d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206a5afbcb4da3bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28dbe00640 1 -- 172.21.15.164:0/881388251 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00a56c0 con 0x7f28dc15ab60 2024-05-08T04:42:28.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28dbe00640 1 -- 172.21.15.164:0/881388251 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d00a5860 con 0x7f28dc15ab60 2024-05-08T04:42:28.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28dbe00640 1 -- 172.21.15.164:0/881388251 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00a5c30 con 0x7f28dc15ab60 2024-05-08T04:42:28.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 -- 172.21.15.164:0/881388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 msgr2=0x7f28dc072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 --2- 172.21.15.164:0/881388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc072100 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0b2b20 tx=0x7f28d001c9f0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 -- 172.21.15.164:0/881388251 shutdown_connections 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 --2- 172.21.15.164:0/881388251 >> v2:172.21.15.172:3300/0 conn(0x7f28dc0cfda0 0x7f28dc0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 --2- 172.21.15.164:0/881388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc072100 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 -- 172.21.15.164:0/881388251 >> 172.21.15.164:0/881388251 conn(0x7f28dc06ac90 msgr2=0x7f28dc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 -- 172.21.15.164:0/881388251 shutdown_connections 2024-05-08T04:42:28.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.408+0000 7f28e4000640 1 -- 172.21.15.164:0/881388251 wait complete. 2024-05-08T04:42:28.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 Processor -- start 2024-05-08T04:42:28.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- start start 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28dc0b26c0 0x7f28dc159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28dc0cfda0 0x7f28dc15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0aeeb0 con 0x7f28dc15ab60 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0aed30 con 0x7f28dc0cfda0 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc167970 con 0x7f28dc0b26c0 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44952/0 (socket says 172.21.15.164:44952) 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 -- 172.21.15.164:0/1808481639 learned_addr learned my addr 172.21.15.164:0/1808481639 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e0c00640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28dc0cfda0 0x7f28dc15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e1600640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28dc0b26c0 0x7f28dc159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 -- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28dc0b26c0 msgr2=0x7f28dc159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28dc0b26c0 0x7f28dc159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 -- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28dc0cfda0 msgr2=0x7f28dc15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28dc0cfda0 0x7f28dc15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 -- 172.21.15.164:0/1808481639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc1f71a0 con 0x7f28dc15ab60 2024-05-08T04:42:28.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e2000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d5d0 tx=0x7f28d400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40125f0 con 0x7f28dc15ab60 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d4019070 con 0x7f28dc15ab60 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40129f0 con 0x7f28dc15ab60 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc15b950 con 0x7f28dc15ab60 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.412+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc1f7450 con 0x7f28dc15ab60 2024-05-08T04:42:28.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc15be70 con 0x7f28dc15ab60 2024-05-08T04:42:28.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f28d4012b90 con 0x7f28dc15ab60 2024-05-08T04:42:28.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28d9600640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 0x7f28bc069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:28.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f28d4086510 con 0x7f28dc15ab60 2024-05-08T04:42:28.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28e1600640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 0x7f28bc069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:28.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.416+0000 7f28e1600640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 0x7f28bc069b70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f28d00ae000 tx=0x7f28d00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:28.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.420+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28d4055f70 con 0x7f28dc15ab60 2024-05-08T04:42:28.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.608+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f28dc06d9a0 con 0x7f28dc15ab60 2024-05-08T04:42:28.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.608+0000 7f28d9600640 1 -- 172.21.15.164:0/1808481639 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f28d405a010 con 0x7f28dc15ab60 2024-05-08T04:42:28.614 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:42:28.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 msgr2=0x7f28bc069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 0x7f28bc069b70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f28d00ae000 tx=0x7f28d00a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:28.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 msgr2=0x7f28dc1f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d5d0 tx=0x7f28d400da90 comp rx=0 tx=0).stop 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 shutdown_connections 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f28bc0676b0 0x7f28bc069b70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28dc15ab60 0x7f28dc1f7c50 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28dc0cfda0 0x7f28dc15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 --2- 172.21.15.164:0/1808481639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28dc0b26c0 0x7f28dc159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 >> 172.21.15.164:0/1808481639 conn(0x7f28dc06ac90 msgr2=0x7f28dc0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 shutdown_connections 2024-05-08T04:42:28.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:28.612+0000 7f28e4000640 1 -- 172.21.15.164:0/1808481639 wait complete. 2024-05-08T04:42:28.635 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:42:28.635 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:42:28.636 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:42:28.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.637 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:42:28.645 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:42:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:42:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:42:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42118sec preferred_lft 42118sec 2024-05-08T04:42:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:42:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:42:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.648 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:42:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:42:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:42:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:28.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.649 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:42:28.686 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:42:28.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:42:28.710 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:42:28.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.711 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:42:28.789 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:28.790 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:42:28.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:42:28.791 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:42:28.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.879 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:42:28.926 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:42:28.927 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:42:28.928 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:42:28.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:28.928 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:42:28.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:42:28.990 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:29.043 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:42:29.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:42:29.043 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:29.098 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:42:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:42:29.186 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:42:29.187 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:42:29.187 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:42:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:42:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:42:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:42:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:42:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:42:29.211 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:42:29.215 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:42:29.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.215 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:29.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.236 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:42:29.299 INFO:teuthology.orchestra.run.smithi164.stdout:client7599 2024-05-08T04:42:29.300 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:29.300 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:29.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:42:29.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 Processor -- start 2024-05-08T04:42:29.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 -- start start 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffb6c0d0780 0x7ffb6c166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c0b42a0 con 0x7ffb6c15bc50 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c0cd4e0 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb71400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cd7c0 con 0x7ffb6c0d0780 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ffb6c15bc50 0x7ffb6c0b4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48158/0 (socket says 172.21.15.164:48158) 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6a800640 1 -- 172.21.15.164:0/4126300244 learned_addr learned my addr 172.21.15.164:0/4126300244 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:29.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6b200640 1 --2- 172.21.15.164:0/4126300244 >> v2:172.21.15.164:3300/0 conn(0x7ffb6c0d0780 0x7ffb6c166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:29.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6b200640 1 -- 172.21.15.164:0/4126300244 >> v1:172.21.15.172:6789/0 conn(0x7ffb6c0ccfe0 legacy=0x7ffb6c0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:29.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6b200640 1 -- 172.21.15.164:0/4126300244 >> v1:172.21.15.164:6789/0 conn(0x7ffb6c15bc50 legacy=0x7ffb6c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb6b200640 1 -- 172.21.15.164:0/4126300244 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c0cf590 con 0x7ffb6c0d0780 2024-05-08T04:42:29.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.536+0000 7ffb69400640 1 -- 172.21.15.164:0/4126300244 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487759635 0 0) 0x7ffb6c0b42a0 con 0x7ffb6c15bc50 2024-05-08T04:42:29.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb6b200640 1 --2- 172.21.15.164:0/4126300244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0d0780 0x7ffb6c166ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009d90 tx=0x7ffb6401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a05efce8f9af86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:29.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb69400640 1 -- 172.21.15.164:0/4126300244 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640b2070 con 0x7ffb6c0d0780 2024-05-08T04:42:29.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb69400640 1 -- 172.21.15.164:0/4126300244 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb640a5930 con 0x7ffb6c0d0780 2024-05-08T04:42:29.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb69400640 1 -- 172.21.15.164:0/4126300244 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640a5d00 con 0x7ffb6c0d0780 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 -- 172.21.15.164:0/4126300244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0d0780 msgr2=0x7ffb6c166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 --2- 172.21.15.164:0/4126300244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0d0780 0x7ffb6c166ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009d90 tx=0x7ffb6401c870 comp rx=0 tx=0).stop 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 -- 172.21.15.164:0/4126300244 shutdown_connections 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 --2- 172.21.15.164:0/4126300244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0d0780 0x7ffb6c166ad0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 -- 172.21.15.164:0/4126300244 >> 172.21.15.164:0/4126300244 conn(0x7ffb6c0b34f0 msgr2=0x7ffb6c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 -- 172.21.15.164:0/4126300244 shutdown_connections 2024-05-08T04:42:29.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.540+0000 7ffb71400640 1 -- 172.21.15.164:0/4126300244 wait complete. 2024-05-08T04:42:29.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 Processor -- start 2024-05-08T04:42:29.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- start start 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 0x7ffb6c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6c15bc50 0x7ffb6c06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b50a0 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b4f20 con 0x7ffb6c15bc50 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c169830 con 0x7ffb6c0d0780 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47134/0 (socket says 172.21.15.164:47134) 2024-05-08T04:42:29.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 -- 172.21.15.164:0/636131144 learned_addr learned my addr 172.21.15.164:0/636131144 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6b200640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6c15bc50 0x7ffb6c06cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb69e00640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 0x7ffb6c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 -- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 msgr2=0x7ffb6c0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 0x7ffb6c0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 -- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6c15bc50 msgr2=0x7ffb6c06cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6c15bc50 0x7ffb6c06cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 -- 172.21.15.164:0/636131144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c154330 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb69e00640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 0x7ffb6c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb6a800640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ffb54007fd0 tx=0x7ffb5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54017070 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb54005d10 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54007310 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c0c7ca0 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c0c8260 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.544+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c15c820 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.548+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7ffb5401a070 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.548+0000 7ffb62a00640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 0x7ffb48069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:29.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.548+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7ffb54085510 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.548+0000 7ffb69e00640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 0x7ffb48069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:29.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.548+0000 7ffb69e00640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 0x7ffb48069b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0b2f00 tx=0x7ffb5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:29.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.552+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb6c154330 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.740+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb6c0b7940 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.740+0000 7ffb62a00640 1 -- 172.21.15.164:0/636131144 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb54054fc0 con 0x7ffb6c0ccfe0 2024-05-08T04:42:29.746 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:42:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 msgr2=0x7ffb48069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 0x7ffb48069b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0b2f00 tx=0x7ffb5c009040 comp rx=0 tx=0).stop 2024-05-08T04:42:29.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 msgr2=0x7ffb6c153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ffb54007fd0 tx=0x7ffb5400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 shutdown_connections 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ffb480676b0 0x7ffb48069b70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb6c0d0780 0x7ffb6c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6c15bc50 0x7ffb6c06cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 --2- 172.21.15.164:0/636131144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb6c0ccfe0 0x7ffb6c153df0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 >> 172.21.15.164:0/636131144 conn(0x7ffb6c0b34f0 msgr2=0x7ffb6c0b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 shutdown_connections 2024-05-08T04:42:29.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:29.744+0000 7ffb71400640 1 -- 172.21.15.164:0/636131144 wait complete. 2024-05-08T04:42:29.767 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:42:29.768 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:42:29.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.768 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:42:29.779 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:42:29.780 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42121sec preferred_lft 42121sec 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.781 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:42:29.782 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:42:29.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.783 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:42:29.845 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:42:29.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.848 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:42:29.903 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:42:29.904 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:42:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:42:29.905 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:42:30.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:30.003 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:42:30.012 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:42:30.012 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:42:30.013 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:42:30.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:30.013 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:42:30.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:42:30.074 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:30.129 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:42:30.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:42:30.130 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:30.185 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:42:30.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:30.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:42:30.276 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:42:30.276 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:42:30.276 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:42:30.291 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:42:30.291 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:42:30.292 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:42:30.292 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:42:30.292 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:42:30.300 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:42:30.300 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:42:30.300 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:42:30.300 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:42:30.301 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:42:30.301 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:42:30.301 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:42:30.301 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:42:30.304 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:42:30.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:30.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:30.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:30.324 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:42:30.376 INFO:teuthology.orchestra.run.smithi172.stdout:client7312 2024-05-08T04:42:30.377 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:30.377 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:30.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:42:30.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 Processor -- start 2024-05-08T04:42:30.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 -- start start 2024-05-08T04:42:30.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff4c40adaa0 0x7ff4c4072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40cdd60 con 0x7ff4c415db40 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40ce040 con 0x7ff4c40cee00 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4c3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c40ce970 con 0x7ff4c40adaa0 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff4c40adaa0 0x7ff4c4072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff4c40adaa0 0x7ff4c4072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47142/0 (socket says 172.21.15.164:47142) 2024-05-08T04:42:30.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 -- 172.21.15.164:0/2250229029 learned_addr learned my addr 172.21.15.164:0/2250229029 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:30.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 -- 172.21.15.164:0/2250229029 >> v1:172.21.15.172:6789/0 conn(0x7ff4c40cee00 legacy=0x7ff4c4070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:30.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 -- 172.21.15.164:0/2250229029 >> v1:172.21.15.164:6789/0 conn(0x7ff4c415db40 legacy=0x7ff4c406cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:30.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.568+0000 7ff4bbe00640 1 -- 172.21.15.164:0/2250229029 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40ce320 con 0x7ff4c40adaa0 2024-05-08T04:42:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4bbe00640 1 --2- 172.21.15.164:0/2250229029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4072b90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009870 tx=0x7ff4ac01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31be39a0b8a5d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c1400640 1 -- 172.21.15.164:0/2250229029 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0aa5f0 con 0x7ff4c40adaa0 2024-05-08T04:42:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c1400640 1 -- 172.21.15.164:0/2250229029 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0af070 con 0x7ff4c40adaa0 2024-05-08T04:42:30.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c1400640 1 -- 172.21.15.164:0/2250229029 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0aa9b0 con 0x7ff4c40adaa0 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- 172.21.15.164:0/2250229029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40adaa0 msgr2=0x7ff4c4072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 --2- 172.21.15.164:0/2250229029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4072b90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009870 tx=0x7ff4ac01c530 comp rx=0 tx=0).stop 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- 172.21.15.164:0/2250229029 shutdown_connections 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 --2- 172.21.15.164:0/2250229029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4072b90 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- 172.21.15.164:0/2250229029 >> 172.21.15.164:0/2250229029 conn(0x7ff4c406ac90 msgr2=0x7ff4c406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- 172.21.15.164:0/2250229029 shutdown_connections 2024-05-08T04:42:30.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- 172.21.15.164:0/2250229029 wait complete. 2024-05-08T04:42:30.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 Processor -- start 2024-05-08T04:42:30.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.572+0000 7ff4c3200640 1 -- start start 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 0x7ff4c4151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40aebd0 con 0x7ff4c40cee00 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40aea50 con 0x7ff4c40adaa0 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4071130 con 0x7ff4c415db40 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47146/0 (socket says 172.21.15.164:47146) 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 -- 172.21.15.164:0/716163164 learned_addr learned my addr 172.21.15.164:0/716163164 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:30.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c1e00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c2800640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 0x7ff4c4151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 -- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 msgr2=0x7ff4c4151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 0x7ff4c4151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 -- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4c40adaa0 msgr2=0x7ff4c4160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 -- 172.21.15.164:0/716163164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4160ab0 con 0x7ff4c40cee00 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c2800640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 0x7ff4c4151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4bbe00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ac040 tx=0x7ff4ac01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0bd5f0 con 0x7ff4c40cee00 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0bd7b0 con 0x7ff4c40cee00 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0bdba0 con 0x7ff4c40cee00 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c415f4d0 con 0x7ff4c40cee00 2024-05-08T04:42:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c415fa90 con 0x7ff4c40cee00 2024-05-08T04:42:30.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.576+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c404ef30 con 0x7ff4c40cee00 2024-05-08T04:42:30.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.580+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7ff4ac0b3600 con 0x7ff4c40cee00 2024-05-08T04:42:30.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.580+0000 7ff4ba000640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 0x7ff4a0069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:30.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.580+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7ff4ac120df0 con 0x7ff4c40cee00 2024-05-08T04:42:30.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.580+0000 7ff4c1e00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 0x7ff4a0069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:30.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.580+0000 7ff4c1e00640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 0x7ff4a0069b70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4007fd0 tx=0x7ff4b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:30.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.584+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4ac0c3060 con 0x7ff4c40cee00 2024-05-08T04:42:30.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:30.776+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4c4066be0 con 0x7ff4c40cee00 2024-05-08T04:42:31.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4ba000640 1 -- 172.21.15.164:0/716163164 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff4ac016150 con 0x7ff4c40cee00 2024-05-08T04:42:31.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 msgr2=0x7ff4a0069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 0x7ff4a0069b70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4007fd0 tx=0x7ff4b4009040 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 msgr2=0x7ff4c4151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ac040 tx=0x7ff4ac01ad70 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 shutdown_connections 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff4a00676b0 0x7ff4a0069b70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4c415db40 0x7ff4c4151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4c40cee00 0x7ff4c4151000 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 --2- 172.21.15.164:0/716163164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4c40adaa0 0x7ff4c4160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 >> 172.21.15.164:0/716163164 conn(0x7ff4c406ac90 msgr2=0x7ff4c406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 shutdown_connections 2024-05-08T04:42:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.456+0000 7ff4c3200640 1 -- 172.21.15.164:0/716163164 wait complete. 2024-05-08T04:42:31.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:42:31.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 Processor -- start 2024-05-08T04:42:31.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 -- start start 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc47c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0cd9f0 con 0x7fc47c15ba70 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0cdcd0 con 0x7fc47c15be50 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.616+0000 7fc481800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c15fe70 con 0x7fc47c0cfda0 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc47c15ba70 0x7fc47c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48184/0 (socket says 172.21.15.164:48184) 2024-05-08T04:42:31.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47aa00640 1 -- 172.21.15.164:0/2573358131 learned_addr learned my addr 172.21.15.164:0/2573358131 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035714773 0 0) 0x7fc47c0cd9f0 con 0x7fc47c15ba70 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc460003530 con 0x7fc47c15ba70 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47a000640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc47c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35894926 0 0) 0x7fc460003530 con 0x7fc47c15ba70 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 msgr2=0x7fc47c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:31.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc47c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 >> v1:172.21.15.172:6789/0 conn(0x7fc47c15be50 legacy=0x7fc47c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c15f820 con 0x7fc47c15ba70 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fc46c001a00 con 0x7fc47c15ba70 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 >> v1:172.21.15.164:6789/0 conn(0x7fc47c15ba70 legacy=0x7fc47c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc4600039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc460004020 0x7fc460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc4600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c15fe70 con 0x7fc460004d50 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46000bb90 con 0x7fc460004020 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46000bd90 con 0x7fc47c0cfda0 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47aa00640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc4600039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc4600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47a000640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc460004020 0x7fc460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 -- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 msgr2=0x7fc4600039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0cfda0 0x7fc4600039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 -- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc460004020 msgr2=0x7fc460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc460004020 0x7fc460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.620+0000 7fc47b400640 1 -- 172.21.15.164:0/2573358131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc460003530 con 0x7fc460004d50 2024-05-08T04:42:31.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc47b400640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc4600051d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc474001780 tx=0x7fc47401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1368422675fd2830 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740013a0 con 0x7fc460004d50 2024-05-08T04:42:31.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4740a9070 con 0x7fc460004d50 2024-05-08T04:42:31.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc479600640 1 -- 172.21.15.164:0/2573358131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474003600 con 0x7fc460004d50 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- 172.21.15.164:0/2573358131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 msgr2=0x7fc4600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 --2- 172.21.15.164:0/2573358131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc4600051d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc474001780 tx=0x7fc47401c420 comp rx=0 tx=0).stop 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc47b400640 1 -- 172.21.15.164:0/2573358131 reap_dead start 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- 172.21.15.164:0/2573358131 shutdown_connections 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- 172.21.15.164:0/2573358131 >> 172.21.15.164:0/2573358131 conn(0x7fc47c06ac90 msgr2=0x7fc47c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- 172.21.15.164:0/2573358131 shutdown_connections 2024-05-08T04:42:31.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- 172.21.15.164:0/2573358131 wait complete. 2024-05-08T04:42:31.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 Processor -- start 2024-05-08T04:42:31.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.624+0000 7fc481800640 1 -- start start 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47c15cc40 0x7fc47c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 0x7fc47c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cd7b0 con 0x7fc460004d50 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cd630 con 0x7fc47c15cc40 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0aeda0 con 0x7fc47c0b11f0 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47170/0 (socket says 172.21.15.164:47170) 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 -- 172.21.15.164:0/1560600747 learned_addr learned my addr 172.21.15.164:0/1560600747 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:31.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47a000640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47c15cc40 0x7fc47c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47b400640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 0x7fc47c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 -- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 msgr2=0x7fc47c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 0x7fc47c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 -- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47c15cc40 msgr2=0x7fc47c15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47c15cc40 0x7fc47c15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 -- 172.21.15.164:0/1560600747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c15d6e0 con 0x7fc460004d50 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47b400640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 0x7fc47c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc47aa00640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc46c007a20 tx=0x7fc46c00a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c017020 con 0x7fc460004d50 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00ae00 con 0x7fc460004d50 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c013840 con 0x7fc460004d50 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c15d9c0 con 0x7fc460004d50 2024-05-08T04:42:31.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c0af220 con 0x7fc460004d50 2024-05-08T04:42:31.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c069d60 con 0x7fc460004d50 2024-05-08T04:42:31.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.628+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fc46c01a020 con 0x7fc460004d50 2024-05-08T04:42:31.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.632+0000 7fc472a00640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 0x7fc45c069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.632+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc46c086dd0 con 0x7fc460004d50 2024-05-08T04:42:31.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.632+0000 7fc47a000640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 0x7fc45c069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.632+0000 7fc47a000640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 0x7fc45c069ba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc47c155850 tx=0x7fc468008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.636+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc46c021020 con 0x7fc460004d50 2024-05-08T04:42:31.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.808+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc47c06e210 con 0x7fc45c0676e0 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc472a00640 1 -- 172.21.15.164:0/1560600747 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc47c06e210 con 0x7fc45c0676e0 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:31.816 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:42:31.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 msgr2=0x7fc45c069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 0x7fc45c069ba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc47c155850 tx=0x7fc468008040 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 msgr2=0x7fc47c14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc46c007a20 tx=0x7fc46c00a6f0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 shutdown_connections 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc45c0676e0 0x7fc45c069ba0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47c0b11f0 0x7fc47c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47c15cc40 0x7fc47c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 --2- 172.21.15.164:0/1560600747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc460004d50 0x7fc47c14a180 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 >> 172.21.15.164:0/1560600747 conn(0x7fc47c06ac90 msgr2=0x7fc47c0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 shutdown_connections 2024-05-08T04:42:31.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.812+0000 7fc481800640 1 -- 172.21.15.164:0/1560600747 wait complete. 2024-05-08T04:42:31.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 Processor -- start 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 -- start start 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcafc15ce10 0x7fcafc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 0x7fcafc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0c6600 con 0x7fcafc15ba20 2024-05-08T04:42:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcafc15fbe0 con 0x7fcafc15ce10 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcb01000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0c6040 con 0x7fcafc06e390 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcafa800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 0x7fcafc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcafc15ce10 0x7fcafc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcafa800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 0x7fcafc0ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:34246/0 (socket says 172.21.15.164:34246) 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf9e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcafc15ba20 0x7fcafc160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:55012/0 (socket says 172.21.15.164:55012) 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcafa800640 1 -- 172.21.15.164:0/2810634609 learned_addr learned my addr 172.21.15.164:0/2810634609 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:31.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf1400640 1 -- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 msgr2=0x7fcafc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf1400640 1 --2- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 0x7fcafc0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf1400640 1 -- 172.21.15.164:0/2810634609 >> v1:172.21.15.172:6789/0 conn(0x7fcafc15ba20 legacy=0x7fcafc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.972+0000 7fcaf1400640 1 -- 172.21.15.164:0/2810634609 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0c6320 con 0x7fcafc15ce10 2024-05-08T04:42:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcaf1400640 1 --2- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc15ce10 0x7fcafc06de50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097c0 tx=0x7fcae401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e644b22fb70d8e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcaf9400640 1 -- 172.21.15.164:0/2810634609 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b0070 con 0x7fcafc15ce10 2024-05-08T04:42:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcaf9400640 1 -- 172.21.15.164:0/2810634609 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae40ab600 con 0x7fcafc15ce10 2024-05-08T04:42:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcaf9400640 1 -- 172.21.15.164:0/2810634609 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40ab9d0 con 0x7fcafc15ce10 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 -- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc15ce10 msgr2=0x7fcafc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 --2- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc15ce10 0x7fcafc06de50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097c0 tx=0x7fcae401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 -- 172.21.15.164:0/2810634609 shutdown_connections 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 --2- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc06e390 0x7fcafc0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 --2- 172.21.15.164:0/2810634609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc15ce10 0x7fcafc06de50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 -- 172.21.15.164:0/2810634609 >> 172.21.15.164:0/2810634609 conn(0x7fcafc0cf770 msgr2=0x7fcafc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 -- 172.21.15.164:0/2810634609 shutdown_connections 2024-05-08T04:42:31.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.976+0000 7fcb01000640 1 -- 172.21.15.164:0/2810634609 wait complete. 2024-05-08T04:42:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 Processor -- start 2024-05-08T04:42:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- start start 2024-05-08T04:42:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc06e390 0x7fcafc0af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc15ba20 0x7fcafc0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0af030 con 0x7fcafc15ce10 2024-05-08T04:42:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0ce0d0 con 0x7fcafc06e390 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc168b70 con 0x7fcafc15ba20 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47186/0 (socket says 172.21.15.164:47186) 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 -- 172.21.15.164:0/1706703164 learned_addr learned my addr 172.21.15.164:0/1706703164 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcaf1400640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc15ba20 0x7fcafc0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcaf9e00640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc06e390 0x7fcafc0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 -- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc15ba20 msgr2=0x7fcafc0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc15ba20 0x7fcafc0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 -- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc06e390 msgr2=0x7fcafc0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc06e390 0x7fcafc0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 -- 172.21.15.164:0/1706703164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0afac0 con 0x7fcafc15ce10 2024-05-08T04:42:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcafa800640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400efc0 tx=0x7fcaf400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400cd20 con 0x7fcafc15ce10 2024-05-08T04:42:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40039a0 con 0x7fcafc15ce10 2024-05-08T04:42:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4003d90 con 0x7fcafc15ce10 2024-05-08T04:42:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc0afda0 con 0x7fcafc15ce10 2024-05-08T04:42:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.980+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc0b0360 con 0x7fcafc15ce10 2024-05-08T04:42:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc072d30 con 0x7fcafc15ce10 2024-05-08T04:42:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fcaf4012050 con 0x7fcafc15ce10 2024-05-08T04:42:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcaf2a00640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 0x7fcad8069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:31.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fcaf4086360 con 0x7fcafc15ce10 2024-05-08T04:42:31.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcaf9e00640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 0x7fcad8069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:31.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.984+0000 7fcaf9e00640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 0x7fcad8069b70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000f70 tx=0x7fcaec007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:31.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:31.988+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcaf4010040 con 0x7fcafc15ce10 2024-05-08T04:42:32.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcafc166940 con 0x7fcafc15ce10 2024-05-08T04:42:32.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcaf2a00640 1 -- 172.21.15.164:0/1706703164 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcaf4056080 con 0x7fcafc15ce10 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 msgr2=0x7fcad8069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 0x7fcad8069b70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000f70 tx=0x7fcaec007e90 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 msgr2=0x7fcafc0b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400efc0 tx=0x7fcaf400c490 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 shutdown_connections 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcad80676b0 0x7fcad8069b70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcafc15ba20 0x7fcafc0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcafc15ce10 0x7fcafc0b0b60 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 --2- 172.21.15.164:0/1706703164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcafc06e390 0x7fcafc0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 >> 172.21.15.164:0/1706703164 conn(0x7fcafc0cf770 msgr2=0x7fcafc15e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 shutdown_connections 2024-05-08T04:42:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.144+0000 7fcb01000640 1 -- 172.21.15.164:0/1706703164 wait complete. 2024-05-08T04:42:32.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp81090 2024-05-08T04:42:32.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 Processor -- start 2024-05-08T04:42:32.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 -- start start 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc364144360 0x7fc3640c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3641627f0 con 0x7fc36406ac90 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3640adc50 con 0x7fc3640cb290 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc369600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640ae860 con 0x7fc364144360 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc36406ac90 0x7fc3641619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48188/0 (socket says 172.21.15.164:48188) 2024-05-08T04:42:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362000640 1 -- 172.21.15.164:0/2303520704 learned_addr learned my addr 172.21.15.164:0/2303520704 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:32.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362a00640 1 --2- 172.21.15.164:0/2303520704 >> v2:172.21.15.172:3300/0 conn(0x7fc364144360 0x7fc3640c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc361600640 1 -- 172.21.15.164:0/2303520704 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79461524 0 0) 0x7fc3641627f0 con 0x7fc36406ac90 2024-05-08T04:42:32.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc361600640 1 -- 172.21.15.164:0/2303520704 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc348003530 con 0x7fc36406ac90 2024-05-08T04:42:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362a00640 1 -- 172.21.15.164:0/2303520704 >> v1:172.21.15.172:6789/0 conn(0x7fc3640cb290 legacy=0x7fc3641620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362a00640 1 -- 172.21.15.164:0/2303520704 >> v1:172.21.15.164:6789/0 conn(0x7fc36406ac90 legacy=0x7fc3641619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.304+0000 7fc362a00640 1 -- 172.21.15.164:0/2303520704 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640adf30 con 0x7fc364144360 2024-05-08T04:42:32.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc362a00640 1 --2- 172.21.15.164:0/2303520704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364144360 0x7fc3640c5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3540097c0 tx=0x7fc35401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824182e9acefa526 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc361600640 1 -- 172.21.15.164:0/2303520704 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b0070 con 0x7fc364144360 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc361600640 1 -- 172.21.15.164:0/2303520704 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3540ab600 con 0x7fc364144360 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc361600640 1 -- 172.21.15.164:0/2303520704 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540ab9d0 con 0x7fc364144360 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 -- 172.21.15.164:0/2303520704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364144360 msgr2=0x7fc3640c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 --2- 172.21.15.164:0/2303520704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364144360 0x7fc3640c5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3540097c0 tx=0x7fc35401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 -- 172.21.15.164:0/2303520704 shutdown_connections 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 --2- 172.21.15.164:0/2303520704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364144360 0x7fc3640c5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 -- 172.21.15.164:0/2303520704 >> 172.21.15.164:0/2303520704 conn(0x7fc3640cf770 msgr2=0x7fc3640cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 -- 172.21.15.164:0/2303520704 shutdown_connections 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.308+0000 7fc369600640 1 -- 172.21.15.164:0/2303520704 wait complete. 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 Processor -- start 2024-05-08T04:42:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- start start 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3640cb290 0x7fc364071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc364144360 0x7fc3640ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406b620 con 0x7fc36406ac90 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406b4a0 con 0x7fc3640cb290 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640c4a60 con 0x7fc364144360 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47200/0 (socket says 172.21.15.164:47200) 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 -- 172.21.15.164:0/2614860114 learned_addr learned my addr 172.21.15.164:0/2614860114 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362000640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3640cb290 0x7fc364071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc363400640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc364144360 0x7fc3640ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 -- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc364144360 msgr2=0x7fc3640ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc364144360 0x7fc3640ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 -- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3640cb290 msgr2=0x7fc364071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3640cb290 0x7fc364071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 -- 172.21.15.164:0/2614860114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640b55f0 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc362a00640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc3540ae040 tx=0x7fc35401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b0050 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3540bc070 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540b5870 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc364072560 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc364139870 con 0x7fc36406ac90 2024-05-08T04:42:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.312+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364072d30 con 0x7fc36406ac90 2024-05-08T04:42:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.316+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fc3540b5a10 con 0x7fc36406ac90 2024-05-08T04:42:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.316+0000 7fc35aa00640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 0x7fc344069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.316+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc3541205f0 con 0x7fc36406ac90 2024-05-08T04:42:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.316+0000 7fc362000640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 0x7fc344069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.316+0000 7fc362000640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 0x7fc344069b70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc3641fa940 tx=0x7fc350009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.320+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc354016090 con 0x7fc36406ac90 2024-05-08T04:42:32.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.500+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp81090", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3640aee20 con 0x7fc3440676b0 2024-05-08T04:42:32.509 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:32.504+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:32.510 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:32.504+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:32.510 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:32.504+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:32.510 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:32.504+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:32.510 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:32.504+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:32.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.520+0000 7fc35aa00640 1 -- 172.21.15.164:0/2614860114 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3640aee20 con 0x7fc3440676b0 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 msgr2=0x7fc344069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 0x7fc344069b70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc3641fa940 tx=0x7fc350009040 comp rx=0 tx=0).stop 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 msgr2=0x7fc3640b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc3540ae040 tx=0x7fc35401ace0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 shutdown_connections 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc3440676b0 0x7fc344069b70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc364144360 0x7fc3640ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3640cb290 0x7fc364071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 --2- 172.21.15.164:0/2614860114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc36406ac90 0x7fc3640b50b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 >> 172.21.15.164:0/2614860114 conn(0x7fc3640cf770 msgr2=0x7fc36413c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:32.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 shutdown_connections 2024-05-08T04:42:32.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.524+0000 7fc369600640 1 -- 172.21.15.164:0/2614860114 wait complete. 2024-05-08T04:42:32.547 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol1540 --group_name subvol_grp81090 2024-05-08T04:42:32.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 Processor -- start 2024-05-08T04:42:32.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- start start 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ca80cdb10 0x7f0ca806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca80cfe60 con 0x7f0ca80ce930 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca80d0140 con 0x7f0ca80d0c20 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca80d0790 con 0x7f0ca80cdb10 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cae800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0ca80ce930 0x7f0ca80cbff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48200/0 (socket says 172.21.15.164:48200) 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cae800640 1 -- 172.21.15.164:0/836848612 learned_addr learned my addr 172.21.15.164:0/836848612 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cade00640 1 --2- 172.21.15.164:0/836848612 >> v2:172.21.15.172:3300/0 conn(0x7f0ca80cdb10 0x7f0ca806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0caca00640 1 -- 172.21.15.164:0/836848612 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873977462 0 0) 0x7f0ca80cfe60 con 0x7f0ca80ce930 2024-05-08T04:42:32.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0caca00640 1 -- 172.21.15.164:0/836848612 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c90003530 con 0x7f0ca80ce930 2024-05-08T04:42:32.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cade00640 1 -- 172.21.15.164:0/836848612 >> v1:172.21.15.172:6789/0 conn(0x7f0ca80d0c20 legacy=0x7f0ca80ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cade00640 1 -- 172.21.15.164:0/836848612 >> v1:172.21.15.164:6789/0 conn(0x7f0ca80ce930 legacy=0x7f0ca80cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cade00640 1 -- 172.21.15.164:0/836848612 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80d0420 con 0x7f0ca80cdb10 2024-05-08T04:42:32.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cade00640 1 --2- 172.21.15.164:0/836848612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca806efe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0097c0 tx=0x7f0c9c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f282601c50e228a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0caca00640 1 -- 172.21.15.164:0/836848612 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0ab430 con 0x7f0ca80cdb10 2024-05-08T04:42:32.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0caca00640 1 -- 172.21.15.164:0/836848612 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0b0070 con 0x7f0ca80cdb10 2024-05-08T04:42:32.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0caca00640 1 -- 172.21.15.164:0/836848612 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0ab7f0 con 0x7f0ca80cdb10 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- 172.21.15.164:0/836848612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 msgr2=0x7f0ca806efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 --2- 172.21.15.164:0/836848612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca806efe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0097c0 tx=0x7f0c9c01c410 comp rx=0 tx=0).stop 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- 172.21.15.164:0/836848612 shutdown_connections 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 --2- 172.21.15.164:0/836848612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca806efe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- 172.21.15.164:0/836848612 >> 172.21.15.164:0/836848612 conn(0x7f0ca80b34f0 msgr2=0x7f0ca80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- 172.21.15.164:0/836848612 shutdown_connections 2024-05-08T04:42:32.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.688+0000 7f0cb0800640 1 -- 172.21.15.164:0/836848612 wait complete. 2024-05-08T04:42:32.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 Processor -- start 2024-05-08T04:42:32.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- start start 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca81f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ca80ce930 0x7f0ca80b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca816c270 con 0x7f0ca80d0c20 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca816c0f0 con 0x7f0ca80cdb10 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca816c3f0 con 0x7f0ca80ce930 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47224/0 (socket says 172.21.15.164:47224) 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 -- 172.21.15.164:0/175908580 learned_addr learned my addr 172.21.15.164:0/175908580 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cade00640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca81f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cad400640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ca80ce930 0x7f0ca80b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 -- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ca80ce930 msgr2=0x7f0ca80b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ca80ce930 0x7f0ca80b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 -- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 msgr2=0x7f0ca81f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca81f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 -- 172.21.15.164:0/175908580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80b67b0 con 0x7f0ca80d0c20 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cae800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400cad0 tx=0x7f0ca400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4017070 con 0x7f0ca80d0c20 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca400ac30 con 0x7f0ca80d0c20 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80b4680 con 0x7f0ca80d0c20 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4012400 con 0x7f0ca80d0c20 2024-05-08T04:42:32.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca80b4c40 con 0x7f0ca80d0c20 2024-05-08T04:42:32.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca809fae0 con 0x7f0ca80d0c20 2024-05-08T04:42:32.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f0ca401a070 con 0x7f0ca80d0c20 2024-05-08T04:42:32.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 0x7f0c8c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:32.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0ca4085f00 con 0x7f0ca80d0c20 2024-05-08T04:42:32.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cade00640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 0x7f0c8c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:32.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.696+0000 7f0cade00640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 0x7f0c8c069b70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0af000 tx=0x7f0c9c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:32.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.700+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ca4055960 con 0x7f0ca80d0c20 2024-05-08T04:42:32.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.876+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol1540", "group_name": "subvol_grp81090", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ca804f010 con 0x7f0c8c0676b0 2024-05-08T04:42:32.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.900+0000 7f0ca2000640 1 -- 172.21.15.164:0/175908580 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ca804f010 con 0x7f0c8c0676b0 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 msgr2=0x7f0c8c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 0x7f0c8c069b70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0af000 tx=0x7f0c9c0a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 msgr2=0x7f0ca80c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400cad0 tx=0x7f0ca400cf90 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 shutdown_connections 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0c8c0676b0 0x7f0c8c069b70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ca80ce930 0x7f0ca80b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ca80d0c20 0x7f0ca80c76a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 --2- 172.21.15.164:0/175908580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ca80cdb10 0x7f0ca81f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 >> 172.21.15.164:0/175908580 conn(0x7f0ca80b34f0 msgr2=0x7f0ca8157040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 shutdown_connections 2024-05-08T04:42:32.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:32.908+0000 7f0cb0800640 1 -- 172.21.15.164:0/175908580 wait complete. 2024-05-08T04:42:32.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:32.927 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp81090/.snap/ancestral_snap_1) 2024-05-08T04:42:32.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:32.956 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp81090) 2024-05-08T04:42:33.006 INFO:teuthology.orchestra.run.smithi164.stdout:1099511627777 2024-05-08T04:42:33.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:33.007 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp81090/subvol1540/.snap/_ancestral_snap_1_1099511627777) 2024-05-08T04:42:33.064 INFO:teuthology.orchestra.run.smithi164.stdout:ff3bd628-e9d0-412a-b02c-454cc75f28ee 2024-05-08T04:42:33.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol1540 _ancestral_snap_1_1099511627777 --group_name subvol_grp81090 2024-05-08T04:42:33.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 Processor -- start 2024-05-08T04:42:33.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 -- start start 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 0x7fc4780ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47815e870 con 0x7fc47815ab10 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4780cf130 con 0x7fc47806cce0 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.252+0000 7fc47dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47806bb80 con 0x7fc4780cecc0 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:34292/0 (socket says 172.21.15.164:34292) 2024-05-08T04:42:33.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 -- 172.21.15.164:0/2977615525 learned_addr learned my addr 172.21.15.164:0/2977615525 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:33.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc476a00640 1 --1- 172.21.15.164:0/2977615525 >> v1:172.21.15.172:6789/0 conn(0x7fc47815ab10 0x7fc4780ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:55036/0 (socket says 172.21.15.164:55036) 2024-05-08T04:42:33.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc46e000640 1 --2- 172.21.15.164:0/2977615525 >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 0x7fc4780ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc476000640 1 -- 172.21.15.164:0/2977615525 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080417585 0 0) 0x7fc47815e870 con 0x7fc47815ab10 2024-05-08T04:42:33.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 -- 172.21.15.164:0/2977615525 >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 msgr2=0x7fc4780ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:33.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 --2- 172.21.15.164:0/2977615525 >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 0x7fc4780ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 -- 172.21.15.164:0/2977615525 >> v1:172.21.15.172:6789/0 conn(0x7fc47815ab10 legacy=0x7fc4780ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 -- 172.21.15.164:0/2977615525 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47815eb50 con 0x7fc4780cecc0 2024-05-08T04:42:33.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc46e000640 1 --2- 172.21.15.164:0/2977615525 >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 0x7fc4780ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:33.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.256+0000 7fc477400640 1 --2- 172.21.15.164:0/2977615525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc470009e50 tx=0x7fc47001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d76556c9209145 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc476000640 1 -- 172.21.15.164:0/2977615525 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4700a5740 con 0x7fc4780cecc0 2024-05-08T04:42:33.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc476000640 1 -- 172.21.15.164:0/2977615525 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4700a58e0 con 0x7fc4780cecc0 2024-05-08T04:42:33.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc476000640 1 -- 172.21.15.164:0/2977615525 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4700a5cb0 con 0x7fc4780cecc0 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- 172.21.15.164:0/2977615525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 msgr2=0x7fc4780ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- 172.21.15.164:0/2977615525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc470009e50 tx=0x7fc47001c970 comp rx=0 tx=0).stop 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- 172.21.15.164:0/2977615525 shutdown_connections 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- 172.21.15.164:0/2977615525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4780cecc0 0x7fc4780ca6e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- 172.21.15.164:0/2977615525 >> v2:172.21.15.172:3300/0 conn(0x7fc47806cce0 0x7fc4780ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- 172.21.15.164:0/2977615525 >> 172.21.15.164:0/2977615525 conn(0x7fc4780cf770 msgr2=0x7fc4780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- 172.21.15.164:0/2977615525 shutdown_connections 2024-05-08T04:42:33.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- 172.21.15.164:0/2977615525 wait complete. 2024-05-08T04:42:33.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 Processor -- start 2024-05-08T04:42:33.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- start start 2024-05-08T04:42:33.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47806cce0 0x7fc478158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4780cecc0 0x7fc4781f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47815ab10 0x7fc478071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780aedf0 con 0x7fc4780cecc0 2024-05-08T04:42:33.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47815aef0 con 0x7fc47815ab10 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc47dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780aef70 con 0x7fc47806cce0 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc46e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4780cecc0 0x7fc4781f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc46e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4780cecc0 0x7fc4781f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47236/0 (socket says 172.21.15.164:47236) 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc46e000640 1 -- 172.21.15.164:0/3927531713 learned_addr learned my addr 172.21.15.164:0/3927531713 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc476a00640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47806cce0 0x7fc478158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.260+0000 7fc477400640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47815ab10 0x7fc478071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 -- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47806cce0 msgr2=0x7fc478158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc47806cce0 0x7fc478158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 -- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47815ab10 msgr2=0x7fc478071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc47815ab10 0x7fc478071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 -- 172.21.15.164:0/3927531713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4781f78b0 con 0x7fc4780cecc0 2024-05-08T04:42:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46e000640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4780cecc0 0x7fc4781f84b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9b0 tx=0x7fc46800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468017070 con 0x7fc4780cecc0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800c9a0 con 0x7fc4780cecc0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800cd70 con 0x7fc4780cecc0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc47dc00640 1 -- 172.21.15.164:0/3927531713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4781f7b90 con 0x7fc4780cecc0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc47dc00640 1 -- 172.21.15.164:0/3927531713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4781f8150 con 0x7fc4780cecc0 2024-05-08T04:42:33.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc47dc00640 1 -- 172.21.15.164:0/3927531713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc478072d30 con 0x7fc4780cecc0 2024-05-08T04:42:33.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7fc46801a070 con 0x7fc4780cecc0 2024-05-08T04:42:33.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc4540676b0 0x7fc454069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc4680863d0 con 0x7fc4780cecc0 2024-05-08T04:42:33.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.264+0000 7fc476a00640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc4540676b0 0x7fc454069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.268+0000 7fc476a00640 1 --2- 172.21.15.164:0/3927531713 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc4540676b0 0x7fc454069b70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc460000f70 tx=0x7fc460007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.268+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc468055df0 con 0x7fc4780cecc0 2024-05-08T04:42:33.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.452+0000 7fc47dc00640 1 -- 172.21.15.164:0/3927531713 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol1540", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp81090", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4780c9b30 con 0x7fc4540676b0 2024-05-08T04:42:33.473 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.472+0000 7ffa56200640 -1 mgr.server reply reply (22) Invalid argument error in rmdir /volumes/subvol_grp81090/subvol1540/.snap/_ancestral_snap_1_1099511627777 2024-05-08T04:42:33.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.472+0000 7fc46f400640 1 -- 172.21.15.164:0/3927531713 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -22 error in rmdir /volumes/subvol_grp81090/subvol1540/.snap/_ancestral_snap_1_1099511627777) v1 ==== 96+0+0 (secure 0 0 0) 0x7fc4780c9b30 con 0x7fc4540676b0 2024-05-08T04:42:33.474 INFO:teuthology.orchestra.run.smithi164.stderr:Error EINVAL: error in rmdir /volumes/subvol_grp81090/subvol1540/.snap/_ancestral_snap_1_1099511627777 2024-05-08T04:42:33.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T04:42:33.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:33.483 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp81090/.snap/ancestral_snap_1) 2024-05-08T04:42:33.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol1540 subvol_grp81090 2024-05-08T04:42:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.688+0000 7f2a43000640 1 Processor -- start 2024-05-08T04:42:33.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.688+0000 7f2a43000640 1 -- start start 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a43000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a3c15ce10 0x7f2a3c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a43000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a43000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0c6600 con 0x7f2a3c15ba20 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a43000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c15fbe0 con 0x7f2a3c06e390 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a43000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0c6040 con 0x7f2a3c15ce10 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47250/0 (socket says 172.21.15.164:47250) 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 -- 172.21.15.164:0/3243440498 learned_addr learned my addr 172.21.15.164:0/3243440498 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a33400640 1 --2- 172.21.15.164:0/3243440498 >> v2:172.21.15.172:3300/0 conn(0x7f2a3c15ce10 0x7f2a3c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 -- 172.21.15.164:0/3243440498 >> v2:172.21.15.172:3300/0 conn(0x7f2a3c15ce10 msgr2=0x7f2a3c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:33.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 --2- 172.21.15.164:0/3243440498 >> v2:172.21.15.172:3300/0 conn(0x7f2a3c15ce10 0x7f2a3c06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 -- 172.21.15.164:0/3243440498 >> v1:172.21.15.164:6789/0 conn(0x7f2a3c15ba20 legacy=0x7f2a3c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a41000640 1 -- 172.21.15.164:0/3243440498 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0c6320 con 0x7f2a3c06e390 2024-05-08T04:42:33.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.692+0000 7f2a3b400640 1 -- 172.21.15.164:0/3243440498 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266998189 0 0) 0x7f2a3c0c6600 con 0x7f2a3c15ba20 2024-05-08T04:42:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a41000640 1 --2- 172.21.15.164:0/3243440498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0b5980 tx=0x7f2a3401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7cf2402c166d9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a3b400640 1 -- 172.21.15.164:0/3243440498 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340a56c0 con 0x7f2a3c06e390 2024-05-08T04:42:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a3b400640 1 -- 172.21.15.164:0/3243440498 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340a5860 con 0x7f2a3c06e390 2024-05-08T04:42:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a3b400640 1 -- 172.21.15.164:0/3243440498 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340a5c30 con 0x7f2a3c06e390 2024-05-08T04:42:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- 172.21.15.164:0/3243440498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 msgr2=0x7f2a3c0ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- 172.21.15.164:0/3243440498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0b5980 tx=0x7f2a3401c9f0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- 172.21.15.164:0/3243440498 shutdown_connections 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- 172.21.15.164:0/3243440498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c0ae6e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- 172.21.15.164:0/3243440498 >> v2:172.21.15.172:3300/0 conn(0x7f2a3c15ce10 0x7f2a3c06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- 172.21.15.164:0/3243440498 >> 172.21.15.164:0/3243440498 conn(0x7f2a3c0cf770 msgr2=0x7f2a3c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- 172.21.15.164:0/3243440498 shutdown_connections 2024-05-08T04:42:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- 172.21.15.164:0/3243440498 wait complete. 2024-05-08T04:42:33.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 Processor -- start 2024-05-08T04:42:33.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- start start 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a3c15ba20 0x7f2a3c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 0x7f2a3c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c168b70 con 0x7f2a3c06e390 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0ce090 con 0x7f2a3c15ce10 2024-05-08T04:42:33.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.696+0000 7f2a43000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c168cf0 con 0x7f2a3c15ba20 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47264/0 (socket says 172.21.15.164:47264) 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 -- 172.21.15.164:0/429605138 learned_addr learned my addr 172.21.15.164:0/429605138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a33400640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a3c15ba20 0x7f2a3c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a41000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 0x7f2a3c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 -- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a3c15ba20 msgr2=0x7f2a3c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a3c15ba20 0x7f2a3c0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 -- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 msgr2=0x7f2a3c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 0x7f2a3c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 -- 172.21.15.164:0/429605138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0c9560 con 0x7f2a3c06e390 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a41000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 0x7f2a3c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a3be00640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c4d0 tx=0x7f2a2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c017070 con 0x7f2a3c06e390 2024-05-08T04:42:33.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c005480 con 0x7f2a3c06e390 2024-05-08T04:42:33.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0c9840 con 0x7f2a3c06e390 2024-05-08T04:42:33.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c003840 con 0x7f2a3c06e390 2024-05-08T04:42:33.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c151000 con 0x7f2a3c06e390 2024-05-08T04:42:33.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c072d30 con 0x7f2a3c06e390 2024-05-08T04:42:33.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f2a2c01a070 con 0x7f2a3c06e390 2024-05-08T04:42:33.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.700+0000 7f2a38c00640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 0x7f2a18069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:33.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.704+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2a2c085580 con 0x7f2a3c06e390 2024-05-08T04:42:33.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.704+0000 7f2a33400640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 0x7f2a18069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:33.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.704+0000 7f2a33400640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 0x7f2a18069b70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c15a6f0 tx=0x7f2a24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:33.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.708+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a2c055740 con 0x7f2a3c06e390 2024-05-08T04:42:33.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.884+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol1540", "group_name": "subvol_grp81090", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a3c04f010 con 0x7f2a180676b0 2024-05-08T04:42:33.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.896+0000 7f2a38c00640 1 -- 172.21.15.164:0/429605138 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2a3c04f010 con 0x7f2a180676b0 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 msgr2=0x7f2a18069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 0x7f2a18069b70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c15a6f0 tx=0x7f2a24008040 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 msgr2=0x7f2a3c159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c4d0 tx=0x7f2a2c00c990 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 shutdown_connections 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a180676b0 0x7f2a18069b70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a3c15ba20 0x7f2a3c0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a3c15ce10 0x7f2a3c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 --2- 172.21.15.164:0/429605138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a3c06e390 0x7f2a3c159bf0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:33.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 >> 172.21.15.164:0/429605138 conn(0x7f2a3c0cf770 msgr2=0x7f2a3c0c5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:33.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 shutdown_connections 2024-05-08T04:42:33.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:33.900+0000 7f2a43000640 1 -- 172.21.15.164:0/429605138 wait complete. 2024-05-08T04:42:33.904 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.904+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.904 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.904+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.905 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.904+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.905 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.904+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.905 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.904+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.921 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.920+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.921 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.920+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.921 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.920+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.921 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.920+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.921 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:42:33.920+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:42:33.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:33.927 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:42:33.941 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:42:33.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp81090 2024-05-08T04:42:34.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 Processor -- start 2024-05-08T04:42:34.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 -- start start 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f668c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0cfe60 con 0x7f668c0ce760 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0d0140 con 0x7f668c0cdb10 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6692e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0d0790 con 0x7f668c0d0c20 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6690e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f668c0ce760 0x7f668c0cbff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48218/0 (socket says 172.21.15.164:48218) 2024-05-08T04:42:34.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6690e00640 1 -- 172.21.15.164:0/2426810913 learned_addr learned my addr 172.21.15.164:0/2426810913 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:34.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f6683400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f668c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034311713 0 0) 0x7f668c0cfe60 con 0x7f668c0ce760 2024-05-08T04:42:34.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6670003530 con 0x7f668c0ce760 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134553939 0 0) 0x7f6670003530 con 0x7f668c0ce760 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 msgr2=0x7f668c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f668c0ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 >> v1:172.21.15.172:6789/0 conn(0x7f668c0cdb10 legacy=0x7f668c06efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.120+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0d0420 con 0x7f668c0ce760 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f6684001a00 con 0x7f668c0ce760 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 >> v1:172.21.15.164:6789/0 conn(0x7f668c0ce760 legacy=0x7f668c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66700039f0 0x7f6670003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700043a0 0x7f6670004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f6670005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0d0790 con 0x7f66700039f0 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667000ba80 con 0x7f66700043a0 2024-05-08T04:42:34.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667000bc80 con 0x7f668c0d0c20 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66700039f0 0x7f6670003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f6690e00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f6670005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f6683400640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700043a0 0x7f6670004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 -- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 msgr2=0x7f6670005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c0d0c20 0x7f6670005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 -- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700043a0 msgr2=0x7f6670004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700043a0 0x7f6670004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 -- 172.21.15.164:0/2426810913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670003530 con 0x7f66700039f0 2024-05-08T04:42:34.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668be00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66700039f0 0x7f6670003dd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f667c001860 tx=0x7f667c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a502cfb1a31a9442 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0b0070 con 0x7f66700039f0 2024-05-08T04:42:34.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0ab620 con 0x7f66700039f0 2024-05-08T04:42:34.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.124+0000 7f668b400640 1 -- 172.21.15.164:0/2426810913 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c001460 con 0x7f66700039f0 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- 172.21.15.164:0/2426810913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66700039f0 msgr2=0x7f6670003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 --2- 172.21.15.164:0/2426810913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66700039f0 0x7f6670003dd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f667c001860 tx=0x7f667c01c520 comp rx=0 tx=0).stop 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 -- 172.21.15.164:0/2426810913 reap_dead start 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- 172.21.15.164:0/2426810913 shutdown_connections 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- 172.21.15.164:0/2426810913 >> 172.21.15.164:0/2426810913 conn(0x7f668c0b34f0 msgr2=0x7f668c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- 172.21.15.164:0/2426810913 shutdown_connections 2024-05-08T04:42:34.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- 172.21.15.164:0/2426810913 wait complete. 2024-05-08T04:42:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 Processor -- start 2024-05-08T04:42:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- start start 2024-05-08T04:42:34.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700039f0 0x7f668c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c14c500 0x7f668c14f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c169ed0 con 0x7f668c14c960 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0ceb90 con 0x7f66700039f0 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6692e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c16a050 con 0x7f668c14c500 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47316/0 (socket says 172.21.15.164:47316) 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6683400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c14c500 0x7f668c14f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 -- 172.21.15.164:0/3199123393 learned_addr learned my addr 172.21.15.164:0/3199123393 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f668be00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700039f0 0x7f668c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 -- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c14c500 msgr2=0x7f668c14f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c14c500 0x7f668c14f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 -- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700039f0 msgr2=0x7f668c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700039f0 0x7f668c0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.128+0000 7f6690e00640 1 -- 172.21.15.164:0/3199123393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0b6c30 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6690e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f668c06df50 tx=0x7f668400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400ec10 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684018070 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c0c8bf0 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684013420 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c0b6ed0 con 0x7f668c14c960 2024-05-08T04:42:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c0b70d0 con 0x7f668c14c960 2024-05-08T04:42:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 87390+0+0 (secure 0 0 0) 0x7f66840135c0 con 0x7f668c14c960 2024-05-08T04:42:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 0x7f6668069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f6684086440 con 0x7f668c14c960 2024-05-08T04:42:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.132+0000 7f668be00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 0x7f6668069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.136+0000 7f668be00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 0x7f6668069ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ad000 tx=0x7f667c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.136+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f668c0b70d0 con 0x7f668c14c960 2024-05-08T04:42:34.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.316+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp81090", "target": ["mon-mgr", ""]}) v1 -- 0x7f668c09fae0 con 0x7f66680676e0 2024-05-08T04:42:34.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.316+0000 7f6688c00640 1 -- 172.21.15.164:0/3199123393 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f668c09fae0 con 0x7f66680676e0 2024-05-08T04:42:34.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 msgr2=0x7f6668069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 0x7f6668069ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ad000 tx=0x7f667c0a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:34.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 msgr2=0x7f668c14f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f668c06df50 tx=0x7f668400e530 comp rx=0 tx=0).stop 2024-05-08T04:42:34.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 shutdown_connections 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f66680676e0 0x7f6668069ba0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f668c14c500 0x7f668c14f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f668c14c960 0x7f668c14f6a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 --2- 172.21.15.164:0/3199123393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66700039f0 0x7f668c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 >> 172.21.15.164:0/3199123393 conn(0x7f668c0b34f0 msgr2=0x7f668c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 shutdown_connections 2024-05-08T04:42:34.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.320+0000 7f6692e00640 1 -- 172.21.15.164:0/3199123393 wait complete. 2024-05-08T04:42:34.341 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:42:34.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:34.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:34.529 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:42:34.529 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:34.529 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:34.538 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:34.539 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:34.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:34.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 Processor -- start 2024-05-08T04:42:34.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 -- start start 2024-05-08T04:42:34.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9400cfda0 0x7fe940160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9400b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9400cd9f0 con 0x7fe94015ba70 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94015fe70 con 0x7fe9400cfda0 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe944e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400cdcd0 con 0x7fe94015be50 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe93de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe94015ba70 0x7fe9400cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48230/0 (socket says 172.21.15.164:48230) 2024-05-08T04:42:34.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.768+0000 7fe93de00640 1 -- 172.21.15.164:0/3031686913 learned_addr learned my addr 172.21.15.164:0/3031686913 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93e800640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9400b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93d400640 1 --2- 172.21.15.164:0/3031686913 >> v2:172.21.15.172:3300/0 conn(0x7fe9400cfda0 0x7fe940160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720407792 0 0) 0x7fe9400cd9f0 con 0x7fe94015ba70 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe924003530 con 0x7fe94015ba70 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091600901 0 0) 0x7fe924003530 con 0x7fe94015ba70 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 msgr2=0x7fe9400b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9400b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 >> v2:172.21.15.172:3300/0 conn(0x7fe9400cfda0 msgr2=0x7fe940160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 --2- 172.21.15.164:0/3031686913 >> v2:172.21.15.172:3300/0 conn(0x7fe9400cfda0 0x7fe940160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94015f820 con 0x7fe94015ba70 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fe930001a00 con 0x7fe94015ba70 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 >> v1:172.21.15.164:6789/0 conn(0x7fe94015ba70 legacy=0x7fe9400cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9240039c0 0x7fe924003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe924004810 0x7fe924004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015fe70 con 0x7fe9240039c0 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400cdcd0 con 0x7fe924004810 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92400b9d0 con 0x7fe94015be50 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9240039c0 0x7fe924003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93d400640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe924004810 0x7fe924004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93e800640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 -- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 msgr2=0x7fe9240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 -- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe924004810 msgr2=0x7fe924004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe924004810 0x7fe924004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93de00640 1 -- 172.21.15.164:0/3031686913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe924003530 con 0x7fe9240039c0 2024-05-08T04:42:34.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.772+0000 7fe93e800640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015be50 0x7fe9240051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe93de00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9240039c0 0x7fe924003de0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe93000a870 tx=0x7fe93001d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757af0bfdedd24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000a4c0 con 0x7fe9240039c0 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9300ad630 con 0x7fe9240039c0 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe93ca00640 1 -- 172.21.15.164:0/3031686913 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300ada20 con 0x7fe9240039c0 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- 172.21.15.164:0/3031686913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9240039c0 msgr2=0x7fe924003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 --2- 172.21.15.164:0/3031686913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9240039c0 0x7fe924003de0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe93000a870 tx=0x7fe93001d370 comp rx=0 tx=0).stop 2024-05-08T04:42:34.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe93e800640 1 -- 172.21.15.164:0/3031686913 reap_dead start 2024-05-08T04:42:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- 172.21.15.164:0/3031686913 shutdown_connections 2024-05-08T04:42:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- 172.21.15.164:0/3031686913 >> 172.21.15.164:0/3031686913 conn(0x7fe94006ac90 msgr2=0x7fe94006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- 172.21.15.164:0/3031686913 shutdown_connections 2024-05-08T04:42:34.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- 172.21.15.164:0/3031686913 wait complete. 2024-05-08T04:42:34.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 Processor -- start 2024-05-08T04:42:34.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- start start 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 0x7fe940144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94006d350 0x7fe94006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400b3d40 con 0x7fe9400b0bc0 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400b3bc0 con 0x7fe9240039c0 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.776+0000 7fe944e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400b3ec0 con 0x7fe94006d350 2024-05-08T04:42:34.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47330/0 (socket says 172.21.15.164:47330) 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 -- 172.21.15.164:0/2915747730 learned_addr learned my addr 172.21.15.164:0/2915747730 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93de00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 0x7fe940144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93e800640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94006d350 0x7fe94006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 -- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94006d350 msgr2=0x7fe94006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94006d350 0x7fe94006d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 -- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 msgr2=0x7fe940144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 0x7fe940144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 -- 172.21.15.164:0/2915747730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe940144c10 con 0x7fe9400b0bc0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93de00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 0x7fe940144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe93d400640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011a30 tx=0x7fe92c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00fb50 con 0x7fe9400b0bc0 2024-05-08T04:42:34.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c00fcf0 con 0x7fe9400b0bc0 2024-05-08T04:42:34.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe940144ef0 con 0x7fe9400b0bc0 2024-05-08T04:42:34.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00fea0 con 0x7fe9400b0bc0 2024-05-08T04:42:34.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9401454b0 con 0x7fe9400b0bc0 2024-05-08T04:42:34.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94006dd10 con 0x7fe9400b0bc0 2024-05-08T04:42:34.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 14) v1 ==== 87441+0+0 (secure 0 0 0) 0x7fe92c012040 con 0x7fe9400b0bc0 2024-05-08T04:42:34.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.780+0000 7fe936000640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 0x7fe920069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:34.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.784+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe92c086920 con 0x7fe9400b0bc0 2024-05-08T04:42:34.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.784+0000 7fe93de00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 0x7fe920069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:34.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.784+0000 7fe93de00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 0x7fe920069c00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ad000 tx=0x7fe930001370 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:34.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.788+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe92c019030 con 0x7fe9400b0bc0 2024-05-08T04:42:34.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.940+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe940069d60 con 0x7fe9400b0bc0 2024-05-08T04:42:34.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe936000640 1 -- 172.21.15.164:0/2915747730 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16802 (secure 0 0 0) 0x7fe92c056970 con 0x7fe9400b0bc0 2024-05-08T04:42:34.944 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:34.944 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":99,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:23.416521+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:34.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 msgr2=0x7fe920069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 0x7fe920069c00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ad000 tx=0x7fe930001370 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 msgr2=0x7fe9400b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011a30 tx=0x7fe92c011ef0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 shutdown_connections 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067740 0x7fe920069c00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94006d350 0x7fe94006d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400b0bc0 0x7fe9400b1040 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 --2- 172.21.15.164:0/2915747730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9240039c0 0x7fe940144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 >> 172.21.15.164:0/2915747730 conn(0x7fe94006ac90 msgr2=0x7fe9400cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 shutdown_connections 2024-05-08T04:42:34.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:34.944+0000 7fe944e00640 1 -- 172.21.15.164:0/2915747730 wait complete. 2024-05-08T04:42:34.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:20.161961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:20.781940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:34.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:35.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 Processor -- start 2024-05-08T04:42:35.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 -- start start 2024-05-08T04:42:35.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46bc0ca850 0x7f46bc0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46bc06dfa0 0x7f46bc0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46bc06d150 con 0x7f46bc15a200 2024-05-08T04:42:35.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46bc06bca0 con 0x7f46bc0ca850 2024-05-08T04:42:35.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.104+0000 7f46c4000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46bc0af150 con 0x7f46bc06dfa0 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46bc0ca850 0x7f46bc0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46bc0ca850 0x7f46bc0aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47344/0 (socket says 172.21.15.164:47344) 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 -- 172.21.15.164:0/3023488233 learned_addr learned my addr 172.21.15.164:0/3023488233 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c2000640 1 --2- 172.21.15.164:0/3023488233 >> v2:172.21.15.172:3300/0 conn(0x7f46bc06dfa0 0x7f46bc0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 -- 172.21.15.164:0/3023488233 >> v2:172.21.15.172:3300/0 conn(0x7f46bc06dfa0 msgr2=0x7f46bc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 --2- 172.21.15.164:0/3023488233 >> v2:172.21.15.172:3300/0 conn(0x7f46bc06dfa0 0x7f46bc0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 -- 172.21.15.164:0/3023488233 >> v1:172.21.15.172:6789/0 conn(0x7f46bc15a200 legacy=0x7f46bc0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 -- 172.21.15.164:0/3023488233 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc06d430 con 0x7f46bc0ca850 2024-05-08T04:42:35.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46c0c00640 1 --2- 172.21.15.164:0/3023488233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc0aeac0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cf1b0 tx=0x7f46a801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66c6da97778dbee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46bbe00640 1 -- 172.21.15.164:0/3023488233 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80b4070 con 0x7f46bc0ca850 2024-05-08T04:42:35.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46bbe00640 1 -- 172.21.15.164:0/3023488233 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80a7b80 con 0x7f46bc0ca850 2024-05-08T04:42:35.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.108+0000 7f46bbe00640 1 -- 172.21.15.164:0/3023488233 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80af440 con 0x7f46bc0ca850 2024-05-08T04:42:35.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- 172.21.15.164:0/3023488233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 msgr2=0x7f46bc0aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- 172.21.15.164:0/3023488233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc0aeac0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cf1b0 tx=0x7f46a801eb30 comp rx=0 tx=0).stop 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- 172.21.15.164:0/3023488233 shutdown_connections 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- 172.21.15.164:0/3023488233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc0aeac0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- 172.21.15.164:0/3023488233 >> v2:172.21.15.172:3300/0 conn(0x7f46bc06dfa0 0x7f46bc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- 172.21.15.164:0/3023488233 >> 172.21.15.164:0/3023488233 conn(0x7f46bc0cf770 msgr2=0x7f46bc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- 172.21.15.164:0/3023488233 shutdown_connections 2024-05-08T04:42:35.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- 172.21.15.164:0/3023488233 wait complete. 2024-05-08T04:42:35.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 Processor -- start 2024-05-08T04:42:35.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- start start 2024-05-08T04:42:35.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 0x7f46bc1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46bc15a200 0x7f46bc0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc0cdd30 con 0x7f46bc0ca850 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc0cdbb0 con 0x7f46bc15a200 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c4000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc0cdeb0 con 0x7f46bc06dfa0 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47348/0 (socket says 172.21.15.164:47348) 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 -- 172.21.15.164:0/1965430217 learned_addr learned my addr 172.21.15.164:0/1965430217 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c2000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46bc15a200 0x7f46bc0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c1600640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 0x7f46bc1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 -- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 msgr2=0x7f46bc1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 0x7f46bc1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 -- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46bc15a200 msgr2=0x7f46bc0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46bc15a200 0x7f46bc0707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 -- 172.21.15.164:0/1965430217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc0b2440 con 0x7f46bc0ca850 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c1600640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 0x7f46bc1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.112+0000 7f46c0c00640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f46a801b740 tx=0x7f46a801b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80b4030 con 0x7f46bc0ca850 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80b6070 con 0x7f46bc0ca850 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc0b26e0 con 0x7f46bc0ca850 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80ad550 con 0x7f46bc0ca850 2024-05-08T04:42:35.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc0b2ca0 con 0x7f46bc0ca850 2024-05-08T04:42:35.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46bc072d30 con 0x7f46bc0ca850 2024-05-08T04:42:35.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 14) v1 ==== 87441+0+0 (secure 0 0 0) 0x7f46a80c1060 con 0x7f46bc0ca850 2024-05-08T04:42:35.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 0x7f469c069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f46a8120370 con 0x7f46bc0ca850 2024-05-08T04:42:35.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.116+0000 7f46c1600640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 0x7f469c069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.120+0000 7f46c1600640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 0x7f469c069c00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f46b0007fd0 tx=0x7f46b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.120+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46a80187e0 con 0x7f46bc0ca850 2024-05-08T04:42:35.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.280+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46bc04f010 con 0x7f46bc0ca850 2024-05-08T04:42:35.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.280+0000 7f46b9600640 1 -- 172.21.15.164:0/1965430217 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46a80f0150 con 0x7f46bc0ca850 2024-05-08T04:42:35.282 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:35.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 msgr2=0x7f469c069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 0x7f469c069c00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f46b0007fd0 tx=0x7f46b0009040 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 msgr2=0x7f46bc070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f46a801b740 tx=0x7f46a801b8d0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 shutdown_connections 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f469c067740 0x7f469c069c00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46bc15a200 0x7f46bc0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc0ca850 0x7f46bc070270 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 --2- 172.21.15.164:0/1965430217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46bc06dfa0 0x7f46bc1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 >> 172.21.15.164:0/1965430217 conn(0x7f46bc0cf770 msgr2=0x7f46bc0af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 shutdown_connections 2024-05-08T04:42:35.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.284+0000 7f46c4000640 1 -- 172.21.15.164:0/1965430217 wait complete. 2024-05-08T04:42:35.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:35.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 Processor -- start 2024-05-08T04:42:35.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 -- start start 2024-05-08T04:42:35.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147806d150 con 0x7f14780ca140 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147806d430 con 0x7f147806dfa0 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.440+0000 7f147e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147806bca0 con 0x7f14780aeac0 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:34434/0 (socket says 172.21.15.164:34434) 2024-05-08T04:42:35.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 -- 172.21.15.164:0/2314134827 learned_addr learned my addr 172.21.15.164:0/2314134827 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:35.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476000640 1 -- 172.21.15.164:0/2314134827 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693783286 0 0) 0x7f147806d150 con 0x7f14780ca140 2024-05-08T04:42:35.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476000640 1 -- 172.21.15.164:0/2314134827 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f145c005740 con 0x7f14780ca140 2024-05-08T04:42:35.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 -- 172.21.15.164:0/2314134827 >> v1:172.21.15.172:6789/0 conn(0x7f147806dfa0 legacy=0x7f147815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 -- 172.21.15.164:0/2314134827 >> v1:172.21.15.164:6789/0 conn(0x7f14780ca140 legacy=0x7f1478164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.444+0000 7f1476a00640 1 -- 172.21.15.164:0/2314134827 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147806d710 con 0x7f14780aeac0 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f1476a00640 1 --2- 172.21.15.164:0/2314134827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14640098a0 tx=0x7f146401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffdfda19835473d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f1476000640 1 -- 172.21.15.164:0/2314134827 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640b0070 con 0x7f14780aeac0 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f1476000640 1 -- 172.21.15.164:0/2314134827 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14640ab600 con 0x7f14780aeac0 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f1476000640 1 -- 172.21.15.164:0/2314134827 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640ab9d0 con 0x7f14780aeac0 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- 172.21.15.164:0/2314134827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 msgr2=0x7f147806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 --2- 172.21.15.164:0/2314134827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14640098a0 tx=0x7f146401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- 172.21.15.164:0/2314134827 shutdown_connections 2024-05-08T04:42:35.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 --2- 172.21.15.164:0/2314134827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f147806ac90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- 172.21.15.164:0/2314134827 >> 172.21.15.164:0/2314134827 conn(0x7f14780cf770 msgr2=0x7f14780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- 172.21.15.164:0/2314134827 shutdown_connections 2024-05-08T04:42:35.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- 172.21.15.164:0/2314134827 wait complete. 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 Processor -- start 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.448+0000 7f147e200640 1 -- start start 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 0x7f1478151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f1478157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780ce000 con 0x7f14780ca140 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780cde80 con 0x7f147806dfa0 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780ce180 con 0x7f14780aeac0 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47362/0 (socket says 172.21.15.164:47362) 2024-05-08T04:42:35.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 -- 172.21.15.164:0/2628525667 learned_addr learned my addr 172.21.15.164:0/2628525667 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1476a00640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f1478157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477400640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 0x7f1478151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 -- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 msgr2=0x7f1478157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f1478157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 -- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 msgr2=0x7f1478151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 0x7f1478151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 -- 172.21.15.164:0/2628525667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780d06f0 con 0x7f14780ca140 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477400640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 0x7f1478151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f1477e00640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f147000c730 tx=0x7f147000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470003b80 con 0x7f14780ca140 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470003d20 con 0x7f14780ca140 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1478150780 con 0x7f14780ca140 2024-05-08T04:42:35.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000aa90 con 0x7f14780ca140 2024-05-08T04:42:35.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1478150d40 con 0x7f14780ca140 2024-05-08T04:42:35.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.452+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478072d30 con 0x7f14780ca140 2024-05-08T04:42:35.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.456+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 14) v1 ==== 87441+0+0 (secure 0 0 0) 0x7f147000ac30 con 0x7f14780ca140 2024-05-08T04:42:35.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.456+0000 7f146f400640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 0x7f1458069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.456+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1470086290 con 0x7f14780ca140 2024-05-08T04:42:35.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.456+0000 7f1477400640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 0x7f1458069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.456+0000 7f1477400640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 0x7f1458069c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1468001710 tx=0x7f1468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.460+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1470056220 con 0x7f14780ca140 2024-05-08T04:42:35.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.616+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f147804f010 con 0x7f14780ca140 2024-05-08T04:42:35.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.616+0000 7f146f400640 1 -- 172.21.15.164:0/2628525667 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1470055b20 con 0x7f14780ca140 2024-05-08T04:42:35.618 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 msgr2=0x7f1458069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 0x7f1458069c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1468001710 tx=0x7f1468009040 comp rx=0 tx=0).stop 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 msgr2=0x7f1478159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f147000c730 tx=0x7f147000cbf0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 shutdown_connections 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1458067740 0x7f1458069c00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14780aeac0 0x7f1478157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14780ca140 0x7f1478159470 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 --2- 172.21.15.164:0/2628525667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f147806dfa0 0x7f1478151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 >> 172.21.15.164:0/2628525667 conn(0x7f14780cf770 msgr2=0x7f147815a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 shutdown_connections 2024-05-08T04:42:35.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.620+0000 7f147e200640 1 -- 172.21.15.164:0/2628525667 wait complete. 2024-05-08T04:42:35.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.776+0000 7fa81b200640 1 Processor -- start 2024-05-08T04:42:35.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.776+0000 7fa81b200640 1 -- start start 2024-05-08T04:42:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa81c0b13d0 0x7fa81c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 0x7fa81c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c0b2940 con 0x7fa81c157ce0 2024-05-08T04:42:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c072390 con 0x7fa81c0b13d0 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0b2c20 con 0x7fa81c071f40 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa81c0b13d0 0x7fa81c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa81c157ce0 0x7fa81c0b1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48252/0 (socket says 172.21.15.164:48252) 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819e00640 1 -- 172.21.15.164:0/2277283966 learned_addr learned my addr 172.21.15.164:0/2277283966 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa81c0b13d0 0x7fa81c071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47376/0 (socket says 172.21.15.164:47376) 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 -- 172.21.15.164:0/2277283966 >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 msgr2=0x7fa81c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81a800640 1 --2- 172.21.15.164:0/2277283966 >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 0x7fa81c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa818a00640 1 -- 172.21.15.164:0/2277283966 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359690410 0 0) 0x7fa81c0b2940 con 0x7fa81c157ce0 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 --2- 172.21.15.164:0/2277283966 >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 0x7fa81c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 -- 172.21.15.164:0/2277283966 >> v1:172.21.15.164:6789/0 conn(0x7fa81c157ce0 legacy=0x7fa81c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 -- 172.21.15.164:0/2277283966 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0b2f00 con 0x7fa81c0b13d0 2024-05-08T04:42:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa81a800640 1 --2- 172.21.15.164:0/2277283966 >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 0x7fa81c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.780+0000 7fa819400640 1 --2- 172.21.15.164:0/2277283966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c0b13d0 0x7fa81c071a00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0cf4e0 tx=0x7fa80801e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c5efabe8df156a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa818a00640 1 -- 172.21.15.164:0/2277283966 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080a86e0 con 0x7fa81c0b13d0 2024-05-08T04:42:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa818a00640 1 -- 172.21.15.164:0/2277283966 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8080a8880 con 0x7fa81c0b13d0 2024-05-08T04:42:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa818a00640 1 -- 172.21.15.164:0/2277283966 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080a8c50 con 0x7fa81c0b13d0 2024-05-08T04:42:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- 172.21.15.164:0/2277283966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c0b13d0 msgr2=0x7fa81c071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- 172.21.15.164:0/2277283966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c0b13d0 0x7fa81c071a00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0cf4e0 tx=0x7fa80801e880 comp rx=0 tx=0).stop 2024-05-08T04:42:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- 172.21.15.164:0/2277283966 shutdown_connections 2024-05-08T04:42:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- 172.21.15.164:0/2277283966 >> v2:172.21.15.172:3300/0 conn(0x7fa81c071f40 0x7fa81c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- 172.21.15.164:0/2277283966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c0b13d0 0x7fa81c071a00 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- 172.21.15.164:0/2277283966 >> 172.21.15.164:0/2277283966 conn(0x7fa81c06ac90 msgr2=0x7fa81c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- 172.21.15.164:0/2277283966 shutdown_connections 2024-05-08T04:42:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- 172.21.15.164:0/2277283966 wait complete. 2024-05-08T04:42:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 Processor -- start 2024-05-08T04:42:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- start start 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa81c071f40 0x7fa81c164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa81c0b13d0 0x7fa81c13db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15c0e0 con 0x7fa81c157ce0 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15bf60 con 0x7fa81c071f40 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15c260 con 0x7fa81c0b13d0 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47390/0 (socket says 172.21.15.164:47390) 2024-05-08T04:42:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa81a800640 1 -- 172.21.15.164:0/1175434950 learned_addr learned my addr 172.21.15.164:0/1175434950 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa819e00640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa81c071f40 0x7fa81c164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.784+0000 7fa819400640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa81c0b13d0 0x7fa81c13db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 -- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa81c0b13d0 msgr2=0x7fa81c13db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa81c0b13d0 0x7fa81c13db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 -- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa81c071f40 msgr2=0x7fa81c164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa81c071f40 0x7fa81c164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 -- 172.21.15.164:0/1175434950 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c13e6e0 con 0x7fa81c157ce0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81a800640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c820 tx=0x7fa81400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814015070 con 0x7fa81c157ce0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814007a70 con 0x7fa81c157ce0 2024-05-08T04:42:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814007e60 con 0x7fa81c157ce0 2024-05-08T04:42:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c1633d0 con 0x7fa81c157ce0 2024-05-08T04:42:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c163990 con 0x7fa81c157ce0 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c04ef30 con 0x7fa81c157ce0 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 14) v1 ==== 87441+0+0 (secure 0 0 0) 0x7fa814018070 con 0x7fa81c157ce0 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 0x7fa7fc069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa814086340 con 0x7fa81c157ce0 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.788+0000 7fa819e00640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 0x7fa7fc069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.792+0000 7fa819e00640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 0x7fa7fc069c00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0012f0 tx=0x7fa80c001690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:35.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.792+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa814055b70 con 0x7fa81c157ce0 2024-05-08T04:42:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.944+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa81c14ea60 con 0x7fa81c157ce0 2024-05-08T04:42:35.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.944+0000 7fa812000640 1 -- 172.21.15.164:0/1175434950 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa814059c10 con 0x7fa81c157ce0 2024-05-08T04:42:35.947 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 msgr2=0x7fa7fc069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 0x7fa7fc069c00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0012f0 tx=0x7fa80c001690 comp rx=0 tx=0).stop 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 msgr2=0x7fa81c152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c820 tx=0x7fa81400cce0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 shutdown_connections 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa7fc067740 0x7fa7fc069c00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa81c0b13d0 0x7fa81c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa81c157ce0 0x7fa81c152ee0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 --2- 172.21.15.164:0/1175434950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa81c071f40 0x7fa81c164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 >> 172.21.15.164:0/1175434950 conn(0x7fa81c06ac90 msgr2=0x7fa81c06cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 shutdown_connections 2024-05-08T04:42:35.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:35.948+0000 7fa81b200640 1 -- 172.21.15.164:0/1175434950 wait complete. 2024-05-08T04:42:35.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:42:36.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 Processor -- start 2024-05-08T04:42:36.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 -- start start 2024-05-08T04:42:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd50415dd70 0x7fd5041619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd50406c7b0 0x7fd504161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5041628b0 con 0x7fd504162420 2024-05-08T04:42:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50406cc70 con 0x7fd50406c7b0 2024-05-08T04:42:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50406cf50 con 0x7fd50415dd70 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd50406c7b0 0x7fd504161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd50406c7b0 0x7fd504161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47398/0 (socket says 172.21.15.164:47398) 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd50a400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd504162420 0x7fd5040cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48264/0 (socket says 172.21.15.164:48264) 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 -- 172.21.15.164:0/2490966666 learned_addr learned my addr 172.21.15.164:0/2490966666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509a00640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd50415dd70 0x7fd5041619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 -- 172.21.15.164:0/2490966666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd50415dd70 msgr2=0x7fd5041619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd50415dd70 0x7fd5041619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.104+0000 7fd509000640 1 -- 172.21.15.164:0/2490966666 >> v1:172.21.15.164:6789/0 conn(0x7fd504162420 legacy=0x7fd5040cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd509000640 1 -- 172.21.15.164:0/2490966666 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040b1570 con 0x7fd50406c7b0 2024-05-08T04:42:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd503e00640 1 -- 172.21.15.164:0/2490966666 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406653957 0 0) 0x7fd4fc0014a0 con 0x7fd504162420 2024-05-08T04:42:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd509000640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504161ee0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009870 tx=0x7fd4f001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cba12c5657ca5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd503e00640 1 -- 172.21.15.164:0/2490966666 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00b5070 con 0x7fd50406c7b0 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd503e00640 1 -- 172.21.15.164:0/2490966666 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f00a8a50 con 0x7fd50406c7b0 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd503e00640 1 -- 172.21.15.164:0/2490966666 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00a8e20 con 0x7fd50406c7b0 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 -- 172.21.15.164:0/2490966666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 msgr2=0x7fd504161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504161ee0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009870 tx=0x7fd4f001ed50 comp rx=0 tx=0).stop 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 -- 172.21.15.164:0/2490966666 shutdown_connections 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504161ee0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 --2- 172.21.15.164:0/2490966666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd50415dd70 0x7fd5041619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 -- 172.21.15.164:0/2490966666 >> 172.21.15.164:0/2490966666 conn(0x7fd5040b34f0 msgr2=0x7fd5040b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 -- 172.21.15.164:0/2490966666 shutdown_connections 2024-05-08T04:42:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.108+0000 7fd50c400640 1 -- 172.21.15.164:0/2490966666 wait complete. 2024-05-08T04:42:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 Processor -- start 2024-05-08T04:42:36.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 -- start start 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 0x7fd5040b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 0x7fd5040af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50415e150 con 0x7fd50406c7b0 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040b4230 con 0x7fd50415dd70 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50415e2d0 con 0x7fd504162420 2024-05-08T04:42:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47400/0 (socket says 172.21.15.164:47400) 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 -- 172.21.15.164:0/3616286232 learned_addr learned my addr 172.21.15.164:0/3616286232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50a400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 0x7fd5040af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509000640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 0x7fd5040b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 -- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 msgr2=0x7fd5040af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 0x7fd5040af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 -- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 msgr2=0x7fd5040b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 0x7fd5040b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 -- 172.21.15.164:0/3616286232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5041e7d40 con 0x7fd50406c7b0 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd50a400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 0x7fd5040af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509000640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 0x7fd5040b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd509a00640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e930 tx=0x7fd4f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017070 con 0x7fd50406c7b0 2024-05-08T04:42:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.112+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f800c9a0 con 0x7fd50406c7b0 2024-05-08T04:42:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041e8020 con 0x7fd50406c7b0 2024-05-08T04:42:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800cd50 con 0x7fd50406c7b0 2024-05-08T04:42:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041e85e0 con 0x7fd50406c7b0 2024-05-08T04:42:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd504069d60 con 0x7fd50406c7b0 2024-05-08T04:42:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 14) v1 ==== 87441+0+0 (secure 0 0 0) 0x7fd4f801a070 con 0x7fd50406c7b0 2024-05-08T04:42:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd501600640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 0x7fd4e4069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd4f80865b0 con 0x7fd50406c7b0 2024-05-08T04:42:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd509000640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 0x7fd4e4069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.116+0000 7fd509000640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 0x7fd4e4069c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00b2000 tx=0x7fd4f00aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.120+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4f8056040 con 0x7fd50406c7b0 2024-05-08T04:42:36.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5040b4010 con 0x7fd50406c7b0 2024-05-08T04:42:36.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd501600640 1 -- 172.21.15.164:0/3616286232 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd4f805a0e0 con 0x7fd50406c7b0 2024-05-08T04:42:36.289 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 msgr2=0x7fd4e4069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 0x7fd4e4069c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00b2000 tx=0x7fd4f00aa070 comp rx=0 tx=0).stop 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 msgr2=0x7fd504158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e930 tx=0x7fd4f800edf0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 shutdown_connections 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd4e4067740 0x7fd4e4069c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd504162420 0x7fd5040af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50415dd70 0x7fd5040b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 --2- 172.21.15.164:0/3616286232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd50406c7b0 0x7fd504158720 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 >> 172.21.15.164:0/3616286232 conn(0x7fd5040b34f0 msgr2=0x7fd5040b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:36.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 shutdown_connections 2024-05-08T04:42:36.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.288+0000 7fd50c400640 1 -- 172.21.15.164:0/3616286232 wait complete. 2024-05-08T04:42:36.310 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:42:36.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:36.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:42:36.337 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:42:36.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:36.338 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:42:36.391 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:36.391 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:36.401 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:36.401 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:36.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.588+0000 7f3450600640 1 Processor -- start 2024-05-08T04:42:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.588+0000 7f3450600640 1 -- start start 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f344815ba70 0x7f3448160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f344815be50 0x7f34480cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f34480cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344815fe70 con 0x7f344815be50 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480cd9f0 con 0x7f344815ba70 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f3450600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34480cdcd0 con 0x7f34480aeda0 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f344815be50 0x7f34480cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f344815be50 0x7f34480cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47410/0 (socket says 172.21.15.164:47410) 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 -- 172.21.15.164:0/2962582500 learned_addr learned my addr 172.21.15.164:0/2962582500 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344dc00640 1 --2- 172.21.15.164:0/2962582500 >> v2:172.21.15.172:3300/0 conn(0x7f344815ba70 0x7f3448160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344e600640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f34480cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 -- 172.21.15.164:0/2962582500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 msgr2=0x7f34480cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:36.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f34480cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 -- 172.21.15.164:0/2962582500 >> v2:172.21.15.172:3300/0 conn(0x7f344815ba70 msgr2=0x7f3448160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:36.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 --2- 172.21.15.164:0/2962582500 >> v2:172.21.15.172:3300/0 conn(0x7f344815ba70 0x7f3448160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.592+0000 7f344d200640 1 -- 172.21.15.164:0/2962582500 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344815f820 con 0x7f344815be50 2024-05-08T04:42:36.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f344d200640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480cf690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f344806bc70 tx=0x7f343401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5dc8bafad416d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3447e00640 1 -- 172.21.15.164:0/2962582500 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340b1070 con 0x7f344815be50 2024-05-08T04:42:36.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3447e00640 1 -- 172.21.15.164:0/2962582500 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34340ac620 con 0x7f344815be50 2024-05-08T04:42:36.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3447e00640 1 -- 172.21.15.164:0/2962582500 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340ac9f0 con 0x7f344815be50 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- 172.21.15.164:0/2962582500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 msgr2=0x7f34480cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480cf690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f344806bc70 tx=0x7f343401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- 172.21.15.164:0/2962582500 shutdown_connections 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f34480cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 --2- 172.21.15.164:0/2962582500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480cf690 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 --2- 172.21.15.164:0/2962582500 >> v2:172.21.15.172:3300/0 conn(0x7f344815ba70 0x7f3448160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- 172.21.15.164:0/2962582500 >> 172.21.15.164:0/2962582500 conn(0x7f344806ac90 msgr2=0x7f344806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- 172.21.15.164:0/2962582500 shutdown_connections 2024-05-08T04:42:36.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- 172.21.15.164:0/2962582500 wait complete. 2024-05-08T04:42:36.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 Processor -- start 2024-05-08T04:42:36.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.596+0000 7f3450600640 1 -- start start 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f3448156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f344815ba70 0x7f34480b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480b3d80 con 0x7f344815be50 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480b3c00 con 0x7f344815ba70 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34480b3f00 con 0x7f34480aeda0 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47424/0 (socket says 172.21.15.164:47424) 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 -- 172.21.15.164:0/4001604970 learned_addr learned my addr 172.21.15.164:0/4001604970 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344dc00640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f3448156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344d200640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f344815ba70 0x7f34480b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 -- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 msgr2=0x7f3448156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f3448156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 -- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f344815ba70 msgr2=0x7f34480b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f344815ba70 0x7f34480b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 -- 172.21.15.164:0/4001604970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448157800 con 0x7f344815be50 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f344e600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3440007e60 tx=0x7f344000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440014620 con 0x7f344815be50 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3440019070 con 0x7f344815be50 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440014a20 con 0x7f344815be50 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34480b15f0 con 0x7f344815be50 2024-05-08T04:42:36.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34480b1bb0 con 0x7f344815be50 2024-05-08T04:42:36.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3448069d60 con 0x7f344815be50 2024-05-08T04:42:36.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.600+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f3440014bc0 con 0x7f344815be50 2024-05-08T04:42:36.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.604+0000 7f3445600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 0x7f3428069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.604+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3440084040 con 0x7f344815be50 2024-05-08T04:42:36.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.604+0000 7f344dc00640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 0x7f3428069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.604+0000 7f344dc00640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 0x7f3428069d20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f343c007a60 tx=0x7f343c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.608+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3440056450 con 0x7f344815be50 2024-05-08T04:42:36.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f344806e210 con 0x7f344815be50 2024-05-08T04:42:36.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3445600640 1 -- 172.21.15.164:0/4001604970 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16802 (secure 0 0 0) 0x7f344005a4f0 con 0x7f344815be50 2024-05-08T04:42:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:36.765 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":99,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:23.416521+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 msgr2=0x7f3428069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 0x7f3428069d20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f343c007a60 tx=0x7f343c008040 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 msgr2=0x7f34480b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3440007e60 tx=0x7f344000e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 shutdown_connections 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3428067860 0x7f3428069d20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f344815be50 0x7f34480b0940 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f344815ba70 0x7f34480b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 --2- 172.21.15.164:0/4001604970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34480aeda0 0x7f3448156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 >> 172.21.15.164:0/4001604970 conn(0x7f344806ac90 msgr2=0x7f344806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:36.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 shutdown_connections 2024-05-08T04:42:36.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.764+0000 7f3450600640 1 -- 172.21.15.164:0/4001604970 wait complete. 2024-05-08T04:42:36.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:20.161961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:20.781940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:36.787 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:36.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.924+0000 7f23b9200640 1 Processor -- start 2024-05-08T04:42:36.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.924+0000 7f23b9200640 1 -- start start 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.924+0000 7f23b9200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23b40cdb10 0x7f23b406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.924+0000 7f23b9200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40cfe60 con 0x7f23b40d0c20 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b9200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40d0140 con 0x7f23b40ce930 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b9200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40d0790 con 0x7f23b40cdb10 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23b40cdb10 0x7f23b406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23b40cdb10 0x7f23b406efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47430/0 (socket says 172.21.15.164:47430) 2024-05-08T04:42:36.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 -- 172.21.15.164:0/3714007241 learned_addr learned my addr 172.21.15.164:0/3714007241 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 -- 172.21.15.164:0/3714007241 >> v1:172.21.15.172:6789/0 conn(0x7f23b40ce930 legacy=0x7f23b40cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:36.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 -- 172.21.15.164:0/3714007241 >> v1:172.21.15.164:6789/0 conn(0x7f23b40d0c20 legacy=0x7f23b40ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 -- 172.21.15.164:0/3714007241 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40d0420 con 0x7f23b40cdb10 2024-05-08T04:42:36.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1e00640 1 --2- 172.21.15.164:0/3714007241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40cdb10 0x7f23b406efe0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009870 tx=0x7f23a401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0f89665bce7db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1400640 1 -- 172.21.15.164:0/3714007241 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40aa610 con 0x7f23b40cdb10 2024-05-08T04:42:36.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1400640 1 -- 172.21.15.164:0/3714007241 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40af070 con 0x7f23b40cdb10 2024-05-08T04:42:36.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.928+0000 7f23b1400640 1 -- 172.21.15.164:0/3714007241 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40aa9d0 con 0x7f23b40cdb10 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- 172.21.15.164:0/3714007241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40cdb10 msgr2=0x7f23b406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 --2- 172.21.15.164:0/3714007241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40cdb10 0x7f23b406efe0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009870 tx=0x7f23a401c490 comp rx=0 tx=0).stop 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- 172.21.15.164:0/3714007241 shutdown_connections 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 --2- 172.21.15.164:0/3714007241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40cdb10 0x7f23b406efe0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- 172.21.15.164:0/3714007241 >> 172.21.15.164:0/3714007241 conn(0x7f23b40b34f0 msgr2=0x7f23b40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- 172.21.15.164:0/3714007241 shutdown_connections 2024-05-08T04:42:36.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- 172.21.15.164:0/3714007241 wait complete. 2024-05-08T04:42:36.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 Processor -- start 2024-05-08T04:42:36.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- start start 2024-05-08T04:42:36.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b40cdb10 0x7f23b4156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 0x7f23b415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b416c270 con 0x7f23b40ce930 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b416c0f0 con 0x7f23b40cdb10 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b416c3f0 con 0x7f23b40d0c20 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47436/0 (socket says 172.21.15.164:47436) 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 -- 172.21.15.164:0/3610465829 learned_addr learned my addr 172.21.15.164:0/3610465829 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b1e00640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b40cdb10 0x7f23b4156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b2800640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 0x7f23b415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 -- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 msgr2=0x7f23b415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 0x7f23b415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 -- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b40cdb10 msgr2=0x7f23b4156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:36.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b40cdb10 0x7f23b4156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23a9400640 1 -- 172.21.15.164:0/3610465829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40c72f0 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.932+0000 7f23b2800640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 0x7f23b415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23a9400640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c7a0 tx=0x7f239c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c016020 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c006d80 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b40c75d0 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c003bc0 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b40c7b90 con 0x7f23b40ce930 2024-05-08T04:42:36.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b409fae0 con 0x7f23b40ce930 2024-05-08T04:42:36.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f239c00a850 con 0x7f23b40ce930 2024-05-08T04:42:36.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 0x7f2390069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:36.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f239c083510 con 0x7f23b40ce930 2024-05-08T04:42:36.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.936+0000 7f23b1e00640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 0x7f2390069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:36.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.940+0000 7f23b1e00640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 0x7f2390069d50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ae000 tx=0x7f23a40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:36.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:36.940+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f239c08a070 con 0x7f23b40ce930 2024-05-08T04:42:37.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23b404f010 con 0x7f23b40ce930 2024-05-08T04:42:37.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23aaa00640 1 -- 172.21.15.164:0/3610465829 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f239c020020 con 0x7f23b40ce930 2024-05-08T04:42:37.096 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:37.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 msgr2=0x7f2390069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 0x7f2390069d50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ae000 tx=0x7f23a40a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 msgr2=0x7f23b4159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c7a0 tx=0x7f239c00cc60 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 shutdown_connections 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2390067890 0x7f2390069d50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23b40d0c20 0x7f23b415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23b40ce930 0x7f23b4159bc0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 --2- 172.21.15.164:0/3610465829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b40cdb10 0x7f23b4156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 >> 172.21.15.164:0/3610465829 conn(0x7f23b40b34f0 msgr2=0x7f23b406d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 shutdown_connections 2024-05-08T04:42:37.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.096+0000 7f23b9200640 1 -- 172.21.15.164:0/3610465829 wait complete. 2024-05-08T04:42:37.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:37.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 Processor -- start 2024-05-08T04:42:37.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 -- start start 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dec144360 0x7f3dec0c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec06ac90 0x7f3dec06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec0adc50 con 0x7f3dec1619d0 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dec0ae860 con 0x7f3dec144360 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3df1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec144740 con 0x7f3dec06ac90 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dec144360 0x7f3dec0c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dec144360 0x7f3dec0c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47452/0 (socket says 172.21.15.164:47452) 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 -- 172.21.15.164:0/4234344522 learned_addr learned my addr 172.21.15.164:0/4234344522 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3de2000640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec06ac90 0x7f3dec06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 -- 172.21.15.164:0/4234344522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec06ac90 msgr2=0x7f3dec06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec06ac90 0x7f3dec06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 -- 172.21.15.164:0/4234344522 >> v1:172.21.15.164:6789/0 conn(0x7f3dec1619d0 legacy=0x7f3dec161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.256+0000 7f3deaa00640 1 -- 172.21.15.164:0/4234344522 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0adf30 con 0x7f3dec144360 2024-05-08T04:42:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3deaa00640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec144360 0x7f3dec0c5040 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009e50 tx=0x7f3ddc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc757cadd8c686ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3dea000640 1 -- 172.21.15.164:0/4234344522 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a5700 con 0x7f3dec144360 2024-05-08T04:42:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3dea000640 1 -- 172.21.15.164:0/4234344522 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0a58a0 con 0x7f3dec144360 2024-05-08T04:42:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3dea000640 1 -- 172.21.15.164:0/4234344522 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a5c70 con 0x7f3dec144360 2024-05-08T04:42:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 -- 172.21.15.164:0/4234344522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec144360 msgr2=0x7f3dec0c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec144360 0x7f3dec0c5040 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009e50 tx=0x7f3ddc01ca30 comp rx=0 tx=0).stop 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 -- 172.21.15.164:0/4234344522 shutdown_connections 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec06ac90 0x7f3dec06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 --2- 172.21.15.164:0/4234344522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec144360 0x7f3dec0c5040 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 -- 172.21.15.164:0/4234344522 >> 172.21.15.164:0/4234344522 conn(0x7f3dec0cf770 msgr2=0x7f3dec0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 -- 172.21.15.164:0/4234344522 shutdown_connections 2024-05-08T04:42:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.260+0000 7f3df1c00640 1 -- 172.21.15.164:0/4234344522 wait complete. 2024-05-08T04:42:37.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 Processor -- start 2024-05-08T04:42:37.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- start start 2024-05-08T04:42:37.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 0x7f3dec06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec1619d0 0x7f3dec15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0c4be0 con 0x7f3dec06ac90 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0c4a60 con 0x7f3dec144360 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec16b580 con 0x7f3dec1619d0 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47466/0 (socket says 172.21.15.164:47466) 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 -- 172.21.15.164:0/3786701071 learned_addr learned my addr 172.21.15.164:0/3786701071 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 -- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec1619d0 msgr2=0x7f3dec15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec1619d0 0x7f3dec15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3de2000640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 0x7f3dec06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 -- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 msgr2=0x7f3dec06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 0x7f3dec06e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 -- 172.21.15.164:0/3786701071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec15d0c0 con 0x7f3dec06ac90 2024-05-08T04:42:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3de2000640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 0x7f3dec06e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3deaa00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc01a660 tx=0x7f3ddc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0bc020 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0b8670 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec0af4a0 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc009510 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.264+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec0afa60 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec072d30 con 0x7f3dec06ac90 2024-05-08T04:42:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f3ddc0bf020 con 0x7f3dec06ac90 2024-05-08T04:42:37.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3de3400640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 0x7f3dc8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3ddc1206e0 con 0x7f3dec06ac90 2024-05-08T04:42:37.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3de2000640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 0x7f3dc8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.268+0000 7f3de2000640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 0x7f3dc8069d50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3dec1fab60 tx=0x7f3dd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.272+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ddc0c7020 con 0x7f3dec06ac90 2024-05-08T04:42:37.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.428+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3dec04f010 con 0x7f3dec06ac90 2024-05-08T04:42:37.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.428+0000 7f3de3400640 1 -- 172.21.15.164:0/3786701071 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ddc016620 con 0x7f3dec06ac90 2024-05-08T04:42:37.430 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 msgr2=0x7f3dc8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 0x7f3dc8069d50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3dec1fab60 tx=0x7f3dd4008040 comp rx=0 tx=0).stop 2024-05-08T04:42:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 msgr2=0x7f3dec15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc01a660 tx=0x7f3ddc0197d0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 shutdown_connections 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3dc8067890 0x7f3dc8069d50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3dec1619d0 0x7f3dec15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dec144360 0x7f3dec06e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 --2- 172.21.15.164:0/3786701071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dec06ac90 0x7f3dec15ca50 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 >> 172.21.15.164:0/3786701071 conn(0x7f3dec0cf770 msgr2=0x7f3dec13d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 shutdown_connections 2024-05-08T04:42:37.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.432+0000 7f3df1c00640 1 -- 172.21.15.164:0/3786701071 wait complete. 2024-05-08T04:42:37.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:37.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 Processor -- start 2024-05-08T04:42:37.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 -- start start 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f485015ab10 0x7f485006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48500cee90 0x7f48500aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485015e870 con 0x7f48500ae6e0 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485006bb80 con 0x7f48500cee90 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f4856400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485015eb50 con 0x7f485015ab10 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.592+0000 7f484fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48500cee90 0x7f48500aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48500cee90 0x7f48500aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47468/0 (socket says 172.21.15.164:47468) 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 -- 172.21.15.164:0/3861869838 learned_addr learned my addr 172.21.15.164:0/3861869838 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484f400640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f485015ab10 0x7f485006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 -- 172.21.15.164:0/3861869838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f485015ab10 msgr2=0x7f485006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f485015ab10 0x7f485006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 -- 172.21.15.164:0/3861869838 >> v1:172.21.15.172:6789/0 conn(0x7f48500ae6e0 legacy=0x7f48500ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 -- 172.21.15.164:0/3861869838 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485015ee30 con 0x7f48500cee90 2024-05-08T04:42:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484fe00640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f48500aeb40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f114dad55c09e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484ea00640 1 -- 172.21.15.164:0/3861869838 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48480ab660 con 0x7f48500cee90 2024-05-08T04:42:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484ea00640 1 -- 172.21.15.164:0/3861869838 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48480b0070 con 0x7f48500cee90 2024-05-08T04:42:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.596+0000 7f484ea00640 1 -- 172.21.15.164:0/3861869838 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48480aba20 con 0x7f48500cee90 2024-05-08T04:42:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- 172.21.15.164:0/3861869838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 msgr2=0x7f48500aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f48500aeb40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- 172.21.15.164:0/3861869838 shutdown_connections 2024-05-08T04:42:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f48500aeb40 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- 172.21.15.164:0/3861869838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f485015ab10 0x7f485006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- 172.21.15.164:0/3861869838 >> 172.21.15.164:0/3861869838 conn(0x7f48500cf770 msgr2=0x7f48500cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- 172.21.15.164:0/3861869838 shutdown_connections 2024-05-08T04:42:37.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- 172.21.15.164:0/3861869838 wait complete. 2024-05-08T04:42:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 Processor -- start 2024-05-08T04:42:37.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- start start 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 0x7f48501516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 0x7f485014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500cb070 con 0x7f48500cee90 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500caef0 con 0x7f485015ab10 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4856400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48500cb1f0 con 0x7f48500ae6e0 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47472/0 (socket says 172.21.15.164:47472) 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 -- 172.21.15.164:0/2600240306 learned_addr learned my addr 172.21.15.164:0/2600240306 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f484f400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 0x7f48501516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f484fe00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 0x7f485014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 -- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 msgr2=0x7f48501516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 0x7f48501516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 -- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 msgr2=0x7f485014fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 0x7f485014fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 -- 172.21.15.164:0/2600240306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4850150640 con 0x7f48500cee90 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f484f400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 0x7f48501516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f484fe00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 0x7f485014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4846a00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f483800c420 tx=0x7f483800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.600+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380052d0 con 0x7f48500cee90 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838005470 con 0x7f48500cee90 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838007260 con 0x7f48500cee90 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850150920 con 0x7f48500cee90 2024-05-08T04:42:37.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4850150ee0 con 0x7f48500cee90 2024-05-08T04:42:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4850072d30 con 0x7f48500cee90 2024-05-08T04:42:37.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f4838007400 con 0x7f48500cee90 2024-05-08T04:42:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4847e00640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 0x7f482c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4838085700 con 0x7f48500cee90 2024-05-08T04:42:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f484f400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 0x7f482c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.604+0000 7f484f400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 0x7f482c069d50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4840003fd0 tx=0x7f48400039e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.608+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4838055820 con 0x7f48500cee90 2024-05-08T04:42:37.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.760+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f485004f010 con 0x7f48500cee90 2024-05-08T04:42:37.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.760+0000 7f4847e00640 1 -- 172.21.15.164:0/2600240306 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4838055120 con 0x7f48500cee90 2024-05-08T04:42:37.763 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 msgr2=0x7f482c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 0x7f482c069d50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4840003fd0 tx=0x7f48400039e0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 msgr2=0x7f485014d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f483800c420 tx=0x7f483800c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 shutdown_connections 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f482c067890 0x7f482c069d50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f485015ab10 0x7f485014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48500cee90 0x7f485014d820 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 --2- 172.21.15.164:0/2600240306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48500ae6e0 0x7f48501516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 >> 172.21.15.164:0/2600240306 conn(0x7f48500cf770 msgr2=0x7f485014eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 shutdown_connections 2024-05-08T04:42:37.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.764+0000 7f4856400640 1 -- 172.21.15.164:0/2600240306 wait complete. 2024-05-08T04:42:37.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:42:37.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 Processor -- start 2024-05-08T04:42:37.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 -- start start 2024-05-08T04:42:37.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff93c15ab60 0x7ff93c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 0x7ff93c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 0x7ff93c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c0d0060 con 0x7ff93c15ab60 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c072640 con 0x7ff93c0b26c0 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff943800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06c770 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff93c15ab60 0x7ff93c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff93c15ab60 0x7ff93c072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47478/0 (socket says 172.21.15.164:47478) 2024-05-08T04:42:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 -- 172.21.15.164:0/1186505507 learned_addr learned my addr 172.21.15.164:0/1186505507 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff941800640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 0x7ff93c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 -- 172.21.15.164:0/1186505507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 msgr2=0x7ff93c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff93be00640 1 --2- 172.21.15.164:0/1186505507 >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 0x7ff93c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 0x7ff93c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 -- 172.21.15.164:0/1186505507 >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 msgr2=0x7ff93c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 --2- 172.21.15.164:0/1186505507 >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 0x7ff93c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff940e00640 1 -- 172.21.15.164:0/1186505507 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c06ca50 con 0x7ff93c15ab60 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff941800640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 0x7ff93c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:37.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.920+0000 7ff93be00640 1 --2- 172.21.15.164:0/1186505507 >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 0x7ff93c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:37.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff940e00640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c15ab60 0x7ff93c072100 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff93c06e880 tx=0x7ff92c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cfac57bf3b9f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff93b400640 1 -- 172.21.15.164:0/1186505507 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0a5670 con 0x7ff93c15ab60 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff93b400640 1 -- 172.21.15.164:0/1186505507 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c0a5810 con 0x7ff93c15ab60 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff93b400640 1 -- 172.21.15.164:0/1186505507 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0a5be0 con 0x7ff93c15ab60 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 -- 172.21.15.164:0/1186505507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c15ab60 msgr2=0x7ff93c072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c15ab60 0x7ff93c072100 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff93c06e880 tx=0x7ff92c01c9f0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 -- 172.21.15.164:0/1186505507 shutdown_connections 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c0cfbd0 0x7ff93c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 --2- 172.21.15.164:0/1186505507 >> v2:172.21.15.172:3300/0 conn(0x7ff93c0b26c0 0x7ff93c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 --2- 172.21.15.164:0/1186505507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c15ab60 0x7ff93c072100 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 -- 172.21.15.164:0/1186505507 >> 172.21.15.164:0/1186505507 conn(0x7ff93c06ac90 msgr2=0x7ff93c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:37.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 -- 172.21.15.164:0/1186505507 shutdown_connections 2024-05-08T04:42:37.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.924+0000 7ff943800640 1 -- 172.21.15.164:0/1186505507 wait complete. 2024-05-08T04:42:37.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 Processor -- start 2024-05-08T04:42:37.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- start start 2024-05-08T04:42:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff93c0b26c0 0x7ff93c14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c15ab60 0x7ff93c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06c550 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06c3d0 con 0x7ff93c0b26c0 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0aed30 con 0x7ff93c15ab60 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47494/0 (socket says 172.21.15.164:47494) 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 -- 172.21.15.164:0/2415682112 learned_addr learned my addr 172.21.15.164:0/2415682112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff940e00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff93c0b26c0 0x7ff93c14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff941800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c15ab60 0x7ff93c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 -- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c15ab60 msgr2=0x7ff93c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c15ab60 0x7ff93c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 -- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff93c0b26c0 msgr2=0x7ff93c14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff93c0b26c0 0x7ff93c14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 -- 172.21.15.164:0/2415682112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c14fc60 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff93be00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7c0 tx=0x7ff92800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928015070 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928007a30 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c14f760 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928007e00 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0ac730 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.928+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c069d60 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.932+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7ff928018070 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.932+0000 7ff938c00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 0x7ff91c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.932+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7ff928083760 con 0x7ff93c0cfbd0 2024-05-08T04:42:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.932+0000 7ff940e00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 0x7ff91c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:37.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.932+0000 7ff940e00640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 0x7ff91c069d20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0ae000 tx=0x7ff92c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:37.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:37.936+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff928055b70 con 0x7ff93c0cfbd0 2024-05-08T04:42:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff93c0b0830 con 0x7ff93c0cfbd0 2024-05-08T04:42:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff938c00640 1 -- 172.21.15.164:0/2415682112 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff928086120 con 0x7ff93c0cfbd0 2024-05-08T04:42:38.104 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:42:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 msgr2=0x7ff91c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 0x7ff91c069d20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0ae000 tx=0x7ff92c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 msgr2=0x7ff93c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7c0 tx=0x7ff92800cc80 comp rx=0 tx=0).stop 2024-05-08T04:42:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 shutdown_connections 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff91c067860 0x7ff91c069d20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff93c15ab60 0x7ff93c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff93c0cfbd0 0x7ff93c152390 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 --2- 172.21.15.164:0/2415682112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff93c0b26c0 0x7ff93c14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 >> 172.21.15.164:0/2415682112 conn(0x7ff93c06ac90 msgr2=0x7ff93c15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 shutdown_connections 2024-05-08T04:42:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.104+0000 7ff943800640 1 -- 172.21.15.164:0/2415682112 wait complete. 2024-05-08T04:42:38.124 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:42:38.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:38.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:42:38.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:42:38.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:42:38.154 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:42:38.211 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:38.211 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:38.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:38.220 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:38.220 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:38.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:38.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:38.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 Processor -- start 2024-05-08T04:42:38.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 -- start start 2024-05-08T04:42:38.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe02c0b26c0 0x7fe02c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c06c770 con 0x7fe02c15ab60 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c06ca50 con 0x7fe02c0cfbd0 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe034800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02c0d0060 con 0x7fe02c0b26c0 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe02c15ab60 0x7fe02c072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48298/0 (socket says 172.21.15.164:48298) 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031e00640 1 -- 172.21.15.164:0/3294566089 learned_addr learned my addr 172.21.15.164:0/3294566089 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031400640 1 --2- 172.21.15.164:0/3294566089 >> v2:172.21.15.164:3300/0 conn(0x7fe02c0b26c0 0x7fe02c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031400640 1 -- 172.21.15.164:0/3294566089 >> v1:172.21.15.172:6789/0 conn(0x7fe02c0cfbd0 legacy=0x7fe02c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:38.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031400640 1 -- 172.21.15.164:0/3294566089 >> v1:172.21.15.164:6789/0 conn(0x7fe02c15ab60 legacy=0x7fe02c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.400+0000 7fe031400640 1 -- 172.21.15.164:0/3294566089 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c06cd30 con 0x7fe02c0b26c0 2024-05-08T04:42:38.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe031400640 1 --2- 172.21.15.164:0/3294566089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c0cf690 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097c0 tx=0x7fe01c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc588cbc61b02ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe030a00640 1 -- 172.21.15.164:0/3294566089 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0b0070 con 0x7fe02c0b26c0 2024-05-08T04:42:38.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe030a00640 1 -- 172.21.15.164:0/3294566089 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c0ab600 con 0x7fe02c0b26c0 2024-05-08T04:42:38.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe030a00640 1 -- 172.21.15.164:0/3294566089 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0ab9d0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 -- 172.21.15.164:0/3294566089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 msgr2=0x7fe02c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 --2- 172.21.15.164:0/3294566089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c0cf690 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097c0 tx=0x7fe01c01c460 comp rx=0 tx=0).stop 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 -- 172.21.15.164:0/3294566089 shutdown_connections 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 --2- 172.21.15.164:0/3294566089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c0cf690 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 -- 172.21.15.164:0/3294566089 >> 172.21.15.164:0/3294566089 conn(0x7fe02c06ac90 msgr2=0x7fe02c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 -- 172.21.15.164:0/3294566089 shutdown_connections 2024-05-08T04:42:38.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.404+0000 7fe034800640 1 -- 172.21.15.164:0/3294566089 wait complete. 2024-05-08T04:42:38.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 Processor -- start 2024-05-08T04:42:38.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- start start 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe02c0cfbd0 0x7fe02c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02c15ab60 0x7fe02c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0aeeb0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0aed30 con 0x7fe02c0cfbd0 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c168980 con 0x7fe02c15ab60 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47524/0 (socket says 172.21.15.164:47524) 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 -- 172.21.15.164:0/3469522468 learned_addr learned my addr 172.21.15.164:0/3469522468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031400640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe02c0cfbd0 0x7fe02c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe032800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02c15ab60 0x7fe02c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 -- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02c15ab60 msgr2=0x7fe02c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02c15ab60 0x7fe02c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 -- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe02c0cfbd0 msgr2=0x7fe02c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe02c0cfbd0 0x7fe02c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 -- 172.21.15.164:0/3469522468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c153980 con 0x7fe02c0b26c0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe031e00640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c3a0 tx=0x7fe02000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020017070 con 0x7fe02c0b26c0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020005cf0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c1566c0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200072f0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c153c20 con 0x7fe02c0b26c0 2024-05-08T04:42:38.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.408+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c069d60 con 0x7fe02c0b26c0 2024-05-08T04:42:38.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.412+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fe02001a070 con 0x7fe02c0b26c0 2024-05-08T04:42:38.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.412+0000 7fe026000640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 0x7fe010069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.412+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe020082c70 con 0x7fe02c0b26c0 2024-05-08T04:42:38.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.412+0000 7fe031400640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 0x7fe010069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.412+0000 7fe031400640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 0x7fe010069d50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ad000 tx=0x7fe01c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.416+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe020054ff0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.616+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe02c156be0 con 0x7fe02c0b26c0 2024-05-08T04:42:38.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.616+0000 7fe026000640 1 -- 172.21.15.164:0/3469522468 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fe020059090 con 0x7fe02c0b26c0 2024-05-08T04:42:38.619 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:38.619 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:38.619 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 63 2024-05-08T04:42:38.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 msgr2=0x7fe010069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 0x7fe010069d50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ad000 tx=0x7fe01c008040 comp rx=0 tx=0).stop 2024-05-08T04:42:38.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 msgr2=0x7fe02c146900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c3a0 tx=0x7fe02000c860 comp rx=0 tx=0).stop 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 shutdown_connections 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe010067890 0x7fe010069d50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe02c15ab60 0x7fe02c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe02c0cfbd0 0x7fe02c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 --2- 172.21.15.164:0/3469522468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe02c0b26c0 0x7fe02c146900 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 >> 172.21.15.164:0/3469522468 conn(0x7fe02c06ac90 msgr2=0x7fe02c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 shutdown_connections 2024-05-08T04:42:38.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.620+0000 7fe034800640 1 -- 172.21.15.164:0/3469522468 wait complete. 2024-05-08T04:42:38.640 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:38.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.772+0000 7fa75bc00640 1 Processor -- start 2024-05-08T04:42:38.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.772+0000 7fa75bc00640 1 -- start start 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa75bc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa7540cfda0 0x7fa7540cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa75bc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75406c770 con 0x7fa7540b26c0 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa75bc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75406ca50 con 0x7fa75415ab60 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa75bc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75406cd30 con 0x7fa7540cfda0 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa74be00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa7540b26c0 0x7fa7540cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:48302/0 (socket says 172.21.15.164:48302) 2024-05-08T04:42:38.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa74be00640 1 -- 172.21.15.164:0/4111697064 learned_addr learned my addr 172.21.15.164:0/4111697064 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:38.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 --2- 172.21.15.164:0/4111697064 >> v2:172.21.15.172:3300/0 conn(0x7fa7540cfda0 0x7fa7540cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449032978 0 0) 0x7fa75406c770 con 0x7fa7540b26c0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa738003530 con 0x7fa7540b26c0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2064755860 0 0) 0x7fa738003530 con 0x7fa7540b26c0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 >> v2:172.21.15.172:3300/0 conn(0x7fa7540cfda0 msgr2=0x7fa7540cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 --2- 172.21.15.164:0/4111697064 >> v2:172.21.15.172:3300/0 conn(0x7fa7540cfda0 0x7fa7540cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 >> v1:172.21.15.172:6789/0 conn(0x7fa75415ab60 legacy=0x7fa754072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75406d010 con 0x7fa7540b26c0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa73c001680 con 0x7fa7540b26c0 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 >> v1:172.21.15.164:6789/0 conn(0x7fa7540b26c0 legacy=0x7fa7540cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7380039c0 0x7fa738003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7380043c0 0x7fa738004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 0x7fa73800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75406cd30 con 0x7fa738005090 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73800bff0 con 0x7fa7380043c0 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73800c1f0 con 0x7fa7380039c0 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 0x7fa73800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa74be00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7380043c0 0x7fa738004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759200640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7380039c0 0x7fa738003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 -- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7380039c0 msgr2=0x7fa738003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7380039c0 0x7fa738003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 -- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7380043c0 msgr2=0x7fa738004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7380043c0 0x7fa738004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.776+0000 7fa759c00640 1 -- 172.21.15.164:0/4111697064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738003530 con 0x7fa738005090 2024-05-08T04:42:38.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa759c00640 1 -- 172.21.15.164:0/4111697064 reap_dead start 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa759c00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 0x7fa73800b9d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009fc0 tx=0x7fa74c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914f071ff17fcfd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c009c10 con 0x7fa738005090 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0a5d10 con 0x7fa738005090 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa753e00640 1 -- 172.21.15.164:0/4111697064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c003be0 con 0x7fa738005090 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 -- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 msgr2=0x7fa73800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 0x7fa73800b9d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009fc0 tx=0x7fa74c01ccc0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 -- 172.21.15.164:0/4111697064 shutdown_connections 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 --2- 172.21.15.164:0/4111697064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa738005090 0x7fa73800b9d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 -- 172.21.15.164:0/4111697064 >> 172.21.15.164:0/4111697064 conn(0x7fa75406ac90 msgr2=0x7fa75406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 -- 172.21.15.164:0/4111697064 shutdown_connections 2024-05-08T04:42:38.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.780+0000 7fa75bc00640 1 -- 172.21.15.164:0/4111697064 wait complete. 2024-05-08T04:42:38.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 Processor -- start 2024-05-08T04:42:38.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- start start 2024-05-08T04:42:38.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa738005090 0x7fa7541f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7541f3e10 0x7fa7540b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540aeeb0 con 0x7fa754146f20 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540aed30 con 0x7fa7541f3e10 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75406c3d0 con 0x7fa738005090 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47542/0 (socket says 172.21.15.164:47542) 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 -- 172.21.15.164:0/3318455358 learned_addr learned my addr 172.21.15.164:0/3318455358 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa759200640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa738005090 0x7fa7541f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa759c00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7541f3e10 0x7fa7540b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 -- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa738005090 msgr2=0x7fa7541f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa738005090 0x7fa7541f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 -- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7541f3e10 msgr2=0x7fa7540b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7541f3e10 0x7fa7540b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 -- 172.21.15.164:0/3318455358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7541f42c0 con 0x7fa754146f20 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa74be00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa73c005820 tx=0x7fa73c012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01c070 con 0x7fa754146f20 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c015600 con 0x7fa754146f20 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7541473e0 con 0x7fa754146f20 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0159f0 con 0x7fa754146f20 2024-05-08T04:42:38.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541479a0 con 0x7fa754146f20 2024-05-08T04:42:38.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.784+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa754069d60 con 0x7fa754146f20 2024-05-08T04:42:38.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.788+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fa73c01f070 con 0x7fa754146f20 2024-05-08T04:42:38.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.788+0000 7fa751600640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 0x7fa730069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:38.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.788+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fa73c085f70 con 0x7fa754146f20 2024-05-08T04:42:38.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.788+0000 7fa759200640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 0x7fa730069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:38.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.788+0000 7fa759200640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 0x7fa730069d20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa744007fd0 tx=0x7fa744009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:38.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.792+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa73c058380 con 0x7fa754146f20 2024-05-08T04:42:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.952+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa75406d9a0 con 0x7fa754146f20 2024-05-08T04:42:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.952+0000 7fa751600640 1 -- 172.21.15.164:0/3318455358 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16916 (secure 0 0 0) 0x7fa73c05c420 con 0x7fa754146f20 2024-05-08T04:42:38.954 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:38.954 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":100,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:38.488862+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.952+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 msgr2=0x7fa730069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.952+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 0x7fa730069d20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa744007fd0 tx=0x7fa744009040 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 msgr2=0x7fa754144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa73c005820 tx=0x7fa73c012630 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 shutdown_connections 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa730067860 0x7fa730069d20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7541f3e10 0x7fa7540b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa754146f20 0x7fa754144ec0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 --2- 172.21.15.164:0/3318455358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa738005090 0x7fa7541f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 >> 172.21.15.164:0/3318455358 conn(0x7fa75406ac90 msgr2=0x7fa75415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 shutdown_connections 2024-05-08T04:42:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:38.956+0000 7fa75bc00640 1 -- 172.21.15.164:0/3318455358 wait complete. 2024-05-08T04:42:38.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:20.161961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:20.781940+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:38.975 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.112+0000 7f6a1c200640 1 Processor -- start 2024-05-08T04:42:39.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.112+0000 7f6a1c200640 1 -- start start 2024-05-08T04:42:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1c200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 0x7f6a140ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 0x7f6a140ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1c200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1415e870 con 0x7f6a1415ab10 2024-05-08T04:42:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1c200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140cf130 con 0x7f6a1406cce0 2024-05-08T04:42:39.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1406bb80 con 0x7f6a140cecc0 2024-05-08T04:42:39.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 0x7f6a140ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a18e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 0x7f6a140ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a19800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6a1415ab10 0x7f6a140ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:55098/0 (socket says 172.21.15.164:55098) 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a19800640 1 -- 172.21.15.164:0/1091743500 learned_addr learned my addr 172.21.15.164:0/1091743500 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1a200640 1 -- 172.21.15.164:0/1091743500 >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 msgr2=0x7f6a140ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1a200640 1 --2- 172.21.15.164:0/1091743500 >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 0x7f6a140ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1a200640 1 -- 172.21.15.164:0/1091743500 >> v1:172.21.15.172:6789/0 conn(0x7f6a1415ab10 legacy=0x7f6a140ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a1a200640 1 -- 172.21.15.164:0/1091743500 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1415eb50 con 0x7f6a140cecc0 2024-05-08T04:42:39.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.116+0000 7f6a18e00640 1 --2- 172.21.15.164:0/1091743500 >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 0x7f6a140ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1a200640 1 --2- 172.21.15.164:0/1091743500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 0x7f6a140ca6e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009d90 tx=0x7f6a0c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40898d480b0ff79c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a13e00640 1 -- 172.21.15.164:0/1091743500 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0b2070 con 0x7f6a140cecc0 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a13e00640 1 -- 172.21.15.164:0/1091743500 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c0a5880 con 0x7f6a140cecc0 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a13e00640 1 -- 172.21.15.164:0/1091743500 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0a5c50 con 0x7f6a140cecc0 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- 172.21.15.164:0/1091743500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 msgr2=0x7f6a140ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1091743500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 0x7f6a140ca6e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009d90 tx=0x7f6a0c01c8b0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- 172.21.15.164:0/1091743500 shutdown_connections 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1091743500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a140cecc0 0x7f6a140ca6e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1091743500 >> v2:172.21.15.172:3300/0 conn(0x7f6a1406cce0 0x7f6a140ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- 172.21.15.164:0/1091743500 >> 172.21.15.164:0/1091743500 conn(0x7f6a140cf770 msgr2=0x7f6a140cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- 172.21.15.164:0/1091743500 shutdown_connections 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- 172.21.15.164:0/1091743500 wait complete. 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 Processor -- start 2024-05-08T04:42:39.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.120+0000 7f6a1c200640 1 -- start start 2024-05-08T04:42:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 0x7f6a14153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a140cecc0 0x7f6a140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140aedf0 con 0x7f6a1415ab10 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1415aef0 con 0x7f6a140cecc0 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140aef70 con 0x7f6a1406cce0 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47550/0 (socket says 172.21.15.164:47550) 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 -- 172.21.15.164:0/1203927909 learned_addr learned my addr 172.21.15.164:0/1203927909 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a18e00640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a140cecc0 0x7f6a140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a19800640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 0x7f6a14153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 -- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 msgr2=0x7f6a14153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 0x7f6a14153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 -- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a140cecc0 msgr2=0x7f6a140b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a140cecc0 0x7f6a140b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 -- 172.21.15.164:0/1203927909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a141554b0 con 0x7f6a1415ab10 2024-05-08T04:42:39.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a19800640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 0x7f6a14153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1a200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0b0040 tx=0x7f6a0c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0b2050 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c0be070 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a14155790 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0ad850 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a141477d0 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a14072d30 con 0x7f6a1415ab10 2024-05-08T04:42:39.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.124+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f6a0c0ad9f0 con 0x7f6a1415ab10 2024-05-08T04:42:39.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.128+0000 7f6a11600640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 0x7f69f4069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.128+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f6a0c11eb20 con 0x7f6a1415ab10 2024-05-08T04:42:39.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.128+0000 7f6a19800640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 0x7f69f4069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.128+0000 7f6a19800640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 0x7f69f4069d20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a08000f70 tx=0x7f6a08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.132+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a0c016560 con 0x7f6a1415ab10 2024-05-08T04:42:39.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.284+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a1404f010 con 0x7f6a1415ab10 2024-05-08T04:42:39.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a11600640 1 -- 172.21.15.164:0/1203927909 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a0c0f1430 con 0x7f6a1415ab10 2024-05-08T04:42:39.287 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:39.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 msgr2=0x7f69f4069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 0x7f69f4069d20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a08000f70 tx=0x7f6a08009040 comp rx=0 tx=0).stop 2024-05-08T04:42:39.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 msgr2=0x7f6a14144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0b0040 tx=0x7f6a0c019650 comp rx=0 tx=0).stop 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 shutdown_connections 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f69f4067860 0x7f69f4069d20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a1415ab10 0x7f6a14144240 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a140cecc0 0x7f6a140b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 --2- 172.21.15.164:0/1203927909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a1406cce0 0x7f6a14153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 >> 172.21.15.164:0/1203927909 conn(0x7f6a140cf770 msgr2=0x7f6a14157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 shutdown_connections 2024-05-08T04:42:39.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.288+0000 7f6a1c200640 1 -- 172.21.15.164:0/1203927909 wait complete. 2024-05-08T04:42:39.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:42:39.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.444+0000 7f07dda00640 1 Processor -- start 2024-05-08T04:42:39.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.444+0000 7f07dda00640 1 -- start start 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07dda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07d8165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07dda00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d815e870 con 0x7f07d815ab10 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07dda00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d815eb50 con 0x7f07d806cce0 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07dda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d806bb80 con 0x7f07d80cee90 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f07d815ab10 0x7f07d80ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46410/0 (socket says 172.21.15.164:46410) 2024-05-08T04:42:39.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6a00640 1 -- 172.21.15.164:0/2161083955 learned_addr learned my addr 172.21.15.164:0/2161083955 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265643673 0 0) 0x7f07d815e870 con 0x7f07d815ab10 2024-05-08T04:42:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d7400640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07d8165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07bc003530 con 0x7f07d815ab10 2024-05-08T04:42:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624737402 0 0) 0x7f07bc003530 con 0x7f07d815ab10 2024-05-08T04:42:39.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 msgr2=0x7f07d8165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07d8165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 >> v1:172.21.15.172:6789/0 conn(0x7f07d806cce0 legacy=0x7f07d80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d815ee30 con 0x7f07d815ab10 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f07c8001680 con 0x7f07d815ab10 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 >> v1:172.21.15.164:6789/0 conn(0x7f07d815ab10 legacy=0x7f07d80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0039f0 0x7f07bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07bc0043a0 0x7f07bc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07bc005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d806bb80 con 0x7f07bc0043a0 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc00bab0 con 0x7f07bc0039f0 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc00bcb0 con 0x7f07d80cee90 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07bc0043a0 0x7f07bc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6a00640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0039f0 0x7f07bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d7400640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07bc005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 -- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 msgr2=0x7f07bc005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80cee90 0x7f07bc005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 -- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0039f0 msgr2=0x7f07bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0039f0 0x7f07bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.448+0000 7f07d6000640 1 -- 172.21.15.164:0/2161083955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc003530 con 0x7f07bc0043a0 2024-05-08T04:42:39.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07d6000640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07bc0043a0 0x7f07bc004870 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001860 tx=0x7f07c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b862a517c724ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4001460 con 0x7f07bc0043a0 2024-05-08T04:42:39.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c40b0070 con 0x7f07bc0043a0 2024-05-08T04:42:39.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07d5600640 1 -- 172.21.15.164:0/2161083955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c40ab830 con 0x7f07bc0043a0 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 -- 172.21.15.164:0/2161083955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07bc0043a0 msgr2=0x7f07bc004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 --2- 172.21.15.164:0/2161083955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07bc0043a0 0x7f07bc004870 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001860 tx=0x7f07c401c4d0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07d7400640 1 -- 172.21.15.164:0/2161083955 reap_dead start 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 -- 172.21.15.164:0/2161083955 shutdown_connections 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 -- 172.21.15.164:0/2161083955 >> 172.21.15.164:0/2161083955 conn(0x7f07d80cf770 msgr2=0x7f07d80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 -- 172.21.15.164:0/2161083955 shutdown_connections 2024-05-08T04:42:39.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.452+0000 7f07dda00640 1 -- 172.21.15.164:0/2161083955 wait complete. 2024-05-08T04:42:39.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 Processor -- start 2024-05-08T04:42:39.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- start start 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0043a0 0x7f07d80c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80c89f0 0x7f07d80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80aef70 con 0x7f07d80b3b50 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80aedf0 con 0x7f07bc0043a0 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d81662e0 con 0x7f07d80c89f0 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57242/0 (socket says 172.21.15.164:57242) 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 -- 172.21.15.164:0/3429705439 learned_addr learned my addr 172.21.15.164:0/3429705439 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d6a00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0043a0 0x7f07d80c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d6000640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80c89f0 0x7f07d80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 -- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80c89f0 msgr2=0x7f07d80c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80c89f0 0x7f07d80c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 -- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0043a0 msgr2=0x7f07d80c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0043a0 0x7f07d80c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 -- 172.21.15.164:0/3429705439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80c9540 con 0x7f07d80b3b50 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07d7400640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f07d0011c80 tx=0x7f07d000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0019070 con 0x7f07d80b3b50 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d00039a0 con 0x7f07d80b3b50 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0003d70 con 0x7f07d80b3b50 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d80c9820 con 0x7f07d80b3b50 2024-05-08T04:42:39.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d80b3630 con 0x7f07d80b3b50 2024-05-08T04:42:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.456+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d80b3830 con 0x7f07d80b3b50 2024-05-08T04:42:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.460+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f07d001c070 con 0x7f07d80b3b50 2024-05-08T04:42:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.460+0000 7f07cea00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 0x7f07b8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.460+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f07d0084400 con 0x7f07d80b3b50 2024-05-08T04:42:39.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.460+0000 7f07d6a00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 0x7f07b8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.460+0000 7f07d6a00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 0x7f07b8069d50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f07c8000c10 tx=0x7f07c8000f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.464+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07d80b3830 con 0x7f07d80b3b50 2024-05-08T04:42:39.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07d8072d30 con 0x7f07d80b3b50 2024-05-08T04:42:39.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07cea00640 1 -- 172.21.15.164:0/3429705439 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07d0010040 con 0x7f07d80b3b50 2024-05-08T04:42:39.625 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:42:39.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 msgr2=0x7f07b8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 0x7f07b8069d50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f07c8000c10 tx=0x7f07c8000f90 comp rx=0 tx=0).stop 2024-05-08T04:42:39.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 msgr2=0x7f07d80b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f07d0011c80 tx=0x7f07d000f580 comp rx=0 tx=0).stop 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 shutdown_connections 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b8067890 0x7f07b8069d50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d80c89f0 0x7f07d80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d80b3b50 0x7f07d80b3fd0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 --2- 172.21.15.164:0/3429705439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07bc0043a0 0x7f07d80c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.624+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 >> 172.21.15.164:0/3429705439 conn(0x7f07d80cf770 msgr2=0x7f07d80ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.628+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 shutdown_connections 2024-05-08T04:42:39.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.628+0000 7f07dda00640 1 -- 172.21.15.164:0/3429705439 wait complete. 2024-05-08T04:42:39.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:42:39.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.784+0000 7f6f29e00640 1 Processor -- start 2024-05-08T04:42:39.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.784+0000 7f6f29e00640 1 -- start start 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f29e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f24160ec0 0x7f6f2406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240ae6c0 0x7f6f2406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f29e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f240d20f0 con 0x7f6f240aeb20 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f29e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2406ecd0 con 0x7f6f24160ec0 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f29e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240d1ea0 con 0x7f6f240ae6c0 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f24160ec0 0x7f6f2406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f24160ec0 0x7f6f2406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57248/0 (socket says 172.21.15.164:57248) 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 -- 172.21.15.164:0/2957927206 learned_addr learned my addr 172.21.15.164:0/2957927206 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f1be00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240ae6c0 0x7f6f2406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 -- 172.21.15.164:0/2957927206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240ae6c0 msgr2=0x7f6f2406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:39.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240ae6c0 0x7f6f2406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 -- 172.21.15.164:0/2957927206 >> v1:172.21.15.172:6789/0 conn(0x7f6f240aeb20 legacy=0x7f6f240b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 -- 172.21.15.164:0/2957927206 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240d23d0 con 0x7f6f24160ec0 2024-05-08T04:42:39.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.788+0000 7f6f28a00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f24160ec0 0x7f6f2406cbe0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009870 tx=0x7f6f1401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1771fd6c1ba1deeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f23e00640 1 -- 172.21.15.164:0/2957927206 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a95f0 con 0x7f6f24160ec0 2024-05-08T04:42:39.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f23e00640 1 -- 172.21.15.164:0/2957927206 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f140a9790 con 0x7f6f24160ec0 2024-05-08T04:42:39.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f23e00640 1 -- 172.21.15.164:0/2957927206 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a9b60 con 0x7f6f24160ec0 2024-05-08T04:42:39.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- 172.21.15.164:0/2957927206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f24160ec0 msgr2=0x7f6f2406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f24160ec0 0x7f6f2406cbe0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009870 tx=0x7f6f1401c530 comp rx=0 tx=0).stop 2024-05-08T04:42:39.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- 172.21.15.164:0/2957927206 shutdown_connections 2024-05-08T04:42:39.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240ae6c0 0x7f6f2406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- 172.21.15.164:0/2957927206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f24160ec0 0x7f6f2406cbe0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- 172.21.15.164:0/2957927206 >> 172.21.15.164:0/2957927206 conn(0x7f6f2409aa20 msgr2=0x7f6f240d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- 172.21.15.164:0/2957927206 shutdown_connections 2024-05-08T04:42:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- 172.21.15.164:0/2957927206 wait complete. 2024-05-08T04:42:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 Processor -- start 2024-05-08T04:42:39.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- start start 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 0x7f6f24154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24160ec0 0x7f6f24155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f24161420 con 0x7f6f240ae6c0 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241612a0 con 0x7f6f24160ec0 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240b0930 con 0x7f6f240aeb20 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f28a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f28a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57250/0 (socket says 172.21.15.164:57250) 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f28a00640 1 -- 172.21.15.164:0/4263726765 learned_addr learned my addr 172.21.15.164:0/4263726765 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f1be00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 0x7f6f24154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.792+0000 7f6f29400640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24160ec0 0x7f6f24155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 -- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 msgr2=0x7f6f24154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 0x7f6f24154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 -- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24160ec0 msgr2=0x7f6f24155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24160ec0 0x7f6f24155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 -- 172.21.15.164:0/4263726765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240b5fa0 con 0x7f6f240ae6c0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f1be00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 0x7f6f24154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f28a00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6f140ab040 tx=0x7f6f1401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a95f0 con 0x7f6f240ae6c0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f140b1820 con 0x7f6f240ae6c0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140b1c10 con 0x7f6f240ae6c0 2024-05-08T04:42:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f240b6280 con 0x7f6f240ae6c0 2024-05-08T04:42:39.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f240b6840 con 0x7f6f240ae6c0 2024-05-08T04:42:39.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2404ef30 con 0x7f6f240ae6c0 2024-05-08T04:42:39.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f6f140c53f0 con 0x7f6f240ae6c0 2024-05-08T04:42:39.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 0x7f6f00069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:39.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f6f1411e490 con 0x7f6f240ae6c0 2024-05-08T04:42:39.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.796+0000 7f6f1be00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 0x7f6f00069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:39.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.800+0000 7f6f1be00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 0x7f6f00069d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009b10 tx=0x7f6f0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:39.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.800+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f140ad020 con 0x7f6f240ae6c0 2024-05-08T04:42:39.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f2409af70 con 0x7f6f240ae6c0 2024-05-08T04:42:39.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f21600640 1 -- 172.21.15.164:0/4263726765 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f14016150 con 0x7f6f240ae6c0 2024-05-08T04:42:39.957 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 msgr2=0x7f6f00069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 0x7f6f00069d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009b10 tx=0x7f6f0c008040 comp rx=0 tx=0).stop 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 msgr2=0x7f6f2414dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6f140ab040 tx=0x7f6f1401ad70 comp rx=0 tx=0).stop 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 shutdown_connections 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6f00067860 0x7f6f00069d20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f240aeb20 0x7f6f24154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f24160ec0 0x7f6f24155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 --2- 172.21.15.164:0/4263726765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f240ae6c0 0x7f6f2414dd50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:39.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 >> 172.21.15.164:0/4263726765 conn(0x7f6f2409aa20 msgr2=0x7f6f240af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:39.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 shutdown_connections 2024-05-08T04:42:39.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:39.956+0000 7f6f29e00640 1 -- 172.21.15.164:0/4263726765 wait complete. 2024-05-08T04:42:39.978 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:40.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 Processor -- start 2024-05-08T04:42:40.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 -- start start 2024-05-08T04:42:40.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65bc1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc06cc70 con 0x7f65bc06c7b0 2024-05-08T04:42:40.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc06cf50 con 0x7f65bc1625f0 2024-05-08T04:42:40.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65c2a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0b1570 con 0x7f65bc15dd70 2024-05-08T04:42:40.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65bb400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f65bc06c7b0 0x7f65bc161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46412/0 (socket says 172.21.15.164:46412) 2024-05-08T04:42:40.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.112+0000 7f65bb400640 1 -- 172.21.15.164:0/1156516834 learned_addr learned my addr 172.21.15.164:0/1156516834 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bbe00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65bc1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159092466 0 0) 0x7f65bc06cc70 con 0x7f65bc06c7b0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a0003530 con 0x7f65bc06c7b0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850187157 0 0) 0x7f65bc06cf50 con 0x7f65bc1625f0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65bc06cc70 con 0x7f65bc1625f0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620405052 0 0) 0x7f65a0003530 con 0x7f65bc06c7b0 2024-05-08T04:42:40.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 msgr2=0x7f65bc1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65bc1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 >> v1:172.21.15.172:6789/0 conn(0x7f65bc1625f0 legacy=0x7f65bc0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0b1850 con 0x7f65bc06c7b0 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f65a8001680 con 0x7f65bc06c7b0 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bbe00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65bc1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 >> v1:172.21.15.164:6789/0 conn(0x7f65bc06c7b0 legacy=0x7f65bc161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65a0003b80 0x7f65a0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0b1570 con 0x7f65a0004810 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a000ba80 con 0x7f65a0003b80 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a000bc50 con 0x7f65bc15dd70 2024-05-08T04:42:40.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bbe00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65a0003b80 0x7f65a0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65c0a00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 -- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 msgr2=0x7f65a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc15dd70 0x7f65a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 -- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65a0003b80 msgr2=0x7f65a0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65a0003b80 0x7f65a0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.116+0000 7f65bb400640 1 -- 172.21.15.164:0/1156516834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0003530 con 0x7f65a0004810 2024-05-08T04:42:40.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65bb400640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65a0004c90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f65a8001ec0 tx=0x7f65a801f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27482ee4cadd2724 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8007c10 con 0x7f65a0004810 2024-05-08T04:42:40.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a80ac070 con 0x7f65a0004810 2024-05-08T04:42:40.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65baa00640 1 -- 172.21.15.164:0/1156516834 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8005b20 con 0x7f65a0004810 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- 172.21.15.164:0/1156516834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 msgr2=0x7f65a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 --2- 172.21.15.164:0/1156516834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65a0004c90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f65a8001ec0 tx=0x7f65a801f250 comp rx=0 tx=0).stop 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c0a00640 1 -- 172.21.15.164:0/1156516834 reap_dead start 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- 172.21.15.164:0/1156516834 shutdown_connections 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- 172.21.15.164:0/1156516834 >> 172.21.15.164:0/1156516834 conn(0x7f65bc0b34f0 msgr2=0x7f65bc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- 172.21.15.164:0/1156516834 shutdown_connections 2024-05-08T04:42:40.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- 172.21.15.164:0/1156516834 wait complete. 2024-05-08T04:42:40.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 Processor -- start 2024-05-08T04:42:40.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.120+0000 7f65c2a00640 1 -- start start 2024-05-08T04:42:40.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 0x7f65bc0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 0x7f65bc1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15e2d0 con 0x7f65a0004810 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15e150 con 0x7f65bc0b5a80 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc1619a0 con 0x7f65bc152c90 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57272/0 (socket says 172.21.15.164:57272) 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 -- 172.21.15.164:0/3627770983 learned_addr learned my addr 172.21.15.164:0/3627770983 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bb400640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 0x7f65bc0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c0a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 0x7f65bc1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 -- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 msgr2=0x7f65bc1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 0x7f65bc1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 -- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 msgr2=0x7f65bc0b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 0x7f65bc0b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 -- 172.21.15.164:0/3627770983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc153630 con 0x7f65a0004810 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c0a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 0x7f65bc1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bb400640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 0x7f65bc0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65bbe00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011910 tx=0x7f65ac011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00faf0 con 0x7f65a0004810 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac00fc90 con 0x7f65a0004810 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0c7510 con 0x7f65a0004810 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac004030 con 0x7f65a0004810 2024-05-08T04:42:40.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc0c7ad0 con 0x7f65a0004810 2024-05-08T04:42:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc069d60 con 0x7f65a0004810 2024-05-08T04:42:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.124+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f65ac012040 con 0x7f65a0004810 2024-05-08T04:42:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.128+0000 7f65b3e00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 0x7f659c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.128+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f65ac083c30 con 0x7f65a0004810 2024-05-08T04:42:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.128+0000 7f65bb400640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 0x7f659c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.128+0000 7f65bb400640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 0x7f659c069d20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f65a80ae000 tx=0x7f65a80b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.128+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65ac019070 con 0x7f65a0004810 2024-05-08T04:42:40.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65bc0b7940 con 0x7f65a0004810 2024-05-08T04:42:40.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65b3e00640 1 -- 172.21.15.164:0/3627770983 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f65ac056690 con 0x7f65a0004810 2024-05-08T04:42:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:40.317 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:40.317 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 63 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 msgr2=0x7f659c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 0x7f659c069d20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f65a80ae000 tx=0x7f65a80b4040 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 msgr2=0x7f65bc0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011910 tx=0x7f65ac011dd0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 shutdown_connections 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f659c067860 0x7f659c069d20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65bc152c90 0x7f65bc1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65bc0b5a80 0x7f65bc0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 --2- 172.21.15.164:0/3627770983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a0004810 0x7f65bc0b4810 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 >> 172.21.15.164:0/3627770983 conn(0x7f65bc0b34f0 msgr2=0x7f65bc15c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 shutdown_connections 2024-05-08T04:42:40.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.316+0000 7f65c2a00640 1 -- 172.21.15.164:0/3627770983 wait complete. 2024-05-08T04:42:40.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:40.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.472+0000 7f049d600640 1 Processor -- start 2024-05-08T04:42:40.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.472+0000 7f049d600640 1 -- start start 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0498144200 0x7f0498161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f04980cfd60 0x7f04980cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cc5f0 0x7f049815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0498162880 con 0x7f04980cfd60 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0498161fe0 con 0x7f0498144200 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f049d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04981622c0 con 0x7f04980cc5f0 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f04980cfd60 0x7f04980cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f04980cfd60 0x7f04980cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57278/0 (socket says 172.21.15.164:57278) 2024-05-08T04:42:40.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 -- 172.21.15.164:0/1040665699 learned_addr learned my addr 172.21.15.164:0/1040665699 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0497400640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cc5f0 0x7f049815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496a00640 1 --2- 172.21.15.164:0/1040665699 >> v2:172.21.15.172:3300/0 conn(0x7f0498144200 0x7f0498161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 -- 172.21.15.164:0/1040665699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cc5f0 msgr2=0x7f049815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cc5f0 0x7f049815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 -- 172.21.15.164:0/1040665699 >> v2:172.21.15.172:3300/0 conn(0x7f0498144200 msgr2=0x7f0498161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 --2- 172.21.15.164:0/1040665699 >> v2:172.21.15.172:3300/0 conn(0x7f0498144200 0x7f0498161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.476+0000 7f0496000640 1 -- 172.21.15.164:0/1040665699 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04981625a0 con 0x7f04980cfd60 2024-05-08T04:42:40.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f0496000640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cfd60 0x7f04980cc0b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0498072820 tx=0x7f048401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894d417338d51c9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f0495600640 1 -- 172.21.15.164:0/1040665699 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840b0070 con 0x7f04980cfd60 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f0495600640 1 -- 172.21.15.164:0/1040665699 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840ab600 con 0x7f04980cfd60 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f0495600640 1 -- 172.21.15.164:0/1040665699 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840ab9d0 con 0x7f04980cfd60 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- 172.21.15.164:0/1040665699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cfd60 msgr2=0x7f04980cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cfd60 0x7f04980cc0b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0498072820 tx=0x7f048401c490 comp rx=0 tx=0).stop 2024-05-08T04:42:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- 172.21.15.164:0/1040665699 shutdown_connections 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cc5f0 0x7f049815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- 172.21.15.164:0/1040665699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cfd60 0x7f04980cc0b0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- 172.21.15.164:0/1040665699 >> v2:172.21.15.172:3300/0 conn(0x7f0498144200 0x7f0498161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- 172.21.15.164:0/1040665699 >> 172.21.15.164:0/1040665699 conn(0x7f049806ac90 msgr2=0x7f049806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- 172.21.15.164:0/1040665699 shutdown_connections 2024-05-08T04:42:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- 172.21.15.164:0/1040665699 wait complete. 2024-05-08T04:42:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 Processor -- start 2024-05-08T04:42:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- start start 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cfd60 0x7f04981f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0498144200 0x7f049813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049806c3d0 con 0x7f04980cc5f0 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cf400 con 0x7f0498144200 2024-05-08T04:42:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f049d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049806c550 con 0x7f04980cfd60 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.480+0000 7f0496a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57290/0 (socket says 172.21.15.164:57290) 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 -- 172.21.15.164:0/707017362 learned_addr learned my addr 172.21.15.164:0/707017362 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496000640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cfd60 0x7f04981f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0497400640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0498144200 0x7f049813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 -- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cfd60 msgr2=0x7f04981f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cfd60 0x7f04981f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 -- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0498144200 msgr2=0x7f049813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0498144200 0x7f049813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 -- 172.21.15.164:0/707017362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04981430a0 con 0x7f04980cc5f0 2024-05-08T04:42:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f0496a00640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f048800d620 tx=0x7f048800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800b620 con 0x7f04980cc5f0 2024-05-08T04:42:40.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488019070 con 0x7f04980cc5f0 2024-05-08T04:42:40.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800ba00 con 0x7f04980cc5f0 2024-05-08T04:42:40.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0498143380 con 0x7f04980cc5f0 2024-05-08T04:42:40.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0498143940 con 0x7f04980cc5f0 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498069d60 con 0x7f04980cc5f0 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f048800bba0 con 0x7f04980cc5f0 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 0x7f0478069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.484+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f0488083cf0 con 0x7f04980cc5f0 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.488+0000 7f0496000640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 0x7f0478069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.488+0000 7f0496000640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 0x7f0478069d50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad000 tx=0x7f0484008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.488+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0488055f10 con 0x7f04980cc5f0 2024-05-08T04:42:40.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.644+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04980add30 con 0x7f04980cc5f0 2024-05-08T04:42:40.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f048ea00640 1 -- 172.21.15.164:0/707017362 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16916 (secure 0 0 0) 0x7f0488059fb0 con 0x7f04980cc5f0 2024-05-08T04:42:40.647 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:40.648 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":100,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:38.488862+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:40.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 msgr2=0x7f0478069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 0x7f0478069d50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04840ad000 tx=0x7f0484008040 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 msgr2=0x7f049813bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f048800d620 tx=0x7f048800dae0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 shutdown_connections 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0478067890 0x7f0478069d50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04980cfd60 0x7f04981f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0498144200 0x7f049813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 --2- 172.21.15.164:0/707017362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04980cc5f0 0x7f049813bcc0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 >> 172.21.15.164:0/707017362 conn(0x7f049806ac90 msgr2=0x7f049806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 shutdown_connections 2024-05-08T04:42:40.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.648+0000 7f049d600640 1 -- 172.21.15.164:0/707017362 wait complete. 2024-05-08T04:42:40.669 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:42:40.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:40.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 Processor -- start 2024-05-08T04:42:40.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 -- start start 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93bc15ba70 0x7f93bc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93bc0cfda0 0x7f93bc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0cd9f0 con 0x7f93bc15be50 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc15fe70 con 0x7f93bc0cfda0 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93c1000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc0cdcd0 con 0x7f93bc15ba70 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93bc0cfda0 0x7f93bc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93bc0cfda0 0x7f93bc0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57300/0 (socket says 172.21.15.164:57300) 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 -- 172.21.15.164:0/3303676820 learned_addr learned my addr 172.21.15.164:0/3303676820 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93b9e00640 1 --2- 172.21.15.164:0/3303676820 >> v2:172.21.15.172:3300/0 conn(0x7f93bc15ba70 0x7f93bc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 -- 172.21.15.164:0/3303676820 >> v2:172.21.15.172:3300/0 conn(0x7f93bc15ba70 msgr2=0x7f93bc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 --2- 172.21.15.164:0/3303676820 >> v2:172.21.15.172:3300/0 conn(0x7f93bc15ba70 0x7f93bc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 -- 172.21.15.164:0/3303676820 >> v1:172.21.15.172:6789/0 conn(0x7f93bc15be50 legacy=0x7f93bc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.804+0000 7f93ba800640 1 -- 172.21.15.164:0/3303676820 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc15f820 con 0x7f93bc0cfda0 2024-05-08T04:42:40.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93ba800640 1 --2- 172.21.15.164:0/3303676820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc0b31a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f93bc06bc70 tx=0x7f93b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf75f8bbd6657fa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93b8a00640 1 -- 172.21.15.164:0/3303676820 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40ab600 con 0x7f93bc0cfda0 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93b8a00640 1 -- 172.21.15.164:0/3303676820 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b40b0070 con 0x7f93bc0cfda0 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93b8a00640 1 -- 172.21.15.164:0/3303676820 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40ab9c0 con 0x7f93bc0cfda0 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 -- 172.21.15.164:0/3303676820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 msgr2=0x7f93bc0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 --2- 172.21.15.164:0/3303676820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc0b31a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f93bc06bc70 tx=0x7f93b401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 -- 172.21.15.164:0/3303676820 shutdown_connections 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 --2- 172.21.15.164:0/3303676820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc0b31a0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 --2- 172.21.15.164:0/3303676820 >> v2:172.21.15.172:3300/0 conn(0x7f93bc15ba70 0x7f93bc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 -- 172.21.15.164:0/3303676820 >> 172.21.15.164:0/3303676820 conn(0x7f93bc06ac90 msgr2=0x7f93bc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 -- 172.21.15.164:0/3303676820 shutdown_connections 2024-05-08T04:42:40.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.808+0000 7f93c1000640 1 -- 172.21.15.164:0/3303676820 wait complete. 2024-05-08T04:42:40.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 Processor -- start 2024-05-08T04:42:40.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- start start 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93bc15ba70 0x7f93bc14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93bc15be50 0x7f93bc070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc168d40 con 0x7f93bc0cfda0 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc168bc0 con 0x7f93bc15ba70 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc168ec0 con 0x7f93bc15be50 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57314/0 (socket says 172.21.15.164:57314) 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 -- 172.21.15.164:0/645991848 learned_addr learned my addr 172.21.15.164:0/645991848 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9400640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93bc15ba70 0x7f93bc14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93ba800640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93bc15be50 0x7f93bc070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 -- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93bc15be50 msgr2=0x7f93bc070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93bc15be50 0x7f93bc070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 -- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93bc15ba70 msgr2=0x7f93bc14c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93bc15ba70 0x7f93bc14c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 -- 172.21.15.164:0/645991848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc152e40 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b9e00640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f93ac007ef0 tx=0x7f93ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac014620 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac019070 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac014a00 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1530e0 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.812+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc0706f0 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc069d60 con 0x7f93bc0cfda0 2024-05-08T04:42:40.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f93ac014ba0 con 0x7f93bc0cfda0 2024-05-08T04:42:40.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93b2000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 0x7f939c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:40.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f93ac0840f0 con 0x7f93bc0cfda0 2024-05-08T04:42:40.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93b9400640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 0x7f939c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:40.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.816+0000 7f93b9400640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 0x7f939c069d20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f93bc157c70 tx=0x7f93a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:40.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:40.820+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93ac056500 con 0x7f93bc0cfda0 2024-05-08T04:42:41.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93bc06e210 con 0x7f93bc0cfda0 2024-05-08T04:42:41.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93b2000640 1 -- 172.21.15.164:0/645991848 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f93ac05a5a0 con 0x7f93bc0cfda0 2024-05-08T04:42:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:41.008 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:41.009 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 63 2024-05-08T04:42:41.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 msgr2=0x7f939c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 0x7f939c069d20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f93bc157c70 tx=0x7f93a8009040 comp rx=0 tx=0).stop 2024-05-08T04:42:41.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 msgr2=0x7f93bc152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f93ac007ef0 tx=0x7f93ac00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 shutdown_connections 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f939c067860 0x7f939c069d20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93bc15be50 0x7f93bc070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93bc15ba70 0x7f93bc14c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 --2- 172.21.15.164:0/645991848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93bc0cfda0 0x7f93bc152900 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 >> 172.21.15.164:0/645991848 conn(0x7f93bc06ac90 msgr2=0x7f93bc06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 shutdown_connections 2024-05-08T04:42:41.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.008+0000 7f93c1000640 1 -- 172.21.15.164:0/645991848 wait complete. 2024-05-08T04:42:41.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:41.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 Processor -- start 2024-05-08T04:42:41.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 -- start start 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f157c15be50 0x7f157c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0cd9f0 con 0x7f157c15ba70 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0cdcd0 con 0x7f157c0cfda0 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f1580c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c15fe70 con 0x7f157c15be50 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f157c15be50 0x7f157c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f157c15be50 0x7f157c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57326/0 (socket says 172.21.15.164:57326) 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 -- 172.21.15.164:0/554776483 learned_addr learned my addr 172.21.15.164:0/554776483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 -- 172.21.15.164:0/554776483 >> v1:172.21.15.172:6789/0 conn(0x7f157c0cfda0 legacy=0x7f157c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 -- 172.21.15.164:0/554776483 >> v1:172.21.15.164:6789/0 conn(0x7f157c15ba70 legacy=0x7f157c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.164+0000 7f157a800640 1 -- 172.21.15.164:0/554776483 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c15f820 con 0x7f157c15be50 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f157a800640 1 --2- 172.21.15.164:0/554776483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15be50 0x7f157c160300 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1574009e50 tx=0x7f157401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d930ac383036211b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1579400640 1 -- 172.21.15.164:0/554776483 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740a57e0 con 0x7f157c15be50 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1579400640 1 -- 172.21.15.164:0/554776483 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15740a5980 con 0x7f157c15be50 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1579400640 1 -- 172.21.15.164:0/554776483 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740a5d50 con 0x7f157c15be50 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 -- 172.21.15.164:0/554776483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15be50 msgr2=0x7f157c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 --2- 172.21.15.164:0/554776483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15be50 0x7f157c160300 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1574009e50 tx=0x7f157401ca00 comp rx=0 tx=0).stop 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 -- 172.21.15.164:0/554776483 shutdown_connections 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 --2- 172.21.15.164:0/554776483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15be50 0x7f157c160300 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 -- 172.21.15.164:0/554776483 >> 172.21.15.164:0/554776483 conn(0x7f157c06ac90 msgr2=0x7f157c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 -- 172.21.15.164:0/554776483 shutdown_connections 2024-05-08T04:42:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.168+0000 7f1580c00640 1 -- 172.21.15.164:0/554776483 wait complete. 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 Processor -- start 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- start start 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f157c0cfda0 0x7f157c14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 0x7f157c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c169f60 con 0x7f157c15ba70 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c169de0 con 0x7f157c15be50 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c16a0e0 con 0x7f157c0cfda0 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57338/0 (socket says 172.21.15.164:57338) 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 -- 172.21.15.164:0/774325032 learned_addr learned my addr 172.21.15.164:0/774325032 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1579e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f157c0cfda0 0x7f157c14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f157a800640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 0x7f157c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 -- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f157c0cfda0 msgr2=0x7f157c14dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f157c0cfda0 0x7f157c14dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 -- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 msgr2=0x7f157c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 0x7f157c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 -- 172.21.15.164:0/774325032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c14e2e0 con 0x7f157c15ba70 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f157a800640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 0x7f157c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1573e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f156400c420 tx=0x7f156400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640052d0 con 0x7f157c15ba70 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564005470 con 0x7f157c15ba70 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564007260 con 0x7f157c15ba70 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c14db30 con 0x7f157c15ba70 2024-05-08T04:42:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.172+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0af2b0 con 0x7f157c15ba70 2024-05-08T04:42:41.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c069d60 con 0x7f157c15ba70 2024-05-08T04:42:41.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f1564007400 con 0x7f157c15ba70 2024-05-08T04:42:41.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1572000640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 0x7f1558069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f1564082e40 con 0x7f157c15ba70 2024-05-08T04:42:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1579e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 0x7f1558069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.176+0000 7f1579e00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 0x7f1558069d20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f157c072550 tx=0x7f156c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.180+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15640558a0 con 0x7f157c15ba70 2024-05-08T04:42:41.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f157c06e210 con 0x7f157c15ba70 2024-05-08T04:42:41.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1572000640 1 -- 172.21.15.164:0/774325032 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f15640551a0 con 0x7f157c15ba70 2024-05-08T04:42:41.368 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:41.368 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:21.793962+0000","modified":"2024-05-08T04:42:24.805370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7443":{"gid":7443,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/4096204626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":4096204626},{"type":"v1","addr":"172.21.15.164:6837","nonce":4096204626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-08T04:42:41.369 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 63 2024-05-08T04:42:41.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 msgr2=0x7f1558069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 0x7f1558069d20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f157c072550 tx=0x7f156c009040 comp rx=0 tx=0).stop 2024-05-08T04:42:41.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 msgr2=0x7f157c0b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f156400c420 tx=0x7f156400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 shutdown_connections 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1558067860 0x7f1558069d20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c15be50 0x7f157c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c15ba70 0x7f157c0b0b10 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 --2- 172.21.15.164:0/774325032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f157c0cfda0 0x7f157c14dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 >> 172.21.15.164:0/774325032 conn(0x7f157c06ac90 msgr2=0x7f157c06ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 shutdown_connections 2024-05-08T04:42:41.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.368+0000 7f1580c00640 1 -- 172.21.15.164:0/774325032 wait complete. 2024-05-08T04:42:41.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:41.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 Processor -- start 2024-05-08T04:42:41.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 -- start start 2024-05-08T04:42:41.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcc6815ba70 0x7fcc68160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc680cfda0 0x7fcc680b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680cd9f0 con 0x7fcc6815be50 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6815fe70 con 0x7fcc6815ba70 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc6e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680cdcd0 con 0x7fcc680cfda0 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcc6815ba70 0x7fcc68160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcc6815ba70 0x7fcc68160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57348/0 (socket says 172.21.15.164:57348) 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 -- 172.21.15.164:0/3384219201 learned_addr learned my addr 172.21.15.164:0/3384219201 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67e00640 1 --2- 172.21.15.164:0/3384219201 >> v2:172.21.15.172:3300/0 conn(0x7fcc680cfda0 0x7fcc680b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 -- 172.21.15.164:0/3384219201 >> v2:172.21.15.172:3300/0 conn(0x7fcc680cfda0 msgr2=0x7fcc680b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 --2- 172.21.15.164:0/3384219201 >> v2:172.21.15.172:3300/0 conn(0x7fcc680cfda0 0x7fcc680b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 -- 172.21.15.164:0/3384219201 >> v1:172.21.15.164:6789/0 conn(0x7fcc6815be50 legacy=0x7fcc680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.528+0000 7fcc67400640 1 -- 172.21.15.164:0/3384219201 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6815f820 con 0x7fcc6815ba70 2024-05-08T04:42:41.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc67400640 1 --2- 172.21.15.164:0/3384219201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815ba70 0x7fcc68160300 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcc6815fde0 tx=0x7fcc5801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfefecb9818111f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc66000640 1 -- 172.21.15.164:0/3384219201 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580ab600 con 0x7fcc6815ba70 2024-05-08T04:42:41.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc66000640 1 -- 172.21.15.164:0/3384219201 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc580b0070 con 0x7fcc6815ba70 2024-05-08T04:42:41.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc66000640 1 -- 172.21.15.164:0/3384219201 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580ab9c0 con 0x7fcc6815ba70 2024-05-08T04:42:41.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 -- 172.21.15.164:0/3384219201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815ba70 msgr2=0x7fcc68160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3384219201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815ba70 0x7fcc68160300 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcc6815fde0 tx=0x7fcc5801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 -- 172.21.15.164:0/3384219201 shutdown_connections 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3384219201 >> v2:172.21.15.172:3300/0 conn(0x7fcc680cfda0 0x7fcc680b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3384219201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815ba70 0x7fcc68160300 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 -- 172.21.15.164:0/3384219201 >> 172.21.15.164:0/3384219201 conn(0x7fcc6806ac90 msgr2=0x7fcc6806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 -- 172.21.15.164:0/3384219201 shutdown_connections 2024-05-08T04:42:41.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 -- 172.21.15.164:0/3384219201 wait complete. 2024-05-08T04:42:41.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.532+0000 7fcc6e000640 1 Processor -- start 2024-05-08T04:42:41.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- start start 2024-05-08T04:42:41.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc680cfda0 0x7fcc681494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6815ba70 0x7fcc68148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68168d40 con 0x7fcc6815be50 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68168bc0 con 0x7fcc680cfda0 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68168ec0 con 0x7fcc6815ba70 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57358/0 (socket says 172.21.15.164:57358) 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 -- 172.21.15.164:0/3550969473 learned_addr learned my addr 172.21.15.164:0/3550969473 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67400640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc680cfda0 0x7fcc681494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc66a00640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6815ba70 0x7fcc68148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 -- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6815ba70 msgr2=0x7fcc68148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6815ba70 0x7fcc68148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 -- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc680cfda0 msgr2=0x7fcc681494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc680cfda0 0x7fcc681494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 -- 172.21.15.164:0/3550969473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc681492d0 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc67e00640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcc60007e40 tx=0x7fcc6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60014620 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc60019070 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680af210 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60014a20 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68150b10 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.536+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68069d60 con 0x7fcc6815be50 2024-05-08T04:42:41.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.540+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fcc60014bc0 con 0x7fcc6815be50 2024-05-08T04:42:41.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.540+0000 7fcc5f400640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 0x7fcc48069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.540+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fcc60084020 con 0x7fcc6815be50 2024-05-08T04:42:41.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.540+0000 7fcc67400640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 0x7fcc48069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.540+0000 7fcc67400640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 0x7fcc48069d20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc580af000 tx=0x7fcc580a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.544+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc60056490 con 0x7fcc6815be50 2024-05-08T04:42:41.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.696+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6806e210 con 0x7fcc6815be50 2024-05-08T04:42:41.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc5f400640 1 -- 172.21.15.164:0/3550969473 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16916 (secure 0 0 0) 0x7fcc6005a530 con 0x7fcc6815be50 2024-05-08T04:42:41.700 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:41.701 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":100,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:38.488862+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:20.161961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:20.781940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:41.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 msgr2=0x7fcc48069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 0x7fcc48069d20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc580af000 tx=0x7fcc580a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 msgr2=0x7fcc68154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcc60007e40 tx=0x7fcc6000e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 shutdown_connections 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcc48067860 0x7fcc48069d20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6815ba70 0x7fcc68148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc6815be50 0x7fcc68154870 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 --2- 172.21.15.164:0/3550969473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc680cfda0 0x7fcc681494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 >> 172.21.15.164:0/3550969473 conn(0x7fcc6806ac90 msgr2=0x7fcc680cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 shutdown_connections 2024-05-08T04:42:41.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.700+0000 7fcc6e000640 1 -- 172.21.15.164:0/3550969473 wait complete. 2024-05-08T04:42:41.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:42:41.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 Processor -- start 2024-05-08T04:42:41.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 -- start start 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 0x7f0cf0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0cf015be50 0x7f0cf00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf00cd9f0 con 0x7f0cf015ba70 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf015fe70 con 0x7f0cf015be50 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.856+0000 7f0cf7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00cdcd0 con 0x7f0cf00cfda0 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cefe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0cf015ba70 0x7f0cf00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46430/0 (socket says 172.21.15.164:46430) 2024-05-08T04:42:41.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cefe00640 1 -- 172.21.15.164:0/10119627 learned_addr learned my addr 172.21.15.164:0/10119627 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:41.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 --2- 172.21.15.164:0/10119627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 0x7f0cf0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cf5000640 1 --2- 172.21.15.164:0/10119627 >> v2:172.21.15.172:3300/0 conn(0x7f0cf015be50 0x7f0cf00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236996070 0 0) 0x7f0cf00cd9f0 con 0x7f0cf015ba70 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4003530 con 0x7f0cf015ba70 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 -- 172.21.15.164:0/10119627 >> v2:172.21.15.172:3300/0 conn(0x7f0cf015be50 msgr2=0x7f0cf00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 --2- 172.21.15.164:0/10119627 >> v2:172.21.15.172:3300/0 conn(0x7f0cf015be50 0x7f0cf00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 -- 172.21.15.164:0/10119627 >> v1:172.21.15.164:6789/0 conn(0x7f0cf015ba70 legacy=0x7f0cf00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359409940 0 0) 0x7f0cd4003530 con 0x7f0cf015ba70 2024-05-08T04:42:41.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 -- 172.21.15.164:0/10119627 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf015f820 con 0x7f0cf00cfda0 2024-05-08T04:42:41.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.860+0000 7f0cef400640 1 --2- 172.21.15.164:0/10119627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 0x7f0cf0160300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097c0 tx=0x7f0ce001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9857c9f975bd766e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00b0070 con 0x7f0cf00cfda0 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00ab400 con 0x7f0cf00cfda0 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0ceea00640 1 -- 172.21.15.164:0/10119627 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8001680 con 0x7f0cf00cfda0 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- 172.21.15.164:0/10119627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 msgr2=0x7f0cf0160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- 172.21.15.164:0/10119627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 0x7f0cf0160300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097c0 tx=0x7f0ce001c390 comp rx=0 tx=0).stop 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- 172.21.15.164:0/10119627 shutdown_connections 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- 172.21.15.164:0/10119627 >> v2:172.21.15.172:3300/0 conn(0x7f0cf015be50 0x7f0cf00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- 172.21.15.164:0/10119627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf00cfda0 0x7f0cf0160300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- 172.21.15.164:0/10119627 >> 172.21.15.164:0/10119627 conn(0x7f0cf006ac90 msgr2=0x7f0cf006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:41.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- 172.21.15.164:0/10119627 shutdown_connections 2024-05-08T04:42:41.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- 172.21.15.164:0/10119627 wait complete. 2024-05-08T04:42:41.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 Processor -- start 2024-05-08T04:42:41.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- start start 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf015ba70 0x7f0cf00adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf015be50 0x7f0cf014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0168d40 con 0x7f0cf00cfda0 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0168bc0 con 0x7f0cf015ba70 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0168ec0 con 0x7f0cf015be50 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cefe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cefe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57368/0 (socket says 172.21.15.164:57368) 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cefe00640 1 -- 172.21.15.164:0/422967037 learned_addr learned my addr 172.21.15.164:0/422967037 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cf5000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf015be50 0x7f0cf014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.864+0000 7f0cef400640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf015ba70 0x7f0cf00adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 -- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf015be50 msgr2=0x7f0cf014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf015be50 0x7f0cf014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 -- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf015ba70 msgr2=0x7f0cf00adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf015ba70 0x7f0cf00adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 -- 172.21.15.164:0/422967037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf014cd50 con 0x7f0cf00cfda0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cefe00640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c750 tx=0x7f0cd800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8005ae0 con 0x7f0cf00cfda0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf00af390 con 0x7f0cf00cfda0 2024-05-08T04:42:41.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8005ca0 con 0x7f0cf00cfda0 2024-05-08T04:42:41.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80072b0 con 0x7f0cf00cfda0 2024-05-08T04:42:41.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf00b03b0 con 0x7f0cf00cfda0 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf0069d60 con 0x7f0cf00cfda0 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f0cd8007450 con 0x7f0cf00cfda0 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 0x7f0ccc069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f0cd8083950 con 0x7f0cf00cfda0 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.868+0000 7f0cef400640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 0x7f0ccc069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:41.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.872+0000 7f0cef400640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 0x7f0ccc069d20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ad000 tx=0x7f0ce00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:41.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:41.872+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0cd8010300 con 0x7f0cf00cfda0 2024-05-08T04:42:42.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.060+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0cf0068e00 con 0x7f0cf00cfda0 2024-05-08T04:42:42.062 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:42:42.060+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:42:42.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.064+0000 7f0ce7e00640 1 -- 172.21.15.164:0/422967037 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0cd8056330 con 0x7f0cf00cfda0 2024-05-08T04:42:42.065 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:42:42.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 msgr2=0x7f0ccc069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 0x7f0ccc069d20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ad000 tx=0x7f0ce00a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 msgr2=0x7f0cf00afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c750 tx=0x7f0cd800cc10 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 shutdown_connections 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0ccc067860 0x7f0ccc069d20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0cf015be50 0x7f0cf014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf015ba70 0x7f0cf00adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 --2- 172.21.15.164:0/422967037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf00cfda0 0x7f0cf00afe70 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 >> 172.21.15.164:0/422967037 conn(0x7f0cf006ac90 msgr2=0x7f0cf006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 shutdown_connections 2024-05-08T04:42:42.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.068+0000 7f0cf7000640 1 -- 172.21.15.164:0/422967037 wait complete. 2024-05-08T04:42:42.089 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:42:42.134 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:42:42.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 Processor -- start 2024-05-08T04:42:42.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 -- start start 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2db40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db415e870 con 0x7f2db40cecc0 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db415eb50 con 0x7f2db415ab10 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2dba200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40cf130 con 0x7f2db406cce0 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2db3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2db40cecc0 0x7f2db40ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46448/0 (socket says 172.21.15.164:46448) 2024-05-08T04:42:42.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2db3e00640 1 -- 172.21.15.164:0/2574509325 learned_addr learned my addr 172.21.15.164:0/2574509325 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2daaa00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2db40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188770352 0 0) 0x7f2db415e870 con 0x7f2db40cecc0 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.228+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d98005740 con 0x7f2db40cecc0 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602357103 0 0) 0x7f2d98005740 con 0x7f2db40cecc0 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 msgr2=0x7f2db40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:42.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2db40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 >> v1:172.21.15.172:6789/0 conn(0x7f2db415ab10 legacy=0x7f2db40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db415ee30 con 0x7f2db40cecc0 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2dac001a00 con 0x7f2db40cecc0 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 >> v1:172.21.15.164:6789/0 conn(0x7f2db40cecc0 legacy=0x7f2db40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2d98005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d98006140 0x7f2d98006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98006f60 0x7f2d980073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40cf130 con 0x7f2d98006140 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9800dcf0 con 0x7f2d98006f60 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9800def0 con 0x7f2db406cce0 2024-05-08T04:42:42.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d98006140 0x7f2d98006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db3400640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2d98005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db3e00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98006f60 0x7f2d980073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 -- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 msgr2=0x7f2d98005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2db406cce0 0x7f2d98005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 -- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98006f60 msgr2=0x7f2d980073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98006f60 0x7f2d980073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 -- 172.21.15.164:0/2574509325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98005740 con 0x7f2d98006140 2024-05-08T04:42:42.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2daaa00640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d98006140 0x7f2d98006a20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009ae0 tx=0x7f2d9c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c036e8da290a32ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0b2070 con 0x7f2d98006140 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0a5660 con 0x7f2d98006140 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.232+0000 7f2db2a00640 1 -- 172.21.15.164:0/2574509325 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0096b0 con 0x7f2d98006140 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- 172.21.15.164:0/2574509325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d98006140 msgr2=0x7f2d98006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 --2- 172.21.15.164:0/2574509325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d98006140 0x7f2d98006a20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009ae0 tx=0x7f2d9c01ca70 comp rx=0 tx=0).stop 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2db3e00640 1 -- 172.21.15.164:0/2574509325 reap_dead start 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- 172.21.15.164:0/2574509325 shutdown_connections 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- 172.21.15.164:0/2574509325 >> 172.21.15.164:0/2574509325 conn(0x7f2db40cf770 msgr2=0x7f2db40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- 172.21.15.164:0/2574509325 shutdown_connections 2024-05-08T04:42:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- 172.21.15.164:0/2574509325 wait complete. 2024-05-08T04:42:42.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 Processor -- start 2024-05-08T04:42:42.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- start start 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d98006140 0x7f2db41533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db406d960 0x7f2db406dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db415f3f0 con 0x7f2db41582a0 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db406bb80 con 0x7f2db406d960 2024-05-08T04:42:42.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.236+0000 7f2dba200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db415aef0 con 0x7f2d98006140 2024-05-08T04:42:42.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d98006140 0x7f2db41533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57386/0 (socket says 172.21.15.164:57386) 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 -- 172.21.15.164:0/1501271400 learned_addr learned my addr 172.21.15.164:0/1501271400 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2daaa00640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db406d960 0x7f2db406dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 -- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d98006140 msgr2=0x7f2db41533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d98006140 0x7f2db41533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 -- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db406d960 msgr2=0x7f2db406dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db406d960 0x7f2db406dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:42.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 -- 172.21.15.164:0/1501271400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40b4e60 con 0x7f2db41582a0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2db3e00640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2db40cf6d0 tx=0x7f2dac007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00ab30 con 0x7f2db41582a0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac018070 con 0x7f2db41582a0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0133f0 con 0x7f2db41582a0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db4158e80 con 0x7f2db41582a0 2024-05-08T04:42:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db4159440 con 0x7f2db41582a0 2024-05-08T04:42:42.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.240+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db4072d30 con 0x7f2db41582a0 2024-05-08T04:42:42.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.244+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f2dac013590 con 0x7f2db41582a0 2024-05-08T04:42:42.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.244+0000 7f2dabe00640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 0x7f2d90069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:42.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.244+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f2dac083d80 con 0x7f2db41582a0 2024-05-08T04:42:42.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.244+0000 7f2db3400640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 0x7f2d90069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:42.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.244+0000 7f2db3400640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 0x7f2d90069d70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2da40014e0 tx=0x7f2da4007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:42.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.248+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dac056190 con 0x7f2db41582a0 2024-05-08T04:42:42.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:42.435+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2db404f010 con 0x7f2db41582a0 2024-05-08T04:42:43.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.063+0000 7f2dabe00640 1 -- 172.21.15.164:0/1501271400 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2dac05a230 con 0x7f2db41582a0 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 msgr2=0x7f2d90069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 0x7f2d90069d70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2da40014e0 tx=0x7f2da4007cd0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 msgr2=0x7f2db4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2db40cf6d0 tx=0x7f2dac007bd0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 shutdown_connections 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2d900678b0 0x7f2d90069d70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db41582a0 0x7f2db4158720 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db406d960 0x7f2db406dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 --2- 172.21.15.164:0/1501271400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d98006140 0x7f2db41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 >> 172.21.15.164:0/1501271400 conn(0x7f2db40cf770 msgr2=0x7f2db4154d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 shutdown_connections 2024-05-08T04:42:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.067+0000 7f2dba200640 1 -- 172.21.15.164:0/1501271400 wait complete. 2024-05-08T04:42:43.092 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:42:43.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 Processor -- start 2024-05-08T04:42:43.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 -- start start 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9fac0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fac0d2720 0x7f9fac0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fac0cf200 con 0x7f9fac0d04a0 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fac0b31e0 con 0x7f9fac0d2720 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac0d0e40 con 0x7f9fac1612a0 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb0a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9fac0d04a0 0x7f9fac0b0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46450/0 (socket says 172.21.15.164:46450) 2024-05-08T04:42:43.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb0a00640 1 -- 172.21.15.164:0/1630903405 learned_addr learned my addr 172.21.15.164:0/1630903405 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:43.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb1e00640 1 --2- 172.21.15.164:0/1630903405 >> v2:172.21.15.172:3300/0 conn(0x7f9fac0d2720 0x7f9fac0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9fac0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116212382 0 0) 0x7f9fac0cf200 con 0x7f9fac0d04a0 2024-05-08T04:42:43.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f94003530 con 0x7f9fac0d04a0 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562907934 0 0) 0x7f9f94003530 con 0x7f9fac0d04a0 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 msgr2=0x7f9fac0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9fac0d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 >> v2:172.21.15.172:3300/0 conn(0x7f9fac0d2720 msgr2=0x7f9fac0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 --2- 172.21.15.164:0/1630903405 >> v2:172.21.15.172:3300/0 conn(0x7f9fac0d2720 0x7f9fac0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac0cf4a0 con 0x7f9fac0d04a0 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f9f98001680 con 0x7f9fac0d04a0 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9fac0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 >> v1:172.21.15.164:6789/0 conn(0x7f9fac0d04a0 legacy=0x7f9fac0b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f940039c0 0x7f9f94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f94004810 0x7f9f94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9f940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f94004620 con 0x7f9f940039c0 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9400ba00 con 0x7f9f94004810 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9400bc00 con 0x7f9fac1612a0 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.279+0000 7f9fb1e00640 1 --2- 172.21.15.164:0/1630903405 >> v2:172.21.15.172:3300/0 conn(0x7f9fac0d2720 0x7f9fac0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f940039c0 0x7f9f94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb0a00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f94004810 0x7f9f94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1e00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9f940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 -- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 msgr2=0x7f9f940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fac1612a0 0x7f9f940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 -- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f94004810 msgr2=0x7f9f94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f94004810 0x7f9f94004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 -- 172.21.15.164:0/1630903405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94003530 con 0x7f9f940039c0 2024-05-08T04:42:43.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1400640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f940039c0 0x7f9f94003de0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000e350 tx=0x7f9fa001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b865f8c17ca275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:43.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00b2070 con 0x7f9f940039c0 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00a5b10 con 0x7f9f940039c0 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fabe00640 1 -- 172.21.15.164:0/1630903405 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0009de0 con 0x7f9f940039c0 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb3e00640 1 -- 172.21.15.164:0/1630903405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f940039c0 msgr2=0x7f9f94003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/1630903405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f940039c0 0x7f9f94003de0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000e350 tx=0x7f9fa001ce00 comp rx=0 tx=0).stop 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.283+0000 7f9fb1e00640 1 -- 172.21.15.164:0/1630903405 reap_dead start 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- 172.21.15.164:0/1630903405 shutdown_connections 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- 172.21.15.164:0/1630903405 >> 172.21.15.164:0/1630903405 conn(0x7f9fac0d3b40 msgr2=0x7f9fac0d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- 172.21.15.164:0/1630903405 shutdown_connections 2024-05-08T04:42:43.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- 172.21.15.164:0/1630903405 wait complete. 2024-05-08T04:42:43.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 Processor -- start 2024-05-08T04:42:43.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- start start 2024-05-08T04:42:43.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f940039c0 0x7f9fac0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fac1f72e0 0x7f9fac0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac161680 con 0x7f9fac0c83d0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac0d4a70 con 0x7f9fac1f72e0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac0b1bb0 con 0x7f9f940039c0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57410/0 (socket says 172.21.15.164:57410) 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 -- 172.21.15.164:0/2728098442 learned_addr learned my addr 172.21.15.164:0/2728098442 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb1400640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f940039c0 0x7f9fac0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb1e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fac1f72e0 0x7f9fac0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 -- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f940039c0 msgr2=0x7f9fac0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f940039c0 0x7f9fac0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 -- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fac1f72e0 msgr2=0x7f9fac0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fac1f72e0 0x7f9fac0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:43.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 -- 172.21.15.164:0/2728098442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac0c90f0 con 0x7f9fac0c83d0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb0a00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007310 tx=0x7f9f980077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800f9c0 con 0x7f9fac0c83d0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f98018070 con 0x7f9fac0c83d0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fac0c93d0 con 0x7f9fac0c83d0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.287+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800fdc0 con 0x7f9fac0c83d0 2024-05-08T04:42:43.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fac0c9990 con 0x7f9fac0c83d0 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac04f010 con 0x7f9fac0c83d0 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f9f9801d460 con 0x7f9fac0c83d0 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fa9600640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 0x7f9f8c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f9f98083b80 con 0x7f9fac0c83d0 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fb1400640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 0x7f9f8c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:43.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.291+0000 7f9fb1400640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 0x7f9f8c069d50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00af000 tx=0x7f9fa0004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:43.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.295+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f980563f0 con 0x7f9fac0c83d0 2024-05-08T04:42:43.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:43.451+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9fac06b510 con 0x7f9fac0c83d0 2024-05-08T04:42:44.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.071+0000 7f9fa9600640 1 -- 172.21.15.164:0/2728098442 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9f98055cf0 con 0x7f9fac0c83d0 2024-05-08T04:42:44.073 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:42:44.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.071+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 msgr2=0x7f9f8c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.071+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 0x7f9f8c069d50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00af000 tx=0x7f9fa0004ac0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.071+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 msgr2=0x7f9fac1f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.071+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007310 tx=0x7f9f980077d0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 shutdown_connections 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f8c067890 0x7f9f8c069d50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fac1f72e0 0x7f9fac0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fac0c83d0 0x7f9fac1f3f30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 --2- 172.21.15.164:0/2728098442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f940039c0 0x7f9fac0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 >> 172.21.15.164:0/2728098442 conn(0x7f9fac0d3b40 msgr2=0x7f9fac161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 shutdown_connections 2024-05-08T04:42:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.075+0000 7f9fb3e00640 1 -- 172.21.15.164:0/2728098442 wait complete. 2024-05-08T04:42:44.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:42:44.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 Processor -- start 2024-05-08T04:42:44.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 -- start start 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f67800cfb70 0x7f678006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67800cff50 0x7f67800b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6780070210 con 0x7f67800b0ec0 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678006ef70 con 0x7f67800cff50 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6785600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678006f250 con 0x7f67800cfb70 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6777400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67800cff50 0x7f67800b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6784c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f67800b0ec0 0x7f678015f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46458/0 (socket says 172.21.15.164:46458) 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6784c00640 1 -- 172.21.15.164:0/2666159206 learned_addr learned my addr 172.21.15.164:0/2666159206 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:44.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f677fe00640 1 --2- 172.21.15.164:0/2666159206 >> v2:172.21.15.172:3300/0 conn(0x7f67800cfb70 0x7f678006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6777400640 1 -- 172.21.15.164:0/2666159206 >> v2:172.21.15.172:3300/0 conn(0x7f67800cfb70 msgr2=0x7f678006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:44.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6777400640 1 --2- 172.21.15.164:0/2666159206 >> v2:172.21.15.172:3300/0 conn(0x7f67800cfb70 0x7f678006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6777400640 1 -- 172.21.15.164:0/2666159206 >> v1:172.21.15.164:6789/0 conn(0x7f67800b0ec0 legacy=0x7f678015f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f6777400640 1 -- 172.21.15.164:0/2666159206 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800704f0 con 0x7f67800cff50 2024-05-08T04:42:44.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.239+0000 7f677f400640 1 -- 172.21.15.164:0/2666159206 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247989375 0 0) 0x7f6780070210 con 0x7f67800b0ec0 2024-05-08T04:42:44.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6777400640 1 --2- 172.21.15.164:0/2666159206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800cff50 0x7f67800b0310 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f67800cf2b0 tx=0x7f676801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e72d087ee0c657 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:44.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f677f400640 1 -- 172.21.15.164:0/2666159206 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67680b2070 con 0x7f67800cff50 2024-05-08T04:42:44.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f677f400640 1 -- 172.21.15.164:0/2666159206 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67680a5980 con 0x7f67800cff50 2024-05-08T04:42:44.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f677f400640 1 -- 172.21.15.164:0/2666159206 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67680a5d50 con 0x7f67800cff50 2024-05-08T04:42:44.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 -- 172.21.15.164:0/2666159206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800cff50 msgr2=0x7f67800b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 --2- 172.21.15.164:0/2666159206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800cff50 0x7f67800b0310 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f67800cf2b0 tx=0x7f676801c8a0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 -- 172.21.15.164:0/2666159206 shutdown_connections 2024-05-08T04:42:44.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 --2- 172.21.15.164:0/2666159206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800cff50 0x7f67800b0310 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 --2- 172.21.15.164:0/2666159206 >> v2:172.21.15.172:3300/0 conn(0x7f67800cfb70 0x7f678006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 -- 172.21.15.164:0/2666159206 >> 172.21.15.164:0/2666159206 conn(0x7f678009aa20 msgr2=0x7f67800b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:44.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 -- 172.21.15.164:0/2666159206 shutdown_connections 2024-05-08T04:42:44.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.243+0000 7f6785600640 1 -- 172.21.15.164:0/2666159206 wait complete. 2024-05-08T04:42:44.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 Processor -- start 2024-05-08T04:42:44.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- start start 2024-05-08T04:42:44.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 0x7f678015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 0x7f67801f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780168e60 con 0x7f67800b0ec0 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780168ce0 con 0x7f67800cfb70 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6780168fe0 con 0x7f67800cff50 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57424/0 (socket says 172.21.15.164:57424) 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 -- 172.21.15.164:0/2876021842 learned_addr learned my addr 172.21.15.164:0/2876021842 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6784c00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 0x7f67801f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6777400640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 0x7f678015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 -- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 msgr2=0x7f67801f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 0x7f67801f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 -- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 msgr2=0x7f678015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 0x7f678015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 -- 172.21.15.164:0/2876021842 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800c6150 con 0x7f67800b0ec0 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6777400640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 0x7f678015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:44.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6784c00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 0x7f67801f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677fe00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f677000e960 tx=0x7f677000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770017070 con 0x7f67800b0ec0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677000ca00 con 0x7f67800b0ec0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67800c5b90 con 0x7f67800b0ec0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000cdb0 con 0x7f67800b0ec0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.247+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678006bf40 con 0x7f67800b0ec0 2024-05-08T04:42:44.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678004ef30 con 0x7f67800b0ec0 2024-05-08T04:42:44.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f677001a070 con 0x7f67800b0ec0 2024-05-08T04:42:44.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f677cc00640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 0x7f675c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:44.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5469+0+0 (secure 0 0 0) 0x7f6770083f10 con 0x7f67800b0ec0 2024-05-08T04:42:44.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f6777400640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 0x7f675c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:44.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.251+0000 7f6777400640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 0x7f675c069d20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67680af000 tx=0x7f67680a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:44.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.255+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6770056780 con 0x7f67800b0ec0 2024-05-08T04:42:44.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:44.419+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f678009afe0 con 0x7f67800b0ec0 2024-05-08T04:42:45.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.187+0000 7f677cc00640 1 -- 172.21.15.164:0/2876021842 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6770056080 con 0x7f67800b0ec0 2024-05-08T04:42:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:42:45.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 msgr2=0x7f675c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 0x7f675c069d20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67680af000 tx=0x7f67680a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:45.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 msgr2=0x7f67801664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f677000e960 tx=0x7f677000ee20 comp rx=0 tx=0).stop 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 shutdown_connections 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f675c067860 0x7f675c069d20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67800cff50 0x7f67801f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67800cfb70 0x7f678015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 --2- 172.21.15.164:0/2876021842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67800b0ec0 0x7f67801664e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 >> 172.21.15.164:0/2876021842 conn(0x7f678009aa20 msgr2=0x7f67800cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 shutdown_connections 2024-05-08T04:42:45.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.191+0000 7f6785600640 1 -- 172.21.15.164:0/2876021842 wait complete. 2024-05-08T04:42:45.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:42:45.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 Processor -- start 2024-05-08T04:42:45.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 -- start start 2024-05-08T04:42:45.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f563c06cce0 0x7f563c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c15e870 con 0x7f563c15ab10 2024-05-08T04:42:45.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0cf130 con 0x7f563c06cce0 2024-05-08T04:42:45.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5643800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c06bb80 con 0x7f563c0cecc0 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5641800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f563be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f563c06cce0 0x7f563c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5640e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f563c15ab10 0x7f563c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:50028/0 (socket says 172.21.15.164:50028) 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5641800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:51562/0 (socket says 172.21.15.164:51562) 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f5640e00640 1 -- 172.21.15.164:0/1192578157 learned_addr learned my addr 172.21.15.164:0/1192578157 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f563be00640 1 -- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 msgr2=0x7f563c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f563be00640 1 --2- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f563be00640 1 -- 172.21.15.164:0/1192578157 >> v1:172.21.15.172:6789/0 conn(0x7f563c15ab10 legacy=0x7f563c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.363+0000 7f563be00640 1 -- 172.21.15.164:0/1192578157 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c15eb50 con 0x7f563c06cce0 2024-05-08T04:42:45.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f563be00640 1 --2- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c06cce0 0x7f563c0ce780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb35b8e89b63702 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f563b400640 1 -- 172.21.15.164:0/1192578157 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280b1070 con 0x7f563c06cce0 2024-05-08T04:42:45.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f563b400640 1 -- 172.21.15.164:0/1192578157 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56280a8050 con 0x7f563c06cce0 2024-05-08T04:42:45.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f563b400640 1 -- 172.21.15.164:0/1192578157 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280ac860 con 0x7f563c06cce0 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 -- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c06cce0 msgr2=0x7f563c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 --2- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c06cce0 0x7f563c0ce780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 -- 172.21.15.164:0/1192578157 shutdown_connections 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 --2- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 --2- 172.21.15.164:0/1192578157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c06cce0 0x7f563c0ce780 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 -- 172.21.15.164:0/1192578157 >> 172.21.15.164:0/1192578157 conn(0x7f563c0cf770 msgr2=0x7f563c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 -- 172.21.15.164:0/1192578157 shutdown_connections 2024-05-08T04:42:45.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.367+0000 7f5643800640 1 -- 172.21.15.164:0/1192578157 wait complete. 2024-05-08T04:42:45.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 Processor -- start 2024-05-08T04:42:45.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- start start 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c15ab10 0x7f563c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0aedf0 con 0x7f563c06cce0 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c15aef0 con 0x7f563c15ab10 2024-05-08T04:42:45.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0aef70 con 0x7f563c0cecc0 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57430/0 (socket says 172.21.15.164:57430) 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 -- 172.21.15.164:0/2161851828 learned_addr learned my addr 172.21.15.164:0/2161851828 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5641800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c15ab10 0x7f563c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f563be00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 -- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 msgr2=0x7f563c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 -- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c15ab10 msgr2=0x7f563c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c15ab10 0x7f563c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 -- 172.21.15.164:0/2161851828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0b58f0 con 0x7f563c06cce0 2024-05-08T04:42:45.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5640e00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f563000c370 tx=0x7f563000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630017070 con 0x7f563c06cce0 2024-05-08T04:42:45.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630005cf0 con 0x7f563c06cce0 2024-05-08T04:42:45.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300072f0 con 0x7f563c06cce0 2024-05-08T04:42:45.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c153d50 con 0x7f563c06cce0 2024-05-08T04:42:45.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c158ef0 con 0x7f563c06cce0 2024-05-08T04:42:45.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.371+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c072d30 con 0x7f563c06cce0 2024-05-08T04:42:45.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.375+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f563001a070 con 0x7f563c06cce0 2024-05-08T04:42:45.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.375+0000 7f5638c00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 0x7f561c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.375+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f5630082c00 con 0x7f563c06cce0 2024-05-08T04:42:45.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.375+0000 7f563be00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 0x7f561c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.375+0000 7f563be00640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 0x7f561c069d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f56280ae000 tx=0x7f56280a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.379+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5630055010 con 0x7f563c06cce0 2024-05-08T04:42:45.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.539+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f563c04f010 con 0x7f563c06cce0 2024-05-08T04:42:45.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.543+0000 7f5638c00640 1 -- 172.21.15.164:0/2161851828 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56300590b0 con 0x7f563c06cce0 2024-05-08T04:42:45.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.543+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 msgr2=0x7f561c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.543+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 0x7f561c069d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f56280ae000 tx=0x7f56280a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 msgr2=0x7f563c071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f563000c370 tx=0x7f563000c830 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 shutdown_connections 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f561c067860 0x7f561c069d20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f563c0cecc0 0x7f563c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f563c15ab10 0x7f563c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 --2- 172.21.15.164:0/2161851828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f563c06cce0 0x7f563c071650 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 >> 172.21.15.164:0/2161851828 conn(0x7f563c0cf770 msgr2=0x7f563c14af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 shutdown_connections 2024-05-08T04:42:45.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.547+0000 7f5643800640 1 -- 172.21.15.164:0/2161851828 wait complete. 2024-05-08T04:42:45.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm' 2024-05-08T04:42:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.699+0000 7faf3a600640 1 Processor -- start 2024-05-08T04:42:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.699+0000 7faf3a600640 1 -- start start 2024-05-08T04:42:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf3a600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf3415be50 0x7faf34160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf3a600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf340cd9f0 con 0x7faf3415ba70 2024-05-08T04:42:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf3a600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf340cdcd0 con 0x7faf340cfda0 2024-05-08T04:42:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf3a600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3415fe70 con 0x7faf3415be50 2024-05-08T04:42:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7faf3415ba70 0x7faf340cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46470/0 (socket says 172.21.15.164:46470) 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33400640 1 -- 172.21.15.164:0/1561312579 learned_addr learned my addr 172.21.15.164:0/1561312579 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33e00640 1 --2- 172.21.15.164:0/1561312579 >> v2:172.21.15.164:3300/0 conn(0x7faf3415be50 0x7faf34160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf32a00640 1 -- 172.21.15.164:0/1561312579 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341705183 0 0) 0x7faf340cd9f0 con 0x7faf3415ba70 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf32a00640 1 -- 172.21.15.164:0/1561312579 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf18003530 con 0x7faf3415ba70 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33e00640 1 -- 172.21.15.164:0/1561312579 >> v1:172.21.15.172:6789/0 conn(0x7faf340cfda0 legacy=0x7faf340b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33e00640 1 -- 172.21.15.164:0/1561312579 >> v1:172.21.15.164:6789/0 conn(0x7faf3415ba70 legacy=0x7faf340cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.703+0000 7faf33e00640 1 -- 172.21.15.164:0/1561312579 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3415f820 con 0x7faf3415be50 2024-05-08T04:42:45.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf33e00640 1 --2- 172.21.15.164:0/1561312579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415be50 0x7faf34160300 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097c0 tx=0x7faf2c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b087cdfe05ad47b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf32a00640 1 -- 172.21.15.164:0/1561312579 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0b2070 con 0x7faf3415be50 2024-05-08T04:42:45.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf32a00640 1 -- 172.21.15.164:0/1561312579 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c0ad680 con 0x7faf3415be50 2024-05-08T04:42:45.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf32a00640 1 -- 172.21.15.164:0/1561312579 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0ada30 con 0x7faf3415be50 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- 172.21.15.164:0/1561312579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415be50 msgr2=0x7faf34160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 --2- 172.21.15.164:0/1561312579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415be50 0x7faf34160300 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097c0 tx=0x7faf2c01c470 comp rx=0 tx=0).stop 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- 172.21.15.164:0/1561312579 shutdown_connections 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 --2- 172.21.15.164:0/1561312579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415be50 0x7faf34160300 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- 172.21.15.164:0/1561312579 >> 172.21.15.164:0/1561312579 conn(0x7faf3406ac90 msgr2=0x7faf3406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- 172.21.15.164:0/1561312579 shutdown_connections 2024-05-08T04:42:45.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- 172.21.15.164:0/1561312579 wait complete. 2024-05-08T04:42:45.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 Processor -- start 2024-05-08T04:42:45.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.707+0000 7faf3a600640 1 -- start start 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf340cfda0 0x7faf3415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 0x7faf3414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf34169f60 con 0x7faf3415ba70 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf34169de0 con 0x7faf3415be50 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3416a0e0 con 0x7faf340cfda0 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57450/0 (socket says 172.21.15.164:57450) 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 -- 172.21.15.164:0/1651031253 learned_addr learned my addr 172.21.15.164:0/1651031253 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf33400640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf340cfda0 0x7faf3415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf33e00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 0x7faf3414afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 -- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf340cfda0 msgr2=0x7faf3415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf340cfda0 0x7faf3415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 -- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 msgr2=0x7faf3414afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 0x7faf3414afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 -- 172.21.15.164:0/1651031253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3415d250 con 0x7faf3415ba70 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf33e00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 0x7faf3414afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2aa00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c500 tx=0x7faf1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c016020 con 0x7faf3415ba70 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c005220 con 0x7faf3415ba70 2024-05-08T04:42:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3415d530 con 0x7faf3415ba70 2024-05-08T04:42:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c007030 con 0x7faf3415ba70 2024-05-08T04:42:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3415daf0 con 0x7faf3415ba70 2024-05-08T04:42:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf34069d60 con 0x7faf3415ba70 2024-05-08T04:42:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.711+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7faf1c0071f0 con 0x7faf3415ba70 2024-05-08T04:42:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.715+0000 7faf2be00640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 0x7faf10069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.715+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7faf1c0833b0 con 0x7faf3415ba70 2024-05-08T04:42:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.715+0000 7faf33400640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 0x7faf10069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.715+0000 7faf33400640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 0x7faf10069d20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf24001570 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:45.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.719+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf1c020020 con 0x7faf3415ba70 2024-05-08T04:42:45.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:45.923+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm"]} v 0) v1 -- 0x7faf3415f600 con 0x7faf3415ba70 2024-05-08T04:42:46.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf2be00640 1 -- 172.21.15.164:0/1651031253 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_rm"]}]=0 v270) v1 ==== 164+0+0 (secure 0 0 0) 0x7faf1c055d90 con 0x7faf3415ba70 2024-05-08T04:42:46.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 msgr2=0x7faf10069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 0x7faf10069d20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf24001570 tx=0x7faf24009040 comp rx=0 tx=0).stop 2024-05-08T04:42:46.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 msgr2=0x7faf3414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c500 tx=0x7faf1c00c9c0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 shutdown_connections 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7faf10067860 0x7faf10069d20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf3415be50 0x7faf3414afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf3415ba70 0x7faf3414a140 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 --2- 172.21.15.164:0/1651031253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf340cfda0 0x7faf3415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 >> 172.21.15.164:0/1651031253 conn(0x7faf3406ac90 msgr2=0x7faf3406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 shutdown_connections 2024-05-08T04:42:46.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.195+0000 7faf3a600640 1 -- 172.21.15.164:0/1651031253 wait complete. 2024-05-08T04:42:46.218 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot rm' command ... ok 2024-05-08T04:42:46.218 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_retain_snapshot_invalid_recreate (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ensure retained subvolume recreate does not leave any incarnations in the subvolume and trash 2024-05-08T04:42:46.218 INFO:tasks.cephfs_test_runner:test_subvolume_retain_snapshot_invalid_recreate (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:42:46.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:46.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.351+0000 7f81c1200640 1 Processor -- start 2024-05-08T04:42:46.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.351+0000 7f81c1200640 1 -- start start 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.351+0000 7f81c1200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81bc0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.351+0000 7f81c1200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0c49e0 con 0x7f81bc0c4420 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.351+0000 7f81c1200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0c4cc0 con 0x7f81bc15fd30 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81c1200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c4fa0 con 0x7f81bc159760 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81baa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f81bc0c4420 0x7f81bc15f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46474/0 (socket says 172.21.15.164:46474) 2024-05-08T04:42:46.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81baa00640 1 -- 172.21.15.164:0/338784037 learned_addr learned my addr 172.21.15.164:0/338784037 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:46.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bb400640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81bc0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935845300 0 0) 0x7f81bc0c49e0 con 0x7f81bc0c4420 2024-05-08T04:42:46.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0003530 con 0x7f81bc0c4420 2024-05-08T04:42:46.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3929841736 0 0) 0x7f81a0003530 con 0x7f81bc0c4420 2024-05-08T04:42:46.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 msgr2=0x7f81bc0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81bc0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 >> v1:172.21.15.172:6789/0 conn(0x7f81bc15fd30 legacy=0x7f81bc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0c5280 con 0x7f81bc0c4420 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f81a8001680 con 0x7f81bc0c4420 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 >> v1:172.21.15.164:6789/0 conn(0x7f81bc0c4420 legacy=0x7f81bc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a00039f0 0x7f81a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0004d50 0x7f81a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c4fa0 con 0x7f81a0004d50 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a000ba80 con 0x7f81a00039f0 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a000bc80 con 0x7f81bc159760 2024-05-08T04:42:46.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0004d50 0x7f81a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bb400640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a00039f0 0x7f81a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81baa00640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 -- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 msgr2=0x7f81a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc159760 0x7f81a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 -- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a00039f0 msgr2=0x7f81a0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a00039f0 0x7f81a0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.355+0000 7f81bbe00640 1 -- 172.21.15.164:0/338784037 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0003530 con 0x7f81a0004d50 2024-05-08T04:42:46.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81bbe00640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0004d50 0x7f81a00051d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f81b4001750 tx=0x7f81b401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2981f57b802fd613 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40013a0 con 0x7f81a0004d50 2024-05-08T04:42:46.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b40052f0 con 0x7f81a0004d50 2024-05-08T04:42:46.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81ba000640 1 -- 172.21.15.164:0/338784037 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4003840 con 0x7f81a0004d50 2024-05-08T04:42:46.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- 172.21.15.164:0/338784037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0004d50 msgr2=0x7f81a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 --2- 172.21.15.164:0/338784037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0004d50 0x7f81a00051d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f81b4001750 tx=0x7f81b401c3e0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81bbe00640 1 -- 172.21.15.164:0/338784037 reap_dead start 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- 172.21.15.164:0/338784037 shutdown_connections 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- 172.21.15.164:0/338784037 >> 172.21.15.164:0/338784037 conn(0x7f81bc06b670 msgr2=0x7f81bc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- 172.21.15.164:0/338784037 shutdown_connections 2024-05-08T04:42:46.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- 172.21.15.164:0/338784037 wait complete. 2024-05-08T04:42:46.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 Processor -- start 2024-05-08T04:42:46.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- start start 2024-05-08T04:42:46.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 0x7f81bc0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 0x7f81bc0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c5ff0 con 0x7f81bc0c6630 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c5e70 con 0x7f81a0004d50 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.359+0000 7f81c1200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c6170 con 0x7f81bc1f1a80 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57464/0 (socket says 172.21.15.164:57464) 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 -- 172.21.15.164:0/1356073989 learned_addr learned my addr 172.21.15.164:0/1356073989 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81bbe00640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 0x7f81bc0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81bb400640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 0x7f81bc0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 -- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 msgr2=0x7f81bc0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 0x7f81bc0c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 -- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 msgr2=0x7f81bc0c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 0x7f81bc0c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 -- 172.21.15.164:0/1356073989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0c6da0 con 0x7f81bc0c6630 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81bb400640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 0x7f81bc0c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81bbe00640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 0x7f81bc0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81baa00640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007290 tx=0x7f81a8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800f940 con 0x7f81bc0c6630 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8018070 con 0x7f81bc0c6630 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800fd40 con 0x7f81bc0c6630 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc13dc70 con 0x7f81bc0c6630 2024-05-08T04:42:46.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc13e230 con 0x7f81bc0c6630 2024-05-08T04:42:46.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc04ef30 con 0x7f81bc0c6630 2024-05-08T04:42:46.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f81a801d460 con 0x7f81bc0c6630 2024-05-08T04:42:46.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.363+0000 7f81b3400640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 0x7f819c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.367+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f81a8083c10 con 0x7f81bc0c6630 2024-05-08T04:42:46.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.367+0000 7f81bb400640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 0x7f819c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.367+0000 7f81bb400640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 0x7f819c069d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0074d0 tx=0x7f81ac00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.367+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81a8056480 con 0x7f81bc0c6630 2024-05-08T04:42:46.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.527+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81bc0681a0 con 0x7f81bc0c6630 2024-05-08T04:42:46.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81b3400640 1 -- 172.21.15.164:0/1356073989 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13218 (secure 0 0 0) 0x7f81a8055d80 con 0x7f81bc0c6630 2024-05-08T04:42:46.531 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:46.531 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":104,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:45.188602+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/4096204626":"2024-05-09T04:42:42.063397+0000","172.21.15.164:6835/4096204626":"2024-05-09T04:42:42.063397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:46.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 msgr2=0x7f819c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 0x7f819c069d50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0074d0 tx=0x7f81ac00b040 comp rx=0 tx=0).stop 2024-05-08T04:42:46.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 msgr2=0x7f81bc1f1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007290 tx=0x7f81a8007750 comp rx=0 tx=0).stop 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 shutdown_connections 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f819c067890 0x7f819c069d50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81bc1f1a80 0x7f81bc0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc0c6630 0x7f81bc1f1540 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 --2- 172.21.15.164:0/1356073989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a0004d50 0x7f81bc0c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 >> 172.21.15.164:0/1356073989 conn(0x7f81bc06b670 msgr2=0x7f81bc071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 shutdown_connections 2024-05-08T04:42:46.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.531+0000 7f81c1200640 1 -- 172.21.15.164:0/1356073989 wait complete. 2024-05-08T04:42:46.553 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:46.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:46.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 Processor -- start 2024-05-08T04:42:46.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 -- start start 2024-05-08T04:42:46.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600b0b90 0x7f166006c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16600ce900 0x7f16600cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166006eea0 con 0x7f166006ce60 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166006c490 con 0x7f16600ce900 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166006f120 con 0x7f16600b0b90 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16600ce900 0x7f16600cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f165f400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f166006ce60 0x7f16600ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46480/0 (socket says 172.21.15.164:46480) 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f165f400640 1 -- 172.21.15.164:0/1978892628 learned_addr learned my addr 172.21.15.164:0/1978892628 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f165fe00640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600b0b90 0x7f166006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665200640 1 -- 172.21.15.164:0/1978892628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600b0b90 msgr2=0x7f166006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665200640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600b0b90 0x7f166006c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665200640 1 -- 172.21.15.164:0/1978892628 >> v1:172.21.15.164:6789/0 conn(0x7f166006ce60 legacy=0x7f16600ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f165ea00640 1 -- 172.21.15.164:0/1978892628 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311595659 0 0) 0x7f1648001120 con 0x7f166006ce60 2024-05-08T04:42:46.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.691+0000 7f1665200640 1 -- 172.21.15.164:0/1978892628 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166006f400 con 0x7f16600ce900 2024-05-08T04:42:46.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665200640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600ce900 0x7f16600cf970 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1658009d90 tx=0x7f165801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d5beb1f060fd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f165ea00640 1 -- 172.21.15.164:0/1978892628 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580b2070 con 0x7f16600ce900 2024-05-08T04:42:46.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f165ea00640 1 -- 172.21.15.164:0/1978892628 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16580a5920 con 0x7f16600ce900 2024-05-08T04:42:46.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f165ea00640 1 -- 172.21.15.164:0/1978892628 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580a5cf0 con 0x7f16600ce900 2024-05-08T04:42:46.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- 172.21.15.164:0/1978892628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600ce900 msgr2=0x7f16600cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600ce900 0x7f16600cf970 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1658009d90 tx=0x7f165801c970 comp rx=0 tx=0).stop 2024-05-08T04:42:46.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- 172.21.15.164:0/1978892628 shutdown_connections 2024-05-08T04:42:46.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600ce900 0x7f16600cf970 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 --2- 172.21.15.164:0/1978892628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600b0b90 0x7f166006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- 172.21.15.164:0/1978892628 >> 172.21.15.164:0/1978892628 conn(0x7f16600b3370 msgr2=0x7f16600b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:46.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- 172.21.15.164:0/1978892628 shutdown_connections 2024-05-08T04:42:46.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- 172.21.15.164:0/1978892628 wait complete. 2024-05-08T04:42:46.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 Processor -- start 2024-05-08T04:42:46.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.695+0000 7f1665c00640 1 -- start start 2024-05-08T04:42:46.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 0x7f16600b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600ce900 0x7f166015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600d0030 con 0x7f16600b0b90 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600cfeb0 con 0x7f166006ce60 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16600d01b0 con 0x7f16600ce900 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57476/0 (socket says 172.21.15.164:57476) 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 -- 172.21.15.164:0/1899714605 learned_addr learned my addr 172.21.15.164:0/1899714605 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165fe00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 0x7f16600b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665200640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600ce900 0x7f166015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 -- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600ce900 msgr2=0x7f166015c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600ce900 0x7f166015c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 -- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 msgr2=0x7f16600b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 0x7f16600b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 -- 172.21.15.164:0/1899714605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166009eac0 con 0x7f16600b0b90 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165fe00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 0x7f16600b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f165f400640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f164800c7b0 tx=0x7f164800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648018070 con 0x7f16600b0b90 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16480072f0 con 0x7f16600b0b90 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166009eda0 con 0x7f16600b0b90 2024-05-08T04:42:46.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16480047f0 con 0x7f16600b0b90 2024-05-08T04:42:46.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166009f360 con 0x7f16600b0b90 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166006b900 con 0x7f16600b0b90 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.699+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f164801b070 con 0x7f16600b0b90 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.703+0000 7f1657e00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 0x7f163c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.703+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f16480853b0 con 0x7f16600b0b90 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.703+0000 7f165fe00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 0x7f163c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:46.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.703+0000 7f165fe00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 0x7f163c069d20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1650007a70 tx=0x7f1650008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:46.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.707+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f166006eea0 con 0x7f16600b0b90 2024-05-08T04:42:46.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.859+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f166004ef30 con 0x7f16600b0b90 2024-05-08T04:42:46.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.859+0000 7f1657e00640 1 -- 172.21.15.164:0/1899714605 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16480557d0 con 0x7f16600b0b90 2024-05-08T04:42:46.862 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 msgr2=0x7f163c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 0x7f163c069d20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1650007a70 tx=0x7f1650008040 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 msgr2=0x7f166009c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f164800c7b0 tx=0x7f164800cc70 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 shutdown_connections 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f163c067860 0x7f163c069d20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16600ce900 0x7f166015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16600b0b90 0x7f166009c820 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 --2- 172.21.15.164:0/1899714605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166006ce60 0x7f16600b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:46.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 >> 172.21.15.164:0/1899714605 conn(0x7f16600b3370 msgr2=0x7f166006fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:46.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 shutdown_connections 2024-05-08T04:42:46.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:46.863+0000 7f1665c00640 1 -- 172.21.15.164:0/1899714605 wait complete. 2024-05-08T04:42:46.883 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate' 2024-05-08T04:42:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 Processor -- start 2024-05-08T04:42:47.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 -- start start 2024-05-08T04:42:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa31c161950 0x7fa31c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa31c15ea20 0x7fa31c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c161fe0 con 0x7fa31c144200 2024-05-08T04:42:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c162880 con 0x7fa31c161950 2024-05-08T04:42:47.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa323e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c1622c0 con 0x7fa31c15ea20 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa320a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa31c161950 0x7fa31c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa321400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa31c144200 0x7fa31c0cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46488/0 (socket says 172.21.15.164:46488) 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa321400640 1 -- 172.21.15.164:0/1006624438 learned_addr learned my addr 172.21.15.164:0/1006624438 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa321e00640 1 --2- 172.21.15.164:0/1006624438 >> v2:172.21.15.172:3300/0 conn(0x7fa31c15ea20 0x7fa31c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa320a00640 1 -- 172.21.15.164:0/1006624438 >> v2:172.21.15.172:3300/0 conn(0x7fa31c15ea20 msgr2=0x7fa31c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa320a00640 1 --2- 172.21.15.164:0/1006624438 >> v2:172.21.15.172:3300/0 conn(0x7fa31c15ea20 0x7fa31c15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa320a00640 1 -- 172.21.15.164:0/1006624438 >> v1:172.21.15.164:6789/0 conn(0x7fa31c144200 legacy=0x7fa31c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:47.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa320a00640 1 -- 172.21.15.164:0/1006624438 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1625a0 con 0x7fa31c161950 2024-05-08T04:42:47.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.019+0000 7fa31be00640 1 -- 172.21.15.164:0/1006624438 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070395 0 0) 0x7fa31c161fe0 con 0x7fa31c144200 2024-05-08T04:42:47.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa320a00640 1 --2- 172.21.15.164:0/1006624438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c161950 0x7fa31c0cfd60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa31c067400 tx=0x7fa30801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3b983ade21963c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa31be00640 1 -- 172.21.15.164:0/1006624438 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080b0070 con 0x7fa31c161950 2024-05-08T04:42:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa31be00640 1 -- 172.21.15.164:0/1006624438 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3080ab600 con 0x7fa31c161950 2024-05-08T04:42:47.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa31be00640 1 -- 172.21.15.164:0/1006624438 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080ab9d0 con 0x7fa31c161950 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 -- 172.21.15.164:0/1006624438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c161950 msgr2=0x7fa31c0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 --2- 172.21.15.164:0/1006624438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c161950 0x7fa31c0cfd60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa31c067400 tx=0x7fa30801c3f0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 -- 172.21.15.164:0/1006624438 shutdown_connections 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 --2- 172.21.15.164:0/1006624438 >> v2:172.21.15.172:3300/0 conn(0x7fa31c15ea20 0x7fa31c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 --2- 172.21.15.164:0/1006624438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c161950 0x7fa31c0cfd60 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 -- 172.21.15.164:0/1006624438 >> 172.21.15.164:0/1006624438 conn(0x7fa31c06ac90 msgr2=0x7fa31c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 -- 172.21.15.164:0/1006624438 shutdown_connections 2024-05-08T04:42:47.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.023+0000 7fa323e00640 1 -- 172.21.15.164:0/1006624438 wait complete. 2024-05-08T04:42:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 Processor -- start 2024-05-08T04:42:47.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- start start 2024-05-08T04:42:47.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 0x7fa31c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 0x7fa31c13e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cf370 con 0x7fa31c144200 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cf1f0 con 0x7fa31c161950 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c16acf0 con 0x7fa31c15ea20 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57492/0 (socket says 172.21.15.164:57492) 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 -- 172.21.15.164:0/3472835489 learned_addr learned my addr 172.21.15.164:0/3472835489 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 0x7fa31c13e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa320a00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 0x7fa31c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 -- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 msgr2=0x7fa31c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:47.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 0x7fa31c15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 -- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 msgr2=0x7fa31c13e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 0x7fa31c13e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 -- 172.21.15.164:0/3472835489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1409b0 con 0x7fa31c144200 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa320a00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 0x7fa31c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 0x7fa31c13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa321400640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7c0 tx=0x7fa31000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310018070 con 0x7fa31c144200 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31000a640 con 0x7fa31c144200 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000aa30 con 0x7fa31c144200 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c142ab0 con 0x7fa31c144200 2024-05-08T04:42:47.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c141fd0 con 0x7fa31c144200 2024-05-08T04:42:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.027+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c069d60 con 0x7fa31c144200 2024-05-08T04:42:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.031+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fa31000abd0 con 0x7fa31c144200 2024-05-08T04:42:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.031+0000 7fa319600640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 0x7fa2fc069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.031+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fa310082eb0 con 0x7fa31c144200 2024-05-08T04:42:47.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.031+0000 7fa320a00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 0x7fa2fc069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:47.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.031+0000 7fa320a00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 0x7fa2fc069d20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3080ad000 tx=0x7fa308008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:47.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.035+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa310055910 con 0x7fa31c144200 2024-05-08T04:42:47.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:47.251+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate"]} v 0) v1 -- 0x7fa31c161550 con 0x7fa31c144200 2024-05-08T04:42:48.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa319600640 1 -- 172.21.15.164:0/3472835489 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate"]}]=0 v272) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa310055210 con 0x7fa31c144200 2024-05-08T04:42:48.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 msgr2=0x7fa2fc069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 0x7fa2fc069d20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3080ad000 tx=0x7fa308008040 comp rx=0 tx=0).stop 2024-05-08T04:42:48.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 msgr2=0x7fa31c1402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7c0 tx=0x7fa31000cc80 comp rx=0 tx=0).stop 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 shutdown_connections 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2fc067860 0x7fa2fc069d20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa31c15ea20 0x7fa31c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31c161950 0x7fa31c13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 --2- 172.21.15.164:0/3472835489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c144200 0x7fa31c1402d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 >> 172.21.15.164:0/3472835489 conn(0x7fa31c06ac90 msgr2=0x7fa31c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 shutdown_connections 2024-05-08T04:42:48.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.203+0000 7fa323e00640 1 -- 172.21.15.164:0/3472835489 wait complete. 2024-05-08T04:42:48.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:42:48.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.367+0000 7f6285c00640 1 Processor -- start 2024-05-08T04:42:48.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f6285c00640 1 -- start start 2024-05-08T04:42:48.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f6285c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f62800d00a0 0x7f62800ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:48.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f6285c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628015f7b0 con 0x7f62800b2600 2024-05-08T04:42:48.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f6285c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628015e320 con 0x7f62800cee50 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f6285c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62800d07e0 con 0x7f62800d00a0 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f62800b2600 0x7f62800cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46500/0 (socket says 172.21.15.164:46500) 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f62800d00a0 0x7f62800ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627e000640 1 -- 172.21.15.164:0/1272635269 learned_addr learned my addr 172.21.15.164:0/1272635269 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627f400640 1 -- 172.21.15.164:0/1272635269 >> v1:172.21.15.172:6789/0 conn(0x7f62800cee50 legacy=0x7f628015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627f400640 1 -- 172.21.15.164:0/1272635269 >> v1:172.21.15.164:6789/0 conn(0x7f62800b2600 legacy=0x7f62800cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627f400640 1 -- 172.21.15.164:0/1272635269 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800b22c0 con 0x7f62800d00a0 2024-05-08T04:42:48.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.371+0000 7f627d600640 1 -- 172.21.15.164:0/1272635269 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972835763 0 0) 0x7f628015f7b0 con 0x7f62800b2600 2024-05-08T04:42:48.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f627f400640 1 --2- 172.21.15.164:0/1272635269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800d00a0 0x7f62800ae7d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6278009d90 tx=0x7f627801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e091f7859bde9594 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:48.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f627d600640 1 -- 172.21.15.164:0/1272635269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780b2070 con 0x7f62800d00a0 2024-05-08T04:42:48.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f627d600640 1 -- 172.21.15.164:0/1272635269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62780a5a20 con 0x7f62800d00a0 2024-05-08T04:42:48.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f627d600640 1 -- 172.21.15.164:0/1272635269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a5df0 con 0x7f62800d00a0 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- 172.21.15.164:0/1272635269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800d00a0 msgr2=0x7f62800ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 --2- 172.21.15.164:0/1272635269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800d00a0 0x7f62800ae7d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6278009d90 tx=0x7f627801c940 comp rx=0 tx=0).stop 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- 172.21.15.164:0/1272635269 shutdown_connections 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 --2- 172.21.15.164:0/1272635269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800d00a0 0x7f62800ae7d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- 172.21.15.164:0/1272635269 >> 172.21.15.164:0/1272635269 conn(0x7f62800b34f0 msgr2=0x7f62800b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- 172.21.15.164:0/1272635269 shutdown_connections 2024-05-08T04:42:48.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- 172.21.15.164:0/1272635269 wait complete. 2024-05-08T04:42:48.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 Processor -- start 2024-05-08T04:42:48.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.375+0000 7f6285c00640 1 -- start start 2024-05-08T04:42:48.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:48.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62800cee50 0x7f628006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:48.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62800d00a0 0x7f628006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62800b1f70 con 0x7f62800b2600 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62800af120 con 0x7f62800d00a0 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628016a280 con 0x7f62800cee50 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57506/0 (socket says 172.21.15.164:57506) 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 -- 172.21.15.164:0/3687304760 learned_addr learned my addr 172.21.15.164:0/3687304760 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627f400640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62800d00a0 0x7f628006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627e000640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62800cee50 0x7f628006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 -- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62800cee50 msgr2=0x7f628006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62800cee50 0x7f628006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 -- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62800d00a0 msgr2=0x7f628006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62800d00a0 0x7f628006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 -- 172.21.15.164:0/3687304760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800c7d80 con 0x7f62800b2600 2024-05-08T04:42:48.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f627ea00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f627000c530 tx=0x7f627000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:48.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270016020 con 0x7f62800b2600 2024-05-08T04:42:48.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6270005220 con 0x7f62800b2600 2024-05-08T04:42:48.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270007030 con 0x7f62800b2600 2024-05-08T04:42:48.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62800c8060 con 0x7f62800b2600 2024-05-08T04:42:48.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628006dcc0 con 0x7f62800b2600 2024-05-08T04:42:48.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628009fb50 con 0x7f62800b2600 2024-05-08T04:42:48.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.379+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f62700071d0 con 0x7f62800b2600 2024-05-08T04:42:48.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.383+0000 7f6276a00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 0x7f6260069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:48.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.383+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f6270083330 con 0x7f62800b2600 2024-05-08T04:42:48.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.383+0000 7f627e000640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 0x7f6260069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:48.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.383+0000 7f627e000640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 0x7f6260069d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f626c007fd0 tx=0x7f626c007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:48.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.387+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6270020020 con 0x7f62800b2600 2024-05-08T04:42:48.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.539+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f62800ce2e0 con 0x7f62800b2600 2024-05-08T04:42:48.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.543+0000 7f6276a00640 1 -- 172.21.15.164:0/3687304760 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f62700074c0 con 0x7f62800b2600 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 msgr2=0x7f6260069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 0x7f6260069d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f626c007fd0 tx=0x7f626c007ba0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 msgr2=0x7f6280155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f627000c530 tx=0x7f627000c9f0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 shutdown_connections 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6260067860 0x7f6260069d20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62800cee50 0x7f628006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62800d00a0 0x7f628006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 --2- 172.21.15.164:0/3687304760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62800b2600 0x7f6280155f80 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:48.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 >> 172.21.15.164:0/3687304760 conn(0x7f62800b34f0 msgr2=0x7f628015ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:48.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 shutdown_connections 2024-05-08T04:42:48.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:48.547+0000 7f6285c00640 1 -- 172.21.15.164:0/3687304760 wait complete. 2024-05-08T04:42:48.568 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:42:48.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:48.666 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:42:48.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:42:48.765 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:48.765 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:48.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:48.775 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:48.776 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:48.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:48.785 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:42:48.785 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:48.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:42:48.831 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:42:48.831 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:42:48.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:42:48.840 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:42:48.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:49.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.011+0000 7fd1b3800640 1 Processor -- start 2024-05-08T04:42:49.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.011+0000 7fd1b3800640 1 -- start start 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b3800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1ac144200 0x7fd1ac161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b3800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1ac15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b3800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac161fe0 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b3800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac162880 con 0x7fd1ac144200 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b3800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1622c0 con 0x7fd1ac15ea20 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd1ac0cfd60 0x7fd1ac0cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46516/0 (socket says 172.21.15.164:46516) 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 -- 172.21.15.164:0/2614188481 learned_addr learned my addr 172.21.15.164:0/2614188481 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b1800640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1ac15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b0e00640 1 --2- 172.21.15.164:0/2614188481 >> v2:172.21.15.172:3300/0 conn(0x7fd1ac144200 0x7fd1ac161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368488036 0 0) 0x7fd1ac161fe0 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd190003530 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 929105210 0 0) 0x7fd190003530 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 msgr2=0x7fd1ac15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1ac15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 >> v2:172.21.15.172:3300/0 conn(0x7fd1ac144200 msgr2=0x7fd1ac161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 --2- 172.21.15.164:0/2614188481 >> v2:172.21.15.172:3300/0 conn(0x7fd1ac144200 0x7fd1ac161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac1625a0 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd1940016a0 con 0x7fd1ac0cfd60 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 >> v1:172.21.15.164:6789/0 conn(0x7fd1ac0cfd60 legacy=0x7fd1ac0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1900039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd190003f40 0x7fd190004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd190004d50 0x7fd1900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac162880 con 0x7fd190003f40 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1622c0 con 0x7fd190004d50 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19000ba20 con 0x7fd1ac15ea20 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd190003f40 0x7fd190004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b0e00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1900039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1b1800640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd190004d50 0x7fd1900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 -- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 msgr2=0x7fd1900039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1ac15ea20 0x7fd1900039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 -- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd190004d50 msgr2=0x7fd1900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd190004d50 0x7fd1900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.015+0000 7fd1a3e00640 1 -- 172.21.15.164:0/2614188481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190003530 con 0x7fd190003f40 2024-05-08T04:42:49.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1a3e00640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd190003f40 0x7fd190004810 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c240 tx=0x7fd19401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935ceec2a70cee6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194007d70 con 0x7fd190003f40 2024-05-08T04:42:49.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1940a9020 con 0x7fd190003f40 2024-05-08T04:42:49.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1abe00640 1 -- 172.21.15.164:0/2614188481 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194005b20 con 0x7fd190003f40 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 -- 172.21.15.164:0/2614188481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd190003f40 msgr2=0x7fd190004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 --2- 172.21.15.164:0/2614188481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd190003f40 0x7fd190004810 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c240 tx=0x7fd19401cc80 comp rx=0 tx=0).stop 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b1800640 1 -- 172.21.15.164:0/2614188481 reap_dead start 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 -- 172.21.15.164:0/2614188481 shutdown_connections 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 -- 172.21.15.164:0/2614188481 >> 172.21.15.164:0/2614188481 conn(0x7fd1ac06ac90 msgr2=0x7fd1ac06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 -- 172.21.15.164:0/2614188481 shutdown_connections 2024-05-08T04:42:49.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.019+0000 7fd1b3800640 1 -- 172.21.15.164:0/2614188481 wait complete. 2024-05-08T04:42:49.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 Processor -- start 2024-05-08T04:42:49.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- start start 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd190003f40 0x7fd1ac160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1ac13de80 0x7fd1ac15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cc940 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cc7c0 con 0x7fd1ac13de80 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cf1f0 con 0x7fd190003f40 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57520/0 (socket says 172.21.15.164:57520) 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 -- 172.21.15.164:0/280181474 learned_addr learned my addr 172.21.15.164:0/280181474 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b0e00640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd190003f40 0x7fd1ac160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1a3e00640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1ac13de80 0x7fd1ac15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 -- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd190003f40 msgr2=0x7fd1ac160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd190003f40 0x7fd1ac160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 -- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1ac13de80 msgr2=0x7fd1ac15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1ac13de80 0x7fd1ac15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 -- 172.21.15.164:0/280181474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac160800 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b1800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0b4d90 tx=0x7fd1a400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400fd10 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4015430 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac160ae0 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4015820 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac1610a0 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.023+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac069d60 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.027+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fd1a40159c0 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.027+0000 7fd1a9600640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 0x7fd188069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.027+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fd1a4084480 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.027+0000 7fd1b0e00640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 0x7fd188069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.027+0000 7fd1b0e00640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 0x7fd188069d20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009be0 tx=0x7fd19c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.031+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1a4019030 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1ac0add30 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1a9600640 1 -- 172.21.15.164:0/280181474 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fd1a4010040 con 0x7fd1ac0b2c10 2024-05-08T04:42:49.221 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:49.221 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6374,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1073429531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1073429531},{"type":"v1","addr":"172.21.15.172:6835","nonce":1073429531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/2013951570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2013951570},{"type":"v1","addr":"172.21.15.172:6834","nonce":2013951570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/161512917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":161512917},{"type":"v1","addr":"172.21.15.164:6836","nonce":161512917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1062117015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1062117015},{"type":"v1","addr":"172.21.15.164:6837","nonce":1062117015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-05-08T04:42:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 65 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 msgr2=0x7fd188069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 0x7fd188069d20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009be0 tx=0x7fd19c009190 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 msgr2=0x7fd1ac0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0b4d90 tx=0x7fd1a400f1b0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 shutdown_connections 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd188067860 0x7fd188069d20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ac0b2c10 0x7fd1ac0b3090 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1ac13de80 0x7fd1ac15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 --2- 172.21.15.164:0/280181474 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd190003f40 0x7fd1ac160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 >> 172.21.15.164:0/280181474 conn(0x7fd1ac06ac90 msgr2=0x7fd1ac142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 shutdown_connections 2024-05-08T04:42:49.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.219+0000 7fd1b3800640 1 -- 172.21.15.164:0/280181474 wait complete. 2024-05-08T04:42:49.243 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:42:49.243 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:42:49.243 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:42:49.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:49.244 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:42:49.244 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:42:49.244 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:42:49.244 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:49.244 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:42:49.244 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:42:49.244 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:42:49.245 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:49.245 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:42:49.245 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:42:49.245 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:42:49.245 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:42:49.260 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:42:49.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:42:49.261 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:42:49.261 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:42:49.262 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:42:49.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:42:49.263 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:42:49.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:42:49.309 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:42:49.310 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:49.311 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:42:49.311 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:49.312 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:42:49.312 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:49.314 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:42:49.315 DEBUG:teuthology.parallel:result is None 2024-05-08T04:42:49.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:42:49.396 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:42:49.397 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:42:49.403 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:42:49.406 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:42:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 Processor -- start 2024-05-08T04:42:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 -- start start 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0fb015bc50 0x7f0fb00b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0fb00ccfe0 0x7f0fb00d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00b4d50 0x7f0fb00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00b42a0 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00cd4e0 con 0x7f0fb015bc50 2024-05-08T04:42:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fb5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00cd7c0 con 0x7f0fb00b4d50 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0fb00ccfe0 0x7f0fb00d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0fb00ccfe0 0x7f0fb00d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56512/0 (socket says 172.21.15.164:56512) 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 -- 172.21.15.164:0/3539007854 learned_addr learned my addr 172.21.15.164:0/3539007854 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0faf400640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00b4d50 0x7f0fb00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0faea00640 1 --2- 172.21.15.164:0/3539007854 >> v2:172.21.15.172:3300/0 conn(0x7f0fb015bc50 0x7f0fb00b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 -- 172.21.15.164:0/3539007854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00b4d50 msgr2=0x7f0fb00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00b4d50 0x7f0fb00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 -- 172.21.15.164:0/3539007854 >> v2:172.21.15.172:3300/0 conn(0x7f0fb015bc50 msgr2=0x7f0fb00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3539007854 >> v2:172.21.15.172:3300/0 conn(0x7f0fb015bc50 0x7f0fb00b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.459+0000 7f0fa6000640 1 -- 172.21.15.164:0/3539007854 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00cf590 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00ccfe0 0x7f0fb00d0070 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0071a00 tx=0x7f0f9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac31547d049c4fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fae000640 1 -- 172.21.15.164:0/3539007854 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980ab620 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fae000640 1 -- 172.21.15.164:0/3539007854 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f980b0070 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fae000640 1 -- 172.21.15.164:0/3539007854 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980ab9e0 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3539007854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00ccfe0 msgr2=0x7f0fb00d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00ccfe0 0x7f0fb00d0070 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0071a00 tx=0x7f0f9801c460 comp rx=0 tx=0).stop 2024-05-08T04:42:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3539007854 shutdown_connections 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00b4d50 0x7f0fb00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3539007854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00ccfe0 0x7f0fb00d0070 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3539007854 >> v2:172.21.15.172:3300/0 conn(0x7f0fb015bc50 0x7f0fb00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3539007854 >> 172.21.15.164:0/3539007854 conn(0x7f0fb00b34f0 msgr2=0x7f0fb00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3539007854 shutdown_connections 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.463+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3539007854 wait complete. 2024-05-08T04:42:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 Processor -- start 2024-05-08T04:42:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- start start 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00ccfe0 0x7f0fb0153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fb015bc50 0x7f0fb01535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00cea90 con 0x7f0fb00b4d50 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00ce910 con 0x7f0fb015bc50 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00cec10 con 0x7f0fb00ccfe0 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56520/0 (socket says 172.21.15.164:56520) 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 -- 172.21.15.164:0/3136968029 learned_addr learned my addr 172.21.15.164:0/3136968029 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00ccfe0 0x7f0fb0153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faf400640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fb015bc50 0x7f0fb01535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 -- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00ccfe0 msgr2=0x7f0fb0153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00ccfe0 0x7f0fb0153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 -- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fb015bc50 msgr2=0x7f0fb01535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fb015bc50 0x7f0fb01535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 -- 172.21.15.164:0/3136968029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb015ac10 con 0x7f0fb00b4d50 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0faea00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007e40 tx=0x7f0fa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0014620 con 0x7f0fb00b4d50 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0019070 con 0x7f0fb00b4d50 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0014a00 con 0x7f0fb00b4d50 2024-05-08T04:42:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb006bf40 con 0x7f0fb00b4d50 2024-05-08T04:42:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb006c500 con 0x7f0fb00b4d50 2024-05-08T04:42:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.467+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb0069d60 con 0x7f0fb00b4d50 2024-05-08T04:42:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.471+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f0fa0014ba0 con 0x7f0fb00b4d50 2024-05-08T04:42:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.471+0000 7f0fa7e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 0x7f0f8c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.471+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0fa0084070 con 0x7f0fb00b4d50 2024-05-08T04:42:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.471+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 0x7f0f8c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.471+0000 7f0fa6000640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 0x7f0f8c069d20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f980af000 tx=0x7f0f980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:49.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.475+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fb015ac10 con 0x7f0fb00b4d50 2024-05-08T04:42:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:49.643+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0fb00ceef0 con 0x7f0fb00b4d50 2024-05-08T04:42:50.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.215+0000 7f0fa7e00640 1 -- 172.21.15.164:0/3136968029 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0fa0056480 con 0x7f0fb00b4d50 2024-05-08T04:42:50.218 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 msgr2=0x7f0f8c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 0x7f0f8c069d20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f980af000 tx=0x7f0f980a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 msgr2=0x7f0fb006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007e40 tx=0x7f0fa000e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 shutdown_connections 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0f8c067860 0x7f0f8c069d20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fb00ccfe0 0x7f0fb0153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fb015bc50 0x7f0fb01535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 --2- 172.21.15.164:0/3136968029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb00b4d50 0x7f0fb006d540 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 >> 172.21.15.164:0/3136968029 conn(0x7f0fb00b34f0 msgr2=0x7f0fb00b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:50.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 shutdown_connections 2024-05-08T04:42:50.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.219+0000 7f0fb5e00640 1 -- 172.21.15.164:0/3136968029 wait complete. 2024-05-08T04:42:50.252 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:42:50.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.387+0000 7fc550600640 1 Processor -- start 2024-05-08T04:42:50.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc550600640 1 -- start start 2024-05-08T04:42:50.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc550600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5480cdfa0 0x7fc5480cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc550600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc548071980 con 0x7fc5480af480 2024-05-08T04:42:50.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc550600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc548071c00 con 0x7fc54806bca0 2024-05-08T04:42:50.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc550600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc548161810 con 0x7fc5480cdfa0 2024-05-08T04:42:50.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54dc00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc5480af480 0x7fc54806cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46366/0 (socket says 172.21.15.164:46366) 2024-05-08T04:42:50.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54dc00640 1 -- 172.21.15.164:0/2577488251 learned_addr learned my addr 172.21.15.164:0/2577488251 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:50.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54d200640 1 --2- 172.21.15.164:0/2577488251 >> v2:172.21.15.164:3300/0 conn(0x7fc5480cdfa0 0x7fc5480cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc547e00640 1 -- 172.21.15.164:0/2577488251 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134453557 0 0) 0x7fc53c001120 con 0x7fc5480af480 2024-05-08T04:42:50.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc547e00640 1 -- 172.21.15.164:0/2577488251 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc530003530 con 0x7fc5480af480 2024-05-08T04:42:50.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54d200640 1 -- 172.21.15.164:0/2577488251 >> v1:172.21.15.172:6789/0 conn(0x7fc54806bca0 legacy=0x7fc5480723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54d200640 1 -- 172.21.15.164:0/2577488251 >> v1:172.21.15.164:6789/0 conn(0x7fc5480af480 legacy=0x7fc54806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.391+0000 7fc54d200640 1 -- 172.21.15.164:0/2577488251 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54806f120 con 0x7fc5480cdfa0 2024-05-08T04:42:50.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc54d200640 1 --2- 172.21.15.164:0/2577488251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5480cdfa0 0x7fc5480cf470 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097c0 tx=0x7fc53401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b096611bff32500 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc547e00640 1 -- 172.21.15.164:0/2577488251 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340b1070 con 0x7fc5480cdfa0 2024-05-08T04:42:50.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc547e00640 1 -- 172.21.15.164:0/2577488251 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340ac410 con 0x7fc5480cdfa0 2024-05-08T04:42:50.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc547e00640 1 -- 172.21.15.164:0/2577488251 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340ac7e0 con 0x7fc5480cdfa0 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- 172.21.15.164:0/2577488251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5480cdfa0 msgr2=0x7fc5480cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 --2- 172.21.15.164:0/2577488251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5480cdfa0 0x7fc5480cf470 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097c0 tx=0x7fc53401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- 172.21.15.164:0/2577488251 shutdown_connections 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 --2- 172.21.15.164:0/2577488251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5480cdfa0 0x7fc5480cf470 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- 172.21.15.164:0/2577488251 >> 172.21.15.164:0/2577488251 conn(0x7fc5480ccab0 msgr2=0x7fc5480ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- 172.21.15.164:0/2577488251 shutdown_connections 2024-05-08T04:42:50.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- 172.21.15.164:0/2577488251 wait complete. 2024-05-08T04:42:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 Processor -- start 2024-05-08T04:42:50.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.395+0000 7fc550600640 1 -- start start 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480af480 0x7fc5480c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480cdfa0 0x7fc5480c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc548071f50 con 0x7fc54806bca0 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54806eed0 con 0x7fc5480af480 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc548072f00 con 0x7fc5480cdfa0 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56540/0 (socket says 172.21.15.164:56540) 2024-05-08T04:42:50.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 -- 172.21.15.164:0/3805970589 learned_addr learned my addr 172.21.15.164:0/3805970589 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:50.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 -- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480cdfa0 msgr2=0x7fc5480c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480cdfa0 0x7fc5480c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 -- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480af480 msgr2=0x7fc5480c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480af480 0x7fc5480c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 -- 172.21.15.164:0/3805970589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480c6b10 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc54dc00640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c460 tx=0x7fc53c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c017070 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c005cf0 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc548156cd0 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c007290 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc548157290 con 0x7fc54806bca0 2024-05-08T04:42:50.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548069d60 con 0x7fc54806bca0 2024-05-08T04:42:50.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.399+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fc53c01a070 con 0x7fc54806bca0 2024-05-08T04:42:50.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.403+0000 7fc545600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 0x7fc528069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.403+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fc53c085340 con 0x7fc54806bca0 2024-05-08T04:42:50.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.403+0000 7fc54d200640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 0x7fc528069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.403+0000 7fc54d200640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 0x7fc528069d20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc5340ae000 tx=0x7fc534004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.407+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc548071980 con 0x7fc54806bca0 2024-05-08T04:42:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.615+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc548149150 con 0x7fc54806bca0 2024-05-08T04:42:50.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc545600640 1 -- 172.21.15.164:0/3805970589 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc53c055730 con 0x7fc54806bca0 2024-05-08T04:42:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:42:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:42:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:42:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:42:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:42:50.638 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:42:50.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 msgr2=0x7fc528069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 0x7fc528069d20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc5340ae000 tx=0x7fc534004740 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 msgr2=0x7fc548148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c460 tx=0x7fc53c00c920 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 shutdown_connections 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc528067860 0x7fc528069d20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5480cdfa0 0x7fc5480c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5480af480 0x7fc5480c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 --2- 172.21.15.164:0/3805970589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc54806bca0 0x7fc548148c10 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 >> 172.21.15.164:0/3805970589 conn(0x7fc5480ccab0 msgr2=0x7fc54815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 shutdown_connections 2024-05-08T04:42:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.619+0000 7fc550600640 1 -- 172.21.15.164:0/3805970589 wait complete. 2024-05-08T04:42:50.641 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:50.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 Processor -- start 2024-05-08T04:42:50.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 -- start start 2024-05-08T04:42:50.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb5406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdb5406b1d0 0x7fdb540af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5406d580 con 0x7fdb541561f0 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5415d040 con 0x7fdb5406b1d0 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb5ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5406d820 con 0x7fdb540ce210 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb53e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdb541561f0 0x7fdb5415c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46368/0 (socket says 172.21.15.164:46368) 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb53e00640 1 -- 172.21.15.164:0/268406066 learned_addr learned my addr 172.21.15.164:0/268406066 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb58c00640 1 --2- 172.21.15.164:0/268406066 >> v2:172.21.15.164:3300/0 conn(0x7fdb5406b1d0 0x7fdb540af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb4b400640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb5406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb53400640 1 -- 172.21.15.164:0/268406066 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203908145 0 0) 0x7fdb5406d580 con 0x7fdb541561f0 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb53400640 1 -- 172.21.15.164:0/268406066 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb38003530 con 0x7fdb541561f0 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb58c00640 1 -- 172.21.15.164:0/268406066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 msgr2=0x7fdb5406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb58c00640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb5406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb58c00640 1 -- 172.21.15.164:0/268406066 >> v1:172.21.15.164:6789/0 conn(0x7fdb541561f0 legacy=0x7fdb5415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.775+0000 7fdb58c00640 1 -- 172.21.15.164:0/268406066 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5406db00 con 0x7fdb5406b1d0 2024-05-08T04:42:50.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb58c00640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5406b1d0 0x7fdb540af4e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009d90 tx=0x7fdb4c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aabf486f407c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb53400640 1 -- 172.21.15.164:0/268406066 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0b2070 con 0x7fdb5406b1d0 2024-05-08T04:42:50.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb53400640 1 -- 172.21.15.164:0/268406066 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c0a5830 con 0x7fdb5406b1d0 2024-05-08T04:42:50.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb53400640 1 -- 172.21.15.164:0/268406066 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0a5c00 con 0x7fdb5406b1d0 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 -- 172.21.15.164:0/268406066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5406b1d0 msgr2=0x7fdb540af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5406b1d0 0x7fdb540af4e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009d90 tx=0x7fdb4c01c970 comp rx=0 tx=0).stop 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 -- 172.21.15.164:0/268406066 shutdown_connections 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5406b1d0 0x7fdb540af4e0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/268406066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb5406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 -- 172.21.15.164:0/268406066 >> 172.21.15.164:0/268406066 conn(0x7fdb54066fa0 msgr2=0x7fdb5406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:50.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 -- 172.21.15.164:0/268406066 shutdown_connections 2024-05-08T04:42:50.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.779+0000 7fdb5ac00640 1 -- 172.21.15.164:0/268406066 wait complete. 2024-05-08T04:42:50.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 Processor -- start 2024-05-08T04:42:50.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- start start 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 0x7fdb540aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb541565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5415ddc0 con 0x7fdb541561f0 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5415dc40 con 0x7fdb5406b1d0 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5415df40 con 0x7fdb540ce210 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56568/0 (socket says 172.21.15.164:56568) 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 -- 172.21.15.164:0/2530988695 learned_addr learned my addr 172.21.15.164:0/2530988695 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb4b400640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb541565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb53e00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 0x7fdb540aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 -- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 msgr2=0x7fdb541565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb541565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 -- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 msgr2=0x7fdb540aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 0x7fdb540aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 -- 172.21.15.164:0/2530988695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb541571b0 con 0x7fdb541561f0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb53e00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 0x7fdb540aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb58c00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0b0040 tx=0x7fdb4c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0b2050 con 0x7fdb541561f0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c0be070 con 0x7fdb541561f0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb54157490 con 0x7fdb541561f0 2024-05-08T04:42:50.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0ada90 con 0x7fdb541561f0 2024-05-08T04:42:50.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb54157a50 con 0x7fdb541561f0 2024-05-08T04:42:50.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.783+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb54069d60 con 0x7fdb541561f0 2024-05-08T04:42:50.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.787+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fdb4c0a56b0 con 0x7fdb541561f0 2024-05-08T04:42:50.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.787+0000 7fdb50c00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 0x7fdb30069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:50.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.787+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fdb4c11ebb0 con 0x7fdb541561f0 2024-05-08T04:42:50.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.787+0000 7fdb53e00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 0x7fdb30069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:50.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.787+0000 7fdb53e00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 0x7fdb30069d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005890 tx=0x7fdb44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:50.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.791+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb4c016560 con 0x7fdb541561f0 2024-05-08T04:42:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb5406eee0 con 0x7fdb541561f0 2024-05-08T04:42:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb50c00640 1 -- 172.21.15.164:0/2530988695 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+13087 (secure 0 0 0) 0x7fdb4c0ab340 con 0x7fdb541561f0 2024-05-08T04:42:50.948 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:50.948 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":108,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:50.226588+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 msgr2=0x7fdb30069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 0x7fdb30069d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005890 tx=0x7fdb44009040 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 msgr2=0x7fdb54156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0b0040 tx=0x7fdb4c019710 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 shutdown_connections 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdb30067860 0x7fdb30069d20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdb540ce210 0x7fdb541565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb541561f0 0x7fdb54156b10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 --2- 172.21.15.164:0/2530988695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb5406b1d0 0x7fdb540aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 >> 172.21.15.164:0/2530988695 conn(0x7fdb54066fa0 msgr2=0x7fdb540cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 shutdown_connections 2024-05-08T04:42:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:50.947+0000 7fdb5ac00640 1 -- 172.21.15.164:0/2530988695 wait complete. 2024-05-08T04:42:50.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:42:50.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:42:51.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.107+0000 7f2051800640 1 Processor -- start 2024-05-08T04:42:51.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.107+0000 7f2051800640 1 -- start start 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.107+0000 7f2051800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f204c15a200 0x7f204c0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.107+0000 7f2051800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c06d150 con 0x7f204c06dfa0 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2051800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c06d430 con 0x7f204c0ca850 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2051800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c06bca0 con 0x7f204c15a200 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f204c06dfa0 0x7f204c0ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46370/0 (socket says 172.21.15.164:46370) 2024-05-08T04:42:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 -- 172.21.15.164:0/5244290 learned_addr learned my addr 172.21.15.164:0/5244290 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:51.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204aa00640 1 --2- 172.21.15.164:0/5244290 >> v2:172.21.15.172:3300/0 conn(0x7f204c15a200 0x7f204c0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657317098 0 0) 0x7f204c06d150 con 0x7f204c06dfa0 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2030003450 con 0x7f204c06dfa0 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 374812333 0 0) 0x7f2030003450 con 0x7f204c06dfa0 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 >> v2:172.21.15.172:3300/0 conn(0x7f204c15a200 msgr2=0x7f204c0aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 --2- 172.21.15.164:0/5244290 >> v2:172.21.15.172:3300/0 conn(0x7f204c15a200 0x7f204c0aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 >> v1:172.21.15.172:6789/0 conn(0x7f204c0ca850 legacy=0x7f204c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c06d710 con 0x7f204c06dfa0 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f2034001680 con 0x7f204c06dfa0 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 >> v1:172.21.15.164:6789/0 conn(0x7f204c06dfa0 legacy=0x7f204c0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030003ab0 0x7f2030003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 0x7f203000acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203000b570 0x7f203000da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203000e2e0 con 0x7f2030008860 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203000e490 con 0x7f203000b570 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203000e690 con 0x7f2030003ab0 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 0x7f203000acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204aa00640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030003ab0 0x7f2030003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204b400640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203000b570 0x7f203000da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 -- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030003ab0 msgr2=0x7f2030003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030003ab0 0x7f2030003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 -- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203000b570 msgr2=0x7f203000da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203000b570 0x7f203000da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f2042000640 1 -- 172.21.15.164:0/5244290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030003450 con 0x7f2030008860 2024-05-08T04:42:51.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.111+0000 7f204b400640 1 -- 172.21.15.164:0/5244290 reap_dead start 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2042000640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 0x7f203000acd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f203400c240 tx=0x7f203401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e14b9c3cd46d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034007d20 con 0x7f2030008860 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20340a9070 con 0x7f2030008860 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f204a000640 1 -- 172.21.15.164:0/5244290 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034005d80 con 0x7f2030008860 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 msgr2=0x7f203000acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 0x7f203000acd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f203400c240 tx=0x7f203401cc80 comp rx=0 tx=0).stop 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- 172.21.15.164:0/5244290 shutdown_connections 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 --2- 172.21.15.164:0/5244290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2030008860 0x7f203000acd0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- 172.21.15.164:0/5244290 >> 172.21.15.164:0/5244290 conn(0x7f204c0cf770 msgr2=0x7f204c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- 172.21.15.164:0/5244290 shutdown_connections 2024-05-08T04:42:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- 172.21.15.164:0/5244290 wait complete. 2024-05-08T04:42:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 Processor -- start 2024-05-08T04:42:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.115+0000 7f2051800640 1 -- start start 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 0x7f204c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 0x7f204c154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15de30 con 0x7f204c155180 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0aeed0 con 0x7f204c1fa9c0 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15dfb0 con 0x7f2030008860 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56586/0 (socket says 172.21.15.164:56586) 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 -- 172.21.15.164:0/3558611436 learned_addr learned my addr 172.21.15.164:0/3558611436 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204aa00640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 0x7f204c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2042000640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 0x7f204c154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 -- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 msgr2=0x7f204c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 0x7f204c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 -- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 msgr2=0x7f204c154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 0x7f204c154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 -- 172.21.15.164:0/3558611436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c154460 con 0x7f204c155180 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2042000640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 0x7f204c154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204aa00640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 0x7f204c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f204b400640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cd2c0 tx=0x7f204400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044019070 con 0x7f204c155180 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044006b60 con 0x7f204c155180 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044003980 con 0x7f204c155180 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c1fb3a0 con 0x7f204c155180 2024-05-08T04:42:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c150960 con 0x7f204c155180 2024-05-08T04:42:51.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c072d30 con 0x7f204c155180 2024-05-08T04:42:51.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.119+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f204401c070 con 0x7f204c155180 2024-05-08T04:42:51.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.123+0000 7f2043400640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 0x7f2028069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.123+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f2044085370 con 0x7f204c155180 2024-05-08T04:42:51.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.123+0000 7f204aa00640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 0x7f2028069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.123+0000 7f204aa00640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 0x7f2028069d20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f204c0736e0 tx=0x7f203c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.127+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2044055a10 con 0x7f204c155180 2024-05-08T04:42:51.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.283+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f204c04f010 con 0x7f204c155180 2024-05-08T04:42:51.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.283+0000 7f2043400640 1 -- 172.21.15.164:0/3558611436 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2044055310 con 0x7f204c155180 2024-05-08T04:42:51.286 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:42:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 msgr2=0x7f2028069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 0x7f2028069d20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f204c0736e0 tx=0x7f203c008040 comp rx=0 tx=0).stop 2024-05-08T04:42:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 msgr2=0x7f204c0b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cd2c0 tx=0x7f204400ec10 comp rx=0 tx=0).stop 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 shutdown_connections 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2028067860 0x7f2028069d20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f204c155180 0x7f204c0b3980 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f204c1fa9c0 0x7f204c154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 --2- 172.21.15.164:0/3558611436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030008860 0x7f204c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 >> 172.21.15.164:0/3558611436 conn(0x7f204c0cf770 msgr2=0x7f204c157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 shutdown_connections 2024-05-08T04:42:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.287+0000 7f2051800640 1 -- 172.21.15.164:0/3558611436 wait complete. 2024-05-08T04:42:51.307 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:42:51.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.439+0000 7ff893800640 1 Processor -- start 2024-05-08T04:42:51.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff893800640 1 -- start start 2024-05-08T04:42:51.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff893800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff88c15ba70 0x7ff88c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff893800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0cd9f0 con 0x7ff88c0cfda0 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff893800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0cdcd0 con 0x7ff88c15be50 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff893800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c15fe70 con 0x7ff88c15ba70 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff891800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff88c0cfda0 0x7ff88c0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46382/0 (socket says 172.21.15.164:46382) 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff891800640 1 -- 172.21.15.164:0/3268553332 learned_addr learned my addr 172.21.15.164:0/3268553332 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff890e00640 1 --2- 172.21.15.164:0/3268553332 >> v2:172.21.15.164:3300/0 conn(0x7ff88c15ba70 0x7ff88c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff890e00640 1 -- 172.21.15.164:0/3268553332 >> v1:172.21.15.172:6789/0 conn(0x7ff88c15be50 legacy=0x7ff88c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff890e00640 1 -- 172.21.15.164:0/3268553332 >> v1:172.21.15.164:6789/0 conn(0x7ff88c0cfda0 legacy=0x7ff88c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff890e00640 1 -- 172.21.15.164:0/3268553332 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c15f820 con 0x7ff88c15ba70 2024-05-08T04:42:51.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.443+0000 7ff88be00640 1 -- 172.21.15.164:0/3268553332 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704066033 0 0) 0x7ff88c0cd9f0 con 0x7ff88c0cfda0 2024-05-08T04:42:51.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff890e00640 1 --2- 172.21.15.164:0/3268553332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c160300 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097c0 tx=0x7ff87c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45ceb82af62e311 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff88be00640 1 -- 172.21.15.164:0/3268553332 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0b2070 con 0x7ff88c15ba70 2024-05-08T04:42:51.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff88be00640 1 -- 172.21.15.164:0/3268553332 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0ad680 con 0x7ff88c15ba70 2024-05-08T04:42:51.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff88be00640 1 -- 172.21.15.164:0/3268553332 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0ada50 con 0x7ff88c15ba70 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- 172.21.15.164:0/3268553332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 msgr2=0x7ff88c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 --2- 172.21.15.164:0/3268553332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c160300 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097c0 tx=0x7ff87c01c470 comp rx=0 tx=0).stop 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- 172.21.15.164:0/3268553332 shutdown_connections 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 --2- 172.21.15.164:0/3268553332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c160300 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- 172.21.15.164:0/3268553332 >> 172.21.15.164:0/3268553332 conn(0x7ff88c06ac90 msgr2=0x7ff88c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- 172.21.15.164:0/3268553332 shutdown_connections 2024-05-08T04:42:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- 172.21.15.164:0/3268553332 wait complete. 2024-05-08T04:42:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 Processor -- start 2024-05-08T04:42:51.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.447+0000 7ff893800640 1 -- start start 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88c0cfda0 0x7ff88c1f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff88c15be50 0x7ff88c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c169f60 con 0x7ff88c15ba70 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c169de0 con 0x7ff88c15be50 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c16a0e0 con 0x7ff88c0cfda0 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56616/0 (socket says 172.21.15.164:56616) 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 -- 172.21.15.164:0/8326820 learned_addr learned my addr 172.21.15.164:0/8326820 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:51.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 -- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88c0cfda0 msgr2=0x7ff88c1f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88c0cfda0 0x7ff88c1f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff891800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff88c15be50 0x7ff88c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 -- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff88c15be50 msgr2=0x7ff88c159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff88c15be50 0x7ff88c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 -- 172.21.15.164:0/8326820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c1faab0 con 0x7ff88c15ba70 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff883e00640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c3f0 tx=0x7ff87400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874017070 con 0x7ff88c15ba70 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8740052d0 con 0x7ff88c15ba70 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874003840 con 0x7ff88c15ba70 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c1f7cc0 con 0x7ff88c15ba70 2024-05-08T04:42:51.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c1f8280 con 0x7ff88c15ba70 2024-05-08T04:42:51.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.451+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c069d60 con 0x7ff88c15ba70 2024-05-08T04:42:51.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.455+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7ff87401a070 con 0x7ff88c15ba70 2024-05-08T04:42:51.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.455+0000 7ff889600640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 0x7ff868069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.455+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4941+0+0 (secure 0 0 0) 0x7ff874082c90 con 0x7ff88c15ba70 2024-05-08T04:42:51.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.455+0000 7ff890e00640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 0x7ff868069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.455+0000 7ff890e00640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 0x7ff868069d20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0af000 tx=0x7ff87c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.459+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff874005480 con 0x7ff88c15ba70 2024-05-08T04:42:51.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff88c0cfb80 con 0x7ff88c15ba70 2024-05-08T04:42:51.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff889600640 1 -- 172.21.15.164:0/8326820 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff8740556f0 con 0x7ff88c15ba70 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:42:51.617 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:42:51.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 msgr2=0x7ff868069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 0x7ff868069d20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0af000 tx=0x7ff87c008040 comp rx=0 tx=0).stop 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 msgr2=0x7ff88c1fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c3f0 tx=0x7ff87400c8b0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 shutdown_connections 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff868067860 0x7ff868069d20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff88c15be50 0x7ff88c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff88c15ba70 0x7ff88c1fb240 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 --2- 172.21.15.164:0/8326820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88c0cfda0 0x7ff88c1f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 >> 172.21.15.164:0/8326820 conn(0x7ff88c06ac90 msgr2=0x7ff88c158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:51.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 shutdown_connections 2024-05-08T04:42:51.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.615+0000 7ff893800640 1 -- 172.21.15.164:0/8326820 wait complete. 2024-05-08T04:42:51.638 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:42:51.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:42:51.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.775+0000 7f4421000640 1 Processor -- start 2024-05-08T04:42:51.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.775+0000 7f4421000640 1 -- start start 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4421000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f441c15ba20 0x7f441c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4421000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 0x7f441c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4421000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0c6600 con 0x7f441c0ae6e0 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4421000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c15fbe0 con 0x7f441c15ba20 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4421000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0c6040 con 0x7f441c06de50 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f441a800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f441c0ae6e0 0x7f441c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:38970/0 (socket says 172.21.15.164:38970) 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f441a800640 1 -- 172.21.15.164:0/205182971 learned_addr learned my addr 172.21.15.164:0/205182971 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:51.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419e00640 1 --2- 172.21.15.164:0/205182971 >> v2:172.21.15.172:3300/0 conn(0x7f441c15ba20 0x7f441c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4411400640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 0x7f441c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419400640 1 -- 172.21.15.164:0/205182971 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105081751 0 0) 0x7f441c0c6600 con 0x7f441c0ae6e0 2024-05-08T04:42:51.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419400640 1 -- 172.21.15.164:0/205182971 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4400003530 con 0x7f441c0ae6e0 2024-05-08T04:42:51.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419e00640 1 -- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 msgr2=0x7f441c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:51.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419e00640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 0x7f441c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419e00640 1 -- 172.21.15.164:0/205182971 >> v1:172.21.15.172:6789/0 conn(0x7f441c0ae6e0 legacy=0x7f441c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4419e00640 1 -- 172.21.15.164:0/205182971 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c0c6320 con 0x7f441c15ba20 2024-05-08T04:42:51.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.779+0000 7f4411400640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 0x7f441c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:51.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4419e00640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c160070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f441c15fd60 tx=0x7f440c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf2606a7566907e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4419400640 1 -- 172.21.15.164:0/205182971 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0b2070 con 0x7f441c15ba20 2024-05-08T04:42:51.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4419400640 1 -- 172.21.15.164:0/205182971 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c0a5980 con 0x7f441c15ba20 2024-05-08T04:42:51.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4419400640 1 -- 172.21.15.164:0/205182971 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0a5d30 con 0x7f441c15ba20 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 msgr2=0x7f441c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c160070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f441c15fd60 tx=0x7f440c01c920 comp rx=0 tx=0).stop 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- 172.21.15.164:0/205182971 shutdown_connections 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c06de50 0x7f441c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 --2- 172.21.15.164:0/205182971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c160070 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- 172.21.15.164:0/205182971 >> 172.21.15.164:0/205182971 conn(0x7f441c0cf770 msgr2=0x7f441c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:51.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- 172.21.15.164:0/205182971 shutdown_connections 2024-05-08T04:42:51.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- 172.21.15.164:0/205182971 wait complete. 2024-05-08T04:42:51.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 Processor -- start 2024-05-08T04:42:51.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.783+0000 7f4421000640 1 -- start start 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c0ae6e0 0x7f441c0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0aec40 con 0x7f441c06de50 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0aeac0 con 0x7f441c15ba20 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0aedc0 con 0x7f441c0ae6e0 2024-05-08T04:42:51.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56632/0 (socket says 172.21.15.164:56632) 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 -- 172.21.15.164:0/3855013875 learned_addr learned my addr 172.21.15.164:0/3855013875 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4411400640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c0ae6e0 0x7f441c0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f441a800640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 -- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c0ae6e0 msgr2=0x7f441c0ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c0ae6e0 0x7f441c0ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 -- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 msgr2=0x7f441c148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:51.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 -- 172.21.15.164:0/3855013875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c0caeb0 con 0x7f441c06de50 2024-05-08T04:42:51.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4419e00640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f440c0b0040 tx=0x7f440c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0b2040 con 0x7f441c06de50 2024-05-08T04:42:51.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c0bf070 con 0x7f441c06de50 2024-05-08T04:42:51.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c0cb190 con 0x7f441c06de50 2024-05-08T04:42:51.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0b8890 con 0x7f441c06de50 2024-05-08T04:42:51.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c0cb750 con 0x7f441c06de50 2024-05-08T04:42:51.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c072d30 con 0x7f441c06de50 2024-05-08T04:42:51.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.787+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f440c0b8a30 con 0x7f441c06de50 2024-05-08T04:42:51.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.791+0000 7f4412a00640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 0x7f43f8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:51.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.791+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f440c11ec10 con 0x7f441c06de50 2024-05-08T04:42:51.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.791+0000 7f4411400640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 0x7f43f8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:51.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.791+0000 7f4411400640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 0x7f43f8069d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4404009580 tx=0x7f4404009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:51.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.795+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f440c016540 con 0x7f441c06de50 2024-05-08T04:42:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:51.947+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f441c04f010 con 0x7f441c06de50 2024-05-08T04:42:52.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4412a00640 1 -- 172.21.15.164:0/3855013875 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f440c0f0ec0 con 0x7f441c06de50 2024-05-08T04:42:52.229 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:42:52.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 msgr2=0x7f43f8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 0x7f43f8069d20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4404009580 tx=0x7f4404009190 comp rx=0 tx=0).stop 2024-05-08T04:42:52.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 msgr2=0x7f441c14ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f440c0b0040 tx=0x7f440c0196c0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 shutdown_connections 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f43f8067860 0x7f43f8069d20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f441c0ae6e0 0x7f441c0ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c15ba20 0x7f441c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 --2- 172.21.15.164:0/3855013875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f441c06de50 0x7f441c14ab60 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 >> 172.21.15.164:0/3855013875 conn(0x7f441c0cf770 msgr2=0x7f441c15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 shutdown_connections 2024-05-08T04:42:52.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.227+0000 7f4421000640 1 -- 172.21.15.164:0/3855013875 wait complete. 2024-05-08T04:42:52.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:42:52.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 Processor -- start 2024-05-08T04:42:52.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 -- start start 2024-05-08T04:42:52.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a0806ac90 0x7f8a0806b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0815f6c0 con 0x7f8a0815b880 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0806db10 con 0x7f8a0815bc60 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0f400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0806b640 con 0x7f8a0806ac90 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a0806ac90 0x7f8a0806b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0ca00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8a0815b880 0x7f8a080cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46386/0 (socket says 172.21.15.164:46386) 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.395+0000 7f8a0ca00640 1 -- 172.21.15.164:0/3692438870 learned_addr learned my addr 172.21.15.164:0/3692438870 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:52.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a0d400640 1 -- 172.21.15.164:0/3692438870 >> v1:172.21.15.172:6789/0 conn(0x7f8a0815bc60 legacy=0x7f8a080cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:52.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a0d400640 1 -- 172.21.15.164:0/3692438870 >> v1:172.21.15.164:6789/0 conn(0x7f8a0815b880 legacy=0x7f8a080cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a0d400640 1 -- 172.21.15.164:0/3692438870 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0806ddf0 con 0x7f8a0806ac90 2024-05-08T04:42:52.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a0d400640 1 --2- 172.21.15.164:0/3692438870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0806b100 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d90 tx=0x7f8a0001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3e0a11fd167fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:52.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a07400640 1 -- 172.21.15.164:0/3692438870 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000b2070 con 0x7f8a0806ac90 2024-05-08T04:42:52.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a07400640 1 -- 172.21.15.164:0/3692438870 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a000a5730 con 0x7f8a0806ac90 2024-05-08T04:42:52.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.399+0000 7f8a07400640 1 -- 172.21.15.164:0/3692438870 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000a5b00 con 0x7f8a0806ac90 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- 172.21.15.164:0/3692438870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 msgr2=0x7f8a0806b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 --2- 172.21.15.164:0/3692438870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0806b100 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d90 tx=0x7f8a0001c7c0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- 172.21.15.164:0/3692438870 shutdown_connections 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 --2- 172.21.15.164:0/3692438870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0806b100 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- 172.21.15.164:0/3692438870 >> 172.21.15.164:0/3692438870 conn(0x7f8a0806bda0 msgr2=0x7f8a0806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:52.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- 172.21.15.164:0/3692438870 shutdown_connections 2024-05-08T04:42:52.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- 172.21.15.164:0/3692438870 wait complete. 2024-05-08T04:42:52.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 Processor -- start 2024-05-08T04:42:52.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- start start 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0815b880 0x7f8a08146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0815bc60 0x7f8a0814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806d8b0 con 0x7f8a0806ac90 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806d730 con 0x7f8a0815b880 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08169860 con 0x7f8a0815bc60 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56638/0 (socket says 172.21.15.164:56638) 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 -- 172.21.15.164:0/1341040226 learned_addr learned my addr 172.21.15.164:0/1341040226 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a07e00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0815b880 0x7f8a08146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0d400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0815bc60 0x7f8a0814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:52.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 -- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0815bc60 msgr2=0x7f8a0814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0815bc60 0x7f8a0814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 -- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0815b880 msgr2=0x7f8a08146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0815b880 0x7f8a08146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.403+0000 7f8a0ca00640 1 -- 172.21.15.164:0/1341040226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0814c3e0 con 0x7f8a0806ac90 2024-05-08T04:42:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a0ca00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c7b0 tx=0x7f89f800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8018070 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f80072d0 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80047f0 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0814c6c0 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0814cc80 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a08072f00 con 0x7f8a0806ac90 2024-05-08T04:42:52.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.407+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f89f800a640 con 0x7f8a0806ac90 2024-05-08T04:42:52.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.411+0000 7f8a04c00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 0x7f89e8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:52.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.411+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5311+0+0 (secure 0 0 0) 0x7f89f8082fc0 con 0x7f8a0806ac90 2024-05-08T04:42:52.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.411+0000 7f8a07e00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 0x7f89e8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:52.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.411+0000 7f8a07e00640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 0x7f89e8069d50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8a081558b0 tx=0x7f89f4009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:52.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.411+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89f80553d0 con 0x7f8a0806ac90 2024-05-08T04:42:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:52.567+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8a0804f010 con 0x7f8a0806ac90 2024-05-08T04:42:53.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.235+0000 7f8a04c00640 1 -- 172.21.15.164:0/1341040226 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7f89f8059470 con 0x7f8a0806ac90 2024-05-08T04:42:53.238 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:42:53.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 msgr2=0x7f89e8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:53.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 0x7f89e8069d50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8a081558b0 tx=0x7f89f4009060 comp rx=0 tx=0).stop 2024-05-08T04:42:53.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 msgr2=0x7f8a0814bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:53.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c7b0 tx=0x7f89f800cc70 comp rx=0 tx=0).stop 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 shutdown_connections 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f89e8067890 0x7f89e8069d50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a0815bc60 0x7f8a0814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a0815b880 0x7f8a08146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 --2- 172.21.15.164:0/1341040226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0806ac90 0x7f8a0814bd20 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 >> 172.21.15.164:0/1341040226 conn(0x7f8a0806bda0 msgr2=0x7f8a0815a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:53.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 shutdown_connections 2024-05-08T04:42:53.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.239+0000 7f8a0f400640 1 -- 172.21.15.164:0/1341040226 wait complete. 2024-05-08T04:42:53.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:42:53.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.415+0000 7f2467600640 1 Processor -- start 2024-05-08T04:42:53.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.415+0000 7f2467600640 1 -- start start 2024-05-08T04:42:53.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 0x7f246015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f246006e7b0 0x7f246006f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f246006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246006e320 con 0x7f246006e7b0 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246006fad0 con 0x7f24600affe0 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2467600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246006da30 con 0x7f246006ac90 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f246006e7b0 0x7f246006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f246006e7b0 0x7f246006f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56648/0 (socket says 172.21.15.164:56648) 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 -- 172.21.15.164:0/2115004932 learned_addr learned my addr 172.21.15.164:0/2115004932 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2464c00640 1 --2- 172.21.15.164:0/2115004932 >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 0x7f246015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2465600640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f246006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 -- 172.21.15.164:0/2115004932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 msgr2=0x7f246006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f246006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 -- 172.21.15.164:0/2115004932 >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 msgr2=0x7f246015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:53.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 --2- 172.21.15.164:0/2115004932 >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 0x7f246015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f245fe00640 1 -- 172.21.15.164:0/2115004932 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246006dcd0 con 0x7f246006e7b0 2024-05-08T04:42:53.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2464c00640 1 --2- 172.21.15.164:0/2115004932 >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 0x7f246015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:53.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.419+0000 7f2465600640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f246006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:53.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f245fe00640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f246006f590 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f24600b5b70 tx=0x7f244c01cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb6854a28fb7b5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:53.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f245f400640 1 -- 172.21.15.164:0/2115004932 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0ab5f0 con 0x7f246006e7b0 2024-05-08T04:42:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f245f400640 1 -- 172.21.15.164:0/2115004932 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c0b0070 con 0x7f246006e7b0 2024-05-08T04:42:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f245f400640 1 -- 172.21.15.164:0/2115004932 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0ab9b0 con 0x7f246006e7b0 2024-05-08T04:42:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- 172.21.15.164:0/2115004932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 msgr2=0x7f246006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f246006f590 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f24600b5b70 tx=0x7f244c01cf80 comp rx=0 tx=0).stop 2024-05-08T04:42:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- 172.21.15.164:0/2115004932 shutdown_connections 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f246006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 --2- 172.21.15.164:0/2115004932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f246006f590 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 --2- 172.21.15.164:0/2115004932 >> v2:172.21.15.172:3300/0 conn(0x7f24600affe0 0x7f246015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- 172.21.15.164:0/2115004932 >> 172.21.15.164:0/2115004932 conn(0x7f24600b2d00 msgr2=0x7f24600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- 172.21.15.164:0/2115004932 shutdown_connections 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- 172.21.15.164:0/2115004932 wait complete. 2024-05-08T04:42:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 Processor -- start 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.423+0000 7f2467600640 1 -- start start 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f24600b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24600affe0 0x7f246015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460072040 con 0x7f246006e7b0 2024-05-08T04:42:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460071ec0 con 0x7f24600affe0 2024-05-08T04:42:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600721c0 con 0x7f246006ac90 2024-05-08T04:42:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56658/0 (socket says 172.21.15.164:56658) 2024-05-08T04:42:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 -- 172.21.15.164:0/3766395366 learned_addr learned my addr 172.21.15.164:0/3766395366 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2464c00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f24600b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2465600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24600affe0 0x7f246015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 -- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 msgr2=0x7f24600b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f24600b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 -- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24600affe0 msgr2=0x7f246015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24600affe0 0x7f246015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 -- 172.21.15.164:0/3766395366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24601f7770 con 0x7f246006e7b0 2024-05-08T04:42:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2464c00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f24600b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245fe00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f244c01abd0 tx=0x7f244c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0be5f0 con 0x7f246006e7b0 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c0be790 con 0x7f246006e7b0 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0beb80 con 0x7f246006e7b0 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24601f7a50 con 0x7f246006e7b0 2024-05-08T04:42:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600b44a0 con 0x7f246006e7b0 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.427+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2460072e80 con 0x7f246006e7b0 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.431+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f244c0c9430 con 0x7f246006e7b0 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.431+0000 7f245cc00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 0x7f2440069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.431+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5693+0+0 (secure 0 0 0) 0x7f244c11e490 con 0x7f246006e7b0 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.431+0000 7f2464c00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 0x7f2440069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.431+0000 7f2464c00640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 0x7f2440069d20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2454006fd0 tx=0x7f2454008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.435+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f244c016110 con 0x7f246006e7b0 2024-05-08T04:42:53.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:53.619+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f24600b5840 con 0x7f246006e7b0 2024-05-08T04:42:54.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.243+0000 7f245cc00640 1 -- 172.21.15.164:0/3766395366 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7f244c0f0810 con 0x7f246006e7b0 2024-05-08T04:42:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 16 and data pool 17 2024-05-08T04:42:54.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 msgr2=0x7f2440069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 0x7f2440069d20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2454006fd0 tx=0x7f2454008040 comp rx=0 tx=0).stop 2024-05-08T04:42:54.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 msgr2=0x7f24601f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f244c01abd0 tx=0x7f244c01ad60 comp rx=0 tx=0).stop 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 shutdown_connections 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2440067860 0x7f2440069d20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24600affe0 0x7f246015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f246006e7b0 0x7f24601f6d90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 --2- 172.21.15.164:0/3766395366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246006ac90 0x7f24600b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.251+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 >> 172.21.15.164:0/3766395366 conn(0x7f24600b2d00 msgr2=0x7f24600b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:54.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.255+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 shutdown_connections 2024-05-08T04:42:54.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.255+0000 7f2467600640 1 -- 172.21.15.164:0/3766395366 wait complete. 2024-05-08T04:42:54.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 Processor -- start 2024-05-08T04:42:54.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 -- start start 2024-05-08T04:42:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb1ec06ac90 0x7fb1ec06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec06e900 con 0x7fb1ec0ccdf0 2024-05-08T04:42:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec06b630 con 0x7fb1ec06ac90 2024-05-08T04:42:54.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.435+0000 7fb1f1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec06d4e0 con 0x7fb1ec06df30 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:35874/0 (socket says 172.21.15.164:35874) 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 -- 172.21.15.164:0/609372664 learned_addr learned my addr 172.21.15.164:0/609372664 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1e2000640 1 --2- 172.21.15.164:0/609372664 >> v2:172.21.15.172:3300/0 conn(0x7fb1ec06ac90 0x7fb1ec06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 -- 172.21.15.164:0/609372664 >> v2:172.21.15.172:3300/0 conn(0x7fb1ec06ac90 msgr2=0x7fb1ec06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 --2- 172.21.15.164:0/609372664 >> v2:172.21.15.172:3300/0 conn(0x7fb1ec06ac90 0x7fb1ec06b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 -- 172.21.15.164:0/609372664 >> v1:172.21.15.172:6789/0 conn(0x7fb1ec0ccdf0 legacy=0x7fb1ec0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 -- 172.21.15.164:0/609372664 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec072320 con 0x7fb1ec06df30 2024-05-08T04:42:54.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1eb400640 1 --2- 172.21.15.164:0/609372664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75ea46eef831e5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1ea000640 1 -- 172.21.15.164:0/609372664 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40b1070 con 0x7fb1ec06df30 2024-05-08T04:42:54.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1ea000640 1 -- 172.21.15.164:0/609372664 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40ac620 con 0x7fb1ec06df30 2024-05-08T04:42:54.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.439+0000 7fb1ea000640 1 -- 172.21.15.164:0/609372664 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40ac9f0 con 0x7fb1ec06df30 2024-05-08T04:42:54.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- 172.21.15.164:0/609372664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 msgr2=0x7fb1ec06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- 172.21.15.164:0/609372664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c4d0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- 172.21.15.164:0/609372664 shutdown_connections 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- 172.21.15.164:0/609372664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06df30 0x7fb1ec06e3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- 172.21.15.164:0/609372664 >> v2:172.21.15.172:3300/0 conn(0x7fb1ec06ac90 0x7fb1ec06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- 172.21.15.164:0/609372664 >> 172.21.15.164:0/609372664 conn(0x7fb1ec066fa0 msgr2=0x7fb1ec06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- 172.21.15.164:0/609372664 shutdown_connections 2024-05-08T04:42:54.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- 172.21.15.164:0/609372664 wait complete. 2024-05-08T04:42:54.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 Processor -- start 2024-05-08T04:42:54.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- start start 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06ac90 0x7fb1ec13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1ec0ccdf0 0x7fb1ec0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0adfa0 con 0x7fb1ec06df30 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec071de0 con 0x7fb1ec0ccdf0 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1f1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec13d0e0 con 0x7fb1ec06ac90 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56672/0 (socket says 172.21.15.164:56672) 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 -- 172.21.15.164:0/3307320712 learned_addr learned my addr 172.21.15.164:0/3307320712 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1eaa00640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06ac90 0x7fb1ec13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1eb400640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1ec0ccdf0 0x7fb1ec0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 -- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06ac90 msgr2=0x7fb1ec13cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06ac90 0x7fb1ec13cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 -- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1ec0ccdf0 msgr2=0x7fb1ec0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1ec0ccdf0 0x7fb1ec0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.443+0000 7fb1e2000640 1 -- 172.21.15.164:0/3307320712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec13f750 con 0x7fb1ec06df30 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e2000640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e930 tx=0x7fb1d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4017070 con 0x7fb1ec06df30 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400ca00 con 0x7fb1ec06df30 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400cdb0 con 0x7fb1ec06df30 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec1fb2d0 con 0x7fb1ec06df30 2024-05-08T04:42:54.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ec15c4b0 con 0x7fb1ec06df30 2024-05-08T04:42:54.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ec069d60 con 0x7fb1ec06df30 2024-05-08T04:42:54.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fb1d401a070 con 0x7fb1ec06df30 2024-05-08T04:42:54.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 0x7fb1c8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb1d40839d0 con 0x7fb1ec06df30 2024-05-08T04:42:54.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.447+0000 7fb1eaa00640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 0x7fb1c8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.451+0000 7fb1eaa00640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 0x7fb1c8069d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001e00 tx=0x7fb1dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.451+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1ec13f750 con 0x7fb1ec06df30 2024-05-08T04:42:54.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.607+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1ec0cdd10 con 0x7fb1ec06df30 2024-05-08T04:42:54.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.607+0000 7fb1e3400640 1 -- 172.21.15.164:0/3307320712 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16834 (secure 0 0 0) 0x7fb1d4055d10 con 0x7fb1ec06df30 2024-05-08T04:42:54.610 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:54.610 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":111,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:53.623356+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:54.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 msgr2=0x7fb1c8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 0x7fb1c8069d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001e00 tx=0x7fb1dc009040 comp rx=0 tx=0).stop 2024-05-08T04:42:54.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 msgr2=0x7fb1ec1571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e930 tx=0x7fb1d400edf0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 shutdown_connections 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb1c8067860 0x7fb1c8069d20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1ec0ccdf0 0x7fb1ec0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1ec06df30 0x7fb1ec1571e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 --2- 172.21.15.164:0/3307320712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1ec06ac90 0x7fb1ec13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 >> 172.21.15.164:0/3307320712 conn(0x7fb1ec066fa0 msgr2=0x7fb1ec160b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 shutdown_connections 2024-05-08T04:42:54.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.611+0000 7fb1f1600640 1 -- 172.21.15.164:0/3307320712 wait complete. 2024-05-08T04:42:54.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:54.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.763+0000 7f506b400640 1 Processor -- start 2024-05-08T04:42:54.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.763+0000 7f506b400640 1 -- start start 2024-05-08T04:42:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f506b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 0x7f50640cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f506b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f50640cfbd0 0x7f50640cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f506b400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506406c770 con 0x7f506415ab60 2024-05-08T04:42:54.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f506b400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640d0060 con 0x7f50640cfbd0 2024-05-08T04:42:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f506b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506406ca50 con 0x7f50640b26c0 2024-05-08T04:42:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5069400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f50640cfbd0 0x7f50640cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5068a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f506415ab60 0x7f5064072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46390/0 (socket says 172.21.15.164:46390) 2024-05-08T04:42:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5068a00640 1 -- 172.21.15.164:0/3580426587 learned_addr learned my addr 172.21.15.164:0/3580426587 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:54.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5063e00640 1 --2- 172.21.15.164:0/3580426587 >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 0x7f50640cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5069400640 1 -- 172.21.15.164:0/3580426587 >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 msgr2=0x7f50640cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5069400640 1 --2- 172.21.15.164:0/3580426587 >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 0x7f50640cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5069400640 1 -- 172.21.15.164:0/3580426587 >> v1:172.21.15.164:6789/0 conn(0x7f506415ab60 legacy=0x7f5064072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5069400640 1 -- 172.21.15.164:0/3580426587 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506406cd30 con 0x7f50640cfbd0 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5063e00640 1 --2- 172.21.15.164:0/3580426587 >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 0x7f50640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:54.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.767+0000 7f5063400640 1 -- 172.21.15.164:0/3580426587 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767736582 0 0) 0x7f506406c770 con 0x7f506415ab60 2024-05-08T04:42:54.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f5069400640 1 --2- 172.21.15.164:0/3580426587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640cfbd0 0x7f50640cd910 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f506406e880 tx=0x7f505c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c20c2ff454335fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f5063400640 1 -- 172.21.15.164:0/3580426587 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c0b0070 con 0x7f50640cfbd0 2024-05-08T04:42:54.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f5063400640 1 -- 172.21.15.164:0/3580426587 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0ab600 con 0x7f50640cfbd0 2024-05-08T04:42:54.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f5063400640 1 -- 172.21.15.164:0/3580426587 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c0ab9d0 con 0x7f50640cfbd0 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- 172.21.15.164:0/3580426587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640cfbd0 msgr2=0x7f50640cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 --2- 172.21.15.164:0/3580426587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640cfbd0 0x7f50640cd910 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f506406e880 tx=0x7f505c01c480 comp rx=0 tx=0).stop 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- 172.21.15.164:0/3580426587 shutdown_connections 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 --2- 172.21.15.164:0/3580426587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50640cfbd0 0x7f50640cd910 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 --2- 172.21.15.164:0/3580426587 >> v2:172.21.15.172:3300/0 conn(0x7f50640b26c0 0x7f50640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- 172.21.15.164:0/3580426587 >> 172.21.15.164:0/3580426587 conn(0x7f506406ac90 msgr2=0x7f506406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- 172.21.15.164:0/3580426587 shutdown_connections 2024-05-08T04:42:54.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- 172.21.15.164:0/3580426587 wait complete. 2024-05-08T04:42:54.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 Processor -- start 2024-05-08T04:42:54.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.771+0000 7f506b400640 1 -- start start 2024-05-08T04:42:54.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 0x7f5064151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 0x7f506414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640aeeb0 con 0x7f506415ab60 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640aed30 con 0x7f50640b26c0 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5064167970 con 0x7f50640cfbd0 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56694/0 (socket says 172.21.15.164:56694) 2024-05-08T04:42:54.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 -- 172.21.15.164:0/1141856427 learned_addr learned my addr 172.21.15.164:0/1141856427 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5068a00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 0x7f5064151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5063e00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 0x7f506414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 -- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 msgr2=0x7f506414f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 0x7f506414f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 -- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 msgr2=0x7f5064151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 0x7f5064151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 -- 172.21.15.164:0/1141856427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064150820 con 0x7f506415ab60 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5068a00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 0x7f5064151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5063e00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 0x7f506414f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5069400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f505c0ae040 tx=0x7f505c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c0b0050 con 0x7f506415ab60 2024-05-08T04:42:54.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0bc070 con 0x7f506415ab60 2024-05-08T04:42:54.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c0b5830 con 0x7f506415ab60 2024-05-08T04:42:54.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5064151990 con 0x7f506415ab60 2024-05-08T04:42:54.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5064151f50 con 0x7f506415ab60 2024-05-08T04:42:54.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5064069d60 con 0x7f506415ab60 2024-05-08T04:42:54.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.775+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f505c0b59d0 con 0x7f506415ab60 2024-05-08T04:42:54.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.779+0000 7f5060c00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 0x7f5044069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:54.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.779+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f505c11e270 con 0x7f506415ab60 2024-05-08T04:42:54.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.779+0000 7f5068a00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 0x7f5044069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:54.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.779+0000 7f5068a00640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 0x7f5044069d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f50540012f0 tx=0x7f5054009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:54.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.783+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5064150820 con 0x7f506415ab60 2024-05-08T04:42:54.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.935+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50640d04b0 con 0x7f506415ab60 2024-05-08T04:42:54.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f5060c00640 1 -- 172.21.15.164:0/1141856427 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f505c016070 con 0x7f506415ab60 2024-05-08T04:42:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:54.940 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":111,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:53.623356+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:54.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 msgr2=0x7f5044069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 0x7f5044069d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f50540012f0 tx=0x7f5054009040 comp rx=0 tx=0).stop 2024-05-08T04:42:54.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 msgr2=0x7f5064154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:54.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f505c0ae040 tx=0x7f505c01acc0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 shutdown_connections 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5044067860 0x7f5044069d20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50640cfbd0 0x7f506414f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506415ab60 0x7f5064154cd0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 --2- 172.21.15.164:0/1141856427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50640b26c0 0x7f5064151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 >> 172.21.15.164:0/1141856427 conn(0x7f506406ac90 msgr2=0x7f50640cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 shutdown_connections 2024-05-08T04:42:54.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:54.939+0000 7f506b400640 1 -- 172.21.15.164:0/1141856427 wait complete. 2024-05-08T04:42:54.962 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:42:55.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 Processor -- start 2024-05-08T04:42:55.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 -- start start 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3dc071460 0x7fa3dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3dc0cd8d0 0x7fa3dc0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0d0630 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b58f0 con 0x7fa3dc071460 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3e2800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b3970 con 0x7fa3dc0cd8d0 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3db400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3dc071460 0x7fa3dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3d3e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa3dc0d00c0 0x7fa3dc071840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46398/0 (socket says 172.21.15.164:46398) 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3d3e00640 1 -- 172.21.15.164:0/54374448 learned_addr learned my addr 172.21.15.164:0/54374448 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3dbe00640 1 --2- 172.21.15.164:0/54374448 >> v2:172.21.15.172:3300/0 conn(0x7fa3dc0cd8d0 0x7fa3dc0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3daa00640 1 -- 172.21.15.164:0/54374448 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740540118 0 0) 0x7fa3dc0d0630 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3db400640 1 -- 172.21.15.164:0/54374448 >> v2:172.21.15.172:3300/0 conn(0x7fa3dc0cd8d0 msgr2=0x7fa3dc0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3db400640 1 --2- 172.21.15.164:0/54374448 >> v2:172.21.15.172:3300/0 conn(0x7fa3dc0cd8d0 0x7fa3dc0cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3db400640 1 -- 172.21.15.164:0/54374448 >> v1:172.21.15.164:6789/0 conn(0x7fa3dc0d00c0 legacy=0x7fa3dc071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:55.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.099+0000 7fa3db400640 1 -- 172.21.15.164:0/54374448 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0b3c50 con 0x7fa3dc071460 2024-05-08T04:42:55.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3db400640 1 --2- 172.21.15.164:0/54374448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc071460 0x7fa3dc0b4f80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba367e09b6425917 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:55.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3daa00640 1 -- 172.21.15.164:0/54374448 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b3070 con 0x7fa3dc071460 2024-05-08T04:42:55.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3daa00640 1 -- 172.21.15.164:0/54374448 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0aa050 con 0x7fa3dc071460 2024-05-08T04:42:55.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3daa00640 1 -- 172.21.15.164:0/54374448 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0ae860 con 0x7fa3dc071460 2024-05-08T04:42:55.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 -- 172.21.15.164:0/54374448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc071460 msgr2=0x7fa3dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:55.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 --2- 172.21.15.164:0/54374448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc071460 0x7fa3dc0b4f80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01e5e0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 -- 172.21.15.164:0/54374448 shutdown_connections 2024-05-08T04:42:55.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 --2- 172.21.15.164:0/54374448 >> v2:172.21.15.172:3300/0 conn(0x7fa3dc0cd8d0 0x7fa3dc0cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 --2- 172.21.15.164:0/54374448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc071460 0x7fa3dc0b4f80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 -- 172.21.15.164:0/54374448 >> 172.21.15.164:0/54374448 conn(0x7fa3dc0b2d00 msgr2=0x7fa3dc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:55.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 -- 172.21.15.164:0/54374448 shutdown_connections 2024-05-08T04:42:55.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.103+0000 7fa3e2800640 1 -- 172.21.15.164:0/54374448 wait complete. 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 Processor -- start 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- start start 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3dc071460 0x7fa3dc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3dc0cd8d0 0x7fa3dc0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b48b0 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b4730 con 0x7fa3dc071460 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc06ac90 con 0x7fa3dc0cd8d0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56720/0 (socket says 172.21.15.164:56720) 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 -- 172.21.15.164:0/1169544854 learned_addr learned my addr 172.21.15.164:0/1169544854 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3d3e00640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3dc0cd8d0 0x7fa3dc0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3db400640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3dc071460 0x7fa3dc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 -- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3dc0cd8d0 msgr2=0x7fa3dc0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3dc0cd8d0 0x7fa3dc0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 -- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3dc071460 msgr2=0x7fa3dc14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3dc071460 0x7fa3dc14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 -- 172.21.15.164:0/1169544854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc159180 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3dbe00640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4007e40 tx=0x7fa3d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:55.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4014620 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d4019070 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4014a20 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc159460 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.107+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc159a20 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc072e80 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fa3d4014bc0 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3d3400640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 0x7fa3b8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:55.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa3d4084080 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3db400640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 0x7fa3b8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:55.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.111+0000 7fa3db400640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 0x7fa3b8069d20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0b0000 tx=0x7fa3cc0a9000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:55.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.115+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3d4056490 con 0x7fa3dc0d00c0 2024-05-08T04:42:55.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:55.303+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa3dc04f010 con 0x7fa3dc0d00c0 2024-05-08T04:42:56.222 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-05-08T04:42:56.219+0000 7f3a84e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:42:56.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.259+0000 7fa3d3400640 1 -- 172.21.15.164:0/1169544854 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa3d405a530 con 0x7fa3dc0d00c0 2024-05-08T04:42:56.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 msgr2=0x7fa3b8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:56.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 0x7fa3b8069d20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0b0000 tx=0x7fa3cc0a9000 comp rx=0 tx=0).stop 2024-05-08T04:42:56.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 msgr2=0x7fa3dc155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:56.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4007e40 tx=0x7fa3d400e4b0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 shutdown_connections 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa3b8067860 0x7fa3b8069d20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3dc0cd8d0 0x7fa3dc0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0d00c0 0x7fa3dc155b60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 --2- 172.21.15.164:0/1169544854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3dc071460 0x7fa3dc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 >> 172.21.15.164:0/1169544854 conn(0x7fa3dc0b2d00 msgr2=0x7fa3dc06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 shutdown_connections 2024-05-08T04:42:56.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.263+0000 7fa3e2800640 1 -- 172.21.15.164:0/1169544854 wait complete. 2024-05-08T04:42:56.286 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:42:56.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:42:56.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.423+0000 7fe38ee00640 1 Processor -- start 2024-05-08T04:42:56.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.423+0000 7fe38ee00640 1 -- start start 2024-05-08T04:42:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe38815a200 0x7fe3880aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38806dfa0 0x7fe3880ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3880ca680 0x7fe38815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3880caaf0 con 0x7fe38815a200 2024-05-08T04:42:56.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38806bca0 con 0x7fe3880ca680 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880af150 con 0x7fe38806dfa0 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe38815a200 0x7fe3880aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe38815a200 0x7fe3880aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56722/0 (socket says 172.21.15.164:56722) 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 -- 172.21.15.164:0/633263116 learned_addr learned my addr 172.21.15.164:0/633263116 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe38ce00640 1 --2- 172.21.15.164:0/633263116 >> v2:172.21.15.172:3300/0 conn(0x7fe3880ca680 0x7fe38815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe37f400640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38806dfa0 0x7fe3880ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 -- 172.21.15.164:0/633263116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38806dfa0 msgr2=0x7fe3880ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38806dfa0 0x7fe3880ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 -- 172.21.15.164:0/633263116 >> v2:172.21.15.172:3300/0 conn(0x7fe3880ca680 msgr2=0x7fe38815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 --2- 172.21.15.164:0/633263116 >> v2:172.21.15.172:3300/0 conn(0x7fe3880ca680 0x7fe38815de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.427+0000 7fe387e00640 1 -- 172.21.15.164:0/633263116 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38806d150 con 0x7fe38815a200 2024-05-08T04:42:56.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe387e00640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe38815a200 0x7fe3880aeac0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe388070130 tx=0x7fe37801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6888bb07af3c9a23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:56.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe387400640 1 -- 172.21.15.164:0/633263116 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780a5730 con 0x7fe38815a200 2024-05-08T04:42:56.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe387400640 1 -- 172.21.15.164:0/633263116 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780a58d0 con 0x7fe38815a200 2024-05-08T04:42:56.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe387400640 1 -- 172.21.15.164:0/633263116 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780a5ca0 con 0x7fe38815a200 2024-05-08T04:42:56.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- 172.21.15.164:0/633263116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe38815a200 msgr2=0x7fe3880aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe38815a200 0x7fe3880aeac0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe388070130 tx=0x7fe37801c930 comp rx=0 tx=0).stop 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- 172.21.15.164:0/633263116 shutdown_connections 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 --2- 172.21.15.164:0/633263116 >> v2:172.21.15.172:3300/0 conn(0x7fe3880ca680 0x7fe38815de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38806dfa0 0x7fe3880ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 --2- 172.21.15.164:0/633263116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe38815a200 0x7fe3880aeac0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- 172.21.15.164:0/633263116 >> 172.21.15.164:0/633263116 conn(0x7fe3880cf770 msgr2=0x7fe3880cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- 172.21.15.164:0/633263116 shutdown_connections 2024-05-08T04:42:56.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- 172.21.15.164:0/633263116 wait complete. 2024-05-08T04:42:56.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 Processor -- start 2024-05-08T04:42:56.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.431+0000 7fe38ee00640 1 -- start start 2024-05-08T04:42:56.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 0x7fe388154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 0x7fe3881562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38815a760 con 0x7fe3880ca680 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38815a5e0 con 0x7fe38806dfa0 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880ce780 con 0x7fe38815a200 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56726/0 (socket says 172.21.15.164:56726) 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 -- 172.21.15.164:0/434878054 learned_addr learned my addr 172.21.15.164:0/434878054 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe387e00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 0x7fe388154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ce00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 0x7fe3881562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 -- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 msgr2=0x7fe3881562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 0x7fe3881562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 -- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 msgr2=0x7fe388154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:56.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 0x7fe388154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 -- 172.21.15.164:0/434878054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880c9b40 con 0x7fe3880ca680 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ce00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 0x7fe3881562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe387e00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 0x7fe388154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe37f400640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe37000d620 tx=0x7fe37000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000b620 con 0x7fe3880ca680 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370019070 con 0x7fe3880ca680 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3880c9860 con 0x7fe3880ca680 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000b9e0 con 0x7fe3880ca680 2024-05-08T04:42:56.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe388155db0 con 0x7fe3880ca680 2024-05-08T04:42:56.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe388072d30 con 0x7fe3880ca680 2024-05-08T04:42:56.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.435+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fe37000bb80 con 0x7fe3880ca680 2024-05-08T04:42:56.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.439+0000 7fe384c00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 0x7fe364069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:56.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.439+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe370083f10 con 0x7fe3880ca680 2024-05-08T04:42:56.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.439+0000 7fe387e00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 0x7fe364069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:56.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.439+0000 7fe387e00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 0x7fe364069d20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe3780ae000 tx=0x7fe3780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:56.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.443+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe370056780 con 0x7fe3880ca680 2024-05-08T04:42:56.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:56.627+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe38804f010 con 0x7fe3880ca680 2024-05-08T04:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.263+0000 7fe384c00640 1 -- 172.21.15.164:0/434878054 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe370056080 con 0x7fe3880ca680 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 msgr2=0x7fe364069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 0x7fe364069d20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe3780ae000 tx=0x7fe3780a7070 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 msgr2=0x7fe3880c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe37000d620 tx=0x7fe37000dae0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 shutdown_connections 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe364067860 0x7fe364069d20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe38815a200 0x7fe3881562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3880ca680 0x7fe3880c8f30 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 --2- 172.21.15.164:0/434878054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38806dfa0 0x7fe388154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 >> 172.21.15.164:0/434878054 conn(0x7fe3880cf770 msgr2=0x7fe3881584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:57.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 shutdown_connections 2024-05-08T04:42:57.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.267+0000 7fe38ee00640 1 -- 172.21.15.164:0/434878054 wait complete. 2024-05-08T04:42:57.288 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:42:57.288 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:42:57.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.423+0000 7f325e800640 1 Processor -- start 2024-05-08T04:42:57.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.423+0000 7f325e800640 1 -- start start 2024-05-08T04:42:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f325e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f325e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 0x7f32580cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f325e800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3258072640 con 0x7f32580b26c0 2024-05-08T04:42:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f325e800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325806c770 con 0x7f325815ab60 2024-05-08T04:42:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f325e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325806ca50 con 0x7f32580cfda0 2024-05-08T04:42:57.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 0x7f32580cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f324ea00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f32580b26c0 0x7f32580cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:38992/0 (socket says 172.21.15.164:38992) 2024-05-08T04:42:57.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:48416/0 (socket says 172.21.15.164:48416) 2024-05-08T04:42:57.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f324ea00640 1 -- 172.21.15.164:0/478104493 learned_addr learned my addr 172.21.15.164:0/478104493 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257e00640 1 -- 172.21.15.164:0/478104493 >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 msgr2=0x7f3258072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257e00640 1 --2- 172.21.15.164:0/478104493 >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257e00640 1 -- 172.21.15.164:0/478104493 >> v1:172.21.15.172:6789/0 conn(0x7f32580b26c0 legacy=0x7f32580cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257e00640 1 -- 172.21.15.164:0/478104493 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325806cd30 con 0x7f32580cfda0 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3256a00640 1 -- 172.21.15.164:0/478104493 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503516353 0 0) 0x7f3258072640 con 0x7f32580b26c0 2024-05-08T04:42:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.427+0000 7f3257400640 1 --2- 172.21.15.164:0/478104493 >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:57.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f3257e00640 1 --2- 172.21.15.164:0/478104493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 0x7f32580cd910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3250009d90 tx=0x7f325001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258b7b9eba948120 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f3256a00640 1 -- 172.21.15.164:0/478104493 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500b2070 con 0x7f32580cfda0 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f3256a00640 1 -- 172.21.15.164:0/478104493 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32500a5830 con 0x7f32580cfda0 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f3256a00640 1 -- 172.21.15.164:0/478104493 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500a5c00 con 0x7f32580cfda0 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- 172.21.15.164:0/478104493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 msgr2=0x7f32580cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- 172.21.15.164:0/478104493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 0x7f32580cd910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3250009d90 tx=0x7f325001c970 comp rx=0 tx=0).stop 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- 172.21.15.164:0/478104493 shutdown_connections 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- 172.21.15.164:0/478104493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580cfda0 0x7f32580cd910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- 172.21.15.164:0/478104493 >> v2:172.21.15.172:3300/0 conn(0x7f325815ab60 0x7f3258072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- 172.21.15.164:0/478104493 >> 172.21.15.164:0/478104493 conn(0x7f325806ac90 msgr2=0x7f325806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:57.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- 172.21.15.164:0/478104493 shutdown_connections 2024-05-08T04:42:57.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- 172.21.15.164:0/478104493 wait complete. 2024-05-08T04:42:57.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 Processor -- start 2024-05-08T04:42:57.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- start start 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580b26c0 0x7f32581552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32580cfda0 0x7f3258158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580aeeb0 con 0x7f325815ab60 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580aed30 con 0x7f32580cfda0 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f325e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580cc1b0 con 0x7f32580b26c0 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.431+0000 7f3257e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56734/0 (socket says 172.21.15.164:56734) 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 -- 172.21.15.164:0/4171542995 learned_addr learned my addr 172.21.15.164:0/4171542995 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:57.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324ea00640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32580cfda0 0x7f3258158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257400640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580b26c0 0x7f32581552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 -- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580b26c0 msgr2=0x7f32581552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580b26c0 0x7f32581552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 -- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32580cfda0 msgr2=0x7f3258158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32580cfda0 0x7f3258158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 -- 172.21.15.164:0/4171542995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258158cd0 con 0x7f325815ab60 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f3257e00640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f32500b0040 tx=0x7f3250019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500b2050 con 0x7f325815ab60 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32500be070 con 0x7f325815ab60 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500ada90 con 0x7f325815ab60 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32581492d0 con 0x7f325815ab60 2024-05-08T04:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3258155d20 con 0x7f325815ab60 2024-05-08T04:42:57.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258069d60 con 0x7f325815ab60 2024-05-08T04:42:57.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f32500a56b0 con 0x7f325815ab60 2024-05-08T04:42:57.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 0x7f3234069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.435+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f325011ebd0 con 0x7f325815ab60 2024-05-08T04:42:57.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.439+0000 7f3257400640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 0x7f3234069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.439+0000 7f3257400640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 0x7f3234069d20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f325815a2c0 tx=0x7f3248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.443+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3250016560 con 0x7f325815ab60 2024-05-08T04:42:57.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.631+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f325806d9a0 con 0x7f325815ab60 2024-05-08T04:42:57.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.631+0000 7f324fe00640 1 -- 172.21.15.164:0/4171542995 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f32500ab340 con 0x7f325815ab60 2024-05-08T04:42:57.635 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:57.635 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:42:57.635 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 72 2024-05-08T04:42:57.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 msgr2=0x7f3234069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 0x7f3234069d20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f325815a2c0 tx=0x7f3248008040 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 msgr2=0x7f3258148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f32500b0040 tx=0x7f3250019710 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 shutdown_connections 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3234067860 0x7f3234069d20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f325815ab60 0x7f3258148d90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32580cfda0 0x7f3258158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 --2- 172.21.15.164:0/4171542995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32580b26c0 0x7f32581552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 >> 172.21.15.164:0/4171542995 conn(0x7f325806ac90 msgr2=0x7f32580cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 shutdown_connections 2024-05-08T04:42:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.635+0000 7f325e800640 1 -- 172.21.15.164:0/4171542995 wait complete. 2024-05-08T04:42:57.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:42:57.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 Processor -- start 2024-05-08T04:42:57.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 -- start start 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f170815ab60 0x7f1708072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17080b26c0 0x7f17080cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1708072640 con 0x7f17080cfbd0 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080d0060 con 0x7f170815ab60 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f170d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170806c770 con 0x7f17080b26c0 2024-05-08T04:42:57.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f170815ab60 0x7f1708072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f170815ab60 0x7f1708072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56736/0 (socket says 172.21.15.164:56736) 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 -- 172.21.15.164:0/3538269895 learned_addr learned my addr 172.21.15.164:0/3538269895 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706000640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17080b26c0 0x7f17080cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 -- 172.21.15.164:0/3538269895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17080b26c0 msgr2=0x7f17080cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17080b26c0 0x7f17080cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 -- 172.21.15.164:0/3538269895 >> v1:172.21.15.172:6789/0 conn(0x7f17080cfbd0 legacy=0x7f17080cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.791+0000 7f1706a00640 1 -- 172.21.15.164:0/3538269895 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170806ca50 con 0x7f170815ab60 2024-05-08T04:42:57.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f1706a00640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f170815ab60 0x7f1708072100 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009870 tx=0x7f16f801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe35f7e31976e928 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f1705600640 1 -- 172.21.15.164:0/3538269895 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f80b2070 con 0x7f170815ab60 2024-05-08T04:42:57.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f1705600640 1 -- 172.21.15.164:0/3538269895 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f80a9070 con 0x7f170815ab60 2024-05-08T04:42:57.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f1705600640 1 -- 172.21.15.164:0/3538269895 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f80ad880 con 0x7f170815ab60 2024-05-08T04:42:57.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 -- 172.21.15.164:0/3538269895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f170815ab60 msgr2=0x7f1708072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f170815ab60 0x7f1708072100 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009870 tx=0x7f16f801c560 comp rx=0 tx=0).stop 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 -- 172.21.15.164:0/3538269895 shutdown_connections 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17080b26c0 0x7f17080cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 --2- 172.21.15.164:0/3538269895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f170815ab60 0x7f1708072100 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 -- 172.21.15.164:0/3538269895 >> 172.21.15.164:0/3538269895 conn(0x7f170806ac90 msgr2=0x7f170806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 -- 172.21.15.164:0/3538269895 shutdown_connections 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.795+0000 7f170d800640 1 -- 172.21.15.164:0/3538269895 wait complete. 2024-05-08T04:42:57.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 Processor -- start 2024-05-08T04:42:57.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- start start 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170815ab60 0x7f1708150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170806c550 con 0x7f17080cfbd0 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170806c3d0 con 0x7f17080b26c0 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080aed30 con 0x7f170815ab60 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56740/0 (socket says 172.21.15.164:56740) 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:48432/0 (socket says 172.21.15.164:48432) 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 -- 172.21.15.164:0/2151864738 learned_addr learned my addr 172.21.15.164:0/2151864738 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1707400640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170815ab60 0x7f1708150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 -- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170815ab60 msgr2=0x7f1708150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170815ab60 0x7f1708150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 -- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 msgr2=0x7f170814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 -- 172.21.15.164:0/2151864738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708148b30 con 0x7f17080cfbd0 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706a00640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f1706000640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d800 tx=0x7f16f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400b6c0 con 0x7f17080cfbd0 2024-05-08T04:42:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4019070 con 0x7f17080cfbd0 2024-05-08T04:42:57.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1708149020 con 0x7f17080cfbd0 2024-05-08T04:42:57.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400bae0 con 0x7f17080cfbd0 2024-05-08T04:42:57.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.799+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170814f9a0 con 0x7f17080cfbd0 2024-05-08T04:42:57.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1708069d60 con 0x7f17080cfbd0 2024-05-08T04:42:57.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f16f400bc80 con 0x7f17080cfbd0 2024-05-08T04:42:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f16fea00640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 0x7f16e8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f16f4083e40 con 0x7f17080cfbd0 2024-05-08T04:42:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f1706a00640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 0x7f16e8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.803+0000 7f1706a00640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 0x7f16e8069d20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f16f80af000 tx=0x7f16f80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:57.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.807+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16f40560c0 con 0x7f17080cfbd0 2024-05-08T04:42:57.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.963+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17080cc7a0 con 0x7f17080cfbd0 2024-05-08T04:42:57.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.963+0000 7f16fea00640 1 -- 172.21.15.164:0/2151864738 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f16f405a160 con 0x7f17080cfbd0 2024-05-08T04:42:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:57.966 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":112,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:42:55.247677+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 msgr2=0x7f16e8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 0x7f16e8069d20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f16f80af000 tx=0x7f16f80a7000 comp rx=0 tx=0).stop 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 msgr2=0x7f17081485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d800 tx=0x7f16f400dcc0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 shutdown_connections 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16e8067860 0x7f16e8069d20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170815ab60 0x7f1708150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17080cfbd0 0x7f17081485f0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 --2- 172.21.15.164:0/2151864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17080b26c0 0x7f170814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:57.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 >> 172.21.15.164:0/2151864738 conn(0x7f170806ac90 msgr2=0x7f1708154220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:57.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 shutdown_connections 2024-05-08T04:42:57.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:57.967+0000 7f170d800640 1 -- 172.21.15.164:0/2151864738 wait complete. 2024-05-08T04:42:57.987 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:42:57.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:58.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.119+0000 7effc3e00640 1 Processor -- start 2024-05-08T04:42:58.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.119+0000 7effc3e00640 1 -- start start 2024-05-08T04:42:58.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 0x7effbc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effbc0cfda0 0x7effbc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc0cd9f0 con 0x7effbc15be50 2024-05-08T04:42:58.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc3e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc15fe70 con 0x7effbc0cfda0 2024-05-08T04:42:58.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0cdcd0 con 0x7effbc15ba70 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effbc0cfda0 0x7effbc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effbc0cfda0 0x7effbc0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56752/0 (socket says 172.21.15.164:56752) 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 -- 172.21.15.164:0/3874457609 learned_addr learned my addr 172.21.15.164:0/3874457609 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1400640 1 --2- 172.21.15.164:0/3874457609 >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 0x7effbc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 -- 172.21.15.164:0/3874457609 >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 msgr2=0x7effbc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:58.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 --2- 172.21.15.164:0/3874457609 >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 0x7effbc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 -- 172.21.15.164:0/3874457609 >> v1:172.21.15.172:6789/0 conn(0x7effbc15be50 legacy=0x7effbc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:58.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1e00640 1 -- 172.21.15.164:0/3874457609 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc15f820 con 0x7effbc0cfda0 2024-05-08T04:42:58.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.123+0000 7effc1400640 1 --2- 172.21.15.164:0/3874457609 >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 0x7effbc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:58.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc1e00640 1 --2- 172.21.15.164:0/3874457609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc0cfda0 0x7effbc0b31a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7effbc06bc70 tx=0x7effb401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4748a2594613c871 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effbbe00640 1 -- 172.21.15.164:0/3874457609 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40ab600 con 0x7effbc0cfda0 2024-05-08T04:42:58.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effbbe00640 1 -- 172.21.15.164:0/3874457609 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40b0070 con 0x7effbc0cfda0 2024-05-08T04:42:58.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effbbe00640 1 -- 172.21.15.164:0/3874457609 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40ab9c0 con 0x7effbc0cfda0 2024-05-08T04:42:58.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- 172.21.15.164:0/3874457609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc0cfda0 msgr2=0x7effbc0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 --2- 172.21.15.164:0/3874457609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc0cfda0 0x7effbc0b31a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7effbc06bc70 tx=0x7effb401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- 172.21.15.164:0/3874457609 shutdown_connections 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 --2- 172.21.15.164:0/3874457609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc0cfda0 0x7effbc0b31a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 --2- 172.21.15.164:0/3874457609 >> v2:172.21.15.172:3300/0 conn(0x7effbc15ba70 0x7effbc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- 172.21.15.164:0/3874457609 >> 172.21.15.164:0/3874457609 conn(0x7effbc06ac90 msgr2=0x7effbc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- 172.21.15.164:0/3874457609 shutdown_connections 2024-05-08T04:42:58.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- 172.21.15.164:0/3874457609 wait complete. 2024-05-08T04:42:58.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 Processor -- start 2024-05-08T04:42:58.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.127+0000 7effc3e00640 1 -- start start 2024-05-08T04:42:58.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effbc0cfda0 0x7effbc071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc15be50 0x7effbc14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc168d40 con 0x7effbc15ba70 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc168bc0 con 0x7effbc15be50 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc168ec0 con 0x7effbc0cfda0 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56758/0 (socket says 172.21.15.164:56758) 2024-05-08T04:42:58.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 -- 172.21.15.164:0/1385906998 learned_addr learned my addr 172.21.15.164:0/1385906998 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc1400640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effbc0cfda0 0x7effbc071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc1e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc15be50 0x7effbc14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 -- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effbc0cfda0 msgr2=0x7effbc071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effbc0cfda0 0x7effbc071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 -- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc15be50 msgr2=0x7effbc14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc15be50 0x7effbc14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 -- 172.21.15.164:0/1385906998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc070750 con 0x7effbc15ba70 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc0a00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7effa800c370 tx=0x7effa800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80052d0 con 0x7effbc15ba70 2024-05-08T04:42:58.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8005470 con 0x7effbc15ba70 2024-05-08T04:42:58.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8007260 con 0x7effbc15ba70 2024-05-08T04:42:58.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc070a30 con 0x7effbc15ba70 2024-05-08T04:42:58.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc070ff0 con 0x7effbc15ba70 2024-05-08T04:42:58.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc069d60 con 0x7effbc15ba70 2024-05-08T04:42:58.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.131+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7effa8007400 con 0x7effbc15ba70 2024-05-08T04:42:58.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.135+0000 7effb9600640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 0x7eff9c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.135+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7effa80856c0 con 0x7effbc15ba70 2024-05-08T04:42:58.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.135+0000 7effc1400640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 0x7eff9c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.135+0000 7effc1400640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 0x7eff9c069d20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7effb0003b90 tx=0x7effb0009890 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.139+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effa8055100 con 0x7effbc15ba70 2024-05-08T04:42:58.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.291+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effbc06e210 con 0x7eff9c067860 2024-05-08T04:42:58.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effb9600640 1 -- 172.21.15.164:0/1385906998 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249796 (secure 0 0 0) 0x7effbc06e210 con 0x7eff9c067860 2024-05-08T04:42:58.302 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:58.303 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 msgr2=0x7eff9c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 0x7eff9c069d20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7effb0003b90 tx=0x7effb0009890 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 msgr2=0x7effbc146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7effa800c370 tx=0x7effa800c830 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 shutdown_connections 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7eff9c067860 0x7eff9c069d20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc15be50 0x7effbc14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effbc15ba70 0x7effbc146b40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 --2- 172.21.15.164:0/1385906998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effbc0cfda0 0x7effbc071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 >> 172.21.15.164:0/1385906998 conn(0x7effbc06ac90 msgr2=0x7effbc15a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:58.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.299+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 shutdown_connections 2024-05-08T04:42:58.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.303+0000 7effc3e00640 1 -- 172.21.15.164:0/1385906998 wait complete. 2024-05-08T04:42:58.323 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-05-08T04:42:57.484254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261468,"kb_used_data":12004,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713252,"statfs":{"total":773094113280,"available":772826370048,"internally_reserved":0,"allocated":12292096,"data_stored":5995905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.254344"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645115+0000","last_change":"2024-05-08T04:42:54.284520+0000","last_active":"2024-05-08T04:42:55.645115+0000","last_peered":"2024-05-08T04:42:55.645115+0000","last_clean":"2024-05-08T04:42:55.645115+0000","last_became_active":"2024-05-08T04:42:54.283787+0000","last_became_peered":"2024-05-08T04:42:54.283787+0000","last_unstale":"2024-05-08T04:42:55.645115+0000","last_undegraded":"2024-05-08T04:42:55.645115+0000","last_fullsized":"2024-05-08T04:42:55.645115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:31.361703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260638+0000","last_change":"2024-05-08T04:42:54.267059+0000","last_active":"2024-05-08T04:42:55.260638+0000","last_peered":"2024-05-08T04:42:55.260638+0000","last_clean":"2024-05-08T04:42:55.260638+0000","last_became_active":"2024-05-08T04:42:54.266748+0000","last_became_peered":"2024-05-08T04:42:54.266748+0000","last_unstale":"2024-05-08T04:42:55.260638+0000","last_undegraded":"2024-05-08T04:42:55.260638+0000","last_fullsized":"2024-05-08T04:42:55.260638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:57.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543833+0000","last_change":"2024-05-08T04:42:54.259254+0000","last_active":"2024-05-08T04:42:55.543833+0000","last_peered":"2024-05-08T04:42:55.543833+0000","last_clean":"2024-05-08T04:42:55.543833+0000","last_became_active":"2024-05-08T04:42:54.258620+0000","last_became_peered":"2024-05-08T04:42:54.258620+0000","last_unstale":"2024-05-08T04:42:55.543833+0000","last_undegraded":"2024-05-08T04:42:55.543833+0000","last_fullsized":"2024-05-08T04:42:55.543833+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644261+0000","last_change":"2024-05-08T04:42:54.276298+0000","last_active":"2024-05-08T04:42:55.644261+0000","last_peered":"2024-05-08T04:42:55.644261+0000","last_clean":"2024-05-08T04:42:55.644261+0000","last_became_active":"2024-05-08T04:42:54.273697+0000","last_became_peered":"2024-05-08T04:42:54.273697+0000","last_unstale":"2024-05-08T04:42:55.644261+0000","last_undegraded":"2024-05-08T04:42:55.644261+0000","last_fullsized":"2024-05-08T04:42:55.644261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:45.521599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644461+0000","last_change":"2024-05-08T04:42:54.259042+0000","last_active":"2024-05-08T04:42:55.644461+0000","last_peered":"2024-05-08T04:42:55.644461+0000","last_clean":"2024-05-08T04:42:55.644461+0000","last_became_active":"2024-05-08T04:42:54.258775+0000","last_became_peered":"2024-05-08T04:42:54.258775+0000","last_unstale":"2024-05-08T04:42:55.644461+0000","last_undegraded":"2024-05-08T04:42:55.644461+0000","last_fullsized":"2024-05-08T04:42:55.644461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:58.137321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258732+0000","last_change":"2024-05-08T04:42:54.258159+0000","last_active":"2024-05-08T04:42:55.258732+0000","last_peered":"2024-05-08T04:42:55.258732+0000","last_clean":"2024-05-08T04:42:55.258732+0000","last_became_active":"2024-05-08T04:42:54.257892+0000","last_became_peered":"2024-05-08T04:42:54.257892+0000","last_unstale":"2024-05-08T04:42:55.258732+0000","last_undegraded":"2024-05-08T04:42:55.258732+0000","last_fullsized":"2024-05-08T04:42:55.258732+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543525+0000","last_change":"2024-05-08T04:42:54.261073+0000","last_active":"2024-05-08T04:42:55.543525+0000","last_peered":"2024-05-08T04:42:55.543525+0000","last_clean":"2024-05-08T04:42:55.543525+0000","last_became_active":"2024-05-08T04:42:54.260783+0000","last_became_peered":"2024-05-08T04:42:54.260783+0000","last_unstale":"2024-05-08T04:42:55.543525+0000","last_undegraded":"2024-05-08T04:42:55.543525+0000","last_fullsized":"2024-05-08T04:42:55.543525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645257+0000","last_change":"2024-05-08T04:42:54.280775+0000","last_active":"2024-05-08T04:42:55.645257+0000","last_peered":"2024-05-08T04:42:55.645257+0000","last_clean":"2024-05-08T04:42:55.645257+0000","last_became_active":"2024-05-08T04:42:54.280501+0000","last_became_peered":"2024-05-08T04:42:54.280501+0000","last_unstale":"2024-05-08T04:42:55.645257+0000","last_undegraded":"2024-05-08T04:42:55.645257+0000","last_fullsized":"2024-05-08T04:42:55.645257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:25.588352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645180+0000","last_change":"2024-05-08T04:42:54.279039+0000","last_active":"2024-05-08T04:42:55.645180+0000","last_peered":"2024-05-08T04:42:55.645180+0000","last_clean":"2024-05-08T04:42:55.645180+0000","last_became_active":"2024-05-08T04:42:54.278516+0000","last_became_peered":"2024-05-08T04:42:54.278516+0000","last_unstale":"2024-05-08T04:42:55.645180+0000","last_undegraded":"2024-05-08T04:42:55.645180+0000","last_fullsized":"2024-05-08T04:42:55.645180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:44.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260298+0000","last_change":"2024-05-08T04:42:54.264244+0000","last_active":"2024-05-08T04:42:55.260298+0000","last_peered":"2024-05-08T04:42:55.260298+0000","last_clean":"2024-05-08T04:42:55.260298+0000","last_became_active":"2024-05-08T04:42:54.261285+0000","last_became_peered":"2024-05-08T04:42:54.261285+0000","last_unstale":"2024-05-08T04:42:55.260298+0000","last_undegraded":"2024-05-08T04:42:55.260298+0000","last_fullsized":"2024-05-08T04:42:55.260298+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:16.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644393+0000","last_change":"2024-05-08T04:42:54.278540+0000","last_active":"2024-05-08T04:42:55.644393+0000","last_peered":"2024-05-08T04:42:55.644393+0000","last_clean":"2024-05-08T04:42:55.644393+0000","last_became_active":"2024-05-08T04:42:54.275676+0000","last_became_peered":"2024-05-08T04:42:54.275676+0000","last_unstale":"2024-05-08T04:42:55.644393+0000","last_undegraded":"2024-05-08T04:42:55.644393+0000","last_fullsized":"2024-05-08T04:42:55.644393+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:02.962699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543936+0000","last_change":"2024-05-08T04:42:54.262625+0000","last_active":"2024-05-08T04:42:55.543936+0000","last_peered":"2024-05-08T04:42:55.543936+0000","last_clean":"2024-05-08T04:42:55.543936+0000","last_became_active":"2024-05-08T04:42:54.262328+0000","last_became_peered":"2024-05-08T04:42:54.262328+0000","last_unstale":"2024-05-08T04:42:55.543936+0000","last_undegraded":"2024-05-08T04:42:55.543936+0000","last_fullsized":"2024-05-08T04:42:55.543936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:37.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543497+0000","last_change":"2024-05-08T04:42:54.263202+0000","last_active":"2024-05-08T04:42:55.543497+0000","last_peered":"2024-05-08T04:42:55.543497+0000","last_clean":"2024-05-08T04:42:55.543497+0000","last_became_active":"2024-05-08T04:42:54.262118+0000","last_became_peered":"2024-05-08T04:42:54.262118+0000","last_unstale":"2024-05-08T04:42:55.543497+0000","last_undegraded":"2024-05-08T04:42:55.543497+0000","last_fullsized":"2024-05-08T04:42:55.543497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643757+0000","last_change":"2024-05-08T04:42:54.258584+0000","last_active":"2024-05-08T04:42:55.643757+0000","last_peered":"2024-05-08T04:42:55.643757+0000","last_clean":"2024-05-08T04:42:55.643757+0000","last_became_active":"2024-05-08T04:42:54.257809+0000","last_became_peered":"2024-05-08T04:42:54.257809+0000","last_unstale":"2024-05-08T04:42:55.643757+0000","last_undegraded":"2024-05-08T04:42:55.643757+0000","last_fullsized":"2024-05-08T04:42:55.643757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.486563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643534+0000","last_change":"2024-05-08T04:42:54.272568+0000","last_active":"2024-05-08T04:42:55.643534+0000","last_peered":"2024-05-08T04:42:55.643534+0000","last_clean":"2024-05-08T04:42:55.643534+0000","last_became_active":"2024-05-08T04:42:54.266872+0000","last_became_peered":"2024-05-08T04:42:54.266872+0000","last_unstale":"2024-05-08T04:42:55.643534+0000","last_undegraded":"2024-05-08T04:42:55.643534+0000","last_fullsized":"2024-05-08T04:42:55.643534+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:19.666610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260434+0000","last_change":"2024-05-08T04:42:54.265221+0000","last_active":"2024-05-08T04:42:55.260434+0000","last_peered":"2024-05-08T04:42:55.260434+0000","last_clean":"2024-05-08T04:42:55.260434+0000","last_became_active":"2024-05-08T04:42:54.263935+0000","last_became_peered":"2024-05-08T04:42:54.263935+0000","last_unstale":"2024-05-08T04:42:55.260434+0000","last_undegraded":"2024-05-08T04:42:55.260434+0000","last_fullsized":"2024-05-08T04:42:55.260434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:24.087153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261472+0000","last_change":"2024-05-08T04:42:54.260979+0000","last_active":"2024-05-08T04:42:55.261472+0000","last_peered":"2024-05-08T04:42:55.261472+0000","last_clean":"2024-05-08T04:42:55.261472+0000","last_became_active":"2024-05-08T04:42:54.260386+0000","last_became_peered":"2024-05-08T04:42:54.260386+0000","last_unstale":"2024-05-08T04:42:55.261472+0000","last_undegraded":"2024-05-08T04:42:55.261472+0000","last_fullsized":"2024-05-08T04:42:55.261472+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:23.982806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543852+0000","last_change":"2024-05-08T04:42:54.267987+0000","last_active":"2024-05-08T04:42:55.543852+0000","last_peered":"2024-05-08T04:42:55.543852+0000","last_clean":"2024-05-08T04:42:55.543852+0000","last_became_active":"2024-05-08T04:42:54.267674+0000","last_became_peered":"2024-05-08T04:42:54.267674+0000","last_unstale":"2024-05-08T04:42:55.543852+0000","last_undegraded":"2024-05-08T04:42:55.543852+0000","last_fullsized":"2024-05-08T04:42:55.543852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:11.284074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259984+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.259984+0000","last_peered":"2024-05-08T04:42:55.259984+0000","last_clean":"2024-05-08T04:42:55.259984+0000","last_became_active":"2024-05-08T04:42:54.261903+0000","last_became_peered":"2024-05-08T04:42:54.261903+0000","last_unstale":"2024-05-08T04:42:55.259984+0000","last_undegraded":"2024-05-08T04:42:55.259984+0000","last_fullsized":"2024-05-08T04:42:55.259984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:49.660293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260474+0000","last_change":"2024-05-08T04:42:54.247853+0000","last_active":"2024-05-08T04:42:55.260474+0000","last_peered":"2024-05-08T04:42:55.260474+0000","last_clean":"2024-05-08T04:42:55.260474+0000","last_became_active":"2024-05-08T04:42:54.247314+0000","last_became_peered":"2024-05-08T04:42:54.247314+0000","last_unstale":"2024-05-08T04:42:55.260474+0000","last_undegraded":"2024-05-08T04:42:55.260474+0000","last_fullsized":"2024-05-08T04:42:55.260474+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.153233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258982+0000","last_change":"2024-05-08T04:42:54.257923+0000","last_active":"2024-05-08T04:42:55.258982+0000","last_peered":"2024-05-08T04:42:55.258982+0000","last_clean":"2024-05-08T04:42:55.258982+0000","last_became_active":"2024-05-08T04:42:54.257524+0000","last_became_peered":"2024-05-08T04:42:54.257524+0000","last_unstale":"2024-05-08T04:42:55.258982+0000","last_undegraded":"2024-05-08T04:42:55.258982+0000","last_fullsized":"2024-05-08T04:42:55.258982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260632+0000","last_change":"2024-05-08T04:42:54.264625+0000","last_active":"2024-05-08T04:42:55.260632+0000","last_peered":"2024-05-08T04:42:55.260632+0000","last_clean":"2024-05-08T04:42:55.260632+0000","last_became_active":"2024-05-08T04:42:54.261407+0000","last_became_peered":"2024-05-08T04:42:54.261407+0000","last_unstale":"2024-05-08T04:42:55.260632+0000","last_undegraded":"2024-05-08T04:42:55.260632+0000","last_fullsized":"2024-05-08T04:42:55.260632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:21.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644944+0000","last_change":"2024-05-08T04:42:54.273444+0000","last_active":"2024-05-08T04:42:55.644944+0000","last_peered":"2024-05-08T04:42:55.644944+0000","last_clean":"2024-05-08T04:42:55.644944+0000","last_became_active":"2024-05-08T04:42:54.273184+0000","last_became_peered":"2024-05-08T04:42:54.273184+0000","last_unstale":"2024-05-08T04:42:55.644944+0000","last_undegraded":"2024-05-08T04:42:55.644944+0000","last_fullsized":"2024-05-08T04:42:55.644944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:18.445404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645146+0000","last_change":"2024-05-08T04:42:54.277145+0000","last_active":"2024-05-08T04:42:55.645146+0000","last_peered":"2024-05-08T04:42:55.645146+0000","last_clean":"2024-05-08T04:42:55.645146+0000","last_became_active":"2024-05-08T04:42:54.274239+0000","last_became_peered":"2024-05-08T04:42:54.274239+0000","last_unstale":"2024-05-08T04:42:55.645146+0000","last_undegraded":"2024-05-08T04:42:55.645146+0000","last_fullsized":"2024-05-08T04:42:55.645146+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260283+0000","last_change":"2024-05-08T04:42:54.257830+0000","last_active":"2024-05-08T04:42:55.260283+0000","last_peered":"2024-05-08T04:42:55.260283+0000","last_clean":"2024-05-08T04:42:55.260283+0000","last_became_active":"2024-05-08T04:42:54.257455+0000","last_became_peered":"2024-05-08T04:42:54.257455+0000","last_unstale":"2024-05-08T04:42:55.260283+0000","last_undegraded":"2024-05-08T04:42:55.260283+0000","last_fullsized":"2024-05-08T04:42:55.260283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:08.717198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259104+0000","last_change":"2024-05-08T04:42:54.247280+0000","last_active":"2024-05-08T04:42:55.259104+0000","last_peered":"2024-05-08T04:42:55.259104+0000","last_clean":"2024-05-08T04:42:55.259104+0000","last_became_active":"2024-05-08T04:42:54.246609+0000","last_became_peered":"2024-05-08T04:42:54.246609+0000","last_unstale":"2024-05-08T04:42:55.259104+0000","last_undegraded":"2024-05-08T04:42:55.259104+0000","last_fullsized":"2024-05-08T04:42:55.259104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643671+0000","last_change":"2024-05-08T04:42:54.270435+0000","last_active":"2024-05-08T04:42:55.643671+0000","last_peered":"2024-05-08T04:42:55.643671+0000","last_clean":"2024-05-08T04:42:55.643671+0000","last_became_active":"2024-05-08T04:42:54.265729+0000","last_became_peered":"2024-05-08T04:42:54.265729+0000","last_unstale":"2024-05-08T04:42:55.643671+0000","last_undegraded":"2024-05-08T04:42:55.643671+0000","last_fullsized":"2024-05-08T04:42:55.643671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:53.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643688+0000","last_change":"2024-05-08T04:42:54.261182+0000","last_active":"2024-05-08T04:42:55.643688+0000","last_peered":"2024-05-08T04:42:55.643688+0000","last_clean":"2024-05-08T04:42:55.643688+0000","last_became_active":"2024-05-08T04:42:54.260895+0000","last_became_peered":"2024-05-08T04:42:54.260895+0000","last_unstale":"2024-05-08T04:42:55.643688+0000","last_undegraded":"2024-05-08T04:42:55.643688+0000","last_fullsized":"2024-05-08T04:42:55.643688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645113+0000","last_change":"2024-05-08T04:42:54.271090+0000","last_active":"2024-05-08T04:42:55.645113+0000","last_peered":"2024-05-08T04:42:55.645113+0000","last_clean":"2024-05-08T04:42:55.645113+0000","last_became_active":"2024-05-08T04:42:54.270874+0000","last_became_peered":"2024-05-08T04:42:54.270874+0000","last_unstale":"2024-05-08T04:42:55.645113+0000","last_undegraded":"2024-05-08T04:42:55.645113+0000","last_fullsized":"2024-05-08T04:42:55.645113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259863+0000","last_change":"2024-05-08T04:42:53.251819+0000","last_active":"2024-05-08T04:42:55.259863+0000","last_peered":"2024-05-08T04:42:55.259863+0000","last_clean":"2024-05-08T04:42:55.259863+0000","last_became_active":"2024-05-08T04:42:53.251490+0000","last_became_peered":"2024-05-08T04:42:53.251490+0000","last_unstale":"2024-05-08T04:42:55.259863+0000","last_undegraded":"2024-05-08T04:42:55.259863+0000","last_fullsized":"2024-05-08T04:42:55.259863+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:07.081658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261626+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.261626+0000","last_peered":"2024-05-08T04:42:55.261626+0000","last_clean":"2024-05-08T04:42:55.261626+0000","last_became_active":"2024-05-08T04:42:54.261894+0000","last_became_peered":"2024-05-08T04:42:54.261894+0000","last_unstale":"2024-05-08T04:42:55.261626+0000","last_undegraded":"2024-05-08T04:42:55.261626+0000","last_fullsized":"2024-05-08T04:42:55.261626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543705+0000","last_change":"2024-05-08T04:42:53.256430+0000","last_active":"2024-05-08T04:42:55.543705+0000","last_peered":"2024-05-08T04:42:55.543705+0000","last_clean":"2024-05-08T04:42:55.543705+0000","last_became_active":"2024-05-08T04:42:53.256154+0000","last_became_peered":"2024-05-08T04:42:53.256154+0000","last_unstale":"2024-05-08T04:42:55.543705+0000","last_undegraded":"2024-05-08T04:42:55.543705+0000","last_fullsized":"2024-05-08T04:42:55.543705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644955+0000","last_change":"2024-05-08T04:42:54.264075+0000","last_active":"2024-05-08T04:42:55.644955+0000","last_peered":"2024-05-08T04:42:55.644955+0000","last_clean":"2024-05-08T04:42:55.644955+0000","last_became_active":"2024-05-08T04:42:54.263803+0000","last_became_peered":"2024-05-08T04:42:54.263803+0000","last_unstale":"2024-05-08T04:42:55.644955+0000","last_undegraded":"2024-05-08T04:42:55.644955+0000","last_fullsized":"2024-05-08T04:42:55.644955+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234430+0000","last_change":"2024-05-08T04:42:52.234430+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234430+0000","last_undegraded":"2024-05-08T04:42:52.234430+0000","last_fullsized":"2024-05-08T04:42:52.234430+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544048+0000","last_change":"2024-05-08T04:42:54.279237+0000","last_active":"2024-05-08T04:42:55.544048+0000","last_peered":"2024-05-08T04:42:55.544048+0000","last_clean":"2024-05-08T04:42:55.544048+0000","last_became_active":"2024-05-08T04:42:54.279021+0000","last_became_peered":"2024-05-08T04:42:54.279021+0000","last_unstale":"2024-05-08T04:42:55.544048+0000","last_undegraded":"2024-05-08T04:42:55.544048+0000","last_fullsized":"2024-05-08T04:42:55.544048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:02.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234541+0000","last_change":"2024-05-08T04:42:52.234541+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234541+0000","last_undegraded":"2024-05-08T04:42:52.234541+0000","last_fullsized":"2024-05-08T04:42:52.234541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644157+0000","last_change":"2024-05-08T04:42:54.270324+0000","last_active":"2024-05-08T04:42:55.644157+0000","last_peered":"2024-05-08T04:42:55.644157+0000","last_clean":"2024-05-08T04:42:55.644157+0000","last_became_active":"2024-05-08T04:42:54.269819+0000","last_became_peered":"2024-05-08T04:42:54.269819+0000","last_unstale":"2024-05-08T04:42:55.644157+0000","last_undegraded":"2024-05-08T04:42:55.644157+0000","last_fullsized":"2024-05-08T04:42:55.644157+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:09.118707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259307+0000","last_change":"2024-05-08T04:42:53.257209+0000","last_active":"2024-05-08T04:42:55.259307+0000","last_peered":"2024-05-08T04:42:55.259307+0000","last_clean":"2024-05-08T04:42:55.259307+0000","last_became_active":"2024-05-08T04:42:53.256908+0000","last_became_peered":"2024-05-08T04:42:53.256908+0000","last_unstale":"2024-05-08T04:42:55.259307+0000","last_undegraded":"2024-05-08T04:42:55.259307+0000","last_fullsized":"2024-05-08T04:42:55.259307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259796+0000","last_change":"2024-05-08T04:42:54.263171+0000","last_active":"2024-05-08T04:42:55.259796+0000","last_peered":"2024-05-08T04:42:55.259796+0000","last_clean":"2024-05-08T04:42:55.259796+0000","last_became_active":"2024-05-08T04:42:54.262760+0000","last_became_peered":"2024-05-08T04:42:54.262760+0000","last_unstale":"2024-05-08T04:42:55.259796+0000","last_undegraded":"2024-05-08T04:42:55.259796+0000","last_fullsized":"2024-05-08T04:42:55.259796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.236833+0000","last_change":"2024-05-08T04:42:52.236833+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.236833+0000","last_undegraded":"2024-05-08T04:42:52.236833+0000","last_fullsized":"2024-05-08T04:42:52.236833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643515+0000","last_change":"2024-05-08T04:42:54.274640+0000","last_active":"2024-05-08T04:42:55.643515+0000","last_peered":"2024-05-08T04:42:55.643515+0000","last_clean":"2024-05-08T04:42:55.643515+0000","last_became_active":"2024-05-08T04:42:54.274209+0000","last_became_peered":"2024-05-08T04:42:54.274209+0000","last_unstale":"2024-05-08T04:42:55.643515+0000","last_undegraded":"2024-05-08T04:42:55.643515+0000","last_fullsized":"2024-05-08T04:42:55.643515+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:36.694738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259005+0000","last_change":"2024-05-08T04:42:53.265323+0000","last_active":"2024-05-08T04:42:55.259005+0000","last_peered":"2024-05-08T04:42:55.259005+0000","last_clean":"2024-05-08T04:42:55.259005+0000","last_became_active":"2024-05-08T04:42:53.265031+0000","last_became_peered":"2024-05-08T04:42:53.265031+0000","last_unstale":"2024-05-08T04:42:55.259005+0000","last_undegraded":"2024-05-08T04:42:55.259005+0000","last_fullsized":"2024-05-08T04:42:55.259005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:19.518749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258582+0000","last_change":"2024-05-08T04:42:54.263167+0000","last_active":"2024-05-08T04:42:55.258582+0000","last_peered":"2024-05-08T04:42:55.258582+0000","last_clean":"2024-05-08T04:42:55.258582+0000","last_became_active":"2024-05-08T04:42:54.262842+0000","last_became_peered":"2024-05-08T04:42:54.262842+0000","last_unstale":"2024-05-08T04:42:55.258582+0000","last_undegraded":"2024-05-08T04:42:55.258582+0000","last_fullsized":"2024-05-08T04:42:55.258582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643593+0000","last_change":"2024-05-08T04:42:53.263261+0000","last_active":"2024-05-08T04:42:55.643593+0000","last_peered":"2024-05-08T04:42:55.643593+0000","last_clean":"2024-05-08T04:42:55.643593+0000","last_became_active":"2024-05-08T04:42:53.262219+0000","last_became_peered":"2024-05-08T04:42:53.262219+0000","last_unstale":"2024-05-08T04:42:55.643593+0000","last_undegraded":"2024-05-08T04:42:55.643593+0000","last_fullsized":"2024-05-08T04:42:55.643593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:20.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260268+0000","last_change":"2024-05-08T04:42:54.269625+0000","last_active":"2024-05-08T04:42:55.260268+0000","last_peered":"2024-05-08T04:42:55.260268+0000","last_clean":"2024-05-08T04:42:55.260268+0000","last_became_active":"2024-05-08T04:42:54.269308+0000","last_became_peered":"2024-05-08T04:42:54.269308+0000","last_unstale":"2024-05-08T04:42:55.260268+0000","last_undegraded":"2024-05-08T04:42:55.260268+0000","last_fullsized":"2024-05-08T04:42:55.260268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.615548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234489+0000","last_change":"2024-05-08T04:42:52.234489+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234489+0000","last_undegraded":"2024-05-08T04:42:52.234489+0000","last_fullsized":"2024-05-08T04:42:52.234489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261712+0000","last_change":"2024-05-08T04:42:54.284605+0000","last_active":"2024-05-08T04:42:55.261712+0000","last_peered":"2024-05-08T04:42:55.261712+0000","last_clean":"2024-05-08T04:42:55.261712+0000","last_became_active":"2024-05-08T04:42:54.284331+0000","last_became_peered":"2024-05-08T04:42:54.284331+0000","last_unstale":"2024-05-08T04:42:55.261712+0000","last_undegraded":"2024-05-08T04:42:55.261712+0000","last_fullsized":"2024-05-08T04:42:55.261712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237857+0000","last_change":"2024-05-08T04:42:52.237857+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237857+0000","last_undegraded":"2024-05-08T04:42:52.237857+0000","last_fullsized":"2024-05-08T04:42:52.237857+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644694+0000","last_change":"2024-05-08T04:42:53.257288+0000","last_active":"2024-05-08T04:42:55.644694+0000","last_peered":"2024-05-08T04:42:55.644694+0000","last_clean":"2024-05-08T04:42:55.644694+0000","last_became_active":"2024-05-08T04:42:53.257014+0000","last_became_peered":"2024-05-08T04:42:53.257014+0000","last_unstale":"2024-05-08T04:42:55.644694+0000","last_undegraded":"2024-05-08T04:42:55.644694+0000","last_fullsized":"2024-05-08T04:42:55.644694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:27.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259886+0000","last_change":"2024-05-08T04:42:54.259828+0000","last_active":"2024-05-08T04:42:55.259886+0000","last_peered":"2024-05-08T04:42:55.259886+0000","last_clean":"2024-05-08T04:42:55.259886+0000","last_became_active":"2024-05-08T04:42:54.259109+0000","last_became_peered":"2024-05-08T04:42:54.259109+0000","last_unstale":"2024-05-08T04:42:55.259886+0000","last_undegraded":"2024-05-08T04:42:55.259886+0000","last_fullsized":"2024-05-08T04:42:55.259886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644648+0000","last_change":"2024-05-08T04:42:53.264438+0000","last_active":"2024-05-08T04:42:55.644648+0000","last_peered":"2024-05-08T04:42:55.644648+0000","last_clean":"2024-05-08T04:42:55.644648+0000","last_became_active":"2024-05-08T04:42:53.264131+0000","last_became_peered":"2024-05-08T04:42:53.264131+0000","last_unstale":"2024-05-08T04:42:55.644648+0000","last_undegraded":"2024-05-08T04:42:55.644648+0000","last_fullsized":"2024-05-08T04:42:55.644648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:15.699103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644435+0000","last_change":"2024-05-08T04:42:53.255665+0000","last_active":"2024-05-08T04:42:55.644435+0000","last_peered":"2024-05-08T04:42:55.644435+0000","last_clean":"2024-05-08T04:42:55.644435+0000","last_became_active":"2024-05-08T04:42:53.255168+0000","last_became_peered":"2024-05-08T04:42:53.255168+0000","last_unstale":"2024-05-08T04:42:55.644435+0000","last_undegraded":"2024-05-08T04:42:55.644435+0000","last_fullsized":"2024-05-08T04:42:55.644435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261312+0000","last_change":"2024-05-08T04:42:53.259031+0000","last_active":"2024-05-08T04:42:55.261312+0000","last_peered":"2024-05-08T04:42:55.261312+0000","last_clean":"2024-05-08T04:42:55.261312+0000","last_became_active":"2024-05-08T04:42:53.258734+0000","last_became_peered":"2024-05-08T04:42:53.258734+0000","last_unstale":"2024-05-08T04:42:55.261312+0000","last_undegraded":"2024-05-08T04:42:55.261312+0000","last_fullsized":"2024-05-08T04:42:55.261312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:07.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233237+0000","last_change":"2024-05-08T04:42:52.233237+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233237+0000","last_undegraded":"2024-05-08T04:42:52.233237+0000","last_fullsized":"2024-05-08T04:42:52.233237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645310+0000","last_change":"2024-05-08T04:42:53.255078+0000","last_active":"2024-05-08T04:42:55.645310+0000","last_peered":"2024-05-08T04:42:55.645310+0000","last_clean":"2024-05-08T04:42:55.645310+0000","last_became_active":"2024-05-08T04:42:53.254350+0000","last_became_peered":"2024-05-08T04:42:53.254350+0000","last_unstale":"2024-05-08T04:42:55.645310+0000","last_undegraded":"2024-05-08T04:42:55.645310+0000","last_fullsized":"2024-05-08T04:42:55.645310+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:59.504019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645155+0000","last_change":"2024-05-08T04:42:54.284624+0000","last_active":"2024-05-08T04:42:55.645155+0000","last_peered":"2024-05-08T04:42:55.645155+0000","last_clean":"2024-05-08T04:42:55.645155+0000","last_became_active":"2024-05-08T04:42:54.284368+0000","last_became_peered":"2024-05-08T04:42:54.284368+0000","last_unstale":"2024-05-08T04:42:55.645155+0000","last_undegraded":"2024-05-08T04:42:55.645155+0000","last_fullsized":"2024-05-08T04:42:55.645155+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233255+0000","last_change":"2024-05-08T04:42:52.233255+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233255+0000","last_undegraded":"2024-05-08T04:42:52.233255+0000","last_fullsized":"2024-05-08T04:42:52.233255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543959+0000","last_change":"2024-05-08T04:42:54.268528+0000","last_active":"2024-05-08T04:42:55.543959+0000","last_peered":"2024-05-08T04:42:55.543959+0000","last_clean":"2024-05-08T04:42:55.543959+0000","last_became_active":"2024-05-08T04:42:54.268247+0000","last_became_peered":"2024-05-08T04:42:54.268247+0000","last_unstale":"2024-05-08T04:42:55.543959+0000","last_undegraded":"2024-05-08T04:42:55.543959+0000","last_fullsized":"2024-05-08T04:42:55.543959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:20.235769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261410+0000","last_change":"2024-05-08T04:42:53.260958+0000","last_active":"2024-05-08T04:42:55.261410+0000","last_peered":"2024-05-08T04:42:55.261410+0000","last_clean":"2024-05-08T04:42:55.261410+0000","last_became_active":"2024-05-08T04:42:53.260621+0000","last_became_peered":"2024-05-08T04:42:53.260621+0000","last_unstale":"2024-05-08T04:42:55.261410+0000","last_undegraded":"2024-05-08T04:42:55.261410+0000","last_fullsized":"2024-05-08T04:42:55.261410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:14.035376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544039+0000","last_change":"2024-05-08T04:42:54.259461+0000","last_active":"2024-05-08T04:42:55.544039+0000","last_peered":"2024-05-08T04:42:55.544039+0000","last_clean":"2024-05-08T04:42:55.544039+0000","last_became_active":"2024-05-08T04:42:54.259005+0000","last_became_peered":"2024-05-08T04:42:54.259005+0000","last_unstale":"2024-05-08T04:42:55.544039+0000","last_undegraded":"2024-05-08T04:42:55.544039+0000","last_fullsized":"2024-05-08T04:42:55.544039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:07.448395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258867+0000","last_change":"2024-05-08T04:42:53.258018+0000","last_active":"2024-05-08T04:42:55.258867+0000","last_peered":"2024-05-08T04:42:55.258867+0000","last_clean":"2024-05-08T04:42:55.258867+0000","last_became_active":"2024-05-08T04:42:53.257357+0000","last_became_peered":"2024-05-08T04:42:53.257357+0000","last_unstale":"2024-05-08T04:42:55.258867+0000","last_undegraded":"2024-05-08T04:42:55.258867+0000","last_fullsized":"2024-05-08T04:42:55.258867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543555+0000","last_change":"2024-05-08T04:42:54.258649+0000","last_active":"2024-05-08T04:42:55.543555+0000","last_peered":"2024-05-08T04:42:55.543555+0000","last_clean":"2024-05-08T04:42:55.543555+0000","last_became_active":"2024-05-08T04:42:54.258367+0000","last_became_peered":"2024-05-08T04:42:54.258367+0000","last_unstale":"2024-05-08T04:42:55.543555+0000","last_undegraded":"2024-05-08T04:42:55.543555+0000","last_fullsized":"2024-05-08T04:42:55.543555+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644524+0000","last_change":"2024-05-08T04:42:53.260251+0000","last_active":"2024-05-08T04:42:55.644524+0000","last_peered":"2024-05-08T04:42:55.644524+0000","last_clean":"2024-05-08T04:42:55.644524+0000","last_became_active":"2024-05-08T04:42:53.259749+0000","last_became_peered":"2024-05-08T04:42:53.259749+0000","last_unstale":"2024-05-08T04:42:55.644524+0000","last_undegraded":"2024-05-08T04:42:55.644524+0000","last_fullsized":"2024-05-08T04:42:55.644524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644062+0000","last_change":"2024-05-08T04:42:54.270229+0000","last_active":"2024-05-08T04:42:55.644062+0000","last_peered":"2024-05-08T04:42:55.644062+0000","last_clean":"2024-05-08T04:42:55.644062+0000","last_became_active":"2024-05-08T04:42:54.267191+0000","last_became_peered":"2024-05-08T04:42:54.267191+0000","last_unstale":"2024-05-08T04:42:55.644062+0000","last_undegraded":"2024-05-08T04:42:55.644062+0000","last_fullsized":"2024-05-08T04:42:55.644062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:16.097496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234678+0000","last_change":"2024-05-08T04:42:52.234678+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234678+0000","last_undegraded":"2024-05-08T04:42:52.234678+0000","last_fullsized":"2024-05-08T04:42:52.234678+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543619+0000","last_change":"2024-05-08T04:42:54.270286+0000","last_active":"2024-05-08T04:42:55.543619+0000","last_peered":"2024-05-08T04:42:55.543619+0000","last_clean":"2024-05-08T04:42:55.543619+0000","last_became_active":"2024-05-08T04:42:54.270019+0000","last_became_peered":"2024-05-08T04:42:54.270019+0000","last_unstale":"2024-05-08T04:42:55.543619+0000","last_undegraded":"2024-05-08T04:42:55.543619+0000","last_fullsized":"2024-05-08T04:42:55.543619+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644073+0000","last_change":"2024-05-08T04:42:53.259343+0000","last_active":"2024-05-08T04:42:55.644073+0000","last_peered":"2024-05-08T04:42:55.644073+0000","last_clean":"2024-05-08T04:42:55.644073+0000","last_became_active":"2024-05-08T04:42:53.258967+0000","last_became_peered":"2024-05-08T04:42:53.258967+0000","last_unstale":"2024-05-08T04:42:55.644073+0000","last_undegraded":"2024-05-08T04:42:55.644073+0000","last_fullsized":"2024-05-08T04:42:55.644073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643660+0000","last_change":"2024-05-08T04:42:53.253264+0000","last_active":"2024-05-08T04:42:55.643660+0000","last_peered":"2024-05-08T04:42:55.643660+0000","last_clean":"2024-05-08T04:42:55.643660+0000","last_became_active":"2024-05-08T04:42:53.252716+0000","last_became_peered":"2024-05-08T04:42:53.252716+0000","last_unstale":"2024-05-08T04:42:55.643660+0000","last_undegraded":"2024-05-08T04:42:55.643660+0000","last_fullsized":"2024-05-08T04:42:55.643660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:56.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275320+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.275003+0000","last_became_peered":"2024-05-08T04:42:54.275003+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237599+0000","last_change":"2024-05-08T04:42:52.237599+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237599+0000","last_undegraded":"2024-05-08T04:42:52.237599+0000","last_fullsized":"2024-05-08T04:42:52.237599+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645304+0000","last_change":"2024-05-08T04:42:54.282790+0000","last_active":"2024-05-08T04:42:55.645304+0000","last_peered":"2024-05-08T04:42:55.645304+0000","last_clean":"2024-05-08T04:42:55.645304+0000","last_became_active":"2024-05-08T04:42:54.282523+0000","last_became_peered":"2024-05-08T04:42:54.282523+0000","last_unstale":"2024-05-08T04:42:55.645304+0000","last_undegraded":"2024-05-08T04:42:55.645304+0000","last_fullsized":"2024-05-08T04:42:55.645304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.467995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261546+0000","last_change":"2024-05-08T04:42:53.252707+0000","last_active":"2024-05-08T04:42:55.261546+0000","last_peered":"2024-05-08T04:42:55.261546+0000","last_clean":"2024-05-08T04:42:55.261546+0000","last_became_active":"2024-05-08T04:42:53.250660+0000","last_became_peered":"2024-05-08T04:42:53.250660+0000","last_unstale":"2024-05-08T04:42:55.261546+0000","last_undegraded":"2024-05-08T04:42:55.261546+0000","last_fullsized":"2024-05-08T04:42:55.261546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:13.437315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644517+0000","last_change":"2024-05-08T04:42:54.277076+0000","last_active":"2024-05-08T04:42:55.644517+0000","last_peered":"2024-05-08T04:42:55.644517+0000","last_clean":"2024-05-08T04:42:55.644517+0000","last_became_active":"2024-05-08T04:42:54.276805+0000","last_became_peered":"2024-05-08T04:42:54.276805+0000","last_unstale":"2024-05-08T04:42:55.644517+0000","last_undegraded":"2024-05-08T04:42:55.644517+0000","last_fullsized":"2024-05-08T04:42:55.644517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:39.976473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644732+0000","last_change":"2024-05-08T04:42:53.252678+0000","last_active":"2024-05-08T04:42:55.644732+0000","last_peered":"2024-05-08T04:42:55.644732+0000","last_clean":"2024-05-08T04:42:55.644732+0000","last_became_active":"2024-05-08T04:42:53.252426+0000","last_became_peered":"2024-05-08T04:42:53.252426+0000","last_unstale":"2024-05-08T04:42:55.644732+0000","last_undegraded":"2024-05-08T04:42:55.644732+0000","last_fullsized":"2024-05-08T04:42:55.644732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:50.935297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275599+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.274979+0000","last_became_peered":"2024-05-08T04:42:54.274979+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:27.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237194+0000","last_change":"2024-05-08T04:42:52.237194+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237194+0000","last_undegraded":"2024-05-08T04:42:52.237194+0000","last_fullsized":"2024-05-08T04:42:52.237194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259403+0000","last_change":"2024-05-08T04:42:54.250195+0000","last_active":"2024-05-08T04:42:55.259403+0000","last_peered":"2024-05-08T04:42:55.259403+0000","last_clean":"2024-05-08T04:42:55.259403+0000","last_became_active":"2024-05-08T04:42:54.249297+0000","last_became_peered":"2024-05-08T04:42:54.249297+0000","last_unstale":"2024-05-08T04:42:55.259403+0000","last_undegraded":"2024-05-08T04:42:55.259403+0000","last_fullsized":"2024-05-08T04:42:55.259403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259421+0000","last_change":"2024-05-08T04:42:53.259450+0000","last_active":"2024-05-08T04:42:55.259421+0000","last_peered":"2024-05-08T04:42:55.259421+0000","last_clean":"2024-05-08T04:42:55.259421+0000","last_became_active":"2024-05-08T04:42:53.259139+0000","last_became_peered":"2024-05-08T04:42:53.259139+0000","last_unstale":"2024-05-08T04:42:55.259421+0000","last_undegraded":"2024-05-08T04:42:55.259421+0000","last_fullsized":"2024-05-08T04:42:55.259421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260413+0000","last_change":"2024-05-08T04:42:54.263700+0000","last_active":"2024-05-08T04:42:55.260413+0000","last_peered":"2024-05-08T04:42:55.260413+0000","last_clean":"2024-05-08T04:42:55.260413+0000","last_became_active":"2024-05-08T04:42:54.263395+0000","last_became_peered":"2024-05-08T04:42:54.263395+0000","last_unstale":"2024-05-08T04:42:55.260413+0000","last_undegraded":"2024-05-08T04:42:55.260413+0000","last_fullsized":"2024-05-08T04:42:55.260413+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:40.094631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259409+0000","last_change":"2024-05-08T04:42:53.258649+0000","last_active":"2024-05-08T04:42:55.259409+0000","last_peered":"2024-05-08T04:42:55.259409+0000","last_clean":"2024-05-08T04:42:55.259409+0000","last_became_active":"2024-05-08T04:42:53.258348+0000","last_became_peered":"2024-05-08T04:42:53.258348+0000","last_unstale":"2024-05-08T04:42:55.259409+0000","last_undegraded":"2024-05-08T04:42:55.259409+0000","last_fullsized":"2024-05-08T04:42:55.259409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:42.312320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543824+0000","last_change":"2024-05-08T04:42:54.266324+0000","last_active":"2024-05-08T04:42:55.543824+0000","last_peered":"2024-05-08T04:42:55.543824+0000","last_clean":"2024-05-08T04:42:55.543824+0000","last_became_active":"2024-05-08T04:42:54.265475+0000","last_became_peered":"2024-05-08T04:42:54.265475+0000","last_unstale":"2024-05-08T04:42:55.543824+0000","last_undegraded":"2024-05-08T04:42:55.543824+0000","last_fullsized":"2024-05-08T04:42:55.543824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233276+0000","last_change":"2024-05-08T04:42:52.233276+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233276+0000","last_undegraded":"2024-05-08T04:42:52.233276+0000","last_fullsized":"2024-05-08T04:42:52.233276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643920+0000","last_change":"2024-05-08T04:42:54.270703+0000","last_active":"2024-05-08T04:42:55.643920+0000","last_peered":"2024-05-08T04:42:55.643920+0000","last_clean":"2024-05-08T04:42:55.643920+0000","last_became_active":"2024-05-08T04:42:54.265604+0000","last_became_peered":"2024-05-08T04:42:54.265604+0000","last_unstale":"2024-05-08T04:42:55.643920+0000","last_undegraded":"2024-05-08T04:42:55.643920+0000","last_fullsized":"2024-05-08T04:42:55.643920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:21.915088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643694+0000","last_change":"2024-05-08T04:42:53.253094+0000","last_active":"2024-05-08T04:42:55.643694+0000","last_peered":"2024-05-08T04:42:55.643694+0000","last_clean":"2024-05-08T04:42:55.643694+0000","last_became_active":"2024-05-08T04:42:53.252291+0000","last_became_peered":"2024-05-08T04:42:53.252291+0000","last_unstale":"2024-05-08T04:42:55.643694+0000","last_undegraded":"2024-05-08T04:42:55.643694+0000","last_fullsized":"2024-05-08T04:42:55.643694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:59.366123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258784+0000","last_change":"2024-05-08T04:42:53.257287+0000","last_active":"2024-05-08T04:42:55.258784+0000","last_peered":"2024-05-08T04:42:55.258784+0000","last_clean":"2024-05-08T04:42:55.258784+0000","last_became_active":"2024-05-08T04:42:53.256851+0000","last_became_peered":"2024-05-08T04:42:53.256851+0000","last_unstale":"2024-05-08T04:42:55.258784+0000","last_undegraded":"2024-05-08T04:42:55.258784+0000","last_fullsized":"2024-05-08T04:42:55.258784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543931+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:55.543931+0000","last_peered":"2024-05-08T04:42:55.543931+0000","last_clean":"2024-05-08T04:42:55.543931+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:55.543931+0000","last_undegraded":"2024-05-08T04:42:55.543931+0000","last_fullsized":"2024-05-08T04:42:55.543931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":1198126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":607212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":607746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":587540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":607266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:42:58.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:58.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 Processor -- start 2024-05-08T04:42:58.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 -- start start 2024-05-08T04:42:58.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f4415ba70 0x7f2f44160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f4415be50 0x7f2f440cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f440cd9f0 con 0x7f2f440aeda0 2024-05-08T04:42:58.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4415fe70 con 0x7f2f4415be50 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f49800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440cdcd0 con 0x7f2f4415ba70 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f43400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2f440aeda0 0x7f2f440b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46408/0 (socket says 172.21.15.164:46408) 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f43400640 1 -- 172.21.15.164:0/2553451742 learned_addr learned my addr 172.21.15.164:0/2553451742 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42000640 1 --2- 172.21.15.164:0/2553451742 >> v2:172.21.15.164:3300/0 conn(0x7f2f4415be50 0x7f2f440cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42a00640 1 --2- 172.21.15.164:0/2553451742 >> v2:172.21.15.172:3300/0 conn(0x7f2f4415ba70 0x7f2f44160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f41600640 1 -- 172.21.15.164:0/2553451742 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867653984 0 0) 0x7f2f440cd9f0 con 0x7f2f440aeda0 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f41600640 1 -- 172.21.15.164:0/2553451742 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f28003530 con 0x7f2f440aeda0 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42000640 1 -- 172.21.15.164:0/2553451742 >> v2:172.21.15.172:3300/0 conn(0x7f2f4415ba70 msgr2=0x7f2f44160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42000640 1 --2- 172.21.15.164:0/2553451742 >> v2:172.21.15.172:3300/0 conn(0x7f2f4415ba70 0x7f2f44160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42000640 1 -- 172.21.15.164:0/2553451742 >> v1:172.21.15.164:6789/0 conn(0x7f2f440aeda0 legacy=0x7f2f440b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.503+0000 7f2f42000640 1 -- 172.21.15.164:0/2553451742 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4415f820 con 0x7f2f4415be50 2024-05-08T04:42:58.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f42000640 1 --2- 172.21.15.164:0/2553451742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415be50 0x7f2f440cf690 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406bc70 tx=0x7f2f3001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e5b1ce00622495 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f41600640 1 -- 172.21.15.164:0/2553451742 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300b2070 con 0x7f2f4415be50 2024-05-08T04:42:58.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f41600640 1 -- 172.21.15.164:0/2553451742 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f300a9070 con 0x7f2f4415be50 2024-05-08T04:42:58.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f41600640 1 -- 172.21.15.164:0/2553451742 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300ad880 con 0x7f2f4415be50 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 -- 172.21.15.164:0/2553451742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415be50 msgr2=0x7f2f440cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 --2- 172.21.15.164:0/2553451742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415be50 0x7f2f440cf690 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406bc70 tx=0x7f2f3001c450 comp rx=0 tx=0).stop 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 -- 172.21.15.164:0/2553451742 shutdown_connections 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 --2- 172.21.15.164:0/2553451742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415be50 0x7f2f440cf690 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 --2- 172.21.15.164:0/2553451742 >> v2:172.21.15.172:3300/0 conn(0x7f2f4415ba70 0x7f2f44160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 -- 172.21.15.164:0/2553451742 >> 172.21.15.164:0/2553451742 conn(0x7f2f4406ac90 msgr2=0x7f2f4406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 -- 172.21.15.164:0/2553451742 shutdown_connections 2024-05-08T04:42:58.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.507+0000 7f2f49800640 1 -- 172.21.15.164:0/2553451742 wait complete. 2024-05-08T04:42:58.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 Processor -- start 2024-05-08T04:42:58.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- start start 2024-05-08T04:42:58.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 0x7f2f4414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 0x7f2f4414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44168d40 con 0x7f2f4415ba70 2024-05-08T04:42:58.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44168bc0 con 0x7f2f4415be50 2024-05-08T04:42:58.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44168ec0 con 0x7f2f440aeda0 2024-05-08T04:42:58.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56784/0 (socket says 172.21.15.164:56784) 2024-05-08T04:42:58.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 -- 172.21.15.164:0/287105049 learned_addr learned my addr 172.21.15.164:0/287105049 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f43400640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 0x7f2f4414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42a00640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 0x7f2f4414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 -- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 msgr2=0x7f2f4414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 0x7f2f4414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 -- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 msgr2=0x7f2f4414db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 0x7f2f4414db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 -- 172.21.15.164:0/287105049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4414e2e0 con 0x7f2f4415ba70 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f43400640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 0x7f2f4414db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42a00640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 0x7f2f4414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f42000640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2f300b0040 tx=0x7f2f300191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300b2040 con 0x7f2f4415ba70 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f300bc070 con 0x7f2f4415ba70 2024-05-08T04:42:58.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4414e5c0 con 0x7f2f4415ba70 2024-05-08T04:42:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300b6410 con 0x7f2f4415ba70 2024-05-08T04:42:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4414eb80 con 0x7f2f4415ba70 2024-05-08T04:42:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.511+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f44069d60 con 0x7f2f4415ba70 2024-05-08T04:42:58.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.515+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f2f300ab070 con 0x7f2f4415ba70 2024-05-08T04:42:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.515+0000 7f2f3aa00640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 0x7f2f24069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.515+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2f3011d390 con 0x7f2f4415ba70 2024-05-08T04:42:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.515+0000 7f2f42a00640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 0x7f2f24069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.515+0000 7f2f42a00640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 0x7f2f24069d20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009ae0 tx=0x7f2f34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.519+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f30016070 con 0x7f2f4415ba70 2024-05-08T04:42:58.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.667+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f4406e210 con 0x7f2f24067860 2024-05-08T04:42:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.675+0000 7f2f3aa00640 1 -- 172.21.15.164:0/287105049 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249796 (secure 0 0 0) 0x7f2f4406e210 con 0x7f2f24067860 2024-05-08T04:42:58.679 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 msgr2=0x7f2f24069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 0x7f2f24069d20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009ae0 tx=0x7f2f34008040 comp rx=0 tx=0).stop 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 msgr2=0x7f2f44070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2f300b0040 tx=0x7f2f300191f0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 shutdown_connections 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f24067860 0x7f2f24069d20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4415be50 0x7f2f4414db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f4415ba70 0x7f2f44070f20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 --2- 172.21.15.164:0/287105049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f440aeda0 0x7f2f4414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 >> 172.21.15.164:0/287105049 conn(0x7f2f4406ac90 msgr2=0x7f2f44158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 shutdown_connections 2024-05-08T04:42:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.679+0000 7f2f49800640 1 -- 172.21.15.164:0/287105049 wait complete. 2024-05-08T04:42:58.699 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-05-08T04:42:57.484254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261468,"kb_used_data":12004,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713252,"statfs":{"total":773094113280,"available":772826370048,"internally_reserved":0,"allocated":12292096,"data_stored":5995905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.254344"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645115+0000","last_change":"2024-05-08T04:42:54.284520+0000","last_active":"2024-05-08T04:42:55.645115+0000","last_peered":"2024-05-08T04:42:55.645115+0000","last_clean":"2024-05-08T04:42:55.645115+0000","last_became_active":"2024-05-08T04:42:54.283787+0000","last_became_peered":"2024-05-08T04:42:54.283787+0000","last_unstale":"2024-05-08T04:42:55.645115+0000","last_undegraded":"2024-05-08T04:42:55.645115+0000","last_fullsized":"2024-05-08T04:42:55.645115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:31.361703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260638+0000","last_change":"2024-05-08T04:42:54.267059+0000","last_active":"2024-05-08T04:42:55.260638+0000","last_peered":"2024-05-08T04:42:55.260638+0000","last_clean":"2024-05-08T04:42:55.260638+0000","last_became_active":"2024-05-08T04:42:54.266748+0000","last_became_peered":"2024-05-08T04:42:54.266748+0000","last_unstale":"2024-05-08T04:42:55.260638+0000","last_undegraded":"2024-05-08T04:42:55.260638+0000","last_fullsized":"2024-05-08T04:42:55.260638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:57.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543833+0000","last_change":"2024-05-08T04:42:54.259254+0000","last_active":"2024-05-08T04:42:55.543833+0000","last_peered":"2024-05-08T04:42:55.543833+0000","last_clean":"2024-05-08T04:42:55.543833+0000","last_became_active":"2024-05-08T04:42:54.258620+0000","last_became_peered":"2024-05-08T04:42:54.258620+0000","last_unstale":"2024-05-08T04:42:55.543833+0000","last_undegraded":"2024-05-08T04:42:55.543833+0000","last_fullsized":"2024-05-08T04:42:55.543833+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644261+0000","last_change":"2024-05-08T04:42:54.276298+0000","last_active":"2024-05-08T04:42:55.644261+0000","last_peered":"2024-05-08T04:42:55.644261+0000","last_clean":"2024-05-08T04:42:55.644261+0000","last_became_active":"2024-05-08T04:42:54.273697+0000","last_became_peered":"2024-05-08T04:42:54.273697+0000","last_unstale":"2024-05-08T04:42:55.644261+0000","last_undegraded":"2024-05-08T04:42:55.644261+0000","last_fullsized":"2024-05-08T04:42:55.644261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:45.521599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644461+0000","last_change":"2024-05-08T04:42:54.259042+0000","last_active":"2024-05-08T04:42:55.644461+0000","last_peered":"2024-05-08T04:42:55.644461+0000","last_clean":"2024-05-08T04:42:55.644461+0000","last_became_active":"2024-05-08T04:42:54.258775+0000","last_became_peered":"2024-05-08T04:42:54.258775+0000","last_unstale":"2024-05-08T04:42:55.644461+0000","last_undegraded":"2024-05-08T04:42:55.644461+0000","last_fullsized":"2024-05-08T04:42:55.644461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:58.137321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258732+0000","last_change":"2024-05-08T04:42:54.258159+0000","last_active":"2024-05-08T04:42:55.258732+0000","last_peered":"2024-05-08T04:42:55.258732+0000","last_clean":"2024-05-08T04:42:55.258732+0000","last_became_active":"2024-05-08T04:42:54.257892+0000","last_became_peered":"2024-05-08T04:42:54.257892+0000","last_unstale":"2024-05-08T04:42:55.258732+0000","last_undegraded":"2024-05-08T04:42:55.258732+0000","last_fullsized":"2024-05-08T04:42:55.258732+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543525+0000","last_change":"2024-05-08T04:42:54.261073+0000","last_active":"2024-05-08T04:42:55.543525+0000","last_peered":"2024-05-08T04:42:55.543525+0000","last_clean":"2024-05-08T04:42:55.543525+0000","last_became_active":"2024-05-08T04:42:54.260783+0000","last_became_peered":"2024-05-08T04:42:54.260783+0000","last_unstale":"2024-05-08T04:42:55.543525+0000","last_undegraded":"2024-05-08T04:42:55.543525+0000","last_fullsized":"2024-05-08T04:42:55.543525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645257+0000","last_change":"2024-05-08T04:42:54.280775+0000","last_active":"2024-05-08T04:42:55.645257+0000","last_peered":"2024-05-08T04:42:55.645257+0000","last_clean":"2024-05-08T04:42:55.645257+0000","last_became_active":"2024-05-08T04:42:54.280501+0000","last_became_peered":"2024-05-08T04:42:54.280501+0000","last_unstale":"2024-05-08T04:42:55.645257+0000","last_undegraded":"2024-05-08T04:42:55.645257+0000","last_fullsized":"2024-05-08T04:42:55.645257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:25.588352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645180+0000","last_change":"2024-05-08T04:42:54.279039+0000","last_active":"2024-05-08T04:42:55.645180+0000","last_peered":"2024-05-08T04:42:55.645180+0000","last_clean":"2024-05-08T04:42:55.645180+0000","last_became_active":"2024-05-08T04:42:54.278516+0000","last_became_peered":"2024-05-08T04:42:54.278516+0000","last_unstale":"2024-05-08T04:42:55.645180+0000","last_undegraded":"2024-05-08T04:42:55.645180+0000","last_fullsized":"2024-05-08T04:42:55.645180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:44.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260298+0000","last_change":"2024-05-08T04:42:54.264244+0000","last_active":"2024-05-08T04:42:55.260298+0000","last_peered":"2024-05-08T04:42:55.260298+0000","last_clean":"2024-05-08T04:42:55.260298+0000","last_became_active":"2024-05-08T04:42:54.261285+0000","last_became_peered":"2024-05-08T04:42:54.261285+0000","last_unstale":"2024-05-08T04:42:55.260298+0000","last_undegraded":"2024-05-08T04:42:55.260298+0000","last_fullsized":"2024-05-08T04:42:55.260298+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:16.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644393+0000","last_change":"2024-05-08T04:42:54.278540+0000","last_active":"2024-05-08T04:42:55.644393+0000","last_peered":"2024-05-08T04:42:55.644393+0000","last_clean":"2024-05-08T04:42:55.644393+0000","last_became_active":"2024-05-08T04:42:54.275676+0000","last_became_peered":"2024-05-08T04:42:54.275676+0000","last_unstale":"2024-05-08T04:42:55.644393+0000","last_undegraded":"2024-05-08T04:42:55.644393+0000","last_fullsized":"2024-05-08T04:42:55.644393+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:02.962699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543936+0000","last_change":"2024-05-08T04:42:54.262625+0000","last_active":"2024-05-08T04:42:55.543936+0000","last_peered":"2024-05-08T04:42:55.543936+0000","last_clean":"2024-05-08T04:42:55.543936+0000","last_became_active":"2024-05-08T04:42:54.262328+0000","last_became_peered":"2024-05-08T04:42:54.262328+0000","last_unstale":"2024-05-08T04:42:55.543936+0000","last_undegraded":"2024-05-08T04:42:55.543936+0000","last_fullsized":"2024-05-08T04:42:55.543936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:37.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543497+0000","last_change":"2024-05-08T04:42:54.263202+0000","last_active":"2024-05-08T04:42:55.543497+0000","last_peered":"2024-05-08T04:42:55.543497+0000","last_clean":"2024-05-08T04:42:55.543497+0000","last_became_active":"2024-05-08T04:42:54.262118+0000","last_became_peered":"2024-05-08T04:42:54.262118+0000","last_unstale":"2024-05-08T04:42:55.543497+0000","last_undegraded":"2024-05-08T04:42:55.543497+0000","last_fullsized":"2024-05-08T04:42:55.543497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643757+0000","last_change":"2024-05-08T04:42:54.258584+0000","last_active":"2024-05-08T04:42:55.643757+0000","last_peered":"2024-05-08T04:42:55.643757+0000","last_clean":"2024-05-08T04:42:55.643757+0000","last_became_active":"2024-05-08T04:42:54.257809+0000","last_became_peered":"2024-05-08T04:42:54.257809+0000","last_unstale":"2024-05-08T04:42:55.643757+0000","last_undegraded":"2024-05-08T04:42:55.643757+0000","last_fullsized":"2024-05-08T04:42:55.643757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.486563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643534+0000","last_change":"2024-05-08T04:42:54.272568+0000","last_active":"2024-05-08T04:42:55.643534+0000","last_peered":"2024-05-08T04:42:55.643534+0000","last_clean":"2024-05-08T04:42:55.643534+0000","last_became_active":"2024-05-08T04:42:54.266872+0000","last_became_peered":"2024-05-08T04:42:54.266872+0000","last_unstale":"2024-05-08T04:42:55.643534+0000","last_undegraded":"2024-05-08T04:42:55.643534+0000","last_fullsized":"2024-05-08T04:42:55.643534+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:19.666610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260434+0000","last_change":"2024-05-08T04:42:54.265221+0000","last_active":"2024-05-08T04:42:55.260434+0000","last_peered":"2024-05-08T04:42:55.260434+0000","last_clean":"2024-05-08T04:42:55.260434+0000","last_became_active":"2024-05-08T04:42:54.263935+0000","last_became_peered":"2024-05-08T04:42:54.263935+0000","last_unstale":"2024-05-08T04:42:55.260434+0000","last_undegraded":"2024-05-08T04:42:55.260434+0000","last_fullsized":"2024-05-08T04:42:55.260434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:24.087153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261472+0000","last_change":"2024-05-08T04:42:54.260979+0000","last_active":"2024-05-08T04:42:55.261472+0000","last_peered":"2024-05-08T04:42:55.261472+0000","last_clean":"2024-05-08T04:42:55.261472+0000","last_became_active":"2024-05-08T04:42:54.260386+0000","last_became_peered":"2024-05-08T04:42:54.260386+0000","last_unstale":"2024-05-08T04:42:55.261472+0000","last_undegraded":"2024-05-08T04:42:55.261472+0000","last_fullsized":"2024-05-08T04:42:55.261472+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:23.982806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543852+0000","last_change":"2024-05-08T04:42:54.267987+0000","last_active":"2024-05-08T04:42:55.543852+0000","last_peered":"2024-05-08T04:42:55.543852+0000","last_clean":"2024-05-08T04:42:55.543852+0000","last_became_active":"2024-05-08T04:42:54.267674+0000","last_became_peered":"2024-05-08T04:42:54.267674+0000","last_unstale":"2024-05-08T04:42:55.543852+0000","last_undegraded":"2024-05-08T04:42:55.543852+0000","last_fullsized":"2024-05-08T04:42:55.543852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:11.284074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259984+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.259984+0000","last_peered":"2024-05-08T04:42:55.259984+0000","last_clean":"2024-05-08T04:42:55.259984+0000","last_became_active":"2024-05-08T04:42:54.261903+0000","last_became_peered":"2024-05-08T04:42:54.261903+0000","last_unstale":"2024-05-08T04:42:55.259984+0000","last_undegraded":"2024-05-08T04:42:55.259984+0000","last_fullsized":"2024-05-08T04:42:55.259984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:49.660293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260474+0000","last_change":"2024-05-08T04:42:54.247853+0000","last_active":"2024-05-08T04:42:55.260474+0000","last_peered":"2024-05-08T04:42:55.260474+0000","last_clean":"2024-05-08T04:42:55.260474+0000","last_became_active":"2024-05-08T04:42:54.247314+0000","last_became_peered":"2024-05-08T04:42:54.247314+0000","last_unstale":"2024-05-08T04:42:55.260474+0000","last_undegraded":"2024-05-08T04:42:55.260474+0000","last_fullsized":"2024-05-08T04:42:55.260474+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.153233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258982+0000","last_change":"2024-05-08T04:42:54.257923+0000","last_active":"2024-05-08T04:42:55.258982+0000","last_peered":"2024-05-08T04:42:55.258982+0000","last_clean":"2024-05-08T04:42:55.258982+0000","last_became_active":"2024-05-08T04:42:54.257524+0000","last_became_peered":"2024-05-08T04:42:54.257524+0000","last_unstale":"2024-05-08T04:42:55.258982+0000","last_undegraded":"2024-05-08T04:42:55.258982+0000","last_fullsized":"2024-05-08T04:42:55.258982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260632+0000","last_change":"2024-05-08T04:42:54.264625+0000","last_active":"2024-05-08T04:42:55.260632+0000","last_peered":"2024-05-08T04:42:55.260632+0000","last_clean":"2024-05-08T04:42:55.260632+0000","last_became_active":"2024-05-08T04:42:54.261407+0000","last_became_peered":"2024-05-08T04:42:54.261407+0000","last_unstale":"2024-05-08T04:42:55.260632+0000","last_undegraded":"2024-05-08T04:42:55.260632+0000","last_fullsized":"2024-05-08T04:42:55.260632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:21.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644944+0000","last_change":"2024-05-08T04:42:54.273444+0000","last_active":"2024-05-08T04:42:55.644944+0000","last_peered":"2024-05-08T04:42:55.644944+0000","last_clean":"2024-05-08T04:42:55.644944+0000","last_became_active":"2024-05-08T04:42:54.273184+0000","last_became_peered":"2024-05-08T04:42:54.273184+0000","last_unstale":"2024-05-08T04:42:55.644944+0000","last_undegraded":"2024-05-08T04:42:55.644944+0000","last_fullsized":"2024-05-08T04:42:55.644944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:18.445404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645146+0000","last_change":"2024-05-08T04:42:54.277145+0000","last_active":"2024-05-08T04:42:55.645146+0000","last_peered":"2024-05-08T04:42:55.645146+0000","last_clean":"2024-05-08T04:42:55.645146+0000","last_became_active":"2024-05-08T04:42:54.274239+0000","last_became_peered":"2024-05-08T04:42:54.274239+0000","last_unstale":"2024-05-08T04:42:55.645146+0000","last_undegraded":"2024-05-08T04:42:55.645146+0000","last_fullsized":"2024-05-08T04:42:55.645146+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260283+0000","last_change":"2024-05-08T04:42:54.257830+0000","last_active":"2024-05-08T04:42:55.260283+0000","last_peered":"2024-05-08T04:42:55.260283+0000","last_clean":"2024-05-08T04:42:55.260283+0000","last_became_active":"2024-05-08T04:42:54.257455+0000","last_became_peered":"2024-05-08T04:42:54.257455+0000","last_unstale":"2024-05-08T04:42:55.260283+0000","last_undegraded":"2024-05-08T04:42:55.260283+0000","last_fullsized":"2024-05-08T04:42:55.260283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:08.717198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259104+0000","last_change":"2024-05-08T04:42:54.247280+0000","last_active":"2024-05-08T04:42:55.259104+0000","last_peered":"2024-05-08T04:42:55.259104+0000","last_clean":"2024-05-08T04:42:55.259104+0000","last_became_active":"2024-05-08T04:42:54.246609+0000","last_became_peered":"2024-05-08T04:42:54.246609+0000","last_unstale":"2024-05-08T04:42:55.259104+0000","last_undegraded":"2024-05-08T04:42:55.259104+0000","last_fullsized":"2024-05-08T04:42:55.259104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643671+0000","last_change":"2024-05-08T04:42:54.270435+0000","last_active":"2024-05-08T04:42:55.643671+0000","last_peered":"2024-05-08T04:42:55.643671+0000","last_clean":"2024-05-08T04:42:55.643671+0000","last_became_active":"2024-05-08T04:42:54.265729+0000","last_became_peered":"2024-05-08T04:42:54.265729+0000","last_unstale":"2024-05-08T04:42:55.643671+0000","last_undegraded":"2024-05-08T04:42:55.643671+0000","last_fullsized":"2024-05-08T04:42:55.643671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:53.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643688+0000","last_change":"2024-05-08T04:42:54.261182+0000","last_active":"2024-05-08T04:42:55.643688+0000","last_peered":"2024-05-08T04:42:55.643688+0000","last_clean":"2024-05-08T04:42:55.643688+0000","last_became_active":"2024-05-08T04:42:54.260895+0000","last_became_peered":"2024-05-08T04:42:54.260895+0000","last_unstale":"2024-05-08T04:42:55.643688+0000","last_undegraded":"2024-05-08T04:42:55.643688+0000","last_fullsized":"2024-05-08T04:42:55.643688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645113+0000","last_change":"2024-05-08T04:42:54.271090+0000","last_active":"2024-05-08T04:42:55.645113+0000","last_peered":"2024-05-08T04:42:55.645113+0000","last_clean":"2024-05-08T04:42:55.645113+0000","last_became_active":"2024-05-08T04:42:54.270874+0000","last_became_peered":"2024-05-08T04:42:54.270874+0000","last_unstale":"2024-05-08T04:42:55.645113+0000","last_undegraded":"2024-05-08T04:42:55.645113+0000","last_fullsized":"2024-05-08T04:42:55.645113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259863+0000","last_change":"2024-05-08T04:42:53.251819+0000","last_active":"2024-05-08T04:42:55.259863+0000","last_peered":"2024-05-08T04:42:55.259863+0000","last_clean":"2024-05-08T04:42:55.259863+0000","last_became_active":"2024-05-08T04:42:53.251490+0000","last_became_peered":"2024-05-08T04:42:53.251490+0000","last_unstale":"2024-05-08T04:42:55.259863+0000","last_undegraded":"2024-05-08T04:42:55.259863+0000","last_fullsized":"2024-05-08T04:42:55.259863+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:07.081658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261626+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.261626+0000","last_peered":"2024-05-08T04:42:55.261626+0000","last_clean":"2024-05-08T04:42:55.261626+0000","last_became_active":"2024-05-08T04:42:54.261894+0000","last_became_peered":"2024-05-08T04:42:54.261894+0000","last_unstale":"2024-05-08T04:42:55.261626+0000","last_undegraded":"2024-05-08T04:42:55.261626+0000","last_fullsized":"2024-05-08T04:42:55.261626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543705+0000","last_change":"2024-05-08T04:42:53.256430+0000","last_active":"2024-05-08T04:42:55.543705+0000","last_peered":"2024-05-08T04:42:55.543705+0000","last_clean":"2024-05-08T04:42:55.543705+0000","last_became_active":"2024-05-08T04:42:53.256154+0000","last_became_peered":"2024-05-08T04:42:53.256154+0000","last_unstale":"2024-05-08T04:42:55.543705+0000","last_undegraded":"2024-05-08T04:42:55.543705+0000","last_fullsized":"2024-05-08T04:42:55.543705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644955+0000","last_change":"2024-05-08T04:42:54.264075+0000","last_active":"2024-05-08T04:42:55.644955+0000","last_peered":"2024-05-08T04:42:55.644955+0000","last_clean":"2024-05-08T04:42:55.644955+0000","last_became_active":"2024-05-08T04:42:54.263803+0000","last_became_peered":"2024-05-08T04:42:54.263803+0000","last_unstale":"2024-05-08T04:42:55.644955+0000","last_undegraded":"2024-05-08T04:42:55.644955+0000","last_fullsized":"2024-05-08T04:42:55.644955+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234430+0000","last_change":"2024-05-08T04:42:52.234430+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234430+0000","last_undegraded":"2024-05-08T04:42:52.234430+0000","last_fullsized":"2024-05-08T04:42:52.234430+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544048+0000","last_change":"2024-05-08T04:42:54.279237+0000","last_active":"2024-05-08T04:42:55.544048+0000","last_peered":"2024-05-08T04:42:55.544048+0000","last_clean":"2024-05-08T04:42:55.544048+0000","last_became_active":"2024-05-08T04:42:54.279021+0000","last_became_peered":"2024-05-08T04:42:54.279021+0000","last_unstale":"2024-05-08T04:42:55.544048+0000","last_undegraded":"2024-05-08T04:42:55.544048+0000","last_fullsized":"2024-05-08T04:42:55.544048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:02.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234541+0000","last_change":"2024-05-08T04:42:52.234541+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234541+0000","last_undegraded":"2024-05-08T04:42:52.234541+0000","last_fullsized":"2024-05-08T04:42:52.234541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644157+0000","last_change":"2024-05-08T04:42:54.270324+0000","last_active":"2024-05-08T04:42:55.644157+0000","last_peered":"2024-05-08T04:42:55.644157+0000","last_clean":"2024-05-08T04:42:55.644157+0000","last_became_active":"2024-05-08T04:42:54.269819+0000","last_became_peered":"2024-05-08T04:42:54.269819+0000","last_unstale":"2024-05-08T04:42:55.644157+0000","last_undegraded":"2024-05-08T04:42:55.644157+0000","last_fullsized":"2024-05-08T04:42:55.644157+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:09.118707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259307+0000","last_change":"2024-05-08T04:42:53.257209+0000","last_active":"2024-05-08T04:42:55.259307+0000","last_peered":"2024-05-08T04:42:55.259307+0000","last_clean":"2024-05-08T04:42:55.259307+0000","last_became_active":"2024-05-08T04:42:53.256908+0000","last_became_peered":"2024-05-08T04:42:53.256908+0000","last_unstale":"2024-05-08T04:42:55.259307+0000","last_undegraded":"2024-05-08T04:42:55.259307+0000","last_fullsized":"2024-05-08T04:42:55.259307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259796+0000","last_change":"2024-05-08T04:42:54.263171+0000","last_active":"2024-05-08T04:42:55.259796+0000","last_peered":"2024-05-08T04:42:55.259796+0000","last_clean":"2024-05-08T04:42:55.259796+0000","last_became_active":"2024-05-08T04:42:54.262760+0000","last_became_peered":"2024-05-08T04:42:54.262760+0000","last_unstale":"2024-05-08T04:42:55.259796+0000","last_undegraded":"2024-05-08T04:42:55.259796+0000","last_fullsized":"2024-05-08T04:42:55.259796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.236833+0000","last_change":"2024-05-08T04:42:52.236833+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.236833+0000","last_undegraded":"2024-05-08T04:42:52.236833+0000","last_fullsized":"2024-05-08T04:42:52.236833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643515+0000","last_change":"2024-05-08T04:42:54.274640+0000","last_active":"2024-05-08T04:42:55.643515+0000","last_peered":"2024-05-08T04:42:55.643515+0000","last_clean":"2024-05-08T04:42:55.643515+0000","last_became_active":"2024-05-08T04:42:54.274209+0000","last_became_peered":"2024-05-08T04:42:54.274209+0000","last_unstale":"2024-05-08T04:42:55.643515+0000","last_undegraded":"2024-05-08T04:42:55.643515+0000","last_fullsized":"2024-05-08T04:42:55.643515+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:36.694738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259005+0000","last_change":"2024-05-08T04:42:53.265323+0000","last_active":"2024-05-08T04:42:55.259005+0000","last_peered":"2024-05-08T04:42:55.259005+0000","last_clean":"2024-05-08T04:42:55.259005+0000","last_became_active":"2024-05-08T04:42:53.265031+0000","last_became_peered":"2024-05-08T04:42:53.265031+0000","last_unstale":"2024-05-08T04:42:55.259005+0000","last_undegraded":"2024-05-08T04:42:55.259005+0000","last_fullsized":"2024-05-08T04:42:55.259005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:19.518749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258582+0000","last_change":"2024-05-08T04:42:54.263167+0000","last_active":"2024-05-08T04:42:55.258582+0000","last_peered":"2024-05-08T04:42:55.258582+0000","last_clean":"2024-05-08T04:42:55.258582+0000","last_became_active":"2024-05-08T04:42:54.262842+0000","last_became_peered":"2024-05-08T04:42:54.262842+0000","last_unstale":"2024-05-08T04:42:55.258582+0000","last_undegraded":"2024-05-08T04:42:55.258582+0000","last_fullsized":"2024-05-08T04:42:55.258582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643593+0000","last_change":"2024-05-08T04:42:53.263261+0000","last_active":"2024-05-08T04:42:55.643593+0000","last_peered":"2024-05-08T04:42:55.643593+0000","last_clean":"2024-05-08T04:42:55.643593+0000","last_became_active":"2024-05-08T04:42:53.262219+0000","last_became_peered":"2024-05-08T04:42:53.262219+0000","last_unstale":"2024-05-08T04:42:55.643593+0000","last_undegraded":"2024-05-08T04:42:55.643593+0000","last_fullsized":"2024-05-08T04:42:55.643593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:20.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260268+0000","last_change":"2024-05-08T04:42:54.269625+0000","last_active":"2024-05-08T04:42:55.260268+0000","last_peered":"2024-05-08T04:42:55.260268+0000","last_clean":"2024-05-08T04:42:55.260268+0000","last_became_active":"2024-05-08T04:42:54.269308+0000","last_became_peered":"2024-05-08T04:42:54.269308+0000","last_unstale":"2024-05-08T04:42:55.260268+0000","last_undegraded":"2024-05-08T04:42:55.260268+0000","last_fullsized":"2024-05-08T04:42:55.260268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.615548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234489+0000","last_change":"2024-05-08T04:42:52.234489+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234489+0000","last_undegraded":"2024-05-08T04:42:52.234489+0000","last_fullsized":"2024-05-08T04:42:52.234489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261712+0000","last_change":"2024-05-08T04:42:54.284605+0000","last_active":"2024-05-08T04:42:55.261712+0000","last_peered":"2024-05-08T04:42:55.261712+0000","last_clean":"2024-05-08T04:42:55.261712+0000","last_became_active":"2024-05-08T04:42:54.284331+0000","last_became_peered":"2024-05-08T04:42:54.284331+0000","last_unstale":"2024-05-08T04:42:55.261712+0000","last_undegraded":"2024-05-08T04:42:55.261712+0000","last_fullsized":"2024-05-08T04:42:55.261712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237857+0000","last_change":"2024-05-08T04:42:52.237857+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237857+0000","last_undegraded":"2024-05-08T04:42:52.237857+0000","last_fullsized":"2024-05-08T04:42:52.237857+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644694+0000","last_change":"2024-05-08T04:42:53.257288+0000","last_active":"2024-05-08T04:42:55.644694+0000","last_peered":"2024-05-08T04:42:55.644694+0000","last_clean":"2024-05-08T04:42:55.644694+0000","last_became_active":"2024-05-08T04:42:53.257014+0000","last_became_peered":"2024-05-08T04:42:53.257014+0000","last_unstale":"2024-05-08T04:42:55.644694+0000","last_undegraded":"2024-05-08T04:42:55.644694+0000","last_fullsized":"2024-05-08T04:42:55.644694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:27.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259886+0000","last_change":"2024-05-08T04:42:54.259828+0000","last_active":"2024-05-08T04:42:55.259886+0000","last_peered":"2024-05-08T04:42:55.259886+0000","last_clean":"2024-05-08T04:42:55.259886+0000","last_became_active":"2024-05-08T04:42:54.259109+0000","last_became_peered":"2024-05-08T04:42:54.259109+0000","last_unstale":"2024-05-08T04:42:55.259886+0000","last_undegraded":"2024-05-08T04:42:55.259886+0000","last_fullsized":"2024-05-08T04:42:55.259886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644648+0000","last_change":"2024-05-08T04:42:53.264438+0000","last_active":"2024-05-08T04:42:55.644648+0000","last_peered":"2024-05-08T04:42:55.644648+0000","last_clean":"2024-05-08T04:42:55.644648+0000","last_became_active":"2024-05-08T04:42:53.264131+0000","last_became_peered":"2024-05-08T04:42:53.264131+0000","last_unstale":"2024-05-08T04:42:55.644648+0000","last_undegraded":"2024-05-08T04:42:55.644648+0000","last_fullsized":"2024-05-08T04:42:55.644648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:15.699103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644435+0000","last_change":"2024-05-08T04:42:53.255665+0000","last_active":"2024-05-08T04:42:55.644435+0000","last_peered":"2024-05-08T04:42:55.644435+0000","last_clean":"2024-05-08T04:42:55.644435+0000","last_became_active":"2024-05-08T04:42:53.255168+0000","last_became_peered":"2024-05-08T04:42:53.255168+0000","last_unstale":"2024-05-08T04:42:55.644435+0000","last_undegraded":"2024-05-08T04:42:55.644435+0000","last_fullsized":"2024-05-08T04:42:55.644435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261312+0000","last_change":"2024-05-08T04:42:53.259031+0000","last_active":"2024-05-08T04:42:55.261312+0000","last_peered":"2024-05-08T04:42:55.261312+0000","last_clean":"2024-05-08T04:42:55.261312+0000","last_became_active":"2024-05-08T04:42:53.258734+0000","last_became_peered":"2024-05-08T04:42:53.258734+0000","last_unstale":"2024-05-08T04:42:55.261312+0000","last_undegraded":"2024-05-08T04:42:55.261312+0000","last_fullsized":"2024-05-08T04:42:55.261312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:07.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233237+0000","last_change":"2024-05-08T04:42:52.233237+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233237+0000","last_undegraded":"2024-05-08T04:42:52.233237+0000","last_fullsized":"2024-05-08T04:42:52.233237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645310+0000","last_change":"2024-05-08T04:42:53.255078+0000","last_active":"2024-05-08T04:42:55.645310+0000","last_peered":"2024-05-08T04:42:55.645310+0000","last_clean":"2024-05-08T04:42:55.645310+0000","last_became_active":"2024-05-08T04:42:53.254350+0000","last_became_peered":"2024-05-08T04:42:53.254350+0000","last_unstale":"2024-05-08T04:42:55.645310+0000","last_undegraded":"2024-05-08T04:42:55.645310+0000","last_fullsized":"2024-05-08T04:42:55.645310+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:59.504019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645155+0000","last_change":"2024-05-08T04:42:54.284624+0000","last_active":"2024-05-08T04:42:55.645155+0000","last_peered":"2024-05-08T04:42:55.645155+0000","last_clean":"2024-05-08T04:42:55.645155+0000","last_became_active":"2024-05-08T04:42:54.284368+0000","last_became_peered":"2024-05-08T04:42:54.284368+0000","last_unstale":"2024-05-08T04:42:55.645155+0000","last_undegraded":"2024-05-08T04:42:55.645155+0000","last_fullsized":"2024-05-08T04:42:55.645155+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233255+0000","last_change":"2024-05-08T04:42:52.233255+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233255+0000","last_undegraded":"2024-05-08T04:42:52.233255+0000","last_fullsized":"2024-05-08T04:42:52.233255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543959+0000","last_change":"2024-05-08T04:42:54.268528+0000","last_active":"2024-05-08T04:42:55.543959+0000","last_peered":"2024-05-08T04:42:55.543959+0000","last_clean":"2024-05-08T04:42:55.543959+0000","last_became_active":"2024-05-08T04:42:54.268247+0000","last_became_peered":"2024-05-08T04:42:54.268247+0000","last_unstale":"2024-05-08T04:42:55.543959+0000","last_undegraded":"2024-05-08T04:42:55.543959+0000","last_fullsized":"2024-05-08T04:42:55.543959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:20.235769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261410+0000","last_change":"2024-05-08T04:42:53.260958+0000","last_active":"2024-05-08T04:42:55.261410+0000","last_peered":"2024-05-08T04:42:55.261410+0000","last_clean":"2024-05-08T04:42:55.261410+0000","last_became_active":"2024-05-08T04:42:53.260621+0000","last_became_peered":"2024-05-08T04:42:53.260621+0000","last_unstale":"2024-05-08T04:42:55.261410+0000","last_undegraded":"2024-05-08T04:42:55.261410+0000","last_fullsized":"2024-05-08T04:42:55.261410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:14.035376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544039+0000","last_change":"2024-05-08T04:42:54.259461+0000","last_active":"2024-05-08T04:42:55.544039+0000","last_peered":"2024-05-08T04:42:55.544039+0000","last_clean":"2024-05-08T04:42:55.544039+0000","last_became_active":"2024-05-08T04:42:54.259005+0000","last_became_peered":"2024-05-08T04:42:54.259005+0000","last_unstale":"2024-05-08T04:42:55.544039+0000","last_undegraded":"2024-05-08T04:42:55.544039+0000","last_fullsized":"2024-05-08T04:42:55.544039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:07.448395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258867+0000","last_change":"2024-05-08T04:42:53.258018+0000","last_active":"2024-05-08T04:42:55.258867+0000","last_peered":"2024-05-08T04:42:55.258867+0000","last_clean":"2024-05-08T04:42:55.258867+0000","last_became_active":"2024-05-08T04:42:53.257357+0000","last_became_peered":"2024-05-08T04:42:53.257357+0000","last_unstale":"2024-05-08T04:42:55.258867+0000","last_undegraded":"2024-05-08T04:42:55.258867+0000","last_fullsized":"2024-05-08T04:42:55.258867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543555+0000","last_change":"2024-05-08T04:42:54.258649+0000","last_active":"2024-05-08T04:42:55.543555+0000","last_peered":"2024-05-08T04:42:55.543555+0000","last_clean":"2024-05-08T04:42:55.543555+0000","last_became_active":"2024-05-08T04:42:54.258367+0000","last_became_peered":"2024-05-08T04:42:54.258367+0000","last_unstale":"2024-05-08T04:42:55.543555+0000","last_undegraded":"2024-05-08T04:42:55.543555+0000","last_fullsized":"2024-05-08T04:42:55.543555+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644524+0000","last_change":"2024-05-08T04:42:53.260251+0000","last_active":"2024-05-08T04:42:55.644524+0000","last_peered":"2024-05-08T04:42:55.644524+0000","last_clean":"2024-05-08T04:42:55.644524+0000","last_became_active":"2024-05-08T04:42:53.259749+0000","last_became_peered":"2024-05-08T04:42:53.259749+0000","last_unstale":"2024-05-08T04:42:55.644524+0000","last_undegraded":"2024-05-08T04:42:55.644524+0000","last_fullsized":"2024-05-08T04:42:55.644524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644062+0000","last_change":"2024-05-08T04:42:54.270229+0000","last_active":"2024-05-08T04:42:55.644062+0000","last_peered":"2024-05-08T04:42:55.644062+0000","last_clean":"2024-05-08T04:42:55.644062+0000","last_became_active":"2024-05-08T04:42:54.267191+0000","last_became_peered":"2024-05-08T04:42:54.267191+0000","last_unstale":"2024-05-08T04:42:55.644062+0000","last_undegraded":"2024-05-08T04:42:55.644062+0000","last_fullsized":"2024-05-08T04:42:55.644062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:16.097496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234678+0000","last_change":"2024-05-08T04:42:52.234678+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234678+0000","last_undegraded":"2024-05-08T04:42:52.234678+0000","last_fullsized":"2024-05-08T04:42:52.234678+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543619+0000","last_change":"2024-05-08T04:42:54.270286+0000","last_active":"2024-05-08T04:42:55.543619+0000","last_peered":"2024-05-08T04:42:55.543619+0000","last_clean":"2024-05-08T04:42:55.543619+0000","last_became_active":"2024-05-08T04:42:54.270019+0000","last_became_peered":"2024-05-08T04:42:54.270019+0000","last_unstale":"2024-05-08T04:42:55.543619+0000","last_undegraded":"2024-05-08T04:42:55.543619+0000","last_fullsized":"2024-05-08T04:42:55.543619+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644073+0000","last_change":"2024-05-08T04:42:53.259343+0000","last_active":"2024-05-08T04:42:55.644073+0000","last_peered":"2024-05-08T04:42:55.644073+0000","last_clean":"2024-05-08T04:42:55.644073+0000","last_became_active":"2024-05-08T04:42:53.258967+0000","last_became_peered":"2024-05-08T04:42:53.258967+0000","last_unstale":"2024-05-08T04:42:55.644073+0000","last_undegraded":"2024-05-08T04:42:55.644073+0000","last_fullsized":"2024-05-08T04:42:55.644073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643660+0000","last_change":"2024-05-08T04:42:53.253264+0000","last_active":"2024-05-08T04:42:55.643660+0000","last_peered":"2024-05-08T04:42:55.643660+0000","last_clean":"2024-05-08T04:42:55.643660+0000","last_became_active":"2024-05-08T04:42:53.252716+0000","last_became_peered":"2024-05-08T04:42:53.252716+0000","last_unstale":"2024-05-08T04:42:55.643660+0000","last_undegraded":"2024-05-08T04:42:55.643660+0000","last_fullsized":"2024-05-08T04:42:55.643660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:56.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275320+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.275003+0000","last_became_peered":"2024-05-08T04:42:54.275003+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237599+0000","last_change":"2024-05-08T04:42:52.237599+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237599+0000","last_undegraded":"2024-05-08T04:42:52.237599+0000","last_fullsized":"2024-05-08T04:42:52.237599+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645304+0000","last_change":"2024-05-08T04:42:54.282790+0000","last_active":"2024-05-08T04:42:55.645304+0000","last_peered":"2024-05-08T04:42:55.645304+0000","last_clean":"2024-05-08T04:42:55.645304+0000","last_became_active":"2024-05-08T04:42:54.282523+0000","last_became_peered":"2024-05-08T04:42:54.282523+0000","last_unstale":"2024-05-08T04:42:55.645304+0000","last_undegraded":"2024-05-08T04:42:55.645304+0000","last_fullsized":"2024-05-08T04:42:55.645304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.467995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261546+0000","last_change":"2024-05-08T04:42:53.252707+0000","last_active":"2024-05-08T04:42:55.261546+0000","last_peered":"2024-05-08T04:42:55.261546+0000","last_clean":"2024-05-08T04:42:55.261546+0000","last_became_active":"2024-05-08T04:42:53.250660+0000","last_became_peered":"2024-05-08T04:42:53.250660+0000","last_unstale":"2024-05-08T04:42:55.261546+0000","last_undegraded":"2024-05-08T04:42:55.261546+0000","last_fullsized":"2024-05-08T04:42:55.261546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:13.437315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644517+0000","last_change":"2024-05-08T04:42:54.277076+0000","last_active":"2024-05-08T04:42:55.644517+0000","last_peered":"2024-05-08T04:42:55.644517+0000","last_clean":"2024-05-08T04:42:55.644517+0000","last_became_active":"2024-05-08T04:42:54.276805+0000","last_became_peered":"2024-05-08T04:42:54.276805+0000","last_unstale":"2024-05-08T04:42:55.644517+0000","last_undegraded":"2024-05-08T04:42:55.644517+0000","last_fullsized":"2024-05-08T04:42:55.644517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:39.976473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644732+0000","last_change":"2024-05-08T04:42:53.252678+0000","last_active":"2024-05-08T04:42:55.644732+0000","last_peered":"2024-05-08T04:42:55.644732+0000","last_clean":"2024-05-08T04:42:55.644732+0000","last_became_active":"2024-05-08T04:42:53.252426+0000","last_became_peered":"2024-05-08T04:42:53.252426+0000","last_unstale":"2024-05-08T04:42:55.644732+0000","last_undegraded":"2024-05-08T04:42:55.644732+0000","last_fullsized":"2024-05-08T04:42:55.644732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:50.935297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275599+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.274979+0000","last_became_peered":"2024-05-08T04:42:54.274979+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:27.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237194+0000","last_change":"2024-05-08T04:42:52.237194+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237194+0000","last_undegraded":"2024-05-08T04:42:52.237194+0000","last_fullsized":"2024-05-08T04:42:52.237194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259403+0000","last_change":"2024-05-08T04:42:54.250195+0000","last_active":"2024-05-08T04:42:55.259403+0000","last_peered":"2024-05-08T04:42:55.259403+0000","last_clean":"2024-05-08T04:42:55.259403+0000","last_became_active":"2024-05-08T04:42:54.249297+0000","last_became_peered":"2024-05-08T04:42:54.249297+0000","last_unstale":"2024-05-08T04:42:55.259403+0000","last_undegraded":"2024-05-08T04:42:55.259403+0000","last_fullsized":"2024-05-08T04:42:55.259403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259421+0000","last_change":"2024-05-08T04:42:53.259450+0000","last_active":"2024-05-08T04:42:55.259421+0000","last_peered":"2024-05-08T04:42:55.259421+0000","last_clean":"2024-05-08T04:42:55.259421+0000","last_became_active":"2024-05-08T04:42:53.259139+0000","last_became_peered":"2024-05-08T04:42:53.259139+0000","last_unstale":"2024-05-08T04:42:55.259421+0000","last_undegraded":"2024-05-08T04:42:55.259421+0000","last_fullsized":"2024-05-08T04:42:55.259421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260413+0000","last_change":"2024-05-08T04:42:54.263700+0000","last_active":"2024-05-08T04:42:55.260413+0000","last_peered":"2024-05-08T04:42:55.260413+0000","last_clean":"2024-05-08T04:42:55.260413+0000","last_became_active":"2024-05-08T04:42:54.263395+0000","last_became_peered":"2024-05-08T04:42:54.263395+0000","last_unstale":"2024-05-08T04:42:55.260413+0000","last_undegraded":"2024-05-08T04:42:55.260413+0000","last_fullsized":"2024-05-08T04:42:55.260413+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:40.094631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259409+0000","last_change":"2024-05-08T04:42:53.258649+0000","last_active":"2024-05-08T04:42:55.259409+0000","last_peered":"2024-05-08T04:42:55.259409+0000","last_clean":"2024-05-08T04:42:55.259409+0000","last_became_active":"2024-05-08T04:42:53.258348+0000","last_became_peered":"2024-05-08T04:42:53.258348+0000","last_unstale":"2024-05-08T04:42:55.259409+0000","last_undegraded":"2024-05-08T04:42:55.259409+0000","last_fullsized":"2024-05-08T04:42:55.259409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:42.312320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543824+0000","last_change":"2024-05-08T04:42:54.266324+0000","last_active":"2024-05-08T04:42:55.543824+0000","last_peered":"2024-05-08T04:42:55.543824+0000","last_clean":"2024-05-08T04:42:55.543824+0000","last_became_active":"2024-05-08T04:42:54.265475+0000","last_became_peered":"2024-05-08T04:42:54.265475+0000","last_unstale":"2024-05-08T04:42:55.543824+0000","last_undegraded":"2024-05-08T04:42:55.543824+0000","last_fullsized":"2024-05-08T04:42:55.543824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233276+0000","last_change":"2024-05-08T04:42:52.233276+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233276+0000","last_undegraded":"2024-05-08T04:42:52.233276+0000","last_fullsized":"2024-05-08T04:42:52.233276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643920+0000","last_change":"2024-05-08T04:42:54.270703+0000","last_active":"2024-05-08T04:42:55.643920+0000","last_peered":"2024-05-08T04:42:55.643920+0000","last_clean":"2024-05-08T04:42:55.643920+0000","last_became_active":"2024-05-08T04:42:54.265604+0000","last_became_peered":"2024-05-08T04:42:54.265604+0000","last_unstale":"2024-05-08T04:42:55.643920+0000","last_undegraded":"2024-05-08T04:42:55.643920+0000","last_fullsized":"2024-05-08T04:42:55.643920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:21.915088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643694+0000","last_change":"2024-05-08T04:42:53.253094+0000","last_active":"2024-05-08T04:42:55.643694+0000","last_peered":"2024-05-08T04:42:55.643694+0000","last_clean":"2024-05-08T04:42:55.643694+0000","last_became_active":"2024-05-08T04:42:53.252291+0000","last_became_peered":"2024-05-08T04:42:53.252291+0000","last_unstale":"2024-05-08T04:42:55.643694+0000","last_undegraded":"2024-05-08T04:42:55.643694+0000","last_fullsized":"2024-05-08T04:42:55.643694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:59.366123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258784+0000","last_change":"2024-05-08T04:42:53.257287+0000","last_active":"2024-05-08T04:42:55.258784+0000","last_peered":"2024-05-08T04:42:55.258784+0000","last_clean":"2024-05-08T04:42:55.258784+0000","last_became_active":"2024-05-08T04:42:53.256851+0000","last_became_peered":"2024-05-08T04:42:53.256851+0000","last_unstale":"2024-05-08T04:42:55.258784+0000","last_undegraded":"2024-05-08T04:42:55.258784+0000","last_fullsized":"2024-05-08T04:42:55.258784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543931+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:55.543931+0000","last_peered":"2024-05-08T04:42:55.543931+0000","last_clean":"2024-05-08T04:42:55.543931+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:55.543931+0000","last_undegraded":"2024-05-08T04:42:55.543931+0000","last_fullsized":"2024-05-08T04:42:55.543931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":1198126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":607212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":607746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":587540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":607266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:42:58.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:58.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:42:58.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:42:58.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.883+0000 7fd037a00640 1 Processor -- start 2024-05-08T04:42:58.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.883+0000 7fd037a00640 1 -- start start 2024-05-08T04:42:58.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd037a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 0x7fd030072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd037a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd0300b26c0 0x7fd0300cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd037a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd030072640 con 0x7fd0300cfbd0 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd037a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300d0060 con 0x7fd0300b26c0 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd037a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03006c770 con 0x7fd03015ab60 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd02fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd0300b26c0 0x7fd0300cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd035a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd0300cfbd0 0x7fd0300cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46418/0 (socket says 172.21.15.164:46418) 2024-05-08T04:42:58.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd035a00640 1 -- 172.21.15.164:0/3223531658 learned_addr learned my addr 172.21.15.164:0/3223531658 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd035000640 1 --2- 172.21.15.164:0/3223531658 >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 0x7fd030072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd02fe00640 1 -- 172.21.15.164:0/3223531658 >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 msgr2=0x7fd030072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd02fe00640 1 --2- 172.21.15.164:0/3223531658 >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 0x7fd030072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd02fe00640 1 -- 172.21.15.164:0/3223531658 >> v1:172.21.15.164:6789/0 conn(0x7fd0300cfbd0 legacy=0x7fd0300cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd02fe00640 1 -- 172.21.15.164:0/3223531658 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03006ca50 con 0x7fd0300b26c0 2024-05-08T04:42:58.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.887+0000 7fd035000640 1 --2- 172.21.15.164:0/3223531658 >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 0x7fd030072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:42:58.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd02fe00640 1 --2- 172.21.15.164:0/3223531658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300b26c0 0x7fd0300cf690 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd0300cd420 tx=0x7fd01c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711847441d7e21e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd02f400640 1 -- 172.21.15.164:0/3223531658 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0b0070 con 0x7fd0300b26c0 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd02f400640 1 -- 172.21.15.164:0/3223531658 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c0ab600 con 0x7fd0300b26c0 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd02f400640 1 -- 172.21.15.164:0/3223531658 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0ab9d0 con 0x7fd0300b26c0 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- 172.21.15.164:0/3223531658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300b26c0 msgr2=0x7fd0300cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- 172.21.15.164:0/3223531658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300b26c0 0x7fd0300cf690 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd0300cd420 tx=0x7fd01c01c480 comp rx=0 tx=0).stop 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- 172.21.15.164:0/3223531658 shutdown_connections 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- 172.21.15.164:0/3223531658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300b26c0 0x7fd0300cf690 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- 172.21.15.164:0/3223531658 >> v2:172.21.15.172:3300/0 conn(0x7fd03015ab60 0x7fd030072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- 172.21.15.164:0/3223531658 >> 172.21.15.164:0/3223531658 conn(0x7fd03006ac90 msgr2=0x7fd03006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:58.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- 172.21.15.164:0/3223531658 shutdown_connections 2024-05-08T04:42:58.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- 172.21.15.164:0/3223531658 wait complete. 2024-05-08T04:42:58.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 Processor -- start 2024-05-08T04:42:58.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- start start 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0300b26c0 0x7fd0300b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd03015ab60 0x7fd03014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006c550 con 0x7fd0300cfbd0 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006c3d0 con 0x7fd0300b26c0 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.891+0000 7fd037a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300aed30 con 0x7fd03015ab60 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56808/0 (socket says 172.21.15.164:56808) 2024-05-08T04:42:58.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 -- 172.21.15.164:0/2204517317 learned_addr learned my addr 172.21.15.164:0/2204517317 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 -- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd03015ab60 msgr2=0x7fd03014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd03015ab60 0x7fd03014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 -- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0300b26c0 msgr2=0x7fd0300b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0300b26c0 0x7fd0300b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 -- 172.21.15.164:0/2204517317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030155380 con 0x7fd0300cfbd0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02fe00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ae040 tx=0x7fd01c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0b8020 con 0x7fd0300cfbd0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c0b4630 con 0x7fd0300cfbd0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd030155660 con 0x7fd0300cfbd0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0b0040 con 0x7fd0300cfbd0 2024-05-08T04:42:58.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030155c20 con 0x7fd0300cfbd0 2024-05-08T04:42:58.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030069d60 con 0x7fd0300cfbd0 2024-05-08T04:42:58.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fd01c0bb020 con 0x7fd0300cfbd0 2024-05-08T04:42:58.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 0x7fd010069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:58.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.895+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd01c11d670 con 0x7fd0300cfbd0 2024-05-08T04:42:58.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.899+0000 7fd035000640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 0x7fd010069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:58.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.899+0000 7fd035000640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 0x7fd010069d20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd020008e00 tx=0x7fd020008330 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:58.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:58.899+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd01c0c2030 con 0x7fd0300cfbd0 2024-05-08T04:42:59.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.051+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd03015ec60 con 0x7fd010067860 2024-05-08T04:42:59.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd02cc00640 1 -- 172.21.15.164:0/2204517317 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249796 (secure 0 0 0) 0x7fd03015ec60 con 0x7fd010067860 2024-05-08T04:42:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:59.063 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:59.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 msgr2=0x7fd010069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 0x7fd010069d20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd020008e00 tx=0x7fd020008330 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 msgr2=0x7fd030154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ae040 tx=0x7fd01c019220 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 shutdown_connections 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd010067860 0x7fd010069d20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd03015ab60 0x7fd03014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0300cfbd0 0x7fd030154be0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 --2- 172.21.15.164:0/2204517317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0300b26c0 0x7fd0300b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 >> 172.21.15.164:0/2204517317 conn(0x7fd03006ac90 msgr2=0x7fd030156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.059+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 shutdown_connections 2024-05-08T04:42:59.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.063+0000 7fd037a00640 1 -- 172.21.15.164:0/2204517317 wait complete. 2024-05-08T04:42:59.082 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-05-08T04:42:57.484254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261468,"kb_used_data":12004,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713252,"statfs":{"total":773094113280,"available":772826370048,"internally_reserved":0,"allocated":12292096,"data_stored":5995905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.254344"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645115+0000","last_change":"2024-05-08T04:42:54.284520+0000","last_active":"2024-05-08T04:42:55.645115+0000","last_peered":"2024-05-08T04:42:55.645115+0000","last_clean":"2024-05-08T04:42:55.645115+0000","last_became_active":"2024-05-08T04:42:54.283787+0000","last_became_peered":"2024-05-08T04:42:54.283787+0000","last_unstale":"2024-05-08T04:42:55.645115+0000","last_undegraded":"2024-05-08T04:42:55.645115+0000","last_fullsized":"2024-05-08T04:42:55.645115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:31.361703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260638+0000","last_change":"2024-05-08T04:42:54.267059+0000","last_active":"2024-05-08T04:42:55.260638+0000","last_peered":"2024-05-08T04:42:55.260638+0000","last_clean":"2024-05-08T04:42:55.260638+0000","last_became_active":"2024-05-08T04:42:54.266748+0000","last_became_peered":"2024-05-08T04:42:54.266748+0000","last_unstale":"2024-05-08T04:42:55.260638+0000","last_undegraded":"2024-05-08T04:42:55.260638+0000","last_fullsized":"2024-05-08T04:42:55.260638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:57.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543833+0000","last_change":"2024-05-08T04:42:54.259254+0000","last_active":"2024-05-08T04:42:55.543833+0000","last_peered":"2024-05-08T04:42:55.543833+0000","last_clean":"2024-05-08T04:42:55.543833+0000","last_became_active":"2024-05-08T04:42:54.258620+0000","last_became_peered":"2024-05-08T04:42:54.258620+0000","last_unstale":"2024-05-08T04:42:55.543833+0000","last_undegraded":"2024-05-08T04:42:55.543833+0000","last_fullsized":"2024-05-08T04:42:55.543833+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644261+0000","last_change":"2024-05-08T04:42:54.276298+0000","last_active":"2024-05-08T04:42:55.644261+0000","last_peered":"2024-05-08T04:42:55.644261+0000","last_clean":"2024-05-08T04:42:55.644261+0000","last_became_active":"2024-05-08T04:42:54.273697+0000","last_became_peered":"2024-05-08T04:42:54.273697+0000","last_unstale":"2024-05-08T04:42:55.644261+0000","last_undegraded":"2024-05-08T04:42:55.644261+0000","last_fullsized":"2024-05-08T04:42:55.644261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:45.521599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644461+0000","last_change":"2024-05-08T04:42:54.259042+0000","last_active":"2024-05-08T04:42:55.644461+0000","last_peered":"2024-05-08T04:42:55.644461+0000","last_clean":"2024-05-08T04:42:55.644461+0000","last_became_active":"2024-05-08T04:42:54.258775+0000","last_became_peered":"2024-05-08T04:42:54.258775+0000","last_unstale":"2024-05-08T04:42:55.644461+0000","last_undegraded":"2024-05-08T04:42:55.644461+0000","last_fullsized":"2024-05-08T04:42:55.644461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:58.137321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258732+0000","last_change":"2024-05-08T04:42:54.258159+0000","last_active":"2024-05-08T04:42:55.258732+0000","last_peered":"2024-05-08T04:42:55.258732+0000","last_clean":"2024-05-08T04:42:55.258732+0000","last_became_active":"2024-05-08T04:42:54.257892+0000","last_became_peered":"2024-05-08T04:42:54.257892+0000","last_unstale":"2024-05-08T04:42:55.258732+0000","last_undegraded":"2024-05-08T04:42:55.258732+0000","last_fullsized":"2024-05-08T04:42:55.258732+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543525+0000","last_change":"2024-05-08T04:42:54.261073+0000","last_active":"2024-05-08T04:42:55.543525+0000","last_peered":"2024-05-08T04:42:55.543525+0000","last_clean":"2024-05-08T04:42:55.543525+0000","last_became_active":"2024-05-08T04:42:54.260783+0000","last_became_peered":"2024-05-08T04:42:54.260783+0000","last_unstale":"2024-05-08T04:42:55.543525+0000","last_undegraded":"2024-05-08T04:42:55.543525+0000","last_fullsized":"2024-05-08T04:42:55.543525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645257+0000","last_change":"2024-05-08T04:42:54.280775+0000","last_active":"2024-05-08T04:42:55.645257+0000","last_peered":"2024-05-08T04:42:55.645257+0000","last_clean":"2024-05-08T04:42:55.645257+0000","last_became_active":"2024-05-08T04:42:54.280501+0000","last_became_peered":"2024-05-08T04:42:54.280501+0000","last_unstale":"2024-05-08T04:42:55.645257+0000","last_undegraded":"2024-05-08T04:42:55.645257+0000","last_fullsized":"2024-05-08T04:42:55.645257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:25.588352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645180+0000","last_change":"2024-05-08T04:42:54.279039+0000","last_active":"2024-05-08T04:42:55.645180+0000","last_peered":"2024-05-08T04:42:55.645180+0000","last_clean":"2024-05-08T04:42:55.645180+0000","last_became_active":"2024-05-08T04:42:54.278516+0000","last_became_peered":"2024-05-08T04:42:54.278516+0000","last_unstale":"2024-05-08T04:42:55.645180+0000","last_undegraded":"2024-05-08T04:42:55.645180+0000","last_fullsized":"2024-05-08T04:42:55.645180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:44.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260298+0000","last_change":"2024-05-08T04:42:54.264244+0000","last_active":"2024-05-08T04:42:55.260298+0000","last_peered":"2024-05-08T04:42:55.260298+0000","last_clean":"2024-05-08T04:42:55.260298+0000","last_became_active":"2024-05-08T04:42:54.261285+0000","last_became_peered":"2024-05-08T04:42:54.261285+0000","last_unstale":"2024-05-08T04:42:55.260298+0000","last_undegraded":"2024-05-08T04:42:55.260298+0000","last_fullsized":"2024-05-08T04:42:55.260298+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:16.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644393+0000","last_change":"2024-05-08T04:42:54.278540+0000","last_active":"2024-05-08T04:42:55.644393+0000","last_peered":"2024-05-08T04:42:55.644393+0000","last_clean":"2024-05-08T04:42:55.644393+0000","last_became_active":"2024-05-08T04:42:54.275676+0000","last_became_peered":"2024-05-08T04:42:54.275676+0000","last_unstale":"2024-05-08T04:42:55.644393+0000","last_undegraded":"2024-05-08T04:42:55.644393+0000","last_fullsized":"2024-05-08T04:42:55.644393+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:02.962699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543936+0000","last_change":"2024-05-08T04:42:54.262625+0000","last_active":"2024-05-08T04:42:55.543936+0000","last_peered":"2024-05-08T04:42:55.543936+0000","last_clean":"2024-05-08T04:42:55.543936+0000","last_became_active":"2024-05-08T04:42:54.262328+0000","last_became_peered":"2024-05-08T04:42:54.262328+0000","last_unstale":"2024-05-08T04:42:55.543936+0000","last_undegraded":"2024-05-08T04:42:55.543936+0000","last_fullsized":"2024-05-08T04:42:55.543936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:37.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543497+0000","last_change":"2024-05-08T04:42:54.263202+0000","last_active":"2024-05-08T04:42:55.543497+0000","last_peered":"2024-05-08T04:42:55.543497+0000","last_clean":"2024-05-08T04:42:55.543497+0000","last_became_active":"2024-05-08T04:42:54.262118+0000","last_became_peered":"2024-05-08T04:42:54.262118+0000","last_unstale":"2024-05-08T04:42:55.543497+0000","last_undegraded":"2024-05-08T04:42:55.543497+0000","last_fullsized":"2024-05-08T04:42:55.543497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643757+0000","last_change":"2024-05-08T04:42:54.258584+0000","last_active":"2024-05-08T04:42:55.643757+0000","last_peered":"2024-05-08T04:42:55.643757+0000","last_clean":"2024-05-08T04:42:55.643757+0000","last_became_active":"2024-05-08T04:42:54.257809+0000","last_became_peered":"2024-05-08T04:42:54.257809+0000","last_unstale":"2024-05-08T04:42:55.643757+0000","last_undegraded":"2024-05-08T04:42:55.643757+0000","last_fullsized":"2024-05-08T04:42:55.643757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.486563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643534+0000","last_change":"2024-05-08T04:42:54.272568+0000","last_active":"2024-05-08T04:42:55.643534+0000","last_peered":"2024-05-08T04:42:55.643534+0000","last_clean":"2024-05-08T04:42:55.643534+0000","last_became_active":"2024-05-08T04:42:54.266872+0000","last_became_peered":"2024-05-08T04:42:54.266872+0000","last_unstale":"2024-05-08T04:42:55.643534+0000","last_undegraded":"2024-05-08T04:42:55.643534+0000","last_fullsized":"2024-05-08T04:42:55.643534+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:19.666610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260434+0000","last_change":"2024-05-08T04:42:54.265221+0000","last_active":"2024-05-08T04:42:55.260434+0000","last_peered":"2024-05-08T04:42:55.260434+0000","last_clean":"2024-05-08T04:42:55.260434+0000","last_became_active":"2024-05-08T04:42:54.263935+0000","last_became_peered":"2024-05-08T04:42:54.263935+0000","last_unstale":"2024-05-08T04:42:55.260434+0000","last_undegraded":"2024-05-08T04:42:55.260434+0000","last_fullsized":"2024-05-08T04:42:55.260434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:24.087153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261472+0000","last_change":"2024-05-08T04:42:54.260979+0000","last_active":"2024-05-08T04:42:55.261472+0000","last_peered":"2024-05-08T04:42:55.261472+0000","last_clean":"2024-05-08T04:42:55.261472+0000","last_became_active":"2024-05-08T04:42:54.260386+0000","last_became_peered":"2024-05-08T04:42:54.260386+0000","last_unstale":"2024-05-08T04:42:55.261472+0000","last_undegraded":"2024-05-08T04:42:55.261472+0000","last_fullsized":"2024-05-08T04:42:55.261472+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:23.982806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543852+0000","last_change":"2024-05-08T04:42:54.267987+0000","last_active":"2024-05-08T04:42:55.543852+0000","last_peered":"2024-05-08T04:42:55.543852+0000","last_clean":"2024-05-08T04:42:55.543852+0000","last_became_active":"2024-05-08T04:42:54.267674+0000","last_became_peered":"2024-05-08T04:42:54.267674+0000","last_unstale":"2024-05-08T04:42:55.543852+0000","last_undegraded":"2024-05-08T04:42:55.543852+0000","last_fullsized":"2024-05-08T04:42:55.543852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:11.284074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259984+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.259984+0000","last_peered":"2024-05-08T04:42:55.259984+0000","last_clean":"2024-05-08T04:42:55.259984+0000","last_became_active":"2024-05-08T04:42:54.261903+0000","last_became_peered":"2024-05-08T04:42:54.261903+0000","last_unstale":"2024-05-08T04:42:55.259984+0000","last_undegraded":"2024-05-08T04:42:55.259984+0000","last_fullsized":"2024-05-08T04:42:55.259984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:49.660293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260474+0000","last_change":"2024-05-08T04:42:54.247853+0000","last_active":"2024-05-08T04:42:55.260474+0000","last_peered":"2024-05-08T04:42:55.260474+0000","last_clean":"2024-05-08T04:42:55.260474+0000","last_became_active":"2024-05-08T04:42:54.247314+0000","last_became_peered":"2024-05-08T04:42:54.247314+0000","last_unstale":"2024-05-08T04:42:55.260474+0000","last_undegraded":"2024-05-08T04:42:55.260474+0000","last_fullsized":"2024-05-08T04:42:55.260474+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.153233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258982+0000","last_change":"2024-05-08T04:42:54.257923+0000","last_active":"2024-05-08T04:42:55.258982+0000","last_peered":"2024-05-08T04:42:55.258982+0000","last_clean":"2024-05-08T04:42:55.258982+0000","last_became_active":"2024-05-08T04:42:54.257524+0000","last_became_peered":"2024-05-08T04:42:54.257524+0000","last_unstale":"2024-05-08T04:42:55.258982+0000","last_undegraded":"2024-05-08T04:42:55.258982+0000","last_fullsized":"2024-05-08T04:42:55.258982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260632+0000","last_change":"2024-05-08T04:42:54.264625+0000","last_active":"2024-05-08T04:42:55.260632+0000","last_peered":"2024-05-08T04:42:55.260632+0000","last_clean":"2024-05-08T04:42:55.260632+0000","last_became_active":"2024-05-08T04:42:54.261407+0000","last_became_peered":"2024-05-08T04:42:54.261407+0000","last_unstale":"2024-05-08T04:42:55.260632+0000","last_undegraded":"2024-05-08T04:42:55.260632+0000","last_fullsized":"2024-05-08T04:42:55.260632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:21.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644944+0000","last_change":"2024-05-08T04:42:54.273444+0000","last_active":"2024-05-08T04:42:55.644944+0000","last_peered":"2024-05-08T04:42:55.644944+0000","last_clean":"2024-05-08T04:42:55.644944+0000","last_became_active":"2024-05-08T04:42:54.273184+0000","last_became_peered":"2024-05-08T04:42:54.273184+0000","last_unstale":"2024-05-08T04:42:55.644944+0000","last_undegraded":"2024-05-08T04:42:55.644944+0000","last_fullsized":"2024-05-08T04:42:55.644944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:18.445404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645146+0000","last_change":"2024-05-08T04:42:54.277145+0000","last_active":"2024-05-08T04:42:55.645146+0000","last_peered":"2024-05-08T04:42:55.645146+0000","last_clean":"2024-05-08T04:42:55.645146+0000","last_became_active":"2024-05-08T04:42:54.274239+0000","last_became_peered":"2024-05-08T04:42:54.274239+0000","last_unstale":"2024-05-08T04:42:55.645146+0000","last_undegraded":"2024-05-08T04:42:55.645146+0000","last_fullsized":"2024-05-08T04:42:55.645146+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260283+0000","last_change":"2024-05-08T04:42:54.257830+0000","last_active":"2024-05-08T04:42:55.260283+0000","last_peered":"2024-05-08T04:42:55.260283+0000","last_clean":"2024-05-08T04:42:55.260283+0000","last_became_active":"2024-05-08T04:42:54.257455+0000","last_became_peered":"2024-05-08T04:42:54.257455+0000","last_unstale":"2024-05-08T04:42:55.260283+0000","last_undegraded":"2024-05-08T04:42:55.260283+0000","last_fullsized":"2024-05-08T04:42:55.260283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:08.717198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259104+0000","last_change":"2024-05-08T04:42:54.247280+0000","last_active":"2024-05-08T04:42:55.259104+0000","last_peered":"2024-05-08T04:42:55.259104+0000","last_clean":"2024-05-08T04:42:55.259104+0000","last_became_active":"2024-05-08T04:42:54.246609+0000","last_became_peered":"2024-05-08T04:42:54.246609+0000","last_unstale":"2024-05-08T04:42:55.259104+0000","last_undegraded":"2024-05-08T04:42:55.259104+0000","last_fullsized":"2024-05-08T04:42:55.259104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643671+0000","last_change":"2024-05-08T04:42:54.270435+0000","last_active":"2024-05-08T04:42:55.643671+0000","last_peered":"2024-05-08T04:42:55.643671+0000","last_clean":"2024-05-08T04:42:55.643671+0000","last_became_active":"2024-05-08T04:42:54.265729+0000","last_became_peered":"2024-05-08T04:42:54.265729+0000","last_unstale":"2024-05-08T04:42:55.643671+0000","last_undegraded":"2024-05-08T04:42:55.643671+0000","last_fullsized":"2024-05-08T04:42:55.643671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:53.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643688+0000","last_change":"2024-05-08T04:42:54.261182+0000","last_active":"2024-05-08T04:42:55.643688+0000","last_peered":"2024-05-08T04:42:55.643688+0000","last_clean":"2024-05-08T04:42:55.643688+0000","last_became_active":"2024-05-08T04:42:54.260895+0000","last_became_peered":"2024-05-08T04:42:54.260895+0000","last_unstale":"2024-05-08T04:42:55.643688+0000","last_undegraded":"2024-05-08T04:42:55.643688+0000","last_fullsized":"2024-05-08T04:42:55.643688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645113+0000","last_change":"2024-05-08T04:42:54.271090+0000","last_active":"2024-05-08T04:42:55.645113+0000","last_peered":"2024-05-08T04:42:55.645113+0000","last_clean":"2024-05-08T04:42:55.645113+0000","last_became_active":"2024-05-08T04:42:54.270874+0000","last_became_peered":"2024-05-08T04:42:54.270874+0000","last_unstale":"2024-05-08T04:42:55.645113+0000","last_undegraded":"2024-05-08T04:42:55.645113+0000","last_fullsized":"2024-05-08T04:42:55.645113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259863+0000","last_change":"2024-05-08T04:42:53.251819+0000","last_active":"2024-05-08T04:42:55.259863+0000","last_peered":"2024-05-08T04:42:55.259863+0000","last_clean":"2024-05-08T04:42:55.259863+0000","last_became_active":"2024-05-08T04:42:53.251490+0000","last_became_peered":"2024-05-08T04:42:53.251490+0000","last_unstale":"2024-05-08T04:42:55.259863+0000","last_undegraded":"2024-05-08T04:42:55.259863+0000","last_fullsized":"2024-05-08T04:42:55.259863+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:07.081658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261626+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.261626+0000","last_peered":"2024-05-08T04:42:55.261626+0000","last_clean":"2024-05-08T04:42:55.261626+0000","last_became_active":"2024-05-08T04:42:54.261894+0000","last_became_peered":"2024-05-08T04:42:54.261894+0000","last_unstale":"2024-05-08T04:42:55.261626+0000","last_undegraded":"2024-05-08T04:42:55.261626+0000","last_fullsized":"2024-05-08T04:42:55.261626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543705+0000","last_change":"2024-05-08T04:42:53.256430+0000","last_active":"2024-05-08T04:42:55.543705+0000","last_peered":"2024-05-08T04:42:55.543705+0000","last_clean":"2024-05-08T04:42:55.543705+0000","last_became_active":"2024-05-08T04:42:53.256154+0000","last_became_peered":"2024-05-08T04:42:53.256154+0000","last_unstale":"2024-05-08T04:42:55.543705+0000","last_undegraded":"2024-05-08T04:42:55.543705+0000","last_fullsized":"2024-05-08T04:42:55.543705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644955+0000","last_change":"2024-05-08T04:42:54.264075+0000","last_active":"2024-05-08T04:42:55.644955+0000","last_peered":"2024-05-08T04:42:55.644955+0000","last_clean":"2024-05-08T04:42:55.644955+0000","last_became_active":"2024-05-08T04:42:54.263803+0000","last_became_peered":"2024-05-08T04:42:54.263803+0000","last_unstale":"2024-05-08T04:42:55.644955+0000","last_undegraded":"2024-05-08T04:42:55.644955+0000","last_fullsized":"2024-05-08T04:42:55.644955+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234430+0000","last_change":"2024-05-08T04:42:52.234430+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234430+0000","last_undegraded":"2024-05-08T04:42:52.234430+0000","last_fullsized":"2024-05-08T04:42:52.234430+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544048+0000","last_change":"2024-05-08T04:42:54.279237+0000","last_active":"2024-05-08T04:42:55.544048+0000","last_peered":"2024-05-08T04:42:55.544048+0000","last_clean":"2024-05-08T04:42:55.544048+0000","last_became_active":"2024-05-08T04:42:54.279021+0000","last_became_peered":"2024-05-08T04:42:54.279021+0000","last_unstale":"2024-05-08T04:42:55.544048+0000","last_undegraded":"2024-05-08T04:42:55.544048+0000","last_fullsized":"2024-05-08T04:42:55.544048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:02.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234541+0000","last_change":"2024-05-08T04:42:52.234541+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234541+0000","last_undegraded":"2024-05-08T04:42:52.234541+0000","last_fullsized":"2024-05-08T04:42:52.234541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644157+0000","last_change":"2024-05-08T04:42:54.270324+0000","last_active":"2024-05-08T04:42:55.644157+0000","last_peered":"2024-05-08T04:42:55.644157+0000","last_clean":"2024-05-08T04:42:55.644157+0000","last_became_active":"2024-05-08T04:42:54.269819+0000","last_became_peered":"2024-05-08T04:42:54.269819+0000","last_unstale":"2024-05-08T04:42:55.644157+0000","last_undegraded":"2024-05-08T04:42:55.644157+0000","last_fullsized":"2024-05-08T04:42:55.644157+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:09.118707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259307+0000","last_change":"2024-05-08T04:42:53.257209+0000","last_active":"2024-05-08T04:42:55.259307+0000","last_peered":"2024-05-08T04:42:55.259307+0000","last_clean":"2024-05-08T04:42:55.259307+0000","last_became_active":"2024-05-08T04:42:53.256908+0000","last_became_peered":"2024-05-08T04:42:53.256908+0000","last_unstale":"2024-05-08T04:42:55.259307+0000","last_undegraded":"2024-05-08T04:42:55.259307+0000","last_fullsized":"2024-05-08T04:42:55.259307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259796+0000","last_change":"2024-05-08T04:42:54.263171+0000","last_active":"2024-05-08T04:42:55.259796+0000","last_peered":"2024-05-08T04:42:55.259796+0000","last_clean":"2024-05-08T04:42:55.259796+0000","last_became_active":"2024-05-08T04:42:54.262760+0000","last_became_peered":"2024-05-08T04:42:54.262760+0000","last_unstale":"2024-05-08T04:42:55.259796+0000","last_undegraded":"2024-05-08T04:42:55.259796+0000","last_fullsized":"2024-05-08T04:42:55.259796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.236833+0000","last_change":"2024-05-08T04:42:52.236833+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.236833+0000","last_undegraded":"2024-05-08T04:42:52.236833+0000","last_fullsized":"2024-05-08T04:42:52.236833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643515+0000","last_change":"2024-05-08T04:42:54.274640+0000","last_active":"2024-05-08T04:42:55.643515+0000","last_peered":"2024-05-08T04:42:55.643515+0000","last_clean":"2024-05-08T04:42:55.643515+0000","last_became_active":"2024-05-08T04:42:54.274209+0000","last_became_peered":"2024-05-08T04:42:54.274209+0000","last_unstale":"2024-05-08T04:42:55.643515+0000","last_undegraded":"2024-05-08T04:42:55.643515+0000","last_fullsized":"2024-05-08T04:42:55.643515+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:36.694738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259005+0000","last_change":"2024-05-08T04:42:53.265323+0000","last_active":"2024-05-08T04:42:55.259005+0000","last_peered":"2024-05-08T04:42:55.259005+0000","last_clean":"2024-05-08T04:42:55.259005+0000","last_became_active":"2024-05-08T04:42:53.265031+0000","last_became_peered":"2024-05-08T04:42:53.265031+0000","last_unstale":"2024-05-08T04:42:55.259005+0000","last_undegraded":"2024-05-08T04:42:55.259005+0000","last_fullsized":"2024-05-08T04:42:55.259005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:19.518749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258582+0000","last_change":"2024-05-08T04:42:54.263167+0000","last_active":"2024-05-08T04:42:55.258582+0000","last_peered":"2024-05-08T04:42:55.258582+0000","last_clean":"2024-05-08T04:42:55.258582+0000","last_became_active":"2024-05-08T04:42:54.262842+0000","last_became_peered":"2024-05-08T04:42:54.262842+0000","last_unstale":"2024-05-08T04:42:55.258582+0000","last_undegraded":"2024-05-08T04:42:55.258582+0000","last_fullsized":"2024-05-08T04:42:55.258582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643593+0000","last_change":"2024-05-08T04:42:53.263261+0000","last_active":"2024-05-08T04:42:55.643593+0000","last_peered":"2024-05-08T04:42:55.643593+0000","last_clean":"2024-05-08T04:42:55.643593+0000","last_became_active":"2024-05-08T04:42:53.262219+0000","last_became_peered":"2024-05-08T04:42:53.262219+0000","last_unstale":"2024-05-08T04:42:55.643593+0000","last_undegraded":"2024-05-08T04:42:55.643593+0000","last_fullsized":"2024-05-08T04:42:55.643593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:20.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260268+0000","last_change":"2024-05-08T04:42:54.269625+0000","last_active":"2024-05-08T04:42:55.260268+0000","last_peered":"2024-05-08T04:42:55.260268+0000","last_clean":"2024-05-08T04:42:55.260268+0000","last_became_active":"2024-05-08T04:42:54.269308+0000","last_became_peered":"2024-05-08T04:42:54.269308+0000","last_unstale":"2024-05-08T04:42:55.260268+0000","last_undegraded":"2024-05-08T04:42:55.260268+0000","last_fullsized":"2024-05-08T04:42:55.260268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.615548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234489+0000","last_change":"2024-05-08T04:42:52.234489+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234489+0000","last_undegraded":"2024-05-08T04:42:52.234489+0000","last_fullsized":"2024-05-08T04:42:52.234489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261712+0000","last_change":"2024-05-08T04:42:54.284605+0000","last_active":"2024-05-08T04:42:55.261712+0000","last_peered":"2024-05-08T04:42:55.261712+0000","last_clean":"2024-05-08T04:42:55.261712+0000","last_became_active":"2024-05-08T04:42:54.284331+0000","last_became_peered":"2024-05-08T04:42:54.284331+0000","last_unstale":"2024-05-08T04:42:55.261712+0000","last_undegraded":"2024-05-08T04:42:55.261712+0000","last_fullsized":"2024-05-08T04:42:55.261712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237857+0000","last_change":"2024-05-08T04:42:52.237857+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237857+0000","last_undegraded":"2024-05-08T04:42:52.237857+0000","last_fullsized":"2024-05-08T04:42:52.237857+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644694+0000","last_change":"2024-05-08T04:42:53.257288+0000","last_active":"2024-05-08T04:42:55.644694+0000","last_peered":"2024-05-08T04:42:55.644694+0000","last_clean":"2024-05-08T04:42:55.644694+0000","last_became_active":"2024-05-08T04:42:53.257014+0000","last_became_peered":"2024-05-08T04:42:53.257014+0000","last_unstale":"2024-05-08T04:42:55.644694+0000","last_undegraded":"2024-05-08T04:42:55.644694+0000","last_fullsized":"2024-05-08T04:42:55.644694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:27.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259886+0000","last_change":"2024-05-08T04:42:54.259828+0000","last_active":"2024-05-08T04:42:55.259886+0000","last_peered":"2024-05-08T04:42:55.259886+0000","last_clean":"2024-05-08T04:42:55.259886+0000","last_became_active":"2024-05-08T04:42:54.259109+0000","last_became_peered":"2024-05-08T04:42:54.259109+0000","last_unstale":"2024-05-08T04:42:55.259886+0000","last_undegraded":"2024-05-08T04:42:55.259886+0000","last_fullsized":"2024-05-08T04:42:55.259886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644648+0000","last_change":"2024-05-08T04:42:53.264438+0000","last_active":"2024-05-08T04:42:55.644648+0000","last_peered":"2024-05-08T04:42:55.644648+0000","last_clean":"2024-05-08T04:42:55.644648+0000","last_became_active":"2024-05-08T04:42:53.264131+0000","last_became_peered":"2024-05-08T04:42:53.264131+0000","last_unstale":"2024-05-08T04:42:55.644648+0000","last_undegraded":"2024-05-08T04:42:55.644648+0000","last_fullsized":"2024-05-08T04:42:55.644648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:15.699103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644435+0000","last_change":"2024-05-08T04:42:53.255665+0000","last_active":"2024-05-08T04:42:55.644435+0000","last_peered":"2024-05-08T04:42:55.644435+0000","last_clean":"2024-05-08T04:42:55.644435+0000","last_became_active":"2024-05-08T04:42:53.255168+0000","last_became_peered":"2024-05-08T04:42:53.255168+0000","last_unstale":"2024-05-08T04:42:55.644435+0000","last_undegraded":"2024-05-08T04:42:55.644435+0000","last_fullsized":"2024-05-08T04:42:55.644435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261312+0000","last_change":"2024-05-08T04:42:53.259031+0000","last_active":"2024-05-08T04:42:55.261312+0000","last_peered":"2024-05-08T04:42:55.261312+0000","last_clean":"2024-05-08T04:42:55.261312+0000","last_became_active":"2024-05-08T04:42:53.258734+0000","last_became_peered":"2024-05-08T04:42:53.258734+0000","last_unstale":"2024-05-08T04:42:55.261312+0000","last_undegraded":"2024-05-08T04:42:55.261312+0000","last_fullsized":"2024-05-08T04:42:55.261312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:07.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233237+0000","last_change":"2024-05-08T04:42:52.233237+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233237+0000","last_undegraded":"2024-05-08T04:42:52.233237+0000","last_fullsized":"2024-05-08T04:42:52.233237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645310+0000","last_change":"2024-05-08T04:42:53.255078+0000","last_active":"2024-05-08T04:42:55.645310+0000","last_peered":"2024-05-08T04:42:55.645310+0000","last_clean":"2024-05-08T04:42:55.645310+0000","last_became_active":"2024-05-08T04:42:53.254350+0000","last_became_peered":"2024-05-08T04:42:53.254350+0000","last_unstale":"2024-05-08T04:42:55.645310+0000","last_undegraded":"2024-05-08T04:42:55.645310+0000","last_fullsized":"2024-05-08T04:42:55.645310+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:59.504019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645155+0000","last_change":"2024-05-08T04:42:54.284624+0000","last_active":"2024-05-08T04:42:55.645155+0000","last_peered":"2024-05-08T04:42:55.645155+0000","last_clean":"2024-05-08T04:42:55.645155+0000","last_became_active":"2024-05-08T04:42:54.284368+0000","last_became_peered":"2024-05-08T04:42:54.284368+0000","last_unstale":"2024-05-08T04:42:55.645155+0000","last_undegraded":"2024-05-08T04:42:55.645155+0000","last_fullsized":"2024-05-08T04:42:55.645155+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233255+0000","last_change":"2024-05-08T04:42:52.233255+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233255+0000","last_undegraded":"2024-05-08T04:42:52.233255+0000","last_fullsized":"2024-05-08T04:42:52.233255+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543959+0000","last_change":"2024-05-08T04:42:54.268528+0000","last_active":"2024-05-08T04:42:55.543959+0000","last_peered":"2024-05-08T04:42:55.543959+0000","last_clean":"2024-05-08T04:42:55.543959+0000","last_became_active":"2024-05-08T04:42:54.268247+0000","last_became_peered":"2024-05-08T04:42:54.268247+0000","last_unstale":"2024-05-08T04:42:55.543959+0000","last_undegraded":"2024-05-08T04:42:55.543959+0000","last_fullsized":"2024-05-08T04:42:55.543959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:20.235769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261410+0000","last_change":"2024-05-08T04:42:53.260958+0000","last_active":"2024-05-08T04:42:55.261410+0000","last_peered":"2024-05-08T04:42:55.261410+0000","last_clean":"2024-05-08T04:42:55.261410+0000","last_became_active":"2024-05-08T04:42:53.260621+0000","last_became_peered":"2024-05-08T04:42:53.260621+0000","last_unstale":"2024-05-08T04:42:55.261410+0000","last_undegraded":"2024-05-08T04:42:55.261410+0000","last_fullsized":"2024-05-08T04:42:55.261410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:14.035376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544039+0000","last_change":"2024-05-08T04:42:54.259461+0000","last_active":"2024-05-08T04:42:55.544039+0000","last_peered":"2024-05-08T04:42:55.544039+0000","last_clean":"2024-05-08T04:42:55.544039+0000","last_became_active":"2024-05-08T04:42:54.259005+0000","last_became_peered":"2024-05-08T04:42:54.259005+0000","last_unstale":"2024-05-08T04:42:55.544039+0000","last_undegraded":"2024-05-08T04:42:55.544039+0000","last_fullsized":"2024-05-08T04:42:55.544039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:07.448395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258867+0000","last_change":"2024-05-08T04:42:53.258018+0000","last_active":"2024-05-08T04:42:55.258867+0000","last_peered":"2024-05-08T04:42:55.258867+0000","last_clean":"2024-05-08T04:42:55.258867+0000","last_became_active":"2024-05-08T04:42:53.257357+0000","last_became_peered":"2024-05-08T04:42:53.257357+0000","last_unstale":"2024-05-08T04:42:55.258867+0000","last_undegraded":"2024-05-08T04:42:55.258867+0000","last_fullsized":"2024-05-08T04:42:55.258867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543555+0000","last_change":"2024-05-08T04:42:54.258649+0000","last_active":"2024-05-08T04:42:55.543555+0000","last_peered":"2024-05-08T04:42:55.543555+0000","last_clean":"2024-05-08T04:42:55.543555+0000","last_became_active":"2024-05-08T04:42:54.258367+0000","last_became_peered":"2024-05-08T04:42:54.258367+0000","last_unstale":"2024-05-08T04:42:55.543555+0000","last_undegraded":"2024-05-08T04:42:55.543555+0000","last_fullsized":"2024-05-08T04:42:55.543555+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644524+0000","last_change":"2024-05-08T04:42:53.260251+0000","last_active":"2024-05-08T04:42:55.644524+0000","last_peered":"2024-05-08T04:42:55.644524+0000","last_clean":"2024-05-08T04:42:55.644524+0000","last_became_active":"2024-05-08T04:42:53.259749+0000","last_became_peered":"2024-05-08T04:42:53.259749+0000","last_unstale":"2024-05-08T04:42:55.644524+0000","last_undegraded":"2024-05-08T04:42:55.644524+0000","last_fullsized":"2024-05-08T04:42:55.644524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644062+0000","last_change":"2024-05-08T04:42:54.270229+0000","last_active":"2024-05-08T04:42:55.644062+0000","last_peered":"2024-05-08T04:42:55.644062+0000","last_clean":"2024-05-08T04:42:55.644062+0000","last_became_active":"2024-05-08T04:42:54.267191+0000","last_became_peered":"2024-05-08T04:42:54.267191+0000","last_unstale":"2024-05-08T04:42:55.644062+0000","last_undegraded":"2024-05-08T04:42:55.644062+0000","last_fullsized":"2024-05-08T04:42:55.644062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:16.097496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.234678+0000","last_change":"2024-05-08T04:42:52.234678+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.234678+0000","last_undegraded":"2024-05-08T04:42:52.234678+0000","last_fullsized":"2024-05-08T04:42:52.234678+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543619+0000","last_change":"2024-05-08T04:42:54.270286+0000","last_active":"2024-05-08T04:42:55.543619+0000","last_peered":"2024-05-08T04:42:55.543619+0000","last_clean":"2024-05-08T04:42:55.543619+0000","last_became_active":"2024-05-08T04:42:54.270019+0000","last_became_peered":"2024-05-08T04:42:54.270019+0000","last_unstale":"2024-05-08T04:42:55.543619+0000","last_undegraded":"2024-05-08T04:42:55.543619+0000","last_fullsized":"2024-05-08T04:42:55.543619+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644073+0000","last_change":"2024-05-08T04:42:53.259343+0000","last_active":"2024-05-08T04:42:55.644073+0000","last_peered":"2024-05-08T04:42:55.644073+0000","last_clean":"2024-05-08T04:42:55.644073+0000","last_became_active":"2024-05-08T04:42:53.258967+0000","last_became_peered":"2024-05-08T04:42:53.258967+0000","last_unstale":"2024-05-08T04:42:55.644073+0000","last_undegraded":"2024-05-08T04:42:55.644073+0000","last_fullsized":"2024-05-08T04:42:55.644073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643660+0000","last_change":"2024-05-08T04:42:53.253264+0000","last_active":"2024-05-08T04:42:55.643660+0000","last_peered":"2024-05-08T04:42:55.643660+0000","last_clean":"2024-05-08T04:42:55.643660+0000","last_became_active":"2024-05-08T04:42:53.252716+0000","last_became_peered":"2024-05-08T04:42:53.252716+0000","last_unstale":"2024-05-08T04:42:55.643660+0000","last_undegraded":"2024-05-08T04:42:55.643660+0000","last_fullsized":"2024-05-08T04:42:55.643660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:56.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275320+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.275003+0000","last_became_peered":"2024-05-08T04:42:54.275003+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237599+0000","last_change":"2024-05-08T04:42:52.237599+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237599+0000","last_undegraded":"2024-05-08T04:42:52.237599+0000","last_fullsized":"2024-05-08T04:42:52.237599+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645304+0000","last_change":"2024-05-08T04:42:54.282790+0000","last_active":"2024-05-08T04:42:55.645304+0000","last_peered":"2024-05-08T04:42:55.645304+0000","last_clean":"2024-05-08T04:42:55.645304+0000","last_became_active":"2024-05-08T04:42:54.282523+0000","last_became_peered":"2024-05-08T04:42:54.282523+0000","last_unstale":"2024-05-08T04:42:55.645304+0000","last_undegraded":"2024-05-08T04:42:55.645304+0000","last_fullsized":"2024-05-08T04:42:55.645304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.467995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261546+0000","last_change":"2024-05-08T04:42:53.252707+0000","last_active":"2024-05-08T04:42:55.261546+0000","last_peered":"2024-05-08T04:42:55.261546+0000","last_clean":"2024-05-08T04:42:55.261546+0000","last_became_active":"2024-05-08T04:42:53.250660+0000","last_became_peered":"2024-05-08T04:42:53.250660+0000","last_unstale":"2024-05-08T04:42:55.261546+0000","last_undegraded":"2024-05-08T04:42:55.261546+0000","last_fullsized":"2024-05-08T04:42:55.261546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:13.437315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644517+0000","last_change":"2024-05-08T04:42:54.277076+0000","last_active":"2024-05-08T04:42:55.644517+0000","last_peered":"2024-05-08T04:42:55.644517+0000","last_clean":"2024-05-08T04:42:55.644517+0000","last_became_active":"2024-05-08T04:42:54.276805+0000","last_became_peered":"2024-05-08T04:42:54.276805+0000","last_unstale":"2024-05-08T04:42:55.644517+0000","last_undegraded":"2024-05-08T04:42:55.644517+0000","last_fullsized":"2024-05-08T04:42:55.644517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:39.976473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644732+0000","last_change":"2024-05-08T04:42:53.252678+0000","last_active":"2024-05-08T04:42:55.644732+0000","last_peered":"2024-05-08T04:42:55.644732+0000","last_clean":"2024-05-08T04:42:55.644732+0000","last_became_active":"2024-05-08T04:42:53.252426+0000","last_became_peered":"2024-05-08T04:42:53.252426+0000","last_unstale":"2024-05-08T04:42:55.644732+0000","last_undegraded":"2024-05-08T04:42:55.644732+0000","last_fullsized":"2024-05-08T04:42:55.644732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:50.935297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275599+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.274979+0000","last_became_peered":"2024-05-08T04:42:54.274979+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:27.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.237194+0000","last_change":"2024-05-08T04:42:52.237194+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.237194+0000","last_undegraded":"2024-05-08T04:42:52.237194+0000","last_fullsized":"2024-05-08T04:42:52.237194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259403+0000","last_change":"2024-05-08T04:42:54.250195+0000","last_active":"2024-05-08T04:42:55.259403+0000","last_peered":"2024-05-08T04:42:55.259403+0000","last_clean":"2024-05-08T04:42:55.259403+0000","last_became_active":"2024-05-08T04:42:54.249297+0000","last_became_peered":"2024-05-08T04:42:54.249297+0000","last_unstale":"2024-05-08T04:42:55.259403+0000","last_undegraded":"2024-05-08T04:42:55.259403+0000","last_fullsized":"2024-05-08T04:42:55.259403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259421+0000","last_change":"2024-05-08T04:42:53.259450+0000","last_active":"2024-05-08T04:42:55.259421+0000","last_peered":"2024-05-08T04:42:55.259421+0000","last_clean":"2024-05-08T04:42:55.259421+0000","last_became_active":"2024-05-08T04:42:53.259139+0000","last_became_peered":"2024-05-08T04:42:53.259139+0000","last_unstale":"2024-05-08T04:42:55.259421+0000","last_undegraded":"2024-05-08T04:42:55.259421+0000","last_fullsized":"2024-05-08T04:42:55.259421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260413+0000","last_change":"2024-05-08T04:42:54.263700+0000","last_active":"2024-05-08T04:42:55.260413+0000","last_peered":"2024-05-08T04:42:55.260413+0000","last_clean":"2024-05-08T04:42:55.260413+0000","last_became_active":"2024-05-08T04:42:54.263395+0000","last_became_peered":"2024-05-08T04:42:54.263395+0000","last_unstale":"2024-05-08T04:42:55.260413+0000","last_undegraded":"2024-05-08T04:42:55.260413+0000","last_fullsized":"2024-05-08T04:42:55.260413+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:40.094631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259409+0000","last_change":"2024-05-08T04:42:53.258649+0000","last_active":"2024-05-08T04:42:55.259409+0000","last_peered":"2024-05-08T04:42:55.259409+0000","last_clean":"2024-05-08T04:42:55.259409+0000","last_became_active":"2024-05-08T04:42:53.258348+0000","last_became_peered":"2024-05-08T04:42:53.258348+0000","last_unstale":"2024-05-08T04:42:55.259409+0000","last_undegraded":"2024-05-08T04:42:55.259409+0000","last_fullsized":"2024-05-08T04:42:55.259409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:42.312320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543824+0000","last_change":"2024-05-08T04:42:54.266324+0000","last_active":"2024-05-08T04:42:55.543824+0000","last_peered":"2024-05-08T04:42:55.543824+0000","last_clean":"2024-05-08T04:42:55.543824+0000","last_became_active":"2024-05-08T04:42:54.265475+0000","last_became_peered":"2024-05-08T04:42:54.265475+0000","last_unstale":"2024-05-08T04:42:55.543824+0000","last_undegraded":"2024-05-08T04:42:55.543824+0000","last_fullsized":"2024-05-08T04:42:55.543824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-05-08T04:42:52.233276+0000","last_change":"2024-05-08T04:42:52.233276+0000","last_active":"2024-05-08T04:42:52.223155+0000","last_peered":"2024-05-08T04:42:52.223155+0000","last_clean":"2024-05-08T04:42:52.223155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:52.233276+0000","last_undegraded":"2024-05-08T04:42:52.233276+0000","last_fullsized":"2024-05-08T04:42:52.233276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643920+0000","last_change":"2024-05-08T04:42:54.270703+0000","last_active":"2024-05-08T04:42:55.643920+0000","last_peered":"2024-05-08T04:42:55.643920+0000","last_clean":"2024-05-08T04:42:55.643920+0000","last_became_active":"2024-05-08T04:42:54.265604+0000","last_became_peered":"2024-05-08T04:42:54.265604+0000","last_unstale":"2024-05-08T04:42:55.643920+0000","last_undegraded":"2024-05-08T04:42:55.643920+0000","last_fullsized":"2024-05-08T04:42:55.643920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:21.915088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643694+0000","last_change":"2024-05-08T04:42:53.253094+0000","last_active":"2024-05-08T04:42:55.643694+0000","last_peered":"2024-05-08T04:42:55.643694+0000","last_clean":"2024-05-08T04:42:55.643694+0000","last_became_active":"2024-05-08T04:42:53.252291+0000","last_became_peered":"2024-05-08T04:42:53.252291+0000","last_unstale":"2024-05-08T04:42:55.643694+0000","last_undegraded":"2024-05-08T04:42:55.643694+0000","last_fullsized":"2024-05-08T04:42:55.643694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:59.366123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T04:42:53.229408+0000","last_change":"2024-05-08T04:42:53.229408+0000","last_active":"2024-05-08T04:42:53.229408+0000","last_peered":"2024-05-08T04:42:53.229408+0000","last_clean":"2024-05-08T04:42:53.229408+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T04:42:53.229408+0000","last_undegraded":"2024-05-08T04:42:53.229408+0000","last_fullsized":"2024-05-08T04:42:53.229408+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258784+0000","last_change":"2024-05-08T04:42:53.257287+0000","last_active":"2024-05-08T04:42:55.258784+0000","last_peered":"2024-05-08T04:42:55.258784+0000","last_clean":"2024-05-08T04:42:55.258784+0000","last_became_active":"2024-05-08T04:42:53.256851+0000","last_became_peered":"2024-05-08T04:42:53.256851+0000","last_unstale":"2024-05-08T04:42:55.258784+0000","last_undegraded":"2024-05-08T04:42:55.258784+0000","last_fullsized":"2024-05-08T04:42:55.258784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543931+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:55.543931+0000","last_peered":"2024-05-08T04:42:55.543931+0000","last_clean":"2024-05-08T04:42:55.543931+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:55.543931+0000","last_undegraded":"2024-05-08T04:42:55.543931+0000","last_fullsized":"2024-05-08T04:42:55.543931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":1198126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":607212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":607746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":587540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":607266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-05-08T04:42:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.234430+0000', 'last_change': '2024-05-08T04:42:52.234430+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.234430+0000', 'last_undegraded': '2024-05-08T04:42:52.234430+0000', 'last_fullsized': '2024-05-08T04:42:52.234430+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.234541+0000', 'last_change': '2024-05-08T04:42:52.234541+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.234541+0000', 'last_undegraded': '2024-05-08T04:42:52.234541+0000', 'last_fullsized': '2024-05-08T04:42:52.234541+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.236833+0000', 'last_change': '2024-05-08T04:42:52.236833+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.236833+0000', 'last_undegraded': '2024-05-08T04:42:52.236833+0000', 'last_fullsized': '2024-05-08T04:42:52.236833+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.234489+0000', 'last_change': '2024-05-08T04:42:52.234489+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.234489+0000', 'last_undegraded': '2024-05-08T04:42:52.234489+0000', 'last_fullsized': '2024-05-08T04:42:52.234489+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.237857+0000', 'last_change': '2024-05-08T04:42:52.237857+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.237857+0000', 'last_undegraded': '2024-05-08T04:42:52.237857+0000', 'last_fullsized': '2024-05-08T04:42:52.237857+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-05-08T04:42:59.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.233237+0000', 'last_change': '2024-05-08T04:42:52.233237+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.233237+0000', 'last_undegraded': '2024-05-08T04:42:52.233237+0000', 'last_fullsized': '2024-05-08T04:42:52.233237+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.233255+0000', 'last_change': '2024-05-08T04:42:52.233255+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.233255+0000', 'last_undegraded': '2024-05-08T04:42:52.233255+0000', 'last_fullsized': '2024-05-08T04:42:52.233255+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.234678+0000', 'last_change': '2024-05-08T04:42:52.234678+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.234678+0000', 'last_undegraded': '2024-05-08T04:42:52.234678+0000', 'last_fullsized': '2024-05-08T04:42:52.234678+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2024-05-08T04:42:59.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.237599+0000', 'last_change': '2024-05-08T04:42:52.237599+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.237599+0000', 'last_undegraded': '2024-05-08T04:42:52.237599+0000', 'last_fullsized': '2024-05-08T04:42:52.237599+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.237194+0000', 'last_change': '2024-05-08T04:42:52.237194+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.237194+0000', 'last_undegraded': '2024-05-08T04:42:52.237194+0000', 'last_fullsized': '2024-05-08T04:42:52.237194+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-05-08T04:42:52.233276+0000', 'last_change': '2024-05-08T04:42:52.233276+0000', 'last_active': '2024-05-08T04:42:52.223155+0000', 'last_peered': '2024-05-08T04:42:52.223155+0000', 'last_clean': '2024-05-08T04:42:52.223155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:52.233276+0000', 'last_undegraded': '2024-05-08T04:42:52.233276+0000', 'last_fullsized': '2024-05-08T04:42:52.233276+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:52.223155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2024-05-08T04:42:59.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T04:42:53.229408+0000', 'last_change': '2024-05-08T04:42:53.229408+0000', 'last_active': '2024-05-08T04:42:53.229408+0000', 'last_peered': '2024-05-08T04:42:53.229408+0000', 'last_clean': '2024-05-08T04:42:53.229408+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T04:42:53.229408+0000', 'last_undegraded': '2024-05-08T04:42:53.229408+0000', 'last_fullsized': '2024-05-08T04:42:53.229408+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'last_clean_scrub_stamp': '2024-05-08T04:42:53.229408+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T04:42:59.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T04:42:59.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.259+0000 7fb02fe00640 1 Processor -- start 2024-05-08T04:42:59.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.259+0000 7fb02fe00640 1 -- start start 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb03015da10 0x7fb030071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb03006ac90 0x7fb03006b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb030072400 con 0x7fb0300d0fb0 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0300b3de0 con 0x7fb03015da10 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03006b620 con 0x7fb03006ac90 2024-05-08T04:42:59.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb03015da10 0x7fb030071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb03015da10 0x7fb030071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40580/0 (socket says 172.21.15.164:40580) 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 -- 172.21.15.164:0/3645194336 learned_addr learned my addr 172.21.15.164:0/3645194336 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02f400640 1 --2- 172.21.15.164:0/3645194336 >> v2:172.21.15.172:3300/0 conn(0x7fb03006ac90 0x7fb03006b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 -- 172.21.15.164:0/3645194336 >> v2:172.21.15.172:3300/0 conn(0x7fb03006ac90 msgr2=0x7fb03006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3645194336 >> v2:172.21.15.172:3300/0 conn(0x7fb03006ac90 0x7fb03006b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 -- 172.21.15.164:0/3645194336 >> v1:172.21.15.172:6789/0 conn(0x7fb0300d0fb0 legacy=0x7fb0300b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:42:59.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.263+0000 7fb02ea00640 1 -- 172.21.15.164:0/3645194336 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300d04d0 con 0x7fb03015da10 2024-05-08T04:42:59.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3645194336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03015da10 0x7fb030071e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb0300710d0 tx=0x7fb02001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b02a2a4283575d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02e000640 1 -- 172.21.15.164:0/3645194336 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200b1070 con 0x7fb03015da10 2024-05-08T04:42:59.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02e000640 1 -- 172.21.15.164:0/3645194336 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0200ac630 con 0x7fb03015da10 2024-05-08T04:42:59.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02e000640 1 -- 172.21.15.164:0/3645194336 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200a8050 con 0x7fb03015da10 2024-05-08T04:42:59.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- 172.21.15.164:0/3645194336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03015da10 msgr2=0x7fb030071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3645194336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03015da10 0x7fb030071e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb0300710d0 tx=0x7fb02001c4e0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- 172.21.15.164:0/3645194336 shutdown_connections 2024-05-08T04:42:59.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3645194336 >> v2:172.21.15.172:3300/0 conn(0x7fb03006ac90 0x7fb03006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3645194336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03015da10 0x7fb030071e20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- 172.21.15.164:0/3645194336 >> 172.21.15.164:0/3645194336 conn(0x7fb030066fa0 msgr2=0x7fb03006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- 172.21.15.164:0/3645194336 shutdown_connections 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- 172.21.15.164:0/3645194336 wait complete. 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 Processor -- start 2024-05-08T04:42:59.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- start start 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 0x7fb030151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 0x7fb03014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300ae220 con 0x7fb0300d0fb0 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300ae0a0 con 0x7fb03015da10 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.267+0000 7fb02fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300ae3a0 con 0x7fb03006ac90 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40584/0 (socket says 172.21.15.164:40584) 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 -- 172.21.15.164:0/3590480209 learned_addr learned my addr 172.21.15.164:0/3590480209 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02f400640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 0x7fb03014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 0x7fb030151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 -- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 msgr2=0x7fb030151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 0x7fb030151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 -- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 msgr2=0x7fb03014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 0x7fb03014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 -- 172.21.15.164:0/3590480209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03014d500 con 0x7fb0300d0fb0 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 0x7fb030151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02f400640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 0x7fb03014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:42:59.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb027e00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c450 tx=0x7fb01800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018012830 con 0x7fb0300d0fb0 2024-05-08T04:42:59.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb018017070 con 0x7fb0300d0fb0 2024-05-08T04:42:59.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018012c30 con 0x7fb0300d0fb0 2024-05-08T04:42:59.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03014eed0 con 0x7fb0300d0fb0 2024-05-08T04:42:59.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb030152060 con 0x7fb0300d0fb0 2024-05-08T04:42:59.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.271+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03004ef30 con 0x7fb0300d0fb0 2024-05-08T04:42:59.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.275+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fb01801c460 con 0x7fb0300d0fb0 2024-05-08T04:42:59.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.275+0000 7fb026a00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 0x7fb00c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.275+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb0180836f0 con 0x7fb0300d0fb0 2024-05-08T04:42:59.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.275+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 0x7fb00c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.275+0000 7fb02ea00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 0x7fb00c069d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb0200ae000 tx=0x7fb020008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.279+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0180561b0 con 0x7fb0300d0fb0 2024-05-08T04:42:59.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb030153800 con 0x7fb0300d0fb0 2024-05-08T04:42:59.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb026a00640 1 -- 172.21.15.164:0/3590480209 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1595 (secure 0 0 0) 0x7fb018055ab0 con 0x7fb0300d0fb0 2024-05-08T04:42:59.488 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:59.489 INFO:teuthology.orchestra.run.smithi164.stdout:{"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1715143096,"num_in_osds":8,"osd_in_since":1715143086,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":75},{"state_name":"creating+peering","count":11},{"state_name":"unknown","count":11}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592778,"bytes_used":267743232,"bytes_avail":772826370048,"bytes_total":773094113280,"unknown_pgs_ratio":0.11340206116437912,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2728,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":7899}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-08T04:40:55.460197+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T04:42:59.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 msgr2=0x7fb00c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 0x7fb00c069d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb0200ae000 tx=0x7fb020008040 comp rx=0 tx=0).stop 2024-05-08T04:42:59.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 msgr2=0x7fb03014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c450 tx=0x7fb01800c910 comp rx=0 tx=0).stop 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 shutdown_connections 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb00c067860 0x7fb00c069d20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03015da10 0x7fb03014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300d0fb0 0x7fb03014f6d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 --2- 172.21.15.164:0/3590480209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03006ac90 0x7fb030151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 >> 172.21.15.164:0/3590480209 conn(0x7fb030066fa0 msgr2=0x7fb0300b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 shutdown_connections 2024-05-08T04:42:59.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.487+0000 7fb02fe00640 1 -- 172.21.15.164:0/3590480209 wait complete. 2024-05-08T04:42:59.510 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T04:42:59.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:42:59.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.643+0000 7f077f200640 1 Processor -- start 2024-05-08T04:42:59.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.643+0000 7f077f200640 1 -- start start 2024-05-08T04:42:59.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f077815ab60 0x7f0778072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0778072640 con 0x7f07780cfda0 2024-05-08T04:42:59.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077806c770 con 0x7f07780b26c0 2024-05-08T04:42:59.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077806ca50 con 0x7f077815ab60 2024-05-08T04:42:59.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077d200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f07780cfda0 0x7f07780cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37280/0 (socket says 172.21.15.164:37280) 2024-05-08T04:42:59.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f077d200640 1 -- 172.21.15.164:0/2299083567 learned_addr learned my addr 172.21.15.164:0/2299083567 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:42:59.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0777e00640 1 --2- 172.21.15.164:0/2299083567 >> v2:172.21.15.172:3300/0 conn(0x7f077815ab60 0x7f0778072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0776a00640 1 -- 172.21.15.164:0/2299083567 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350393062 0 0) 0x7f0778072640 con 0x7f07780cfda0 2024-05-08T04:42:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0776a00640 1 -- 172.21.15.164:0/2299083567 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f075c005740 con 0x7f07780cfda0 2024-05-08T04:42:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0777e00640 1 -- 172.21.15.164:0/2299083567 >> v1:172.21.15.172:6789/0 conn(0x7f07780b26c0 legacy=0x7f07780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0777e00640 1 -- 172.21.15.164:0/2299083567 >> v1:172.21.15.164:6789/0 conn(0x7f07780cfda0 legacy=0x7f07780cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.647+0000 7f0777e00640 1 -- 172.21.15.164:0/2299083567 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077806cd30 con 0x7f077815ab60 2024-05-08T04:42:59.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0777e00640 1 --2- 172.21.15.164:0/2299083567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077815ab60 0x7f0778072100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07680097c0 tx=0x7f076801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a633216834eacc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0776a00640 1 -- 172.21.15.164:0/2299083567 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680b0070 con 0x7f077815ab60 2024-05-08T04:42:59.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0776a00640 1 -- 172.21.15.164:0/2299083567 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07680ab600 con 0x7f077815ab60 2024-05-08T04:42:59.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0776a00640 1 -- 172.21.15.164:0/2299083567 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680ab9d0 con 0x7f077815ab60 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- 172.21.15.164:0/2299083567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077815ab60 msgr2=0x7f0778072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 --2- 172.21.15.164:0/2299083567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077815ab60 0x7f0778072100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f07680097c0 tx=0x7f076801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- 172.21.15.164:0/2299083567 shutdown_connections 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 --2- 172.21.15.164:0/2299083567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077815ab60 0x7f0778072100 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- 172.21.15.164:0/2299083567 >> 172.21.15.164:0/2299083567 conn(0x7f077806ac90 msgr2=0x7f077806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- 172.21.15.164:0/2299083567 shutdown_connections 2024-05-08T04:42:59.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- 172.21.15.164:0/2299083567 wait complete. 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 Processor -- start 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- start start 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 0x7f0778159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ab60 0x7f077815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780aeeb0 con 0x7f07780cfda0 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780aed30 con 0x7f07780b26c0 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f077f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07780cc1b0 con 0x7f077815ab60 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0777400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.651+0000 7f0777400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40596/0 (socket says 172.21.15.164:40596) 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 -- 172.21.15.164:0/4048287185 learned_addr learned my addr 172.21.15.164:0/4048287185 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:42:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777e00640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 0x7f0778159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f077d200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ab60 0x7f077815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 -- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ab60 msgr2=0x7f077815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ab60 0x7f077815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 -- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 msgr2=0x7f0778159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 0x7f0778159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 -- 172.21.15.164:0/4048287185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077815a270 con 0x7f07780cfda0 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777e00640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 0x7f0778159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f0777400640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f076400c420 tx=0x7f076400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640052d0 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764005470 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764007260 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077814ebe0 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07781476f0 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0778069d60 con 0x7f07780cfda0 2024-05-08T04:42:59.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.655+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f0764007400 con 0x7f07780cfda0 2024-05-08T04:42:59.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.659+0000 7f076fe00640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 0x7f0758069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:42:59.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.659+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f07640856e0 con 0x7f07780cfda0 2024-05-08T04:42:59.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.659+0000 7f0777e00640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 0x7f0758069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:42:59.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.659+0000 7f0777e00640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 0x7f0758069d20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07680ad000 tx=0x7f07680a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:42:59.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.663+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07640551a0 con 0x7f07780cfda0 2024-05-08T04:42:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.815+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f077806d9a0 con 0x7f0758067860 2024-05-08T04:42:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.819+0000 7f076fe00640 1 -- 172.21.15.164:0/4048287185 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7f077806d9a0 con 0x7f0758067860 2024-05-08T04:42:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:42:59.826 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:42:59.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 msgr2=0x7f0758069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 0x7f0758069d20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07680ad000 tx=0x7f07680a6000 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 msgr2=0x7f0778146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f076400c420 tx=0x7f076400c8e0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 shutdown_connections 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0758067860 0x7f0758069d20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077815ab60 0x7f077815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780cfda0 0x7f0778146930 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 --2- 172.21.15.164:0/4048287185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07780b26c0 0x7f0778159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 >> 172.21.15.164:0/4048287185 conn(0x7f077806ac90 msgr2=0x7f077806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 shutdown_connections 2024-05-08T04:42:59.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:42:59.823+0000 7f077f200640 1 -- 172.21.15.164:0/4048287185 wait complete. 2024-05-08T04:42:59.845 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-05-08T04:42:59.484668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261572,"kb_used_data":12104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713148,"statfs":{"total":773094113280,"available":772826263552,"internally_reserved":0,"allocated":12394496,"data_stored":6042692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.246056"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645115+0000","last_change":"2024-05-08T04:42:54.284520+0000","last_active":"2024-05-08T04:42:55.645115+0000","last_peered":"2024-05-08T04:42:55.645115+0000","last_clean":"2024-05-08T04:42:55.645115+0000","last_became_active":"2024-05-08T04:42:54.283787+0000","last_became_peered":"2024-05-08T04:42:54.283787+0000","last_unstale":"2024-05-08T04:42:55.645115+0000","last_undegraded":"2024-05-08T04:42:55.645115+0000","last_fullsized":"2024-05-08T04:42:55.645115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:31.361703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260638+0000","last_change":"2024-05-08T04:42:54.267059+0000","last_active":"2024-05-08T04:42:55.260638+0000","last_peered":"2024-05-08T04:42:55.260638+0000","last_clean":"2024-05-08T04:42:55.260638+0000","last_became_active":"2024-05-08T04:42:54.266748+0000","last_became_peered":"2024-05-08T04:42:54.266748+0000","last_unstale":"2024-05-08T04:42:55.260638+0000","last_undegraded":"2024-05-08T04:42:55.260638+0000","last_fullsized":"2024-05-08T04:42:55.260638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:57.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543833+0000","last_change":"2024-05-08T04:42:54.259254+0000","last_active":"2024-05-08T04:42:55.543833+0000","last_peered":"2024-05-08T04:42:55.543833+0000","last_clean":"2024-05-08T04:42:55.543833+0000","last_became_active":"2024-05-08T04:42:54.258620+0000","last_became_peered":"2024-05-08T04:42:54.258620+0000","last_unstale":"2024-05-08T04:42:55.543833+0000","last_undegraded":"2024-05-08T04:42:55.543833+0000","last_fullsized":"2024-05-08T04:42:55.543833+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644261+0000","last_change":"2024-05-08T04:42:54.276298+0000","last_active":"2024-05-08T04:42:55.644261+0000","last_peered":"2024-05-08T04:42:55.644261+0000","last_clean":"2024-05-08T04:42:55.644261+0000","last_became_active":"2024-05-08T04:42:54.273697+0000","last_became_peered":"2024-05-08T04:42:54.273697+0000","last_unstale":"2024-05-08T04:42:55.644261+0000","last_undegraded":"2024-05-08T04:42:55.644261+0000","last_fullsized":"2024-05-08T04:42:55.644261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:45.521599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644231+0000","last_change":"2024-05-08T04:42:54.271374+0000","last_active":"2024-05-08T04:42:55.644231+0000","last_peered":"2024-05-08T04:42:55.644231+0000","last_clean":"2024-05-08T04:42:55.644231+0000","last_became_active":"2024-05-08T04:42:54.267689+0000","last_became_peered":"2024-05-08T04:42:54.267689+0000","last_unstale":"2024-05-08T04:42:55.644231+0000","last_undegraded":"2024-05-08T04:42:55.644231+0000","last_fullsized":"2024-05-08T04:42:55.644231+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:16.960439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644461+0000","last_change":"2024-05-08T04:42:54.259042+0000","last_active":"2024-05-08T04:42:55.644461+0000","last_peered":"2024-05-08T04:42:55.644461+0000","last_clean":"2024-05-08T04:42:55.644461+0000","last_became_active":"2024-05-08T04:42:54.258775+0000","last_became_peered":"2024-05-08T04:42:54.258775+0000","last_unstale":"2024-05-08T04:42:55.644461+0000","last_undegraded":"2024-05-08T04:42:55.644461+0000","last_fullsized":"2024-05-08T04:42:55.644461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:58.137321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258732+0000","last_change":"2024-05-08T04:42:54.258159+0000","last_active":"2024-05-08T04:42:55.258732+0000","last_peered":"2024-05-08T04:42:55.258732+0000","last_clean":"2024-05-08T04:42:55.258732+0000","last_became_active":"2024-05-08T04:42:54.257892+0000","last_became_peered":"2024-05-08T04:42:54.257892+0000","last_unstale":"2024-05-08T04:42:55.258732+0000","last_undegraded":"2024-05-08T04:42:55.258732+0000","last_fullsized":"2024-05-08T04:42:55.258732+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543525+0000","last_change":"2024-05-08T04:42:54.261073+0000","last_active":"2024-05-08T04:42:55.543525+0000","last_peered":"2024-05-08T04:42:55.543525+0000","last_clean":"2024-05-08T04:42:55.543525+0000","last_became_active":"2024-05-08T04:42:54.260783+0000","last_became_peered":"2024-05-08T04:42:54.260783+0000","last_unstale":"2024-05-08T04:42:55.543525+0000","last_undegraded":"2024-05-08T04:42:55.543525+0000","last_fullsized":"2024-05-08T04:42:55.543525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645257+0000","last_change":"2024-05-08T04:42:54.280775+0000","last_active":"2024-05-08T04:42:55.645257+0000","last_peered":"2024-05-08T04:42:55.645257+0000","last_clean":"2024-05-08T04:42:55.645257+0000","last_became_active":"2024-05-08T04:42:54.280501+0000","last_became_peered":"2024-05-08T04:42:54.280501+0000","last_unstale":"2024-05-08T04:42:55.645257+0000","last_undegraded":"2024-05-08T04:42:55.645257+0000","last_fullsized":"2024-05-08T04:42:55.645257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:25.588352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645180+0000","last_change":"2024-05-08T04:42:54.279039+0000","last_active":"2024-05-08T04:42:55.645180+0000","last_peered":"2024-05-08T04:42:55.645180+0000","last_clean":"2024-05-08T04:42:55.645180+0000","last_became_active":"2024-05-08T04:42:54.278516+0000","last_became_peered":"2024-05-08T04:42:54.278516+0000","last_unstale":"2024-05-08T04:42:55.645180+0000","last_undegraded":"2024-05-08T04:42:55.645180+0000","last_fullsized":"2024-05-08T04:42:55.645180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:44.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260298+0000","last_change":"2024-05-08T04:42:54.264244+0000","last_active":"2024-05-08T04:42:55.260298+0000","last_peered":"2024-05-08T04:42:55.260298+0000","last_clean":"2024-05-08T04:42:55.260298+0000","last_became_active":"2024-05-08T04:42:54.261285+0000","last_became_peered":"2024-05-08T04:42:54.261285+0000","last_unstale":"2024-05-08T04:42:55.260298+0000","last_undegraded":"2024-05-08T04:42:55.260298+0000","last_fullsized":"2024-05-08T04:42:55.260298+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:16.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644393+0000","last_change":"2024-05-08T04:42:54.278540+0000","last_active":"2024-05-08T04:42:55.644393+0000","last_peered":"2024-05-08T04:42:55.644393+0000","last_clean":"2024-05-08T04:42:55.644393+0000","last_became_active":"2024-05-08T04:42:54.275676+0000","last_became_peered":"2024-05-08T04:42:54.275676+0000","last_unstale":"2024-05-08T04:42:55.644393+0000","last_undegraded":"2024-05-08T04:42:55.644393+0000","last_fullsized":"2024-05-08T04:42:55.644393+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:02.962699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543936+0000","last_change":"2024-05-08T04:42:54.262625+0000","last_active":"2024-05-08T04:42:55.543936+0000","last_peered":"2024-05-08T04:42:55.543936+0000","last_clean":"2024-05-08T04:42:55.543936+0000","last_became_active":"2024-05-08T04:42:54.262328+0000","last_became_peered":"2024-05-08T04:42:54.262328+0000","last_unstale":"2024-05-08T04:42:55.543936+0000","last_undegraded":"2024-05-08T04:42:55.543936+0000","last_fullsized":"2024-05-08T04:42:55.543936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:37.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543497+0000","last_change":"2024-05-08T04:42:54.263202+0000","last_active":"2024-05-08T04:42:55.543497+0000","last_peered":"2024-05-08T04:42:55.543497+0000","last_clean":"2024-05-08T04:42:55.543497+0000","last_became_active":"2024-05-08T04:42:54.262118+0000","last_became_peered":"2024-05-08T04:42:54.262118+0000","last_unstale":"2024-05-08T04:42:55.543497+0000","last_undegraded":"2024-05-08T04:42:55.543497+0000","last_fullsized":"2024-05-08T04:42:55.543497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643757+0000","last_change":"2024-05-08T04:42:54.258584+0000","last_active":"2024-05-08T04:42:55.643757+0000","last_peered":"2024-05-08T04:42:55.643757+0000","last_clean":"2024-05-08T04:42:55.643757+0000","last_became_active":"2024-05-08T04:42:54.257809+0000","last_became_peered":"2024-05-08T04:42:54.257809+0000","last_unstale":"2024-05-08T04:42:55.643757+0000","last_undegraded":"2024-05-08T04:42:55.643757+0000","last_fullsized":"2024-05-08T04:42:55.643757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.486563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643534+0000","last_change":"2024-05-08T04:42:54.272568+0000","last_active":"2024-05-08T04:42:55.643534+0000","last_peered":"2024-05-08T04:42:55.643534+0000","last_clean":"2024-05-08T04:42:55.643534+0000","last_became_active":"2024-05-08T04:42:54.266872+0000","last_became_peered":"2024-05-08T04:42:54.266872+0000","last_unstale":"2024-05-08T04:42:55.643534+0000","last_undegraded":"2024-05-08T04:42:55.643534+0000","last_fullsized":"2024-05-08T04:42:55.643534+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:19.666610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260434+0000","last_change":"2024-05-08T04:42:54.265221+0000","last_active":"2024-05-08T04:42:55.260434+0000","last_peered":"2024-05-08T04:42:55.260434+0000","last_clean":"2024-05-08T04:42:55.260434+0000","last_became_active":"2024-05-08T04:42:54.263935+0000","last_became_peered":"2024-05-08T04:42:54.263935+0000","last_unstale":"2024-05-08T04:42:55.260434+0000","last_undegraded":"2024-05-08T04:42:55.260434+0000","last_fullsized":"2024-05-08T04:42:55.260434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:24.087153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261472+0000","last_change":"2024-05-08T04:42:54.260979+0000","last_active":"2024-05-08T04:42:55.261472+0000","last_peered":"2024-05-08T04:42:55.261472+0000","last_clean":"2024-05-08T04:42:55.261472+0000","last_became_active":"2024-05-08T04:42:54.260386+0000","last_became_peered":"2024-05-08T04:42:54.260386+0000","last_unstale":"2024-05-08T04:42:55.261472+0000","last_undegraded":"2024-05-08T04:42:55.261472+0000","last_fullsized":"2024-05-08T04:42:55.261472+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:23.982806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543852+0000","last_change":"2024-05-08T04:42:54.267987+0000","last_active":"2024-05-08T04:42:55.543852+0000","last_peered":"2024-05-08T04:42:55.543852+0000","last_clean":"2024-05-08T04:42:55.543852+0000","last_became_active":"2024-05-08T04:42:54.267674+0000","last_became_peered":"2024-05-08T04:42:54.267674+0000","last_unstale":"2024-05-08T04:42:55.543852+0000","last_undegraded":"2024-05-08T04:42:55.543852+0000","last_fullsized":"2024-05-08T04:42:55.543852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:11.284074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259984+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.259984+0000","last_peered":"2024-05-08T04:42:55.259984+0000","last_clean":"2024-05-08T04:42:55.259984+0000","last_became_active":"2024-05-08T04:42:54.261903+0000","last_became_peered":"2024-05-08T04:42:54.261903+0000","last_unstale":"2024-05-08T04:42:55.259984+0000","last_undegraded":"2024-05-08T04:42:55.259984+0000","last_fullsized":"2024-05-08T04:42:55.259984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:49.660293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260474+0000","last_change":"2024-05-08T04:42:54.247853+0000","last_active":"2024-05-08T04:42:55.260474+0000","last_peered":"2024-05-08T04:42:55.260474+0000","last_clean":"2024-05-08T04:42:55.260474+0000","last_became_active":"2024-05-08T04:42:54.247314+0000","last_became_peered":"2024-05-08T04:42:54.247314+0000","last_unstale":"2024-05-08T04:42:55.260474+0000","last_undegraded":"2024-05-08T04:42:55.260474+0000","last_fullsized":"2024-05-08T04:42:55.260474+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.153233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260564+0000","last_change":"2024-05-08T04:42:54.263380+0000","last_active":"2024-05-08T04:42:55.260564+0000","last_peered":"2024-05-08T04:42:55.260564+0000","last_clean":"2024-05-08T04:42:55.260564+0000","last_became_active":"2024-05-08T04:42:54.262268+0000","last_became_peered":"2024-05-08T04:42:54.262268+0000","last_unstale":"2024-05-08T04:42:55.260564+0000","last_undegraded":"2024-05-08T04:42:55.260564+0000","last_fullsized":"2024-05-08T04:42:55.260564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258982+0000","last_change":"2024-05-08T04:42:54.257923+0000","last_active":"2024-05-08T04:42:55.258982+0000","last_peered":"2024-05-08T04:42:55.258982+0000","last_clean":"2024-05-08T04:42:55.258982+0000","last_became_active":"2024-05-08T04:42:54.257524+0000","last_became_peered":"2024-05-08T04:42:54.257524+0000","last_unstale":"2024-05-08T04:42:55.258982+0000","last_undegraded":"2024-05-08T04:42:55.258982+0000","last_fullsized":"2024-05-08T04:42:55.258982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260354+0000","last_change":"2024-05-08T04:42:54.258040+0000","last_active":"2024-05-08T04:42:55.260354+0000","last_peered":"2024-05-08T04:42:55.260354+0000","last_clean":"2024-05-08T04:42:55.260354+0000","last_became_active":"2024-05-08T04:42:54.257775+0000","last_became_peered":"2024-05-08T04:42:54.257775+0000","last_unstale":"2024-05-08T04:42:55.260354+0000","last_undegraded":"2024-05-08T04:42:55.260354+0000","last_fullsized":"2024-05-08T04:42:55.260354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:17.593712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260632+0000","last_change":"2024-05-08T04:42:54.264625+0000","last_active":"2024-05-08T04:42:55.260632+0000","last_peered":"2024-05-08T04:42:55.260632+0000","last_clean":"2024-05-08T04:42:55.260632+0000","last_became_active":"2024-05-08T04:42:54.261407+0000","last_became_peered":"2024-05-08T04:42:54.261407+0000","last_unstale":"2024-05-08T04:42:55.260632+0000","last_undegraded":"2024-05-08T04:42:55.260632+0000","last_fullsized":"2024-05-08T04:42:55.260632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:21.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644944+0000","last_change":"2024-05-08T04:42:54.273444+0000","last_active":"2024-05-08T04:42:55.644944+0000","last_peered":"2024-05-08T04:42:55.644944+0000","last_clean":"2024-05-08T04:42:55.644944+0000","last_became_active":"2024-05-08T04:42:54.273184+0000","last_became_peered":"2024-05-08T04:42:54.273184+0000","last_unstale":"2024-05-08T04:42:55.644944+0000","last_undegraded":"2024-05-08T04:42:55.644944+0000","last_fullsized":"2024-05-08T04:42:55.644944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:18.445404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645146+0000","last_change":"2024-05-08T04:42:54.277145+0000","last_active":"2024-05-08T04:42:55.645146+0000","last_peered":"2024-05-08T04:42:55.645146+0000","last_clean":"2024-05-08T04:42:55.645146+0000","last_became_active":"2024-05-08T04:42:54.274239+0000","last_became_peered":"2024-05-08T04:42:54.274239+0000","last_unstale":"2024-05-08T04:42:55.645146+0000","last_undegraded":"2024-05-08T04:42:55.645146+0000","last_fullsized":"2024-05-08T04:42:55.645146+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260902+0000","last_change":"2024-05-08T04:42:54.279227+0000","last_active":"2024-05-08T04:42:55.260902+0000","last_peered":"2024-05-08T04:42:55.260902+0000","last_clean":"2024-05-08T04:42:55.260902+0000","last_became_active":"2024-05-08T04:42:54.278953+0000","last_became_peered":"2024-05-08T04:42:54.278953+0000","last_unstale":"2024-05-08T04:42:55.260902+0000","last_undegraded":"2024-05-08T04:42:55.260902+0000","last_fullsized":"2024-05-08T04:42:55.260902+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260283+0000","last_change":"2024-05-08T04:42:54.257830+0000","last_active":"2024-05-08T04:42:55.260283+0000","last_peered":"2024-05-08T04:42:55.260283+0000","last_clean":"2024-05-08T04:42:55.260283+0000","last_became_active":"2024-05-08T04:42:54.257455+0000","last_became_peered":"2024-05-08T04:42:54.257455+0000","last_unstale":"2024-05-08T04:42:55.260283+0000","last_undegraded":"2024-05-08T04:42:55.260283+0000","last_fullsized":"2024-05-08T04:42:55.260283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:08.717198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259104+0000","last_change":"2024-05-08T04:42:54.247280+0000","last_active":"2024-05-08T04:42:55.259104+0000","last_peered":"2024-05-08T04:42:55.259104+0000","last_clean":"2024-05-08T04:42:55.259104+0000","last_became_active":"2024-05-08T04:42:54.246609+0000","last_became_peered":"2024-05-08T04:42:54.246609+0000","last_unstale":"2024-05-08T04:42:55.259104+0000","last_undegraded":"2024-05-08T04:42:55.259104+0000","last_fullsized":"2024-05-08T04:42:55.259104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643671+0000","last_change":"2024-05-08T04:42:54.270435+0000","last_active":"2024-05-08T04:42:55.643671+0000","last_peered":"2024-05-08T04:42:55.643671+0000","last_clean":"2024-05-08T04:42:55.643671+0000","last_became_active":"2024-05-08T04:42:54.265729+0000","last_became_peered":"2024-05-08T04:42:54.265729+0000","last_unstale":"2024-05-08T04:42:55.643671+0000","last_undegraded":"2024-05-08T04:42:55.643671+0000","last_fullsized":"2024-05-08T04:42:55.643671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:53.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643688+0000","last_change":"2024-05-08T04:42:54.261182+0000","last_active":"2024-05-08T04:42:55.643688+0000","last_peered":"2024-05-08T04:42:55.643688+0000","last_clean":"2024-05-08T04:42:55.643688+0000","last_became_active":"2024-05-08T04:42:54.260895+0000","last_became_peered":"2024-05-08T04:42:54.260895+0000","last_unstale":"2024-05-08T04:42:55.643688+0000","last_undegraded":"2024-05-08T04:42:55.643688+0000","last_fullsized":"2024-05-08T04:42:55.643688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645113+0000","last_change":"2024-05-08T04:42:54.271090+0000","last_active":"2024-05-08T04:42:55.645113+0000","last_peered":"2024-05-08T04:42:55.645113+0000","last_clean":"2024-05-08T04:42:55.645113+0000","last_became_active":"2024-05-08T04:42:54.270874+0000","last_became_peered":"2024-05-08T04:42:54.270874+0000","last_unstale":"2024-05-08T04:42:55.645113+0000","last_undegraded":"2024-05-08T04:42:55.645113+0000","last_fullsized":"2024-05-08T04:42:55.645113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259863+0000","last_change":"2024-05-08T04:42:53.251819+0000","last_active":"2024-05-08T04:42:55.259863+0000","last_peered":"2024-05-08T04:42:55.259863+0000","last_clean":"2024-05-08T04:42:55.259863+0000","last_became_active":"2024-05-08T04:42:53.251490+0000","last_became_peered":"2024-05-08T04:42:53.251490+0000","last_unstale":"2024-05-08T04:42:55.259863+0000","last_undegraded":"2024-05-08T04:42:55.259863+0000","last_fullsized":"2024-05-08T04:42:55.259863+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:07.081658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261626+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.261626+0000","last_peered":"2024-05-08T04:42:55.261626+0000","last_clean":"2024-05-08T04:42:55.261626+0000","last_became_active":"2024-05-08T04:42:54.261894+0000","last_became_peered":"2024-05-08T04:42:54.261894+0000","last_unstale":"2024-05-08T04:42:55.261626+0000","last_undegraded":"2024-05-08T04:42:55.261626+0000","last_fullsized":"2024-05-08T04:42:55.261626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543705+0000","last_change":"2024-05-08T04:42:53.256430+0000","last_active":"2024-05-08T04:42:55.543705+0000","last_peered":"2024-05-08T04:42:55.543705+0000","last_clean":"2024-05-08T04:42:55.543705+0000","last_became_active":"2024-05-08T04:42:53.256154+0000","last_became_peered":"2024-05-08T04:42:53.256154+0000","last_unstale":"2024-05-08T04:42:55.543705+0000","last_undegraded":"2024-05-08T04:42:55.543705+0000","last_fullsized":"2024-05-08T04:42:55.543705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644955+0000","last_change":"2024-05-08T04:42:54.264075+0000","last_active":"2024-05-08T04:42:55.644955+0000","last_peered":"2024-05-08T04:42:55.644955+0000","last_clean":"2024-05-08T04:42:55.644955+0000","last_became_active":"2024-05-08T04:42:54.263803+0000","last_became_peered":"2024-05-08T04:42:54.263803+0000","last_unstale":"2024-05-08T04:42:55.644955+0000","last_undegraded":"2024-05-08T04:42:55.644955+0000","last_fullsized":"2024-05-08T04:42:55.644955+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644201+0000","last_change":"2024-05-08T04:42:53.257257+0000","last_active":"2024-05-08T04:42:55.644201+0000","last_peered":"2024-05-08T04:42:55.644201+0000","last_clean":"2024-05-08T04:42:55.644201+0000","last_became_active":"2024-05-08T04:42:53.256901+0000","last_became_peered":"2024-05-08T04:42:53.256901+0000","last_unstale":"2024-05-08T04:42:55.644201+0000","last_undegraded":"2024-05-08T04:42:55.644201+0000","last_fullsized":"2024-05-08T04:42:55.644201+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544048+0000","last_change":"2024-05-08T04:42:54.279237+0000","last_active":"2024-05-08T04:42:55.544048+0000","last_peered":"2024-05-08T04:42:55.544048+0000","last_clean":"2024-05-08T04:42:55.544048+0000","last_became_active":"2024-05-08T04:42:54.279021+0000","last_became_peered":"2024-05-08T04:42:54.279021+0000","last_unstale":"2024-05-08T04:42:55.544048+0000","last_undegraded":"2024-05-08T04:42:55.544048+0000","last_fullsized":"2024-05-08T04:42:55.544048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:02.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643992+0000","last_change":"2024-05-08T04:42:53.254748+0000","last_active":"2024-05-08T04:42:55.643992+0000","last_peered":"2024-05-08T04:42:55.643992+0000","last_clean":"2024-05-08T04:42:55.643992+0000","last_became_active":"2024-05-08T04:42:53.253422+0000","last_became_peered":"2024-05-08T04:42:53.253422+0000","last_unstale":"2024-05-08T04:42:55.643992+0000","last_undegraded":"2024-05-08T04:42:55.643992+0000","last_fullsized":"2024-05-08T04:42:55.643992+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:23.946191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644157+0000","last_change":"2024-05-08T04:42:54.270324+0000","last_active":"2024-05-08T04:42:55.644157+0000","last_peered":"2024-05-08T04:42:55.644157+0000","last_clean":"2024-05-08T04:42:55.644157+0000","last_became_active":"2024-05-08T04:42:54.269819+0000","last_became_peered":"2024-05-08T04:42:54.269819+0000","last_unstale":"2024-05-08T04:42:55.644157+0000","last_undegraded":"2024-05-08T04:42:55.644157+0000","last_fullsized":"2024-05-08T04:42:55.644157+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:09.118707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259307+0000","last_change":"2024-05-08T04:42:53.257209+0000","last_active":"2024-05-08T04:42:55.259307+0000","last_peered":"2024-05-08T04:42:55.259307+0000","last_clean":"2024-05-08T04:42:55.259307+0000","last_became_active":"2024-05-08T04:42:53.256908+0000","last_became_peered":"2024-05-08T04:42:53.256908+0000","last_unstale":"2024-05-08T04:42:55.259307+0000","last_undegraded":"2024-05-08T04:42:55.259307+0000","last_fullsized":"2024-05-08T04:42:55.259307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259796+0000","last_change":"2024-05-08T04:42:54.263171+0000","last_active":"2024-05-08T04:42:55.259796+0000","last_peered":"2024-05-08T04:42:55.259796+0000","last_clean":"2024-05-08T04:42:55.259796+0000","last_became_active":"2024-05-08T04:42:54.262760+0000","last_became_peered":"2024-05-08T04:42:54.262760+0000","last_unstale":"2024-05-08T04:42:55.259796+0000","last_undegraded":"2024-05-08T04:42:55.259796+0000","last_fullsized":"2024-05-08T04:42:55.259796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260608+0000","last_change":"2024-05-08T04:42:53.261630+0000","last_active":"2024-05-08T04:42:55.260608+0000","last_peered":"2024-05-08T04:42:55.260608+0000","last_clean":"2024-05-08T04:42:55.260608+0000","last_became_active":"2024-05-08T04:42:53.261358+0000","last_became_peered":"2024-05-08T04:42:53.261358+0000","last_unstale":"2024-05-08T04:42:55.260608+0000","last_undegraded":"2024-05-08T04:42:55.260608+0000","last_fullsized":"2024-05-08T04:42:55.260608+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:07.652299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643515+0000","last_change":"2024-05-08T04:42:54.274640+0000","last_active":"2024-05-08T04:42:55.643515+0000","last_peered":"2024-05-08T04:42:55.643515+0000","last_clean":"2024-05-08T04:42:55.643515+0000","last_became_active":"2024-05-08T04:42:54.274209+0000","last_became_peered":"2024-05-08T04:42:54.274209+0000","last_unstale":"2024-05-08T04:42:55.643515+0000","last_undegraded":"2024-05-08T04:42:55.643515+0000","last_fullsized":"2024-05-08T04:42:55.643515+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:36.694738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259005+0000","last_change":"2024-05-08T04:42:53.265323+0000","last_active":"2024-05-08T04:42:55.259005+0000","last_peered":"2024-05-08T04:42:55.259005+0000","last_clean":"2024-05-08T04:42:55.259005+0000","last_became_active":"2024-05-08T04:42:53.265031+0000","last_became_peered":"2024-05-08T04:42:53.265031+0000","last_unstale":"2024-05-08T04:42:55.259005+0000","last_undegraded":"2024-05-08T04:42:55.259005+0000","last_fullsized":"2024-05-08T04:42:55.259005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:19.518749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258582+0000","last_change":"2024-05-08T04:42:54.263167+0000","last_active":"2024-05-08T04:42:55.258582+0000","last_peered":"2024-05-08T04:42:55.258582+0000","last_clean":"2024-05-08T04:42:55.258582+0000","last_became_active":"2024-05-08T04:42:54.262842+0000","last_became_peered":"2024-05-08T04:42:54.262842+0000","last_unstale":"2024-05-08T04:42:55.258582+0000","last_undegraded":"2024-05-08T04:42:55.258582+0000","last_fullsized":"2024-05-08T04:42:55.258582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643593+0000","last_change":"2024-05-08T04:42:53.263261+0000","last_active":"2024-05-08T04:42:55.643593+0000","last_peered":"2024-05-08T04:42:55.643593+0000","last_clean":"2024-05-08T04:42:55.643593+0000","last_became_active":"2024-05-08T04:42:53.262219+0000","last_became_peered":"2024-05-08T04:42:53.262219+0000","last_unstale":"2024-05-08T04:42:55.643593+0000","last_undegraded":"2024-05-08T04:42:55.643593+0000","last_fullsized":"2024-05-08T04:42:55.643593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:20.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260268+0000","last_change":"2024-05-08T04:42:54.269625+0000","last_active":"2024-05-08T04:42:55.260268+0000","last_peered":"2024-05-08T04:42:55.260268+0000","last_clean":"2024-05-08T04:42:55.260268+0000","last_became_active":"2024-05-08T04:42:54.269308+0000","last_became_peered":"2024-05-08T04:42:54.269308+0000","last_unstale":"2024-05-08T04:42:55.260268+0000","last_undegraded":"2024-05-08T04:42:55.260268+0000","last_fullsized":"2024-05-08T04:42:55.260268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.615548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644470+0000","last_change":"2024-05-08T04:42:53.254347+0000","last_active":"2024-05-08T04:42:55.644470+0000","last_peered":"2024-05-08T04:42:55.644470+0000","last_clean":"2024-05-08T04:42:55.644470+0000","last_became_active":"2024-05-08T04:42:53.254090+0000","last_became_peered":"2024-05-08T04:42:53.254090+0000","last_unstale":"2024-05-08T04:42:55.644470+0000","last_undegraded":"2024-05-08T04:42:55.644470+0000","last_fullsized":"2024-05-08T04:42:55.644470+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261712+0000","last_change":"2024-05-08T04:42:54.284605+0000","last_active":"2024-05-08T04:42:55.261712+0000","last_peered":"2024-05-08T04:42:55.261712+0000","last_clean":"2024-05-08T04:42:55.261712+0000","last_became_active":"2024-05-08T04:42:54.284331+0000","last_became_peered":"2024-05-08T04:42:54.284331+0000","last_unstale":"2024-05-08T04:42:55.261712+0000","last_undegraded":"2024-05-08T04:42:55.261712+0000","last_fullsized":"2024-05-08T04:42:55.261712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260792+0000","last_change":"2024-05-08T04:42:53.265080+0000","last_active":"2024-05-08T04:42:55.260792+0000","last_peered":"2024-05-08T04:42:55.260792+0000","last_clean":"2024-05-08T04:42:55.260792+0000","last_became_active":"2024-05-08T04:42:53.263632+0000","last_became_peered":"2024-05-08T04:42:53.263632+0000","last_unstale":"2024-05-08T04:42:55.260792+0000","last_undegraded":"2024-05-08T04:42:55.260792+0000","last_fullsized":"2024-05-08T04:42:55.260792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260920+0000","last_change":"2024-05-08T04:42:54.254357+0000","last_active":"2024-05-08T04:42:55.260920+0000","last_peered":"2024-05-08T04:42:55.260920+0000","last_clean":"2024-05-08T04:42:55.260920+0000","last_became_active":"2024-05-08T04:42:54.254083+0000","last_became_peered":"2024-05-08T04:42:54.254083+0000","last_unstale":"2024-05-08T04:42:55.260920+0000","last_undegraded":"2024-05-08T04:42:55.260920+0000","last_fullsized":"2024-05-08T04:42:55.260920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644694+0000","last_change":"2024-05-08T04:42:53.257288+0000","last_active":"2024-05-08T04:42:55.644694+0000","last_peered":"2024-05-08T04:42:55.644694+0000","last_clean":"2024-05-08T04:42:55.644694+0000","last_became_active":"2024-05-08T04:42:53.257014+0000","last_became_peered":"2024-05-08T04:42:53.257014+0000","last_unstale":"2024-05-08T04:42:55.644694+0000","last_undegraded":"2024-05-08T04:42:55.644694+0000","last_fullsized":"2024-05-08T04:42:55.644694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:27.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259886+0000","last_change":"2024-05-08T04:42:54.259828+0000","last_active":"2024-05-08T04:42:55.259886+0000","last_peered":"2024-05-08T04:42:55.259886+0000","last_clean":"2024-05-08T04:42:55.259886+0000","last_became_active":"2024-05-08T04:42:54.259109+0000","last_became_peered":"2024-05-08T04:42:54.259109+0000","last_unstale":"2024-05-08T04:42:55.259886+0000","last_undegraded":"2024-05-08T04:42:55.259886+0000","last_fullsized":"2024-05-08T04:42:55.259886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644648+0000","last_change":"2024-05-08T04:42:53.264438+0000","last_active":"2024-05-08T04:42:55.644648+0000","last_peered":"2024-05-08T04:42:55.644648+0000","last_clean":"2024-05-08T04:42:55.644648+0000","last_became_active":"2024-05-08T04:42:53.264131+0000","last_became_peered":"2024-05-08T04:42:53.264131+0000","last_unstale":"2024-05-08T04:42:55.644648+0000","last_undegraded":"2024-05-08T04:42:55.644648+0000","last_fullsized":"2024-05-08T04:42:55.644648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:15.699103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260599+0000","last_change":"2024-05-08T04:42:54.251858+0000","last_active":"2024-05-08T04:42:55.260599+0000","last_peered":"2024-05-08T04:42:55.260599+0000","last_clean":"2024-05-08T04:42:55.260599+0000","last_became_active":"2024-05-08T04:42:54.251600+0000","last_became_peered":"2024-05-08T04:42:54.251600+0000","last_unstale":"2024-05-08T04:42:55.260599+0000","last_undegraded":"2024-05-08T04:42:55.260599+0000","last_fullsized":"2024-05-08T04:42:55.260599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644435+0000","last_change":"2024-05-08T04:42:53.255665+0000","last_active":"2024-05-08T04:42:55.644435+0000","last_peered":"2024-05-08T04:42:55.644435+0000","last_clean":"2024-05-08T04:42:55.644435+0000","last_became_active":"2024-05-08T04:42:53.255168+0000","last_became_peered":"2024-05-08T04:42:53.255168+0000","last_unstale":"2024-05-08T04:42:55.644435+0000","last_undegraded":"2024-05-08T04:42:55.644435+0000","last_fullsized":"2024-05-08T04:42:55.644435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260674+0000","last_change":"2024-05-08T04:42:54.260677+0000","last_active":"2024-05-08T04:42:55.260674+0000","last_peered":"2024-05-08T04:42:55.260674+0000","last_clean":"2024-05-08T04:42:55.260674+0000","last_became_active":"2024-05-08T04:42:54.259329+0000","last_became_peered":"2024-05-08T04:42:54.259329+0000","last_unstale":"2024-05-08T04:42:55.260674+0000","last_undegraded":"2024-05-08T04:42:55.260674+0000","last_fullsized":"2024-05-08T04:42:55.260674+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261312+0000","last_change":"2024-05-08T04:42:53.259031+0000","last_active":"2024-05-08T04:42:55.261312+0000","last_peered":"2024-05-08T04:42:55.261312+0000","last_clean":"2024-05-08T04:42:55.261312+0000","last_became_active":"2024-05-08T04:42:53.258734+0000","last_became_peered":"2024-05-08T04:42:53.258734+0000","last_unstale":"2024-05-08T04:42:55.261312+0000","last_undegraded":"2024-05-08T04:42:55.261312+0000","last_fullsized":"2024-05-08T04:42:55.261312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:07.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260382+0000","last_change":"2024-05-08T04:42:54.255601+0000","last_active":"2024-05-08T04:42:55.260382+0000","last_peered":"2024-05-08T04:42:55.260382+0000","last_clean":"2024-05-08T04:42:55.260382+0000","last_became_active":"2024-05-08T04:42:54.255333+0000","last_became_peered":"2024-05-08T04:42:54.255333+0000","last_unstale":"2024-05-08T04:42:55.260382+0000","last_undegraded":"2024-05-08T04:42:55.260382+0000","last_fullsized":"2024-05-08T04:42:55.260382+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:28.195601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643788+0000","last_change":"2024-05-08T04:42:53.254175+0000","last_active":"2024-05-08T04:42:55.643788+0000","last_peered":"2024-05-08T04:42:55.643788+0000","last_clean":"2024-05-08T04:42:55.643788+0000","last_became_active":"2024-05-08T04:42:53.253505+0000","last_became_peered":"2024-05-08T04:42:53.253505+0000","last_unstale":"2024-05-08T04:42:55.643788+0000","last_undegraded":"2024-05-08T04:42:55.643788+0000","last_fullsized":"2024-05-08T04:42:55.643788+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644424+0000","last_change":"2024-05-08T04:42:54.283984+0000","last_active":"2024-05-08T04:42:55.644424+0000","last_peered":"2024-05-08T04:42:55.644424+0000","last_clean":"2024-05-08T04:42:55.644424+0000","last_became_active":"2024-05-08T04:42:54.283630+0000","last_became_peered":"2024-05-08T04:42:54.283630+0000","last_unstale":"2024-05-08T04:42:55.644424+0000","last_undegraded":"2024-05-08T04:42:55.644424+0000","last_fullsized":"2024-05-08T04:42:55.644424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645310+0000","last_change":"2024-05-08T04:42:53.255078+0000","last_active":"2024-05-08T04:42:55.645310+0000","last_peered":"2024-05-08T04:42:55.645310+0000","last_clean":"2024-05-08T04:42:55.645310+0000","last_became_active":"2024-05-08T04:42:53.254350+0000","last_became_peered":"2024-05-08T04:42:53.254350+0000","last_unstale":"2024-05-08T04:42:55.645310+0000","last_undegraded":"2024-05-08T04:42:55.645310+0000","last_fullsized":"2024-05-08T04:42:55.645310+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:59.504019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645155+0000","last_change":"2024-05-08T04:42:54.284624+0000","last_active":"2024-05-08T04:42:55.645155+0000","last_peered":"2024-05-08T04:42:55.645155+0000","last_clean":"2024-05-08T04:42:55.645155+0000","last_became_active":"2024-05-08T04:42:54.284368+0000","last_became_peered":"2024-05-08T04:42:54.284368+0000","last_unstale":"2024-05-08T04:42:55.645155+0000","last_undegraded":"2024-05-08T04:42:55.645155+0000","last_fullsized":"2024-05-08T04:42:55.645155+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644356+0000","last_change":"2024-05-08T04:42:53.259653+0000","last_active":"2024-05-08T04:42:55.644356+0000","last_peered":"2024-05-08T04:42:55.644356+0000","last_clean":"2024-05-08T04:42:55.644356+0000","last_became_active":"2024-05-08T04:42:53.259407+0000","last_became_peered":"2024-05-08T04:42:53.259407+0000","last_unstale":"2024-05-08T04:42:55.644356+0000","last_undegraded":"2024-05-08T04:42:55.644356+0000","last_fullsized":"2024-05-08T04:42:55.644356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543959+0000","last_change":"2024-05-08T04:42:54.268528+0000","last_active":"2024-05-08T04:42:55.543959+0000","last_peered":"2024-05-08T04:42:55.543959+0000","last_clean":"2024-05-08T04:42:55.543959+0000","last_became_active":"2024-05-08T04:42:54.268247+0000","last_became_peered":"2024-05-08T04:42:54.268247+0000","last_unstale":"2024-05-08T04:42:55.543959+0000","last_undegraded":"2024-05-08T04:42:55.543959+0000","last_fullsized":"2024-05-08T04:42:55.543959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:20.235769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261410+0000","last_change":"2024-05-08T04:42:53.260958+0000","last_active":"2024-05-08T04:42:55.261410+0000","last_peered":"2024-05-08T04:42:55.261410+0000","last_clean":"2024-05-08T04:42:55.261410+0000","last_became_active":"2024-05-08T04:42:53.260621+0000","last_became_peered":"2024-05-08T04:42:53.260621+0000","last_unstale":"2024-05-08T04:42:55.261410+0000","last_undegraded":"2024-05-08T04:42:55.261410+0000","last_fullsized":"2024-05-08T04:42:55.261410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:14.035376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544039+0000","last_change":"2024-05-08T04:42:54.259461+0000","last_active":"2024-05-08T04:42:55.544039+0000","last_peered":"2024-05-08T04:42:55.544039+0000","last_clean":"2024-05-08T04:42:55.544039+0000","last_became_active":"2024-05-08T04:42:54.259005+0000","last_became_peered":"2024-05-08T04:42:54.259005+0000","last_unstale":"2024-05-08T04:42:55.544039+0000","last_undegraded":"2024-05-08T04:42:55.544039+0000","last_fullsized":"2024-05-08T04:42:55.544039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:07.448395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258867+0000","last_change":"2024-05-08T04:42:53.258018+0000","last_active":"2024-05-08T04:42:55.258867+0000","last_peered":"2024-05-08T04:42:55.258867+0000","last_clean":"2024-05-08T04:42:55.258867+0000","last_became_active":"2024-05-08T04:42:53.257357+0000","last_became_peered":"2024-05-08T04:42:53.257357+0000","last_unstale":"2024-05-08T04:42:55.258867+0000","last_undegraded":"2024-05-08T04:42:55.258867+0000","last_fullsized":"2024-05-08T04:42:55.258867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543555+0000","last_change":"2024-05-08T04:42:54.258649+0000","last_active":"2024-05-08T04:42:55.543555+0000","last_peered":"2024-05-08T04:42:55.543555+0000","last_clean":"2024-05-08T04:42:55.543555+0000","last_became_active":"2024-05-08T04:42:54.258367+0000","last_became_peered":"2024-05-08T04:42:54.258367+0000","last_unstale":"2024-05-08T04:42:55.543555+0000","last_undegraded":"2024-05-08T04:42:55.543555+0000","last_fullsized":"2024-05-08T04:42:55.543555+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644524+0000","last_change":"2024-05-08T04:42:53.260251+0000","last_active":"2024-05-08T04:42:55.644524+0000","last_peered":"2024-05-08T04:42:55.644524+0000","last_clean":"2024-05-08T04:42:55.644524+0000","last_became_active":"2024-05-08T04:42:53.259749+0000","last_became_peered":"2024-05-08T04:42:53.259749+0000","last_unstale":"2024-05-08T04:42:55.644524+0000","last_undegraded":"2024-05-08T04:42:55.644524+0000","last_fullsized":"2024-05-08T04:42:55.644524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644062+0000","last_change":"2024-05-08T04:42:54.270229+0000","last_active":"2024-05-08T04:42:55.644062+0000","last_peered":"2024-05-08T04:42:55.644062+0000","last_clean":"2024-05-08T04:42:55.644062+0000","last_became_active":"2024-05-08T04:42:54.267191+0000","last_became_peered":"2024-05-08T04:42:54.267191+0000","last_unstale":"2024-05-08T04:42:55.644062+0000","last_undegraded":"2024-05-08T04:42:55.644062+0000","last_fullsized":"2024-05-08T04:42:55.644062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:16.097496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643837+0000","last_change":"2024-05-08T04:42:53.255758+0000","last_active":"2024-05-08T04:42:55.643837+0000","last_peered":"2024-05-08T04:42:55.643837+0000","last_clean":"2024-05-08T04:42:55.643837+0000","last_became_active":"2024-05-08T04:42:53.255511+0000","last_became_peered":"2024-05-08T04:42:53.255511+0000","last_unstale":"2024-05-08T04:42:55.643837+0000","last_undegraded":"2024-05-08T04:42:55.643837+0000","last_fullsized":"2024-05-08T04:42:55.643837+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543619+0000","last_change":"2024-05-08T04:42:54.270286+0000","last_active":"2024-05-08T04:42:55.543619+0000","last_peered":"2024-05-08T04:42:55.543619+0000","last_clean":"2024-05-08T04:42:55.543619+0000","last_became_active":"2024-05-08T04:42:54.270019+0000","last_became_peered":"2024-05-08T04:42:54.270019+0000","last_unstale":"2024-05-08T04:42:55.543619+0000","last_undegraded":"2024-05-08T04:42:55.543619+0000","last_fullsized":"2024-05-08T04:42:55.543619+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644073+0000","last_change":"2024-05-08T04:42:53.259343+0000","last_active":"2024-05-08T04:42:55.644073+0000","last_peered":"2024-05-08T04:42:55.644073+0000","last_clean":"2024-05-08T04:42:55.644073+0000","last_became_active":"2024-05-08T04:42:53.258967+0000","last_became_peered":"2024-05-08T04:42:53.258967+0000","last_unstale":"2024-05-08T04:42:55.644073+0000","last_undegraded":"2024-05-08T04:42:55.644073+0000","last_fullsized":"2024-05-08T04:42:55.644073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260450+0000","last_change":"2024-05-08T04:42:54.267912+0000","last_active":"2024-05-08T04:42:55.260450+0000","last_peered":"2024-05-08T04:42:55.260450+0000","last_clean":"2024-05-08T04:42:55.260450+0000","last_became_active":"2024-05-08T04:42:54.267648+0000","last_became_peered":"2024-05-08T04:42:54.267648+0000","last_unstale":"2024-05-08T04:42:55.260450+0000","last_undegraded":"2024-05-08T04:42:55.260450+0000","last_fullsized":"2024-05-08T04:42:55.260450+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643660+0000","last_change":"2024-05-08T04:42:53.253264+0000","last_active":"2024-05-08T04:42:55.643660+0000","last_peered":"2024-05-08T04:42:55.643660+0000","last_clean":"2024-05-08T04:42:55.643660+0000","last_became_active":"2024-05-08T04:42:53.252716+0000","last_became_peered":"2024-05-08T04:42:53.252716+0000","last_unstale":"2024-05-08T04:42:55.643660+0000","last_undegraded":"2024-05-08T04:42:55.643660+0000","last_fullsized":"2024-05-08T04:42:55.643660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:56.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275320+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.275003+0000","last_became_peered":"2024-05-08T04:42:54.275003+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260852+0000","last_change":"2024-05-08T04:42:53.264575+0000","last_active":"2024-05-08T04:42:55.260852+0000","last_peered":"2024-05-08T04:42:55.260852+0000","last_clean":"2024-05-08T04:42:55.260852+0000","last_became_active":"2024-05-08T04:42:53.264163+0000","last_became_peered":"2024-05-08T04:42:53.264163+0000","last_unstale":"2024-05-08T04:42:55.260852+0000","last_undegraded":"2024-05-08T04:42:55.260852+0000","last_fullsized":"2024-05-08T04:42:55.260852+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:37.443525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645304+0000","last_change":"2024-05-08T04:42:54.282790+0000","last_active":"2024-05-08T04:42:55.645304+0000","last_peered":"2024-05-08T04:42:55.645304+0000","last_clean":"2024-05-08T04:42:55.645304+0000","last_became_active":"2024-05-08T04:42:54.282523+0000","last_became_peered":"2024-05-08T04:42:54.282523+0000","last_unstale":"2024-05-08T04:42:55.645304+0000","last_undegraded":"2024-05-08T04:42:55.645304+0000","last_fullsized":"2024-05-08T04:42:55.645304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.467995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261546+0000","last_change":"2024-05-08T04:42:53.252707+0000","last_active":"2024-05-08T04:42:55.261546+0000","last_peered":"2024-05-08T04:42:55.261546+0000","last_clean":"2024-05-08T04:42:55.261546+0000","last_became_active":"2024-05-08T04:42:53.250660+0000","last_became_peered":"2024-05-08T04:42:53.250660+0000","last_unstale":"2024-05-08T04:42:55.261546+0000","last_undegraded":"2024-05-08T04:42:55.261546+0000","last_fullsized":"2024-05-08T04:42:55.261546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:13.437315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644517+0000","last_change":"2024-05-08T04:42:54.277076+0000","last_active":"2024-05-08T04:42:55.644517+0000","last_peered":"2024-05-08T04:42:55.644517+0000","last_clean":"2024-05-08T04:42:55.644517+0000","last_became_active":"2024-05-08T04:42:54.276805+0000","last_became_peered":"2024-05-08T04:42:54.276805+0000","last_unstale":"2024-05-08T04:42:55.644517+0000","last_undegraded":"2024-05-08T04:42:55.644517+0000","last_fullsized":"2024-05-08T04:42:55.644517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:39.976473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644732+0000","last_change":"2024-05-08T04:42:53.252678+0000","last_active":"2024-05-08T04:42:55.644732+0000","last_peered":"2024-05-08T04:42:55.644732+0000","last_clean":"2024-05-08T04:42:55.644732+0000","last_became_active":"2024-05-08T04:42:53.252426+0000","last_became_peered":"2024-05-08T04:42:53.252426+0000","last_unstale":"2024-05-08T04:42:55.644732+0000","last_undegraded":"2024-05-08T04:42:55.644732+0000","last_fullsized":"2024-05-08T04:42:55.644732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:50.935297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275599+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.274979+0000","last_became_peered":"2024-05-08T04:42:54.274979+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:27.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260948+0000","last_change":"2024-05-08T04:42:53.262988+0000","last_active":"2024-05-08T04:42:55.260948+0000","last_peered":"2024-05-08T04:42:55.260948+0000","last_clean":"2024-05-08T04:42:55.260948+0000","last_became_active":"2024-05-08T04:42:53.262721+0000","last_became_peered":"2024-05-08T04:42:53.262721+0000","last_unstale":"2024-05-08T04:42:55.260948+0000","last_undegraded":"2024-05-08T04:42:55.260948+0000","last_fullsized":"2024-05-08T04:42:55.260948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:07.710944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259403+0000","last_change":"2024-05-08T04:42:54.250195+0000","last_active":"2024-05-08T04:42:55.259403+0000","last_peered":"2024-05-08T04:42:55.259403+0000","last_clean":"2024-05-08T04:42:55.259403+0000","last_became_active":"2024-05-08T04:42:54.249297+0000","last_became_peered":"2024-05-08T04:42:54.249297+0000","last_unstale":"2024-05-08T04:42:55.259403+0000","last_undegraded":"2024-05-08T04:42:55.259403+0000","last_fullsized":"2024-05-08T04:42:55.259403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259421+0000","last_change":"2024-05-08T04:42:53.259450+0000","last_active":"2024-05-08T04:42:55.259421+0000","last_peered":"2024-05-08T04:42:55.259421+0000","last_clean":"2024-05-08T04:42:55.259421+0000","last_became_active":"2024-05-08T04:42:53.259139+0000","last_became_peered":"2024-05-08T04:42:53.259139+0000","last_unstale":"2024-05-08T04:42:55.259421+0000","last_undegraded":"2024-05-08T04:42:55.259421+0000","last_fullsized":"2024-05-08T04:42:55.259421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260413+0000","last_change":"2024-05-08T04:42:54.263700+0000","last_active":"2024-05-08T04:42:55.260413+0000","last_peered":"2024-05-08T04:42:55.260413+0000","last_clean":"2024-05-08T04:42:55.260413+0000","last_became_active":"2024-05-08T04:42:54.263395+0000","last_became_peered":"2024-05-08T04:42:54.263395+0000","last_unstale":"2024-05-08T04:42:55.260413+0000","last_undegraded":"2024-05-08T04:42:55.260413+0000","last_fullsized":"2024-05-08T04:42:55.260413+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:40.094631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259409+0000","last_change":"2024-05-08T04:42:53.258649+0000","last_active":"2024-05-08T04:42:55.259409+0000","last_peered":"2024-05-08T04:42:55.259409+0000","last_clean":"2024-05-08T04:42:55.259409+0000","last_became_active":"2024-05-08T04:42:53.258348+0000","last_became_peered":"2024-05-08T04:42:53.258348+0000","last_unstale":"2024-05-08T04:42:55.259409+0000","last_undegraded":"2024-05-08T04:42:55.259409+0000","last_fullsized":"2024-05-08T04:42:55.259409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:42.312320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543824+0000","last_change":"2024-05-08T04:42:54.266324+0000","last_active":"2024-05-08T04:42:55.543824+0000","last_peered":"2024-05-08T04:42:55.543824+0000","last_clean":"2024-05-08T04:42:55.543824+0000","last_became_active":"2024-05-08T04:42:54.265475+0000","last_became_peered":"2024-05-08T04:42:54.265475+0000","last_unstale":"2024-05-08T04:42:55.543824+0000","last_undegraded":"2024-05-08T04:42:55.543824+0000","last_fullsized":"2024-05-08T04:42:55.543824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643830+0000","last_change":"2024-05-08T04:42:53.253674+0000","last_active":"2024-05-08T04:42:55.643830+0000","last_peered":"2024-05-08T04:42:55.643830+0000","last_clean":"2024-05-08T04:42:55.643830+0000","last_became_active":"2024-05-08T04:42:53.253392+0000","last_became_peered":"2024-05-08T04:42:53.253392+0000","last_unstale":"2024-05-08T04:42:55.643830+0000","last_undegraded":"2024-05-08T04:42:55.643830+0000","last_fullsized":"2024-05-08T04:42:55.643830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643920+0000","last_change":"2024-05-08T04:42:54.270703+0000","last_active":"2024-05-08T04:42:55.643920+0000","last_peered":"2024-05-08T04:42:55.643920+0000","last_clean":"2024-05-08T04:42:55.643920+0000","last_became_active":"2024-05-08T04:42:54.265604+0000","last_became_peered":"2024-05-08T04:42:54.265604+0000","last_unstale":"2024-05-08T04:42:55.643920+0000","last_undegraded":"2024-05-08T04:42:55.643920+0000","last_fullsized":"2024-05-08T04:42:55.643920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:21.915088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643694+0000","last_change":"2024-05-08T04:42:53.253094+0000","last_active":"2024-05-08T04:42:55.643694+0000","last_peered":"2024-05-08T04:42:55.643694+0000","last_clean":"2024-05-08T04:42:55.643694+0000","last_became_active":"2024-05-08T04:42:53.252291+0000","last_became_peered":"2024-05-08T04:42:53.252291+0000","last_unstale":"2024-05-08T04:42:55.643694+0000","last_undegraded":"2024-05-08T04:42:55.643694+0000","last_fullsized":"2024-05-08T04:42:55.643694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:59.366123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260367+0000","last_change":"2024-05-08T04:42:54.254877+0000","last_active":"2024-05-08T04:42:55.260367+0000","last_peered":"2024-05-08T04:42:55.260367+0000","last_clean":"2024-05-08T04:42:55.260367+0000","last_became_active":"2024-05-08T04:42:54.254581+0000","last_became_peered":"2024-05-08T04:42:54.254581+0000","last_unstale":"2024-05-08T04:42:55.260367+0000","last_undegraded":"2024-05-08T04:42:55.260367+0000","last_fullsized":"2024-05-08T04:42:55.260367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:48.608906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258784+0000","last_change":"2024-05-08T04:42:53.257287+0000","last_active":"2024-05-08T04:42:55.258784+0000","last_peered":"2024-05-08T04:42:55.258784+0000","last_clean":"2024-05-08T04:42:55.258784+0000","last_became_active":"2024-05-08T04:42:53.256851+0000","last_became_peered":"2024-05-08T04:42:53.256851+0000","last_unstale":"2024-05-08T04:42:55.258784+0000","last_undegraded":"2024-05-08T04:42:55.258784+0000","last_fullsized":"2024-05-08T04:42:55.258784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543931+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:55.543931+0000","last_peered":"2024-05-08T04:42:55.543931+0000","last_clean":"2024-05-08T04:42:55.543931+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:55.543931+0000","last_undegraded":"2024-05-08T04:42:55.543931+0000","last_fullsized":"2024-05-08T04:42:55.543931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":1198126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":607212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":607746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":608824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":607266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:43:02.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:43:02.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 Processor -- start 2024-05-08T04:43:02.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 -- start start 2024-05-08T04:43:02.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe7f8144200 0x7fe7f8161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:02.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f8161fe0 con 0x7fe7f80cfd60 2024-05-08T04:43:02.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f81622c0 con 0x7fe7f815ea20 2024-05-08T04:43:02.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7fde00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f8162880 con 0x7fe7f8144200 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7ee000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe7f80cfd60 0x7fe7f80cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37288/0 (socket says 172.21.15.164:37288) 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7f6a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe7f8144200 0x7fe7f8161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7ee000640 1 -- 172.21.15.164:0/3344692365 learned_addr learned my addr 172.21.15.164:0/3344692365 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.995+0000 7fe7f6a00640 1 -- 172.21.15.164:0/3344692365 >> v1:172.21.15.172:6789/0 conn(0x7fe7f815ea20 legacy=0x7fe7f80ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6a00640 1 -- 172.21.15.164:0/3344692365 >> v1:172.21.15.164:6789/0 conn(0x7fe7f80cfd60 legacy=0x7fe7f80cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:02.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6a00640 1 -- 172.21.15.164:0/3344692365 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f81625a0 con 0x7fe7f8144200 2024-05-08T04:43:03.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6a00640 1 --2- 172.21.15.164:0/3344692365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f8161950 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097c0 tx=0x7fe7e001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d70e6c9a8be2f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6000640 1 -- 172.21.15.164:0/3344692365 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00b0070 con 0x7fe7f8144200 2024-05-08T04:43:03.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6000640 1 -- 172.21.15.164:0/3344692365 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e00ab600 con 0x7fe7f8144200 2024-05-08T04:43:03.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7f6000640 1 -- 172.21.15.164:0/3344692365 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00ab9d0 con 0x7fe7f8144200 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 -- 172.21.15.164:0/3344692365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 msgr2=0x7fe7f8161950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3344692365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f8161950 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00097c0 tx=0x7fe7e001c260 comp rx=0 tx=0).stop 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 -- 172.21.15.164:0/3344692365 shutdown_connections 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3344692365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f8161950 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 -- 172.21.15.164:0/3344692365 >> 172.21.15.164:0/3344692365 conn(0x7fe7f806ac90 msgr2=0x7fe7f806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 -- 172.21.15.164:0/3344692365 shutdown_connections 2024-05-08T04:43:03.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:02.999+0000 7fe7fde00640 1 -- 172.21.15.164:0/3344692365 wait complete. 2024-05-08T04:43:03.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 Processor -- start 2024-05-08T04:43:03.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- start start 2024-05-08T04:43:03.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7f80cfd60 0x7fe7f81f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 0x7fe7f815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80cc7c0 con 0x7fe7f8144200 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80d01d0 con 0x7fe7f815ea20 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f80cc940 con 0x7fe7f80cfd60 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40606/0 (socket says 172.21.15.164:40606) 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 -- 172.21.15.164:0/3942529638 learned_addr learned my addr 172.21.15.164:0/3942529638 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7f6a00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7f80cfd60 0x7fe7f81f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7f7400640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 0x7fe7f815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 -- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7f80cfd60 msgr2=0x7fe7f81f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7f80cfd60 0x7fe7f81f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 -- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 msgr2=0x7fe7f815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 0x7fe7f815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 -- 172.21.15.164:0/3942529638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8158b80 con 0x7fe7f8144200 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7f7400640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 0x7fe7f815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7ee000640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c780 tx=0x7fe7e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8018070 con 0x7fe7f8144200 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80072d0 con 0x7fe7f8144200 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80047f0 con 0x7fe7f8144200 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f8158e60 con 0x7fe7f8144200 2024-05-08T04:43:03.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.003+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f8159420 con 0x7fe7f8144200 2024-05-08T04:43:03.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f8069d60 con 0x7fe7f8144200 2024-05-08T04:43:03.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fe7e801b070 con 0x7fe7f8144200 2024-05-08T04:43:03.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7efe00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 0x7fe7d4069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe7e80856d0 con 0x7fe7f8144200 2024-05-08T04:43:03.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7f6a00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 0x7fe7d4069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.007+0000 7fe7f6a00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 0x7fe7d4069d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ad000 tx=0x7fe7e00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.011+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7e80552e0 con 0x7fe7f8144200 2024-05-08T04:43:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.163+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7f813bcf0 con 0x7fe7d40678b0 2024-05-08T04:43:03.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.171+0000 7fe7efe00640 1 -- 172.21.15.164:0/3942529638 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fe7f813bcf0 con 0x7fe7d40678b0 2024-05-08T04:43:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:03.176 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 msgr2=0x7fe7d4069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 0x7fe7d4069d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ad000 tx=0x7fe7e00a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 msgr2=0x7fe7f815a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c780 tx=0x7fe7e800cc40 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 shutdown_connections 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe7d40678b0 0x7fe7d4069d70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7f815ea20 0x7fe7f815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f8144200 0x7fe7f815a0a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 --2- 172.21.15.164:0/3942529638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7f80cfd60 0x7fe7f81f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 >> 172.21.15.164:0/3942529638 conn(0x7fe7f806ac90 msgr2=0x7fe7f80adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 shutdown_connections 2024-05-08T04:43:03.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.175+0000 7fe7fde00640 1 -- 172.21.15.164:0/3942529638 wait complete. 2024-05-08T04:43:03.196 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-08T04:43:01.485100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261572,"kb_used_data":12104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713148,"statfs":{"total":773094113280,"available":772826263552,"internally_reserved":0,"allocated":12394496,"data_stored":6042692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001766"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645115+0000","last_change":"2024-05-08T04:42:54.284520+0000","last_active":"2024-05-08T04:42:55.645115+0000","last_peered":"2024-05-08T04:42:55.645115+0000","last_clean":"2024-05-08T04:42:55.645115+0000","last_became_active":"2024-05-08T04:42:54.283787+0000","last_became_peered":"2024-05-08T04:42:54.283787+0000","last_unstale":"2024-05-08T04:42:55.645115+0000","last_undegraded":"2024-05-08T04:42:55.645115+0000","last_fullsized":"2024-05-08T04:42:55.645115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:31.361703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260638+0000","last_change":"2024-05-08T04:42:54.267059+0000","last_active":"2024-05-08T04:42:55.260638+0000","last_peered":"2024-05-08T04:42:55.260638+0000","last_clean":"2024-05-08T04:42:55.260638+0000","last_became_active":"2024-05-08T04:42:54.266748+0000","last_became_peered":"2024-05-08T04:42:54.266748+0000","last_unstale":"2024-05-08T04:42:55.260638+0000","last_undegraded":"2024-05-08T04:42:55.260638+0000","last_fullsized":"2024-05-08T04:42:55.260638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:57.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543833+0000","last_change":"2024-05-08T04:42:54.259254+0000","last_active":"2024-05-08T04:42:55.543833+0000","last_peered":"2024-05-08T04:42:55.543833+0000","last_clean":"2024-05-08T04:42:55.543833+0000","last_became_active":"2024-05-08T04:42:54.258620+0000","last_became_peered":"2024-05-08T04:42:54.258620+0000","last_unstale":"2024-05-08T04:42:55.543833+0000","last_undegraded":"2024-05-08T04:42:55.543833+0000","last_fullsized":"2024-05-08T04:42:55.543833+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644261+0000","last_change":"2024-05-08T04:42:54.276298+0000","last_active":"2024-05-08T04:42:55.644261+0000","last_peered":"2024-05-08T04:42:55.644261+0000","last_clean":"2024-05-08T04:42:55.644261+0000","last_became_active":"2024-05-08T04:42:54.273697+0000","last_became_peered":"2024-05-08T04:42:54.273697+0000","last_unstale":"2024-05-08T04:42:55.644261+0000","last_undegraded":"2024-05-08T04:42:55.644261+0000","last_fullsized":"2024-05-08T04:42:55.644261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:45.521599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644231+0000","last_change":"2024-05-08T04:42:54.271374+0000","last_active":"2024-05-08T04:42:55.644231+0000","last_peered":"2024-05-08T04:42:55.644231+0000","last_clean":"2024-05-08T04:42:55.644231+0000","last_became_active":"2024-05-08T04:42:54.267689+0000","last_became_peered":"2024-05-08T04:42:54.267689+0000","last_unstale":"2024-05-08T04:42:55.644231+0000","last_undegraded":"2024-05-08T04:42:55.644231+0000","last_fullsized":"2024-05-08T04:42:55.644231+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:16.960439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644461+0000","last_change":"2024-05-08T04:42:54.259042+0000","last_active":"2024-05-08T04:42:55.644461+0000","last_peered":"2024-05-08T04:42:55.644461+0000","last_clean":"2024-05-08T04:42:55.644461+0000","last_became_active":"2024-05-08T04:42:54.258775+0000","last_became_peered":"2024-05-08T04:42:54.258775+0000","last_unstale":"2024-05-08T04:42:55.644461+0000","last_undegraded":"2024-05-08T04:42:55.644461+0000","last_fullsized":"2024-05-08T04:42:55.644461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:58.137321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258732+0000","last_change":"2024-05-08T04:42:54.258159+0000","last_active":"2024-05-08T04:42:55.258732+0000","last_peered":"2024-05-08T04:42:55.258732+0000","last_clean":"2024-05-08T04:42:55.258732+0000","last_became_active":"2024-05-08T04:42:54.257892+0000","last_became_peered":"2024-05-08T04:42:54.257892+0000","last_unstale":"2024-05-08T04:42:55.258732+0000","last_undegraded":"2024-05-08T04:42:55.258732+0000","last_fullsized":"2024-05-08T04:42:55.258732+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543525+0000","last_change":"2024-05-08T04:42:54.261073+0000","last_active":"2024-05-08T04:42:55.543525+0000","last_peered":"2024-05-08T04:42:55.543525+0000","last_clean":"2024-05-08T04:42:55.543525+0000","last_became_active":"2024-05-08T04:42:54.260783+0000","last_became_peered":"2024-05-08T04:42:54.260783+0000","last_unstale":"2024-05-08T04:42:55.543525+0000","last_undegraded":"2024-05-08T04:42:55.543525+0000","last_fullsized":"2024-05-08T04:42:55.543525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645257+0000","last_change":"2024-05-08T04:42:54.280775+0000","last_active":"2024-05-08T04:42:55.645257+0000","last_peered":"2024-05-08T04:42:55.645257+0000","last_clean":"2024-05-08T04:42:55.645257+0000","last_became_active":"2024-05-08T04:42:54.280501+0000","last_became_peered":"2024-05-08T04:42:54.280501+0000","last_unstale":"2024-05-08T04:42:55.645257+0000","last_undegraded":"2024-05-08T04:42:55.645257+0000","last_fullsized":"2024-05-08T04:42:55.645257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:25.588352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645180+0000","last_change":"2024-05-08T04:42:54.279039+0000","last_active":"2024-05-08T04:42:55.645180+0000","last_peered":"2024-05-08T04:42:55.645180+0000","last_clean":"2024-05-08T04:42:55.645180+0000","last_became_active":"2024-05-08T04:42:54.278516+0000","last_became_peered":"2024-05-08T04:42:54.278516+0000","last_unstale":"2024-05-08T04:42:55.645180+0000","last_undegraded":"2024-05-08T04:42:55.645180+0000","last_fullsized":"2024-05-08T04:42:55.645180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:44.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260298+0000","last_change":"2024-05-08T04:42:54.264244+0000","last_active":"2024-05-08T04:42:55.260298+0000","last_peered":"2024-05-08T04:42:55.260298+0000","last_clean":"2024-05-08T04:42:55.260298+0000","last_became_active":"2024-05-08T04:42:54.261285+0000","last_became_peered":"2024-05-08T04:42:54.261285+0000","last_unstale":"2024-05-08T04:42:55.260298+0000","last_undegraded":"2024-05-08T04:42:55.260298+0000","last_fullsized":"2024-05-08T04:42:55.260298+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:16.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644393+0000","last_change":"2024-05-08T04:42:54.278540+0000","last_active":"2024-05-08T04:42:55.644393+0000","last_peered":"2024-05-08T04:42:55.644393+0000","last_clean":"2024-05-08T04:42:55.644393+0000","last_became_active":"2024-05-08T04:42:54.275676+0000","last_became_peered":"2024-05-08T04:42:54.275676+0000","last_unstale":"2024-05-08T04:42:55.644393+0000","last_undegraded":"2024-05-08T04:42:55.644393+0000","last_fullsized":"2024-05-08T04:42:55.644393+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:02.962699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543936+0000","last_change":"2024-05-08T04:42:54.262625+0000","last_active":"2024-05-08T04:42:55.543936+0000","last_peered":"2024-05-08T04:42:55.543936+0000","last_clean":"2024-05-08T04:42:55.543936+0000","last_became_active":"2024-05-08T04:42:54.262328+0000","last_became_peered":"2024-05-08T04:42:54.262328+0000","last_unstale":"2024-05-08T04:42:55.543936+0000","last_undegraded":"2024-05-08T04:42:55.543936+0000","last_fullsized":"2024-05-08T04:42:55.543936+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:37.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543497+0000","last_change":"2024-05-08T04:42:54.263202+0000","last_active":"2024-05-08T04:42:55.543497+0000","last_peered":"2024-05-08T04:42:55.543497+0000","last_clean":"2024-05-08T04:42:55.543497+0000","last_became_active":"2024-05-08T04:42:54.262118+0000","last_became_peered":"2024-05-08T04:42:54.262118+0000","last_unstale":"2024-05-08T04:42:55.543497+0000","last_undegraded":"2024-05-08T04:42:55.543497+0000","last_fullsized":"2024-05-08T04:42:55.543497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643757+0000","last_change":"2024-05-08T04:42:54.258584+0000","last_active":"2024-05-08T04:42:55.643757+0000","last_peered":"2024-05-08T04:42:55.643757+0000","last_clean":"2024-05-08T04:42:55.643757+0000","last_became_active":"2024-05-08T04:42:54.257809+0000","last_became_peered":"2024-05-08T04:42:54.257809+0000","last_unstale":"2024-05-08T04:42:55.643757+0000","last_undegraded":"2024-05-08T04:42:55.643757+0000","last_fullsized":"2024-05-08T04:42:55.643757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.486563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643534+0000","last_change":"2024-05-08T04:42:54.272568+0000","last_active":"2024-05-08T04:42:55.643534+0000","last_peered":"2024-05-08T04:42:55.643534+0000","last_clean":"2024-05-08T04:42:55.643534+0000","last_became_active":"2024-05-08T04:42:54.266872+0000","last_became_peered":"2024-05-08T04:42:54.266872+0000","last_unstale":"2024-05-08T04:42:55.643534+0000","last_undegraded":"2024-05-08T04:42:55.643534+0000","last_fullsized":"2024-05-08T04:42:55.643534+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:19.666610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260434+0000","last_change":"2024-05-08T04:42:54.265221+0000","last_active":"2024-05-08T04:42:55.260434+0000","last_peered":"2024-05-08T04:42:55.260434+0000","last_clean":"2024-05-08T04:42:55.260434+0000","last_became_active":"2024-05-08T04:42:54.263935+0000","last_became_peered":"2024-05-08T04:42:54.263935+0000","last_unstale":"2024-05-08T04:42:55.260434+0000","last_undegraded":"2024-05-08T04:42:55.260434+0000","last_fullsized":"2024-05-08T04:42:55.260434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:24.087153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261472+0000","last_change":"2024-05-08T04:42:54.260979+0000","last_active":"2024-05-08T04:42:55.261472+0000","last_peered":"2024-05-08T04:42:55.261472+0000","last_clean":"2024-05-08T04:42:55.261472+0000","last_became_active":"2024-05-08T04:42:54.260386+0000","last_became_peered":"2024-05-08T04:42:54.260386+0000","last_unstale":"2024-05-08T04:42:55.261472+0000","last_undegraded":"2024-05-08T04:42:55.261472+0000","last_fullsized":"2024-05-08T04:42:55.261472+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:23.982806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543852+0000","last_change":"2024-05-08T04:42:54.267987+0000","last_active":"2024-05-08T04:42:55.543852+0000","last_peered":"2024-05-08T04:42:55.543852+0000","last_clean":"2024-05-08T04:42:55.543852+0000","last_became_active":"2024-05-08T04:42:54.267674+0000","last_became_peered":"2024-05-08T04:42:54.267674+0000","last_unstale":"2024-05-08T04:42:55.543852+0000","last_undegraded":"2024-05-08T04:42:55.543852+0000","last_fullsized":"2024-05-08T04:42:55.543852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:11.284074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259984+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.259984+0000","last_peered":"2024-05-08T04:42:55.259984+0000","last_clean":"2024-05-08T04:42:55.259984+0000","last_became_active":"2024-05-08T04:42:54.261903+0000","last_became_peered":"2024-05-08T04:42:54.261903+0000","last_unstale":"2024-05-08T04:42:55.259984+0000","last_undegraded":"2024-05-08T04:42:55.259984+0000","last_fullsized":"2024-05-08T04:42:55.259984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:49.660293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260474+0000","last_change":"2024-05-08T04:42:54.247853+0000","last_active":"2024-05-08T04:42:55.260474+0000","last_peered":"2024-05-08T04:42:55.260474+0000","last_clean":"2024-05-08T04:42:55.260474+0000","last_became_active":"2024-05-08T04:42:54.247314+0000","last_became_peered":"2024-05-08T04:42:54.247314+0000","last_unstale":"2024-05-08T04:42:55.260474+0000","last_undegraded":"2024-05-08T04:42:55.260474+0000","last_fullsized":"2024-05-08T04:42:55.260474+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.153233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260564+0000","last_change":"2024-05-08T04:42:54.263380+0000","last_active":"2024-05-08T04:42:55.260564+0000","last_peered":"2024-05-08T04:42:55.260564+0000","last_clean":"2024-05-08T04:42:55.260564+0000","last_became_active":"2024-05-08T04:42:54.262268+0000","last_became_peered":"2024-05-08T04:42:54.262268+0000","last_unstale":"2024-05-08T04:42:55.260564+0000","last_undegraded":"2024-05-08T04:42:55.260564+0000","last_fullsized":"2024-05-08T04:42:55.260564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258982+0000","last_change":"2024-05-08T04:42:54.257923+0000","last_active":"2024-05-08T04:42:55.258982+0000","last_peered":"2024-05-08T04:42:55.258982+0000","last_clean":"2024-05-08T04:42:55.258982+0000","last_became_active":"2024-05-08T04:42:54.257524+0000","last_became_peered":"2024-05-08T04:42:54.257524+0000","last_unstale":"2024-05-08T04:42:55.258982+0000","last_undegraded":"2024-05-08T04:42:55.258982+0000","last_fullsized":"2024-05-08T04:42:55.258982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260354+0000","last_change":"2024-05-08T04:42:54.258040+0000","last_active":"2024-05-08T04:42:55.260354+0000","last_peered":"2024-05-08T04:42:55.260354+0000","last_clean":"2024-05-08T04:42:55.260354+0000","last_became_active":"2024-05-08T04:42:54.257775+0000","last_became_peered":"2024-05-08T04:42:54.257775+0000","last_unstale":"2024-05-08T04:42:55.260354+0000","last_undegraded":"2024-05-08T04:42:55.260354+0000","last_fullsized":"2024-05-08T04:42:55.260354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:17.593712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260632+0000","last_change":"2024-05-08T04:42:54.264625+0000","last_active":"2024-05-08T04:42:55.260632+0000","last_peered":"2024-05-08T04:42:55.260632+0000","last_clean":"2024-05-08T04:42:55.260632+0000","last_became_active":"2024-05-08T04:42:54.261407+0000","last_became_peered":"2024-05-08T04:42:54.261407+0000","last_unstale":"2024-05-08T04:42:55.260632+0000","last_undegraded":"2024-05-08T04:42:55.260632+0000","last_fullsized":"2024-05-08T04:42:55.260632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:21.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644944+0000","last_change":"2024-05-08T04:42:54.273444+0000","last_active":"2024-05-08T04:42:55.644944+0000","last_peered":"2024-05-08T04:42:55.644944+0000","last_clean":"2024-05-08T04:42:55.644944+0000","last_became_active":"2024-05-08T04:42:54.273184+0000","last_became_peered":"2024-05-08T04:42:54.273184+0000","last_unstale":"2024-05-08T04:42:55.644944+0000","last_undegraded":"2024-05-08T04:42:55.644944+0000","last_fullsized":"2024-05-08T04:42:55.644944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:18.445404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645146+0000","last_change":"2024-05-08T04:42:54.277145+0000","last_active":"2024-05-08T04:42:55.645146+0000","last_peered":"2024-05-08T04:42:55.645146+0000","last_clean":"2024-05-08T04:42:55.645146+0000","last_became_active":"2024-05-08T04:42:54.274239+0000","last_became_peered":"2024-05-08T04:42:54.274239+0000","last_unstale":"2024-05-08T04:42:55.645146+0000","last_undegraded":"2024-05-08T04:42:55.645146+0000","last_fullsized":"2024-05-08T04:42:55.645146+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260902+0000","last_change":"2024-05-08T04:42:54.279227+0000","last_active":"2024-05-08T04:42:55.260902+0000","last_peered":"2024-05-08T04:42:55.260902+0000","last_clean":"2024-05-08T04:42:55.260902+0000","last_became_active":"2024-05-08T04:42:54.278953+0000","last_became_peered":"2024-05-08T04:42:54.278953+0000","last_unstale":"2024-05-08T04:42:55.260902+0000","last_undegraded":"2024-05-08T04:42:55.260902+0000","last_fullsized":"2024-05-08T04:42:55.260902+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260283+0000","last_change":"2024-05-08T04:42:54.257830+0000","last_active":"2024-05-08T04:42:55.260283+0000","last_peered":"2024-05-08T04:42:55.260283+0000","last_clean":"2024-05-08T04:42:55.260283+0000","last_became_active":"2024-05-08T04:42:54.257455+0000","last_became_peered":"2024-05-08T04:42:54.257455+0000","last_unstale":"2024-05-08T04:42:55.260283+0000","last_undegraded":"2024-05-08T04:42:55.260283+0000","last_fullsized":"2024-05-08T04:42:55.260283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:08.717198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259104+0000","last_change":"2024-05-08T04:42:54.247280+0000","last_active":"2024-05-08T04:42:55.259104+0000","last_peered":"2024-05-08T04:42:55.259104+0000","last_clean":"2024-05-08T04:42:55.259104+0000","last_became_active":"2024-05-08T04:42:54.246609+0000","last_became_peered":"2024-05-08T04:42:54.246609+0000","last_unstale":"2024-05-08T04:42:55.259104+0000","last_undegraded":"2024-05-08T04:42:55.259104+0000","last_fullsized":"2024-05-08T04:42:55.259104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:48:17.490916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643671+0000","last_change":"2024-05-08T04:42:54.270435+0000","last_active":"2024-05-08T04:42:55.643671+0000","last_peered":"2024-05-08T04:42:55.643671+0000","last_clean":"2024-05-08T04:42:55.643671+0000","last_became_active":"2024-05-08T04:42:54.265729+0000","last_became_peered":"2024-05-08T04:42:54.265729+0000","last_unstale":"2024-05-08T04:42:55.643671+0000","last_undegraded":"2024-05-08T04:42:55.643671+0000","last_fullsized":"2024-05-08T04:42:55.643671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:53.630938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643688+0000","last_change":"2024-05-08T04:42:54.261182+0000","last_active":"2024-05-08T04:42:55.643688+0000","last_peered":"2024-05-08T04:42:55.643688+0000","last_clean":"2024-05-08T04:42:55.643688+0000","last_became_active":"2024-05-08T04:42:54.260895+0000","last_became_peered":"2024-05-08T04:42:54.260895+0000","last_unstale":"2024-05-08T04:42:55.643688+0000","last_undegraded":"2024-05-08T04:42:55.643688+0000","last_fullsized":"2024-05-08T04:42:55.643688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645113+0000","last_change":"2024-05-08T04:42:54.271090+0000","last_active":"2024-05-08T04:42:55.645113+0000","last_peered":"2024-05-08T04:42:55.645113+0000","last_clean":"2024-05-08T04:42:55.645113+0000","last_became_active":"2024-05-08T04:42:54.270874+0000","last_became_peered":"2024-05-08T04:42:54.270874+0000","last_unstale":"2024-05-08T04:42:55.645113+0000","last_undegraded":"2024-05-08T04:42:55.645113+0000","last_fullsized":"2024-05-08T04:42:55.645113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:04.064320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259863+0000","last_change":"2024-05-08T04:42:53.251819+0000","last_active":"2024-05-08T04:42:55.259863+0000","last_peered":"2024-05-08T04:42:55.259863+0000","last_clean":"2024-05-08T04:42:55.259863+0000","last_became_active":"2024-05-08T04:42:53.251490+0000","last_became_peered":"2024-05-08T04:42:53.251490+0000","last_unstale":"2024-05-08T04:42:55.259863+0000","last_undegraded":"2024-05-08T04:42:55.259863+0000","last_fullsized":"2024-05-08T04:42:55.259863+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:07.081658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261626+0000","last_change":"2024-05-08T04:42:54.262175+0000","last_active":"2024-05-08T04:42:55.261626+0000","last_peered":"2024-05-08T04:42:55.261626+0000","last_clean":"2024-05-08T04:42:55.261626+0000","last_became_active":"2024-05-08T04:42:54.261894+0000","last_became_peered":"2024-05-08T04:42:54.261894+0000","last_unstale":"2024-05-08T04:42:55.261626+0000","last_undegraded":"2024-05-08T04:42:55.261626+0000","last_fullsized":"2024-05-08T04:42:55.261626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543705+0000","last_change":"2024-05-08T04:42:53.256430+0000","last_active":"2024-05-08T04:42:55.543705+0000","last_peered":"2024-05-08T04:42:55.543705+0000","last_clean":"2024-05-08T04:42:55.543705+0000","last_became_active":"2024-05-08T04:42:53.256154+0000","last_became_peered":"2024-05-08T04:42:53.256154+0000","last_unstale":"2024-05-08T04:42:55.543705+0000","last_undegraded":"2024-05-08T04:42:55.543705+0000","last_fullsized":"2024-05-08T04:42:55.543705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644955+0000","last_change":"2024-05-08T04:42:54.264075+0000","last_active":"2024-05-08T04:42:55.644955+0000","last_peered":"2024-05-08T04:42:55.644955+0000","last_clean":"2024-05-08T04:42:55.644955+0000","last_became_active":"2024-05-08T04:42:54.263803+0000","last_became_peered":"2024-05-08T04:42:54.263803+0000","last_unstale":"2024-05-08T04:42:55.644955+0000","last_undegraded":"2024-05-08T04:42:55.644955+0000","last_fullsized":"2024-05-08T04:42:55.644955+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644201+0000","last_change":"2024-05-08T04:42:53.257257+0000","last_active":"2024-05-08T04:42:55.644201+0000","last_peered":"2024-05-08T04:42:55.644201+0000","last_clean":"2024-05-08T04:42:55.644201+0000","last_became_active":"2024-05-08T04:42:53.256901+0000","last_became_peered":"2024-05-08T04:42:53.256901+0000","last_unstale":"2024-05-08T04:42:55.644201+0000","last_undegraded":"2024-05-08T04:42:55.644201+0000","last_fullsized":"2024-05-08T04:42:55.644201+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544048+0000","last_change":"2024-05-08T04:42:54.279237+0000","last_active":"2024-05-08T04:42:55.544048+0000","last_peered":"2024-05-08T04:42:55.544048+0000","last_clean":"2024-05-08T04:42:55.544048+0000","last_became_active":"2024-05-08T04:42:54.279021+0000","last_became_peered":"2024-05-08T04:42:54.279021+0000","last_unstale":"2024-05-08T04:42:55.544048+0000","last_undegraded":"2024-05-08T04:42:55.544048+0000","last_fullsized":"2024-05-08T04:42:55.544048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:04:02.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643992+0000","last_change":"2024-05-08T04:42:53.254748+0000","last_active":"2024-05-08T04:42:55.643992+0000","last_peered":"2024-05-08T04:42:55.643992+0000","last_clean":"2024-05-08T04:42:55.643992+0000","last_became_active":"2024-05-08T04:42:53.253422+0000","last_became_peered":"2024-05-08T04:42:53.253422+0000","last_unstale":"2024-05-08T04:42:55.643992+0000","last_undegraded":"2024-05-08T04:42:55.643992+0000","last_fullsized":"2024-05-08T04:42:55.643992+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:23.946191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644157+0000","last_change":"2024-05-08T04:42:54.270324+0000","last_active":"2024-05-08T04:42:55.644157+0000","last_peered":"2024-05-08T04:42:55.644157+0000","last_clean":"2024-05-08T04:42:55.644157+0000","last_became_active":"2024-05-08T04:42:54.269819+0000","last_became_peered":"2024-05-08T04:42:54.269819+0000","last_unstale":"2024-05-08T04:42:55.644157+0000","last_undegraded":"2024-05-08T04:42:55.644157+0000","last_fullsized":"2024-05-08T04:42:55.644157+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:09.118707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259307+0000","last_change":"2024-05-08T04:42:53.257209+0000","last_active":"2024-05-08T04:42:55.259307+0000","last_peered":"2024-05-08T04:42:55.259307+0000","last_clean":"2024-05-08T04:42:55.259307+0000","last_became_active":"2024-05-08T04:42:53.256908+0000","last_became_peered":"2024-05-08T04:42:53.256908+0000","last_unstale":"2024-05-08T04:42:55.259307+0000","last_undegraded":"2024-05-08T04:42:55.259307+0000","last_fullsized":"2024-05-08T04:42:55.259307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259796+0000","last_change":"2024-05-08T04:42:54.263171+0000","last_active":"2024-05-08T04:42:55.259796+0000","last_peered":"2024-05-08T04:42:55.259796+0000","last_clean":"2024-05-08T04:42:55.259796+0000","last_became_active":"2024-05-08T04:42:54.262760+0000","last_became_peered":"2024-05-08T04:42:54.262760+0000","last_unstale":"2024-05-08T04:42:55.259796+0000","last_undegraded":"2024-05-08T04:42:55.259796+0000","last_fullsized":"2024-05-08T04:42:55.259796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260608+0000","last_change":"2024-05-08T04:42:53.261630+0000","last_active":"2024-05-08T04:42:55.260608+0000","last_peered":"2024-05-08T04:42:55.260608+0000","last_clean":"2024-05-08T04:42:55.260608+0000","last_became_active":"2024-05-08T04:42:53.261358+0000","last_became_peered":"2024-05-08T04:42:53.261358+0000","last_unstale":"2024-05-08T04:42:55.260608+0000","last_undegraded":"2024-05-08T04:42:55.260608+0000","last_fullsized":"2024-05-08T04:42:55.260608+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:07.652299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643515+0000","last_change":"2024-05-08T04:42:54.274640+0000","last_active":"2024-05-08T04:42:55.643515+0000","last_peered":"2024-05-08T04:42:55.643515+0000","last_clean":"2024-05-08T04:42:55.643515+0000","last_became_active":"2024-05-08T04:42:54.274209+0000","last_became_peered":"2024-05-08T04:42:54.274209+0000","last_unstale":"2024-05-08T04:42:55.643515+0000","last_undegraded":"2024-05-08T04:42:55.643515+0000","last_fullsized":"2024-05-08T04:42:55.643515+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:36.694738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259005+0000","last_change":"2024-05-08T04:42:53.265323+0000","last_active":"2024-05-08T04:42:55.259005+0000","last_peered":"2024-05-08T04:42:55.259005+0000","last_clean":"2024-05-08T04:42:55.259005+0000","last_became_active":"2024-05-08T04:42:53.265031+0000","last_became_peered":"2024-05-08T04:42:53.265031+0000","last_unstale":"2024-05-08T04:42:55.259005+0000","last_undegraded":"2024-05-08T04:42:55.259005+0000","last_fullsized":"2024-05-08T04:42:55.259005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:19.518749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258582+0000","last_change":"2024-05-08T04:42:54.263167+0000","last_active":"2024-05-08T04:42:55.258582+0000","last_peered":"2024-05-08T04:42:55.258582+0000","last_clean":"2024-05-08T04:42:55.258582+0000","last_became_active":"2024-05-08T04:42:54.262842+0000","last_became_peered":"2024-05-08T04:42:54.262842+0000","last_unstale":"2024-05-08T04:42:55.258582+0000","last_undegraded":"2024-05-08T04:42:55.258582+0000","last_fullsized":"2024-05-08T04:42:55.258582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:37.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643593+0000","last_change":"2024-05-08T04:42:53.263261+0000","last_active":"2024-05-08T04:42:55.643593+0000","last_peered":"2024-05-08T04:42:55.643593+0000","last_clean":"2024-05-08T04:42:55.643593+0000","last_became_active":"2024-05-08T04:42:53.262219+0000","last_became_peered":"2024-05-08T04:42:53.262219+0000","last_unstale":"2024-05-08T04:42:55.643593+0000","last_undegraded":"2024-05-08T04:42:55.643593+0000","last_fullsized":"2024-05-08T04:42:55.643593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:20.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260268+0000","last_change":"2024-05-08T04:42:54.269625+0000","last_active":"2024-05-08T04:42:55.260268+0000","last_peered":"2024-05-08T04:42:55.260268+0000","last_clean":"2024-05-08T04:42:55.260268+0000","last_became_active":"2024-05-08T04:42:54.269308+0000","last_became_peered":"2024-05-08T04:42:54.269308+0000","last_unstale":"2024-05-08T04:42:55.260268+0000","last_undegraded":"2024-05-08T04:42:55.260268+0000","last_fullsized":"2024-05-08T04:42:55.260268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.615548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644470+0000","last_change":"2024-05-08T04:42:53.254347+0000","last_active":"2024-05-08T04:42:55.644470+0000","last_peered":"2024-05-08T04:42:55.644470+0000","last_clean":"2024-05-08T04:42:55.644470+0000","last_became_active":"2024-05-08T04:42:53.254090+0000","last_became_peered":"2024-05-08T04:42:53.254090+0000","last_unstale":"2024-05-08T04:42:55.644470+0000","last_undegraded":"2024-05-08T04:42:55.644470+0000","last_fullsized":"2024-05-08T04:42:55.644470+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261712+0000","last_change":"2024-05-08T04:42:54.284605+0000","last_active":"2024-05-08T04:42:55.261712+0000","last_peered":"2024-05-08T04:42:55.261712+0000","last_clean":"2024-05-08T04:42:55.261712+0000","last_became_active":"2024-05-08T04:42:54.284331+0000","last_became_peered":"2024-05-08T04:42:54.284331+0000","last_unstale":"2024-05-08T04:42:55.261712+0000","last_undegraded":"2024-05-08T04:42:55.261712+0000","last_fullsized":"2024-05-08T04:42:55.261712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260792+0000","last_change":"2024-05-08T04:42:53.265080+0000","last_active":"2024-05-08T04:42:55.260792+0000","last_peered":"2024-05-08T04:42:55.260792+0000","last_clean":"2024-05-08T04:42:55.260792+0000","last_became_active":"2024-05-08T04:42:53.263632+0000","last_became_peered":"2024-05-08T04:42:53.263632+0000","last_unstale":"2024-05-08T04:42:55.260792+0000","last_undegraded":"2024-05-08T04:42:55.260792+0000","last_fullsized":"2024-05-08T04:42:55.260792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:48.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260920+0000","last_change":"2024-05-08T04:42:54.254357+0000","last_active":"2024-05-08T04:42:55.260920+0000","last_peered":"2024-05-08T04:42:55.260920+0000","last_clean":"2024-05-08T04:42:55.260920+0000","last_became_active":"2024-05-08T04:42:54.254083+0000","last_became_peered":"2024-05-08T04:42:54.254083+0000","last_unstale":"2024-05-08T04:42:55.260920+0000","last_undegraded":"2024-05-08T04:42:55.260920+0000","last_fullsized":"2024-05-08T04:42:55.260920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:16.705357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644694+0000","last_change":"2024-05-08T04:42:53.257288+0000","last_active":"2024-05-08T04:42:55.644694+0000","last_peered":"2024-05-08T04:42:55.644694+0000","last_clean":"2024-05-08T04:42:55.644694+0000","last_became_active":"2024-05-08T04:42:53.257014+0000","last_became_peered":"2024-05-08T04:42:53.257014+0000","last_unstale":"2024-05-08T04:42:55.644694+0000","last_undegraded":"2024-05-08T04:42:55.644694+0000","last_fullsized":"2024-05-08T04:42:55.644694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:27.189348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259886+0000","last_change":"2024-05-08T04:42:54.259828+0000","last_active":"2024-05-08T04:42:55.259886+0000","last_peered":"2024-05-08T04:42:55.259886+0000","last_clean":"2024-05-08T04:42:55.259886+0000","last_became_active":"2024-05-08T04:42:54.259109+0000","last_became_peered":"2024-05-08T04:42:54.259109+0000","last_unstale":"2024-05-08T04:42:55.259886+0000","last_undegraded":"2024-05-08T04:42:55.259886+0000","last_fullsized":"2024-05-08T04:42:55.259886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644648+0000","last_change":"2024-05-08T04:42:53.264438+0000","last_active":"2024-05-08T04:42:55.644648+0000","last_peered":"2024-05-08T04:42:55.644648+0000","last_clean":"2024-05-08T04:42:55.644648+0000","last_became_active":"2024-05-08T04:42:53.264131+0000","last_became_peered":"2024-05-08T04:42:53.264131+0000","last_unstale":"2024-05-08T04:42:55.644648+0000","last_undegraded":"2024-05-08T04:42:55.644648+0000","last_fullsized":"2024-05-08T04:42:55.644648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:15.699103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260599+0000","last_change":"2024-05-08T04:42:54.251858+0000","last_active":"2024-05-08T04:42:55.260599+0000","last_peered":"2024-05-08T04:42:55.260599+0000","last_clean":"2024-05-08T04:42:55.260599+0000","last_became_active":"2024-05-08T04:42:54.251600+0000","last_became_peered":"2024-05-08T04:42:54.251600+0000","last_unstale":"2024-05-08T04:42:55.260599+0000","last_undegraded":"2024-05-08T04:42:55.260599+0000","last_fullsized":"2024-05-08T04:42:55.260599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:06:55.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644435+0000","last_change":"2024-05-08T04:42:53.255665+0000","last_active":"2024-05-08T04:42:55.644435+0000","last_peered":"2024-05-08T04:42:55.644435+0000","last_clean":"2024-05-08T04:42:55.644435+0000","last_became_active":"2024-05-08T04:42:53.255168+0000","last_became_peered":"2024-05-08T04:42:53.255168+0000","last_unstale":"2024-05-08T04:42:55.644435+0000","last_undegraded":"2024-05-08T04:42:55.644435+0000","last_fullsized":"2024-05-08T04:42:55.644435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260674+0000","last_change":"2024-05-08T04:42:54.260677+0000","last_active":"2024-05-08T04:42:55.260674+0000","last_peered":"2024-05-08T04:42:55.260674+0000","last_clean":"2024-05-08T04:42:55.260674+0000","last_became_active":"2024-05-08T04:42:54.259329+0000","last_became_peered":"2024-05-08T04:42:54.259329+0000","last_unstale":"2024-05-08T04:42:55.260674+0000","last_undegraded":"2024-05-08T04:42:55.260674+0000","last_fullsized":"2024-05-08T04:42:55.260674+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261312+0000","last_change":"2024-05-08T04:42:53.259031+0000","last_active":"2024-05-08T04:42:55.261312+0000","last_peered":"2024-05-08T04:42:55.261312+0000","last_clean":"2024-05-08T04:42:55.261312+0000","last_became_active":"2024-05-08T04:42:53.258734+0000","last_became_peered":"2024-05-08T04:42:53.258734+0000","last_unstale":"2024-05-08T04:42:55.261312+0000","last_undegraded":"2024-05-08T04:42:55.261312+0000","last_fullsized":"2024-05-08T04:42:55.261312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:07.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260382+0000","last_change":"2024-05-08T04:42:54.255601+0000","last_active":"2024-05-08T04:42:55.260382+0000","last_peered":"2024-05-08T04:42:55.260382+0000","last_clean":"2024-05-08T04:42:55.260382+0000","last_became_active":"2024-05-08T04:42:54.255333+0000","last_became_peered":"2024-05-08T04:42:54.255333+0000","last_unstale":"2024-05-08T04:42:55.260382+0000","last_undegraded":"2024-05-08T04:42:55.260382+0000","last_fullsized":"2024-05-08T04:42:55.260382+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:28.195601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643788+0000","last_change":"2024-05-08T04:42:53.254175+0000","last_active":"2024-05-08T04:42:55.643788+0000","last_peered":"2024-05-08T04:42:55.643788+0000","last_clean":"2024-05-08T04:42:55.643788+0000","last_became_active":"2024-05-08T04:42:53.253505+0000","last_became_peered":"2024-05-08T04:42:53.253505+0000","last_unstale":"2024-05-08T04:42:55.643788+0000","last_undegraded":"2024-05-08T04:42:55.643788+0000","last_fullsized":"2024-05-08T04:42:55.643788+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:23.470898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644424+0000","last_change":"2024-05-08T04:42:54.283984+0000","last_active":"2024-05-08T04:42:55.644424+0000","last_peered":"2024-05-08T04:42:55.644424+0000","last_clean":"2024-05-08T04:42:55.644424+0000","last_became_active":"2024-05-08T04:42:54.283630+0000","last_became_peered":"2024-05-08T04:42:54.283630+0000","last_unstale":"2024-05-08T04:42:55.644424+0000","last_undegraded":"2024-05-08T04:42:55.644424+0000","last_fullsized":"2024-05-08T04:42:55.644424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:27.818505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645310+0000","last_change":"2024-05-08T04:42:53.255078+0000","last_active":"2024-05-08T04:42:55.645310+0000","last_peered":"2024-05-08T04:42:55.645310+0000","last_clean":"2024-05-08T04:42:55.645310+0000","last_became_active":"2024-05-08T04:42:53.254350+0000","last_became_peered":"2024-05-08T04:42:53.254350+0000","last_unstale":"2024-05-08T04:42:55.645310+0000","last_undegraded":"2024-05-08T04:42:55.645310+0000","last_fullsized":"2024-05-08T04:42:55.645310+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:59.504019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645155+0000","last_change":"2024-05-08T04:42:54.284624+0000","last_active":"2024-05-08T04:42:55.645155+0000","last_peered":"2024-05-08T04:42:55.645155+0000","last_clean":"2024-05-08T04:42:55.645155+0000","last_became_active":"2024-05-08T04:42:54.284368+0000","last_became_peered":"2024-05-08T04:42:54.284368+0000","last_unstale":"2024-05-08T04:42:55.645155+0000","last_undegraded":"2024-05-08T04:42:55.645155+0000","last_fullsized":"2024-05-08T04:42:55.645155+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:53.288054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644356+0000","last_change":"2024-05-08T04:42:53.259653+0000","last_active":"2024-05-08T04:42:55.644356+0000","last_peered":"2024-05-08T04:42:55.644356+0000","last_clean":"2024-05-08T04:42:55.644356+0000","last_became_active":"2024-05-08T04:42:53.259407+0000","last_became_peered":"2024-05-08T04:42:53.259407+0000","last_unstale":"2024-05-08T04:42:55.644356+0000","last_undegraded":"2024-05-08T04:42:55.644356+0000","last_fullsized":"2024-05-08T04:42:55.644356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:33.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543959+0000","last_change":"2024-05-08T04:42:54.268528+0000","last_active":"2024-05-08T04:42:55.543959+0000","last_peered":"2024-05-08T04:42:55.543959+0000","last_clean":"2024-05-08T04:42:55.543959+0000","last_became_active":"2024-05-08T04:42:54.268247+0000","last_became_peered":"2024-05-08T04:42:54.268247+0000","last_unstale":"2024-05-08T04:42:55.543959+0000","last_undegraded":"2024-05-08T04:42:55.543959+0000","last_fullsized":"2024-05-08T04:42:55.543959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:20.235769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261410+0000","last_change":"2024-05-08T04:42:53.260958+0000","last_active":"2024-05-08T04:42:55.261410+0000","last_peered":"2024-05-08T04:42:55.261410+0000","last_clean":"2024-05-08T04:42:55.261410+0000","last_became_active":"2024-05-08T04:42:53.260621+0000","last_became_peered":"2024-05-08T04:42:53.260621+0000","last_unstale":"2024-05-08T04:42:55.261410+0000","last_undegraded":"2024-05-08T04:42:55.261410+0000","last_fullsized":"2024-05-08T04:42:55.261410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:14.035376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.544039+0000","last_change":"2024-05-08T04:42:54.259461+0000","last_active":"2024-05-08T04:42:55.544039+0000","last_peered":"2024-05-08T04:42:55.544039+0000","last_clean":"2024-05-08T04:42:55.544039+0000","last_became_active":"2024-05-08T04:42:54.259005+0000","last_became_peered":"2024-05-08T04:42:54.259005+0000","last_unstale":"2024-05-08T04:42:55.544039+0000","last_undegraded":"2024-05-08T04:42:55.544039+0000","last_fullsized":"2024-05-08T04:42:55.544039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:07.448395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258867+0000","last_change":"2024-05-08T04:42:53.258018+0000","last_active":"2024-05-08T04:42:55.258867+0000","last_peered":"2024-05-08T04:42:55.258867+0000","last_clean":"2024-05-08T04:42:55.258867+0000","last_became_active":"2024-05-08T04:42:53.257357+0000","last_became_peered":"2024-05-08T04:42:53.257357+0000","last_unstale":"2024-05-08T04:42:55.258867+0000","last_undegraded":"2024-05-08T04:42:55.258867+0000","last_fullsized":"2024-05-08T04:42:55.258867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543555+0000","last_change":"2024-05-08T04:42:54.258649+0000","last_active":"2024-05-08T04:42:55.543555+0000","last_peered":"2024-05-08T04:42:55.543555+0000","last_clean":"2024-05-08T04:42:55.543555+0000","last_became_active":"2024-05-08T04:42:54.258367+0000","last_became_peered":"2024-05-08T04:42:54.258367+0000","last_unstale":"2024-05-08T04:42:55.543555+0000","last_undegraded":"2024-05-08T04:42:55.543555+0000","last_fullsized":"2024-05-08T04:42:55.543555+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:55.900111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644524+0000","last_change":"2024-05-08T04:42:53.260251+0000","last_active":"2024-05-08T04:42:55.644524+0000","last_peered":"2024-05-08T04:42:55.644524+0000","last_clean":"2024-05-08T04:42:55.644524+0000","last_became_active":"2024-05-08T04:42:53.259749+0000","last_became_peered":"2024-05-08T04:42:53.259749+0000","last_unstale":"2024-05-08T04:42:55.644524+0000","last_undegraded":"2024-05-08T04:42:55.644524+0000","last_fullsized":"2024-05-08T04:42:55.644524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644062+0000","last_change":"2024-05-08T04:42:54.270229+0000","last_active":"2024-05-08T04:42:55.644062+0000","last_peered":"2024-05-08T04:42:55.644062+0000","last_clean":"2024-05-08T04:42:55.644062+0000","last_became_active":"2024-05-08T04:42:54.267191+0000","last_became_peered":"2024-05-08T04:42:54.267191+0000","last_unstale":"2024-05-08T04:42:55.644062+0000","last_undegraded":"2024-05-08T04:42:55.644062+0000","last_fullsized":"2024-05-08T04:42:55.644062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:16.097496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643837+0000","last_change":"2024-05-08T04:42:53.255758+0000","last_active":"2024-05-08T04:42:55.643837+0000","last_peered":"2024-05-08T04:42:55.643837+0000","last_clean":"2024-05-08T04:42:55.643837+0000","last_became_active":"2024-05-08T04:42:53.255511+0000","last_became_peered":"2024-05-08T04:42:53.255511+0000","last_unstale":"2024-05-08T04:42:55.643837+0000","last_undegraded":"2024-05-08T04:42:55.643837+0000","last_fullsized":"2024-05-08T04:42:55.643837+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543619+0000","last_change":"2024-05-08T04:42:54.270286+0000","last_active":"2024-05-08T04:42:55.543619+0000","last_peered":"2024-05-08T04:42:55.543619+0000","last_clean":"2024-05-08T04:42:55.543619+0000","last_became_active":"2024-05-08T04:42:54.270019+0000","last_became_peered":"2024-05-08T04:42:54.270019+0000","last_unstale":"2024-05-08T04:42:55.543619+0000","last_undegraded":"2024-05-08T04:42:55.543619+0000","last_fullsized":"2024-05-08T04:42:55.543619+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:38.698071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644073+0000","last_change":"2024-05-08T04:42:53.259343+0000","last_active":"2024-05-08T04:42:55.644073+0000","last_peered":"2024-05-08T04:42:55.644073+0000","last_clean":"2024-05-08T04:42:55.644073+0000","last_became_active":"2024-05-08T04:42:53.258967+0000","last_became_peered":"2024-05-08T04:42:53.258967+0000","last_unstale":"2024-05-08T04:42:55.644073+0000","last_undegraded":"2024-05-08T04:42:55.644073+0000","last_fullsized":"2024-05-08T04:42:55.644073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:13.039135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260450+0000","last_change":"2024-05-08T04:42:54.267912+0000","last_active":"2024-05-08T04:42:55.260450+0000","last_peered":"2024-05-08T04:42:55.260450+0000","last_clean":"2024-05-08T04:42:55.260450+0000","last_became_active":"2024-05-08T04:42:54.267648+0000","last_became_peered":"2024-05-08T04:42:54.267648+0000","last_unstale":"2024-05-08T04:42:55.260450+0000","last_undegraded":"2024-05-08T04:42:55.260450+0000","last_fullsized":"2024-05-08T04:42:55.260450+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:51.941550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643660+0000","last_change":"2024-05-08T04:42:53.253264+0000","last_active":"2024-05-08T04:42:55.643660+0000","last_peered":"2024-05-08T04:42:55.643660+0000","last_clean":"2024-05-08T04:42:55.643660+0000","last_became_active":"2024-05-08T04:42:53.252716+0000","last_became_peered":"2024-05-08T04:42:53.252716+0000","last_unstale":"2024-05-08T04:42:55.643660+0000","last_undegraded":"2024-05-08T04:42:55.643660+0000","last_fullsized":"2024-05-08T04:42:55.643660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:56.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275320+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.275003+0000","last_became_peered":"2024-05-08T04:42:54.275003+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:25.398785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260852+0000","last_change":"2024-05-08T04:42:53.264575+0000","last_active":"2024-05-08T04:42:55.260852+0000","last_peered":"2024-05-08T04:42:55.260852+0000","last_clean":"2024-05-08T04:42:55.260852+0000","last_became_active":"2024-05-08T04:42:53.264163+0000","last_became_peered":"2024-05-08T04:42:53.264163+0000","last_unstale":"2024-05-08T04:42:55.260852+0000","last_undegraded":"2024-05-08T04:42:55.260852+0000","last_fullsized":"2024-05-08T04:42:55.260852+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:37.443525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.645304+0000","last_change":"2024-05-08T04:42:54.282790+0000","last_active":"2024-05-08T04:42:55.645304+0000","last_peered":"2024-05-08T04:42:55.645304+0000","last_clean":"2024-05-08T04:42:55.645304+0000","last_became_active":"2024-05-08T04:42:54.282523+0000","last_became_peered":"2024-05-08T04:42:54.282523+0000","last_unstale":"2024-05-08T04:42:55.645304+0000","last_undegraded":"2024-05-08T04:42:55.645304+0000","last_fullsized":"2024-05-08T04:42:55.645304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.467995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261546+0000","last_change":"2024-05-08T04:42:53.252707+0000","last_active":"2024-05-08T04:42:55.261546+0000","last_peered":"2024-05-08T04:42:55.261546+0000","last_clean":"2024-05-08T04:42:55.261546+0000","last_became_active":"2024-05-08T04:42:53.250660+0000","last_became_peered":"2024-05-08T04:42:53.250660+0000","last_unstale":"2024-05-08T04:42:55.261546+0000","last_undegraded":"2024-05-08T04:42:55.261546+0000","last_fullsized":"2024-05-08T04:42:55.261546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:13.437315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644517+0000","last_change":"2024-05-08T04:42:54.277076+0000","last_active":"2024-05-08T04:42:55.644517+0000","last_peered":"2024-05-08T04:42:55.644517+0000","last_clean":"2024-05-08T04:42:55.644517+0000","last_became_active":"2024-05-08T04:42:54.276805+0000","last_became_peered":"2024-05-08T04:42:54.276805+0000","last_unstale":"2024-05-08T04:42:55.644517+0000","last_undegraded":"2024-05-08T04:42:55.644517+0000","last_fullsized":"2024-05-08T04:42:55.644517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:39.976473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.644732+0000","last_change":"2024-05-08T04:42:53.252678+0000","last_active":"2024-05-08T04:42:55.644732+0000","last_peered":"2024-05-08T04:42:55.644732+0000","last_clean":"2024-05-08T04:42:55.644732+0000","last_became_active":"2024-05-08T04:42:53.252426+0000","last_became_peered":"2024-05-08T04:42:53.252426+0000","last_unstale":"2024-05-08T04:42:55.644732+0000","last_undegraded":"2024-05-08T04:42:55.644732+0000","last_fullsized":"2024-05-08T04:42:55.644732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:50.935297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.261536+0000","last_change":"2024-05-08T04:42:54.275599+0000","last_active":"2024-05-08T04:42:55.261536+0000","last_peered":"2024-05-08T04:42:55.261536+0000","last_clean":"2024-05-08T04:42:55.261536+0000","last_became_active":"2024-05-08T04:42:54.274979+0000","last_became_peered":"2024-05-08T04:42:54.274979+0000","last_unstale":"2024-05-08T04:42:55.261536+0000","last_undegraded":"2024-05-08T04:42:55.261536+0000","last_fullsized":"2024-05-08T04:42:55.261536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:27.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260948+0000","last_change":"2024-05-08T04:42:53.262988+0000","last_active":"2024-05-08T04:42:55.260948+0000","last_peered":"2024-05-08T04:42:55.260948+0000","last_clean":"2024-05-08T04:42:55.260948+0000","last_became_active":"2024-05-08T04:42:53.262721+0000","last_became_peered":"2024-05-08T04:42:53.262721+0000","last_unstale":"2024-05-08T04:42:55.260948+0000","last_undegraded":"2024-05-08T04:42:55.260948+0000","last_fullsized":"2024-05-08T04:42:55.260948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:07.710944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259403+0000","last_change":"2024-05-08T04:42:54.250195+0000","last_active":"2024-05-08T04:42:55.259403+0000","last_peered":"2024-05-08T04:42:55.259403+0000","last_clean":"2024-05-08T04:42:55.259403+0000","last_became_active":"2024-05-08T04:42:54.249297+0000","last_became_peered":"2024-05-08T04:42:54.249297+0000","last_unstale":"2024-05-08T04:42:55.259403+0000","last_undegraded":"2024-05-08T04:42:55.259403+0000","last_fullsized":"2024-05-08T04:42:55.259403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.658552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259421+0000","last_change":"2024-05-08T04:42:53.259450+0000","last_active":"2024-05-08T04:42:55.259421+0000","last_peered":"2024-05-08T04:42:55.259421+0000","last_clean":"2024-05-08T04:42:55.259421+0000","last_became_active":"2024-05-08T04:42:53.259139+0000","last_became_peered":"2024-05-08T04:42:53.259139+0000","last_unstale":"2024-05-08T04:42:55.259421+0000","last_undegraded":"2024-05-08T04:42:55.259421+0000","last_fullsized":"2024-05-08T04:42:55.259421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:34.539406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260413+0000","last_change":"2024-05-08T04:42:54.263700+0000","last_active":"2024-05-08T04:42:55.260413+0000","last_peered":"2024-05-08T04:42:55.260413+0000","last_clean":"2024-05-08T04:42:55.260413+0000","last_became_active":"2024-05-08T04:42:54.263395+0000","last_became_peered":"2024-05-08T04:42:54.263395+0000","last_unstale":"2024-05-08T04:42:55.260413+0000","last_undegraded":"2024-05-08T04:42:55.260413+0000","last_fullsized":"2024-05-08T04:42:55.260413+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:40.094631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.259409+0000","last_change":"2024-05-08T04:42:53.258649+0000","last_active":"2024-05-08T04:42:55.259409+0000","last_peered":"2024-05-08T04:42:55.259409+0000","last_clean":"2024-05-08T04:42:55.259409+0000","last_became_active":"2024-05-08T04:42:53.258348+0000","last_became_peered":"2024-05-08T04:42:53.258348+0000","last_unstale":"2024-05-08T04:42:55.259409+0000","last_undegraded":"2024-05-08T04:42:55.259409+0000","last_fullsized":"2024-05-08T04:42:55.259409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:42.312320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543824+0000","last_change":"2024-05-08T04:42:54.266324+0000","last_active":"2024-05-08T04:42:55.543824+0000","last_peered":"2024-05-08T04:42:55.543824+0000","last_clean":"2024-05-08T04:42:55.543824+0000","last_became_active":"2024-05-08T04:42:54.265475+0000","last_became_peered":"2024-05-08T04:42:54.265475+0000","last_unstale":"2024-05-08T04:42:55.543824+0000","last_undegraded":"2024-05-08T04:42:55.543824+0000","last_fullsized":"2024-05-08T04:42:55.543824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:00.510272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643830+0000","last_change":"2024-05-08T04:42:53.253674+0000","last_active":"2024-05-08T04:42:55.643830+0000","last_peered":"2024-05-08T04:42:55.643830+0000","last_clean":"2024-05-08T04:42:55.643830+0000","last_became_active":"2024-05-08T04:42:53.253392+0000","last_became_peered":"2024-05-08T04:42:53.253392+0000","last_unstale":"2024-05-08T04:42:55.643830+0000","last_undegraded":"2024-05-08T04:42:55.643830+0000","last_fullsized":"2024-05-08T04:42:55.643830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:21.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643920+0000","last_change":"2024-05-08T04:42:54.270703+0000","last_active":"2024-05-08T04:42:55.643920+0000","last_peered":"2024-05-08T04:42:55.643920+0000","last_clean":"2024-05-08T04:42:55.643920+0000","last_became_active":"2024-05-08T04:42:54.265604+0000","last_became_peered":"2024-05-08T04:42:54.265604+0000","last_unstale":"2024-05-08T04:42:55.643920+0000","last_undegraded":"2024-05-08T04:42:55.643920+0000","last_fullsized":"2024-05-08T04:42:55.643920+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:21.915088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.643694+0000","last_change":"2024-05-08T04:42:53.253094+0000","last_active":"2024-05-08T04:42:55.643694+0000","last_peered":"2024-05-08T04:42:55.643694+0000","last_clean":"2024-05-08T04:42:55.643694+0000","last_became_active":"2024-05-08T04:42:53.252291+0000","last_became_peered":"2024-05-08T04:42:53.252291+0000","last_unstale":"2024-05-08T04:42:55.643694+0000","last_undegraded":"2024-05-08T04:42:55.643694+0000","last_fullsized":"2024-05-08T04:42:55.643694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:59.366123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.260367+0000","last_change":"2024-05-08T04:42:54.254877+0000","last_active":"2024-05-08T04:42:55.260367+0000","last_peered":"2024-05-08T04:42:55.260367+0000","last_clean":"2024-05-08T04:42:55.260367+0000","last_became_active":"2024-05-08T04:42:54.254581+0000","last_became_peered":"2024-05-08T04:42:54.254581+0000","last_unstale":"2024-05-08T04:42:55.260367+0000","last_undegraded":"2024-05-08T04:42:55.260367+0000","last_fullsized":"2024-05-08T04:42:55.260367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:53.229408+0000","last_clean_scrub_stamp":"2024-05-08T04:42:53.229408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:48.608906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.258784+0000","last_change":"2024-05-08T04:42:53.257287+0000","last_active":"2024-05-08T04:42:55.258784+0000","last_peered":"2024-05-08T04:42:55.258784+0000","last_clean":"2024-05-08T04:42:55.258784+0000","last_became_active":"2024-05-08T04:42:53.256851+0000","last_became_peered":"2024-05-08T04:42:53.256851+0000","last_unstale":"2024-05-08T04:42:55.258784+0000","last_undegraded":"2024-05-08T04:42:55.258784+0000","last_fullsized":"2024-05-08T04:42:55.258784+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:42:52.223155+0000","last_clean_scrub_stamp":"2024-05-08T04:42:52.223155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:03.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-08T04:42:55.543931+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:42:55.543931+0000","last_peered":"2024-05-08T04:42:55.543931+0000","last_clean":"2024-05-08T04:42:55.543931+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:42:55.543931+0000","last_undegraded":"2024-05-08T04:42:55.543931+0000","last_fullsized":"2024-05-08T04:42:55.543931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":1198126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":607212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":607746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":608824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":607142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":607266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:43:03.200 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:43:03.200 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:43:03.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 Processor -- start 2024-05-08T04:43:03.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 -- start start 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0840adaa0 0x7fb084072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0840cee00 0x7fb0840cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840cdd60 con 0x7fb08415db40 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840ce970 con 0x7fb0840adaa0 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.379+0000 7fb089a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840ce040 con 0x7fb0840cee00 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0840adaa0 0x7fb084072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb083e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb08415db40 0x7fb08406cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37292/0 (socket says 172.21.15.164:37292) 2024-05-08T04:43:03.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb083e00640 1 -- 172.21.15.164:0/2683378403 learned_addr learned my addr 172.21.15.164:0/2683378403 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb089000640 1 --2- 172.21.15.164:0/2683378403 >> v2:172.21.15.172:3300/0 conn(0x7fb0840cee00 0x7fb0840cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 -- 172.21.15.164:0/2683378403 >> v2:172.21.15.172:3300/0 conn(0x7fb0840cee00 msgr2=0x7fb0840cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 --2- 172.21.15.164:0/2683378403 >> v2:172.21.15.172:3300/0 conn(0x7fb0840cee00 0x7fb0840cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 -- 172.21.15.164:0/2683378403 >> v1:172.21.15.164:6789/0 conn(0x7fb08415db40 legacy=0x7fb08406cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 -- 172.21.15.164:0/2683378403 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0840ce320 con 0x7fb0840adaa0 2024-05-08T04:43:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb07b400640 1 --2- 172.21.15.164:0/2683378403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0840adaa0 0x7fb084072b90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb084073470 tx=0x7fb06c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347219d70c35363e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb083400640 1 -- 172.21.15.164:0/2683378403 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0b1070 con 0x7fb0840adaa0 2024-05-08T04:43:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb083400640 1 -- 172.21.15.164:0/2683378403 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c0ac410 con 0x7fb0840adaa0 2024-05-08T04:43:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.383+0000 7fb083400640 1 -- 172.21.15.164:0/2683378403 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0ac7e0 con 0x7fb0840adaa0 2024-05-08T04:43:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- 172.21.15.164:0/2683378403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0840adaa0 msgr2=0x7fb084072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- 172.21.15.164:0/2683378403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0840adaa0 0x7fb084072b90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb084073470 tx=0x7fb06c01c450 comp rx=0 tx=0).stop 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- 172.21.15.164:0/2683378403 shutdown_connections 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- 172.21.15.164:0/2683378403 >> v2:172.21.15.172:3300/0 conn(0x7fb0840cee00 0x7fb0840cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- 172.21.15.164:0/2683378403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0840adaa0 0x7fb084072b90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- 172.21.15.164:0/2683378403 >> 172.21.15.164:0/2683378403 conn(0x7fb08406ac90 msgr2=0x7fb08406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- 172.21.15.164:0/2683378403 shutdown_connections 2024-05-08T04:43:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- 172.21.15.164:0/2683378403 wait complete. 2024-05-08T04:43:03.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 Processor -- start 2024-05-08T04:43:03.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- start start 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0840adaa0 0x7fb084159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0840cee00 0x7fb08415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840aebd0 con 0x7fb08415db40 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840aea50 con 0x7fb0840cee00 2024-05-08T04:43:03.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08416ab10 con 0x7fb0840adaa0 2024-05-08T04:43:03.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb083e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0840adaa0 0x7fb084159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40624/0 (socket says 172.21.15.164:40624) 2024-05-08T04:43:03.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 -- 172.21.15.164:0/2703248277 learned_addr learned my addr 172.21.15.164:0/2703248277 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:03.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb07b400640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0840cee00 0x7fb08415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 -- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0840adaa0 msgr2=0x7fb084159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0840adaa0 0x7fb084159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 -- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0840cee00 msgr2=0x7fb08415f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0840cee00 0x7fb08415f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.387+0000 7fb089000640 1 -- 172.21.15.164:0/2703248277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08415f560 con 0x7fb08415db40 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb089000640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d560 tx=0x7fb07c00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0125f0 con 0x7fb08415db40 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c019070 con 0x7fb08415db40 2024-05-08T04:43:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0129f0 con 0x7fb08415db40 2024-05-08T04:43:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08414f790 con 0x7fb08415db40 2024-05-08T04:43:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0840b3b20 con 0x7fb08415db40 2024-05-08T04:43:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08404ef30 con 0x7fb08415db40 2024-05-08T04:43:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fb07c012b90 con 0x7fb08415db40 2024-05-08T04:43:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 0x7fb060069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb07c083dc0 con 0x7fb08415db40 2024-05-08T04:43:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.391+0000 7fb083e00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 0x7fb060069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.395+0000 7fb083e00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 0x7fb060069d20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb0740012f0 tx=0x7fb074001a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.395+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb08414f790 con 0x7fb08415db40 2024-05-08T04:43:03.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.603+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb084068170 con 0x7fb08415db40 2024-05-08T04:43:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.607+0000 7fb080c00640 1 -- 172.21.15.164:0/2703248277 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb07c055fe0 con 0x7fb08415db40 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:43:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 msgr2=0x7fb060069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 0x7fb060069d20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb0740012f0 tx=0x7fb074001a40 comp rx=0 tx=0).stop 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 msgr2=0x7fb08414f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d560 tx=0x7fb07c00da20 comp rx=0 tx=0).stop 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 shutdown_connections 2024-05-08T04:43:03.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb060067860 0x7fb060069d20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb08415db40 0x7fb08414f250 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0840cee00 0x7fb08415f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 --2- 172.21.15.164:0/2703248277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0840adaa0 0x7fb084159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 >> 172.21.15.164:0/2703248277 conn(0x7fb08406ac90 msgr2=0x7fb0840b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 shutdown_connections 2024-05-08T04:43:03.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.611+0000 7fb089a00640 1 -- 172.21.15.164:0/2703248277 wait complete. 2024-05-08T04:43:03.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:03.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 Processor -- start 2024-05-08T04:43:03.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 -- start start 2024-05-08T04:43:03.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 0x7f93f80af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f806d580 con 0x7f93f81561f0 2024-05-08T04:43:03.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f815d040 con 0x7f93f806b1d0 2024-05-08T04:43:03.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f9400400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f806d820 con 0x7f93f80ce210 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fe400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 0x7f93f80af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:34602/0 (socket says 172.21.15.164:34602) 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 -- 172.21.15.164:0/601546539 learned_addr learned my addr 172.21.15.164:0/601546539 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 -- 172.21.15.164:0/601546539 >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 msgr2=0x7f93f80af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93f7e00640 1 -- 172.21.15.164:0/601546539 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771530746 0 0) 0x7f93f806d580 con 0x7f93f81561f0 2024-05-08T04:43:03.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 --2- 172.21.15.164:0/601546539 >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 0x7f93f80af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 -- 172.21.15.164:0/601546539 >> v1:172.21.15.172:6789/0 conn(0x7f93f81561f0 legacy=0x7f93f815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fd000640 1 -- 172.21.15.164:0/601546539 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f806db00 con 0x7f93f80ce210 2024-05-08T04:43:03.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.767+0000 7f93fe400640 1 --2- 172.21.15.164:0/601546539 >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 0x7f93f80af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:03.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f93fd000640 1 --2- 172.21.15.164:0/601546539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009870 tx=0x7f93e401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b778331d8d2e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f93f7e00640 1 -- 172.21.15.164:0/601546539 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40b2070 con 0x7f93f80ce210 2024-05-08T04:43:03.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f93f7e00640 1 -- 172.21.15.164:0/601546539 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e40a9070 con 0x7f93f80ce210 2024-05-08T04:43:03.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f93f7e00640 1 -- 172.21.15.164:0/601546539 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40ad880 con 0x7f93f80ce210 2024-05-08T04:43:03.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 -- 172.21.15.164:0/601546539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 msgr2=0x7f93f806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 --2- 172.21.15.164:0/601546539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009870 tx=0x7f93e401c560 comp rx=0 tx=0).stop 2024-05-08T04:43:03.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 -- 172.21.15.164:0/601546539 shutdown_connections 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 --2- 172.21.15.164:0/601546539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f806ac90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 --2- 172.21.15.164:0/601546539 >> v2:172.21.15.172:3300/0 conn(0x7f93f806b1d0 0x7f93f80af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 -- 172.21.15.164:0/601546539 >> 172.21.15.164:0/601546539 conn(0x7f93f8066fa0 msgr2=0x7f93f806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 -- 172.21.15.164:0/601546539 shutdown_connections 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.771+0000 7f9400400640 1 -- 172.21.15.164:0/601546539 wait complete. 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 Processor -- start 2024-05-08T04:43:03.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- start start 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93f806b1d0 0x7f93f813efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f815ddc0 con 0x7f93f81561f0 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f815dc40 con 0x7f93f806b1d0 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f815df40 con 0x7f93f80ce210 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40640/0 (socket says 172.21.15.164:40640) 2024-05-08T04:43:03.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 -- 172.21.15.164:0/1205335957 learned_addr learned my addr 172.21.15.164:0/1205335957 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fda00640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93f806b1d0 0x7f93f813efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fd000640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 -- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 msgr2=0x7f93f815a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f815a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 -- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93f806b1d0 msgr2=0x7f93f813efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93f806b1d0 0x7f93f813efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 -- 172.21.15.164:0/1205335957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f815b550 con 0x7f93f81561f0 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93fe400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f93f000efc0 tx=0x7f93f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0018030 con 0x7f93f81561f0 2024-05-08T04:43:03.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000cde0 con 0x7f93f81561f0 2024-05-08T04:43:03.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0003bc0 con 0x7f93f81561f0 2024-05-08T04:43:03.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8156780 con 0x7f93f81561f0 2024-05-08T04:43:03.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f8156d40 con 0x7f93f81561f0 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.775+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f813ead0 con 0x7f93f81561f0 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.779+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f93f001b070 con 0x7f93f81561f0 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.779+0000 7f93f5600640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 0x7f93d8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.779+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f93f0084400 con 0x7f93f81561f0 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.779+0000 7f93fda00640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 0x7f93d8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:03.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.779+0000 7f93fda00640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 0x7f93d8069d20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007fd0 tx=0x7f93ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:03.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.783+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93f813ead0 con 0x7f93f81561f0 2024-05-08T04:43:03.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93f80b5210 con 0x7f93f81561f0 2024-05-08T04:43:03.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f93f5600640 1 -- 172.21.15.164:0/1205335957 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f93f0022070 con 0x7f93f81561f0 2024-05-08T04:43:03.972 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:03.972 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:43:03.972 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 72 2024-05-08T04:43:03.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 msgr2=0x7f93d8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 0x7f93d8069d20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007fd0 tx=0x7f93ec009040 comp rx=0 tx=0).stop 2024-05-08T04:43:03.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 msgr2=0x7f93f815acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:03.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f93f000efc0 tx=0x7f93f0009f90 comp rx=0 tx=0).stop 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 shutdown_connections 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f93d8067860 0x7f93d8069d20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93f80ce210 0x7f93f815a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f81561f0 0x7f93f815acf0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 --2- 172.21.15.164:0/1205335957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93f806b1d0 0x7f93f813efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 >> 172.21.15.164:0/1205335957 conn(0x7f93f8066fa0 msgr2=0x7f93f80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 shutdown_connections 2024-05-08T04:43:03.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:03.971+0000 7f9400400640 1 -- 172.21.15.164:0/1205335957 wait complete. 2024-05-08T04:43:04.397 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:42:53.623714+0000', 'modified': '2024-05-08T04:42:56.632348+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/1496523549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6835', 'nonce': 1496523549}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1496523549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:43:04.397 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:43:04.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T04:43:04.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 Processor -- start 2024-05-08T04:43:04.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 -- start start 2024-05-08T04:43:04.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3600b06b0 0x7fe36009ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe3600ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3600cc350 con 0x7fe36015cd80 2024-05-08T04:43:04.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3600ceac0 con 0x7fe3600b06b0 2024-05-08T04:43:04.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe366000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600d1ec0 con 0x7fe3600ce110 2024-05-08T04:43:04.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe365600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe3600ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe35fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3600b06b0 0x7fe36009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe364c00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe36015cd80 0x7fe36015d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46642/0 (socket says 172.21.15.164:46642) 2024-05-08T04:43:04.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe364c00640 1 -- 172.21.15.164:0/2841813234 learned_addr learned my addr 172.21.15.164:0/2841813234 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:04.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe35fe00640 1 -- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 msgr2=0x7fe3600ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe35fe00640 1 --2- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe3600ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe35fe00640 1 -- 172.21.15.164:0/2841813234 >> v1:172.21.15.172:6789/0 conn(0x7fe36015cd80 legacy=0x7fe36015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.535+0000 7fe35fe00640 1 -- 172.21.15.164:0/2841813234 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600d2110 con 0x7fe3600b06b0 2024-05-08T04:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe35fe00640 1 --2- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3600b06b0 0x7fe36009ae30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d2040 tx=0x7fe34c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9061df7dd1e00ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe35f400640 1 -- 172.21.15.164:0/2841813234 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0b0070 con 0x7fe3600b06b0 2024-05-08T04:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe35f400640 1 -- 172.21.15.164:0/2841813234 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0ab600 con 0x7fe3600b06b0 2024-05-08T04:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe35f400640 1 -- 172.21.15.164:0/2841813234 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ab9d0 con 0x7fe3600b06b0 2024-05-08T04:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3600b06b0 msgr2=0x7fe36009ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 --2- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3600b06b0 0x7fe36009ae30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d2040 tx=0x7fe34c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- 172.21.15.164:0/2841813234 shutdown_connections 2024-05-08T04:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 --2- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe3600ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 --2- 172.21.15.164:0/2841813234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3600b06b0 0x7fe36009ae30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- 172.21.15.164:0/2841813234 >> 172.21.15.164:0/2841813234 conn(0x7fe36009aa20 msgr2=0x7fe3600d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- 172.21.15.164:0/2841813234 shutdown_connections 2024-05-08T04:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- 172.21.15.164:0/2841813234 wait complete. 2024-05-08T04:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 Processor -- start 2024-05-08T04:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.539+0000 7fe366000640 1 -- start start 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe360153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe36015cd80 0x7fe360154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600cd5e0 con 0x7fe3600b06b0 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600cc960 con 0x7fe36015cd80 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600cd760 con 0x7fe3600ce110 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40650/0 (socket says 172.21.15.164:40650) 2024-05-08T04:43:04.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 -- 172.21.15.164:0/3733671202 learned_addr learned my addr 172.21.15.164:0/3733671202 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe35fe00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe360153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe365600640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe36015cd80 0x7fe360154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 -- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 msgr2=0x7fe360153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe360153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 -- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe36015cd80 msgr2=0x7fe360154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe36015cd80 0x7fe360154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 -- 172.21.15.164:0/3733671202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360154a00 con 0x7fe3600b06b0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe364c00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe35000cb10 tx=0x7fe35000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe35cc00640 1 -- 172.21.15.164:0/3733671202 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350017070 con 0x7fe3600b06b0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe35cc00640 1 -- 172.21.15.164:0/3733671202 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000ac70 con 0x7fe3600b06b0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe360154720 con 0x7fe3600b06b0 2024-05-08T04:43:04.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe35cc00640 1 -- 172.21.15.164:0/3733671202 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350012620 con 0x7fe3600b06b0 2024-05-08T04:43:04.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.543+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3600cac40 con 0x7fe3600b06b0 2024-05-08T04:43:04.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35cc00640 1 -- 172.21.15.164:0/3733671202 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fe35001a070 con 0x7fe3600b06b0 2024-05-08T04:43:04.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35cc00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 0x7fe340069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35cc00640 1 -- 172.21.15.164:0/3733671202 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe350083630 con 0x7fe3600b06b0 2024-05-08T04:43:04.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 Processor -- start 2024-05-08T04:43:04.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35fe00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 0x7fe340069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 -- start start 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35fe00640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 0x7fe340069d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0ad000 tx=0x7fe34c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55d920 0x5642ee55dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55e240 0x5642ee5139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee3fb5f0 con 0x5642ee513f30 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee362da0 con 0x5642ee55d920 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642ee2eb150 con 0x5642ee55e240 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40652/0 (socket says 172.21.15.164:40652) 2024-05-08T04:43:04.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 -- 172.21.15.164:0/698050115 learned_addr learned my addr 172.21.15.164:0/698050115 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe364c00640 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55d920 0x5642ee55dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe35fe00640 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55e240 0x5642ee5139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 -- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55e240 msgr2=0x5642ee5139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55e240 0x5642ee5139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 -- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55d920 msgr2=0x5642ee55dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55d920 0x5642ee55dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 -- 172.21.15.164:0/698050115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ee304ca0 con 0x5642ee513f30 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.547+0000 7fe365600640 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b80 tx=0x7fe358009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739b500de4de0ce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe355600640 1 -- 172.21.15.164:0/698050115 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004230 con 0x5642ee513f30 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe355600640 1 -- 172.21.15.164:0/698050115 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580043d0 con 0x5642ee513f30 2024-05-08T04:43:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe355600640 1 -- 172.21.15.164:0/698050115 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358011be0 con 0x5642ee513f30 2024-05-08T04:43:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/698050115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 msgr2=0x5642ee514310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b80 tx=0x7fe358009190 comp rx=0 tx=0).stop 2024-05-08T04:43:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/698050115 shutdown_connections 2024-05-08T04:43:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55e240 0x5642ee5139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee514310 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- 172.21.15.164:0/698050115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55d920 0x5642ee55dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/698050115 >> 172.21.15.164:0/698050115 conn(0x5642ee51a2a0 msgr2=0x5642ee5122d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/698050115 shutdown_connections 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/698050115 wait complete. 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- start start 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 Processor -- start 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee564df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee517aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee31d990 con 0x5642ee55d920 2024-05-08T04:43:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee35ee40 con 0x5642ee55e240 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642ee2ee900 con 0x5642ee513f30 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 10 client.0 _ll_register_callbacks cb 0x5642ee4161c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40668/0 (socket says 172.21.15.164:40668) 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 -- 172.21.15.164:0/1639856626 learned_addr learned my addr 172.21.15.164:0/1639856626 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe365600640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee517aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe364c00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee564df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 -- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 msgr2=0x5642ee564df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee564df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 -- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 msgr2=0x5642ee517aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee517aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 -- 172.21.15.164:0/1639856626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ee31b630 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009580 tx=0x7fe34c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b40f0f87c51a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe34a000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0b0040 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe34c0b67f0 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.551+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3580043d0 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe34c0d0050 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 10 client.8064 fetch_fsmap learned FSMap version 72 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5642ee2eb3a0 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580043d0 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe34c0d3020 con 0x5642ee55d920 2024-05-08T04:43:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 10 client.8064 fetch_fsmap finished waiting for FSMap version 72 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 10 client.8064 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 20 client.8064 populate_metadata read hostname 'smithi164' 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee519160 0x5642ee56ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 4 client.8064 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 --> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642ee2fea50 con 0x5642ee519160 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe364c00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee519160 0x5642ee56ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe364c00640 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee519160 0x5642ee56ee50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1afb47d8629cc5a server_cookie=416c61e85f6f4ec4 in_seq=0 out_seq=0 2024-05-08T04:43:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 10 client.8064 ms_handle_connect on v2:172.21.15.164:6835/1496523549 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 1 -- 172.21.15.164:0/1639856626 <== mds.0 v2:172.21.15.164:6835/1496523549 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642ee2fea50 con 0x5642ee519160 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 10 client.8064 handle_command_reply: tid=0 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 client.8064 shutdown 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 msgr2=0x5642ee565330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009580 tx=0x7fe34c01ace0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 shutdown_connections 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee519160 0x5642ee56ee50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee517aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee565330 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe34a000640 0 client.8064 ms_handle_reset on v2:172.21.15.164:6835/1496523549 2024-05-08T04:43:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1639856626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee564df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 >> 172.21.15.164:0/1639856626 conn(0x5642ee51a2a0 msgr2=0x5642ee5129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 shutdown_connections 2024-05-08T04:43:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 1 -- 172.21.15.164:0/1639856626 wait complete. 2024-05-08T04:43:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.555+0000 7fe36ab78000 20 client.8064 trim_cache size 0 max 16384 2024-05-08T04:43:04.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 Processor -- start 2024-05-08T04:43:04.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 -- start start 2024-05-08T04:43:04.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee5b9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360201f10 con 0x5642ee55d920 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360201310 con 0x5642ee55e240 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe360201910 con 0x5642ee513f30 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40674/0 (socket says 172.21.15.164:40674) 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 -- 172.21.15.164:0/1991403391 learned_addr learned my addr 172.21.15.164:0/1991403391 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe364c00640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee5b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe365600640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 -- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 msgr2=0x5642ee5b9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee5b9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 -- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 msgr2=0x5642ee5b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 -- 172.21.15.164:0/1991403391 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ee55e070 con 0x5642ee55d920 2024-05-08T04:43:04.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe365600640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe364c00640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee5b9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe34c01c2d0 tx=0x7fe34c0ba070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcdb3842c53c482 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe354c00640 1 -- 172.21.15.164:0/1991403391 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0b0040 con 0x5642ee55d920 2024-05-08T04:43:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe354c00640 1 -- 172.21.15.164:0/1991403391 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0c8070 con 0x5642ee55d920 2024-05-08T04:43:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.571+0000 7fe354c00640 1 -- 172.21.15.164:0/1991403391 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0c3460 con 0x5642ee55d920 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1991403391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 msgr2=0x5642ee5b9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe34c01c2d0 tx=0x7fe34c0ba070 comp rx=0 tx=0).stop 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1991403391 shutdown_connections 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee55d920 0x5642ee5b9a50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1991403391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee513f30 0x5642ee5b9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1991403391 >> 172.21.15.164:0/1991403391 conn(0x5642ee51a2a0 msgr2=0x5642ee517220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1991403391 shutdown_connections 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1991403391 wait complete. 2024-05-08T04:43:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- start start 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 Processor -- start 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55d920 0x5642ee5a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee3673d0 con 0x5642ee513f30 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ee41ed50 con 0x5642ee55e240 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642ee2fd980 con 0x5642ee55d920 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 10 client.0 _ll_register_callbacks cb 0x7fe360072560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40680/0 (socket says 172.21.15.164:40680) 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 -- 172.21.15.164:0/1310009629 learned_addr learned my addr 172.21.15.164:0/1310009629 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55d920 0x5642ee5a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe365600640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 -- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55d920 msgr2=0x5642ee5a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55d920 0x5642ee5a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 -- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 msgr2=0x5642ee5a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 -- 172.21.15.164:0/1310009629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ee3e4600 con 0x5642ee513f30 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe365600640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe364c00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe3500556f0 tx=0x7fe350059c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c371fa61d27bf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:43:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500830b0 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe350083250 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe34c0c8070 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe35005dc70 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 10 client.8070 fetch_fsmap learned FSMap version 72 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5642ee2f79b0 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0c8070 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe350054e40 con 0x5642ee513f30 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 10 client.8070 fetch_fsmap finished waiting for FSMap version 72 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 20 client.8070 populate_metadata read hostname 'smithi164' 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee5a3530 0x5642ee5a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 4 client.8070 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 --> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5642ee3702e0 con 0x5642ee5a3530 2024-05-08T04:43:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.575+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee5a3530 0x5642ee5a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe35fe00640 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee5a3530 0x5642ee5a3910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d2acd6075822fb server_cookie=f65d620f002fd42e in_seq=0 out_seq=0 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe34a000640 10 client.8070 ms_handle_connect on v2:172.21.15.164:6835/1496523549 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe34a000640 1 -- 172.21.15.164:0/1310009629 <== mds.0 v2:172.21.15.164:6835/1496523549 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5642ee3702e0 con 0x5642ee5a3530 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe34a000640 10 client.8070 handle_command_reply: tid=0 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 client.8070 shutdown 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 msgr2=0x5642ee5a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe3500556f0 tx=0x7fe350059c00 comp rx=0 tx=0).stop 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 shutdown_connections 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5642ee55d920 0x5642ee5a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:6835/1496523549,v1:172.21.15.164:6837/1496523549] conn(0x5642ee5a3530 0x5642ee5a3910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642ee55e240 0x5642ee5a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 --2- 172.21.15.164:0/1310009629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642ee513f30 0x5642ee5a2030 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe34a000640 0 client.8070 ms_handle_reset on v2:172.21.15.164:6835/1496523549 2024-05-08T04:43:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 >> 172.21.15.164:0/1310009629 conn(0x5642ee51a2a0 msgr2=0x5642ee5659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 shutdown_connections 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 1 -- 172.21.15.164:0/1310009629 wait complete. 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe36ab78000 20 client.8070 trim_cache size 0 max 16384 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 7899, 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:43:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 10.332653785, 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 72, 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 112, 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 112, 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 15.178827639 2024-05-08T04:43:04.587 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 msgr2=0x7fe340069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 0x7fe340069d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0ad000 tx=0x7fe34c008040 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 msgr2=0x7fe36006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe35000cb10 tx=0x7fe35000a060 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 shutdown_connections 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe340067890 0x7fe340069d50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3600ce110 0x7fe360153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe36015cd80 0x7fe360154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 --2- 172.21.15.164:0/3733671202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3600b06b0 0x7fe36006c000 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 >> 172.21.15.164:0/3733671202 conn(0x7fe36009aa20 msgr2=0x7fe36006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 shutdown_connections 2024-05-08T04:43:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.579+0000 7fe366000640 1 -- 172.21.15.164:0/3733671202 wait complete. 2024-05-08T04:43:04.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:43:04.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.731+0000 7fd17e200640 1 Processor -- start 2024-05-08T04:43:04.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.731+0000 7fd17e200640 1 -- start start 2024-05-08T04:43:04.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd17e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17806ac90 0x7fd17806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd17e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd17806df30 0x7fd17806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd17e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17806e900 con 0x7fd1780ccdf0 2024-05-08T04:43:04.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd17e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17806b630 con 0x7fd17806df30 2024-05-08T04:43:04.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd17e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17806d4e0 con 0x7fd17806ac90 2024-05-08T04:43:04.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd1780ccdf0 0x7fd1780ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37304/0 (socket says 172.21.15.164:37304) 2024-05-08T04:43:04.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177400640 1 -- 172.21.15.164:0/874392997 learned_addr learned my addr 172.21.15.164:0/874392997 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:04.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd176a00640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17806ac90 0x7fd17806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177e00640 1 --2- 172.21.15.164:0/874392997 >> v2:172.21.15.172:3300/0 conn(0x7fd17806df30 0x7fd17806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd176000640 1 -- 172.21.15.164:0/874392997 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461972624 0 0) 0x7fd168001120 con 0x7fd1780ccdf0 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd176000640 1 -- 172.21.15.164:0/874392997 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c003530 con 0x7fd1780ccdf0 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177e00640 1 -- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17806ac90 msgr2=0x7fd17806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177e00640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17806ac90 0x7fd17806b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177e00640 1 -- 172.21.15.164:0/874392997 >> v1:172.21.15.164:6789/0 conn(0x7fd1780ccdf0 legacy=0x7fd1780ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.735+0000 7fd177e00640 1 -- 172.21.15.164:0/874392997 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178072320 con 0x7fd17806df30 2024-05-08T04:43:04.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd177e00640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd17806e3c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd170009d90 tx=0x7fd17001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bab2dc897fd342 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd176000640 1 -- 172.21.15.164:0/874392997 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700b2070 con 0x7fd17806df30 2024-05-08T04:43:04.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd176000640 1 -- 172.21.15.164:0/874392997 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1700a5890 con 0x7fd17806df30 2024-05-08T04:43:04.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd176000640 1 -- 172.21.15.164:0/874392997 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700a5c60 con 0x7fd17806df30 2024-05-08T04:43:04.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 -- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 msgr2=0x7fd17806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd17806e3c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd170009d90 tx=0x7fd17001c8e0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 -- 172.21.15.164:0/874392997 shutdown_connections 2024-05-08T04:43:04.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd17806e3c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 --2- 172.21.15.164:0/874392997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17806ac90 0x7fd17806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 -- 172.21.15.164:0/874392997 >> 172.21.15.164:0/874392997 conn(0x7fd178066fa0 msgr2=0x7fd17806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 -- 172.21.15.164:0/874392997 shutdown_connections 2024-05-08T04:43:04.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 -- 172.21.15.164:0/874392997 wait complete. 2024-05-08T04:43:04.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.739+0000 7fd17e200640 1 Processor -- start 2024-05-08T04:43:04.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- start start 2024-05-08T04:43:04.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd1781529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780ccdf0 0x7fd17815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780adfa0 con 0x7fd17806ac90 2024-05-08T04:43:04.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178071de0 con 0x7fd17806df30 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd178153990 con 0x7fd1780ccdf0 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40694/0 (socket says 172.21.15.164:40694) 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 -- 172.21.15.164:0/907994369 learned_addr learned my addr 172.21.15.164:0/907994369 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177e00640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780ccdf0 0x7fd17815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd176a00640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd1781529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 -- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780ccdf0 msgr2=0x7fd17815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780ccdf0 0x7fd17815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 -- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 msgr2=0x7fd1781529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd1781529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 -- 172.21.15.164:0/907994369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1781536b0 con 0x7fd17806ac90 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd177400640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c3f0 tx=0x7fd16800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168017070 con 0x7fd17806ac90 2024-05-08T04:43:04.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168005cf0 con 0x7fd17806ac90 2024-05-08T04:43:04.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178157700 con 0x7fd17806ac90 2024-05-08T04:43:04.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168007310 con 0x7fd17806ac90 2024-05-08T04:43:04.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178157cc0 con 0x7fd17806ac90 2024-05-08T04:43:04.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.743+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178069d60 con 0x7fd17806ac90 2024-05-08T04:43:04.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.747+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fd16801a070 con 0x7fd17806ac90 2024-05-08T04:43:04.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.747+0000 7fd16f400640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 0x7fd158069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:04.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.747+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd168082c40 con 0x7fd17806ac90 2024-05-08T04:43:04.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.747+0000 7fd176a00640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 0x7fd158069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:04.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.747+0000 7fd176a00640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 0x7fd158069d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd17815af90 tx=0x7fd164009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:04.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.751+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd17806e900 con 0x7fd17806ac90 2024-05-08T04:43:04.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.939+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1780cdd10 con 0x7fd17806ac90 2024-05-08T04:43:04.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.939+0000 7fd16f400640 1 -- 172.21.15.164:0/907994369 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1680550b0 con 0x7fd17806ac90 2024-05-08T04:43:04.942 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:43:04.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 msgr2=0x7fd158069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 0x7fd158069d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd17815af90 tx=0x7fd164009190 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 msgr2=0x7fd1780b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c3f0 tx=0x7fd16800c8b0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 shutdown_connections 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd158067860 0x7fd158069d20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780ccdf0 0x7fd17815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17806df30 0x7fd1781529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 --2- 172.21.15.164:0/907994369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd17806ac90 0x7fd1780b24b0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 >> 172.21.15.164:0/907994369 conn(0x7fd178066fa0 msgr2=0x7fd1780ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 shutdown_connections 2024-05-08T04:43:04.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:04.943+0000 7fd17e200640 1 -- 172.21.15.164:0/907994369 wait complete. 2024-05-08T04:43:04.963 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:43:04.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:43:04.963 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:43:04.964 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:04.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:04.964 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42082sec preferred_lft 42082sec 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:43:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:43:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:04.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:04.977 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:43:05.036 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:43:05.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.038 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:43:05.094 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:43:05.094 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:43:05.094 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:43:05.094 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:05.096 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:43:05.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:43:05.097 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:43:05.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.188 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:43:05.245 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:43:05.245 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:43:05.245 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:05.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:43:05.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:05.310 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:05.363 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:05.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:05.363 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:05.418 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:43:05.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:05.515 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:05.515 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:05.515 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:05.530 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:05.530 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:05.530 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:05.530 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:05.531 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:05.539 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:05.539 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:05.539 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:05.539 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:05.540 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:05.540 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:43:05.540 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:43:05.540 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:43:05.543 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:43:05.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.543 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:05.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:05.563 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:43:05.627 INFO:teuthology.orchestra.run.smithi164.stdout:client7732 2024-05-08T04:43:05.628 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:05.628 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:05.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:43:05.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 Processor -- start 2024-05-08T04:43:05.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 -- start start 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 0x7f6c9c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9c06c770 con 0x7f6c9c15ab60 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0d0060 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6ca1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c06ca50 con 0x7f6c9c0b26c0 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40708/0 (socket says 172.21.15.164:40708) 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 -- 172.21.15.164:0/4160971802 learned_addr learned my addr 172.21.15.164:0/4160971802 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c92000640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 0x7f6c9c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 -- 172.21.15.164:0/4160971802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 msgr2=0x7f6c9c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 0x7f6c9c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 -- 172.21.15.164:0/4160971802 >> v1:172.21.15.172:6789/0 conn(0x7f6c9c15ab60 legacy=0x7f6c9c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:05.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c9b400640 1 -- 172.21.15.164:0/4160971802 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c06cd30 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.859+0000 7f6c92000640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 0x7f6c9c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:05.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6c9b400640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009870 tx=0x7f6c9401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9413893d4ce287b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:05.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6c9a000640 1 -- 172.21.15.164:0/4160971802 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940b2070 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6c9a000640 1 -- 172.21.15.164:0/4160971802 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c940a9070 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6c9a000640 1 -- 172.21.15.164:0/4160971802 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940ad880 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 -- 172.21.15.164:0/4160971802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 msgr2=0x7f6c9c0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009870 tx=0x7f6c9401c560 comp rx=0 tx=0).stop 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 -- 172.21.15.164:0/4160971802 shutdown_connections 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c0cd910 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 --2- 172.21.15.164:0/4160971802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c0b26c0 0x7f6c9c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 -- 172.21.15.164:0/4160971802 >> 172.21.15.164:0/4160971802 conn(0x7f6c9c06ac90 msgr2=0x7f6c9c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 -- 172.21.15.164:0/4160971802 shutdown_connections 2024-05-08T04:43:05.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.863+0000 7f6ca1600640 1 -- 172.21.15.164:0/4160971802 wait complete. 2024-05-08T04:43:05.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 Processor -- start 2024-05-08T04:43:05.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- start start 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 0x7f6c9c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c15ab60 0x7f6c9c14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0aeeb0 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0aed30 con 0x7f6c9c0b26c0 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c167970 con 0x7f6c9c15ab60 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40724/0 (socket says 172.21.15.164:40724) 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 -- 172.21.15.164:0/155127163 learned_addr learned my addr 172.21.15.164:0/155127163 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c9b400640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c15ab60 0x7f6c9c14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c9aa00640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 0x7f6c9c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 -- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c15ab60 msgr2=0x7f6c9c14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c15ab60 0x7f6c9c14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 -- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 msgr2=0x7f6c9c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 0x7f6c9c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 -- 172.21.15.164:0/155127163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c1583b0 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c9aa00640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 0x7f6c9c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:05.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c92000640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400e840 tx=0x7f6c8400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:05.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017070 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400c8d0 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c158690 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400f040 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.867+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c158c50 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c069d60 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f6c8401a070 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6c93400640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 0x7f6c78069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:05.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f6c84083640 con 0x7f6c9c0cfbd0 2024-05-08T04:43:05.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6c9aa00640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 0x7f6c78069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:05.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.871+0000 7f6c9aa00640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 0x7f6c78069d20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c001d70 tx=0x7f6c8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:05.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:05.875+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c84055860 con 0x7f6c9c0cfbd0 2024-05-08T04:43:06.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.063+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c9c06d9a0 con 0x7f6c9c0cfbd0 2024-05-08T04:43:06.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.063+0000 7f6c93400640 1 -- 172.21.15.164:0/155127163 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c84059900 con 0x7f6c9c0cfbd0 2024-05-08T04:43:06.067 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 msgr2=0x7f6c78069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 0x7f6c78069d20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c001d70 tx=0x7f6c8c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 msgr2=0x7f6c9c14f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400e840 tx=0x7f6c8400ed00 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 shutdown_connections 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6c78067860 0x7f6c78069d20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c9c15ab60 0x7f6c9c14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c9c0cfbd0 0x7f6c9c14f370 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 --2- 172.21.15.164:0/155127163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9c0b26c0 0x7f6c9c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 >> 172.21.15.164:0/155127163 conn(0x7f6c9c06ac90 msgr2=0x7f6c9c156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:06.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 shutdown_connections 2024-05-08T04:43:06.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.067+0000 7f6ca1600640 1 -- 172.21.15.164:0/155127163 wait complete. 2024-05-08T04:43:06.088 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:43:06.089 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:43:06.090 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:43:06.090 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:06.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.090 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:43:06.101 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:43:06.101 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:43:06.102 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:43:06.102 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.102 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:43:06.102 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.102 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.103 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42085sec preferred_lft 42085sec 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.104 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:43:06.105 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:06.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.106 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:43:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:43:06.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.171 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:43:06.231 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:43:06.231 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:43:06.231 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:43:06.231 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:43:06.231 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:43:06.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:43:06.233 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:43:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.336 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:43:06.344 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:43:06.345 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:43:06.346 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:06.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.346 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:43:06.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:06.410 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:06.461 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:06.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:06.462 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:06.517 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:43:06.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.517 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:06.611 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:06.611 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:06.611 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:06.626 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:06.626 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:06.626 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:06.626 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:06.627 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:43:06.635 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:43:06.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:43:06.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:06.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:06.659 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:43:06.711 INFO:teuthology.orchestra.run.smithi172.stdout:client7741 2024-05-08T04:43:06.712 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:06.712 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:06.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:43:06.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.899+0000 7f1b4a200640 1 Processor -- start 2024-05-08T04:43:06.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.899+0000 7f1b4a200640 1 -- start start 2024-05-08T04:43:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b4a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4415ab10 0x7f1b4406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b4a200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b440cee90 0x7f1b440aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b4a200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4415e870 con 0x7f1b440ae6e0 2024-05-08T04:43:06.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b4a200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4406bb80 con 0x7f1b440cee90 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b4a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4415eb50 con 0x7f1b4415ab10 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b440cee90 0x7f1b440aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b440cee90 0x7f1b440aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40736/0 (socket says 172.21.15.164:40736) 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 -- 172.21.15.164:0/1121643766 learned_addr learned my addr 172.21.15.164:0/1121643766 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43400640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4415ab10 0x7f1b4406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 -- 172.21.15.164:0/1121643766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4415ab10 msgr2=0x7f1b4406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4415ab10 0x7f1b4406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 -- 172.21.15.164:0/1121643766 >> v1:172.21.15.172:6789/0 conn(0x7f1b440ae6e0 legacy=0x7f1b440ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.903+0000 7f1b43e00640 1 -- 172.21.15.164:0/1121643766 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4415ee30 con 0x7f1b440cee90 2024-05-08T04:43:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b43e00640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b440cee90 0x7f1b440aeb40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00bb10 tx=0x7f1b3c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d47c980b2f2a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b42a00640 1 -- 172.21.15.164:0/1121643766 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0b4070 con 0x7f1b440cee90 2024-05-08T04:43:06.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b42a00640 1 -- 172.21.15.164:0/1121643766 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c0a7c00 con 0x7f1b440cee90 2024-05-08T04:43:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b42a00640 1 -- 172.21.15.164:0/1121643766 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0af440 con 0x7f1b440cee90 2024-05-08T04:43:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- 172.21.15.164:0/1121643766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b440cee90 msgr2=0x7f1b440aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b440cee90 0x7f1b440aeb40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00bb10 tx=0x7f1b3c01eb70 comp rx=0 tx=0).stop 2024-05-08T04:43:06.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- 172.21.15.164:0/1121643766 shutdown_connections 2024-05-08T04:43:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b440cee90 0x7f1b440aeb40 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 --2- 172.21.15.164:0/1121643766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4415ab10 0x7f1b4406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- 172.21.15.164:0/1121643766 >> 172.21.15.164:0/1121643766 conn(0x7f1b440cf770 msgr2=0x7f1b440cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- 172.21.15.164:0/1121643766 shutdown_connections 2024-05-08T04:43:06.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- 172.21.15.164:0/1121643766 wait complete. 2024-05-08T04:43:06.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 Processor -- start 2024-05-08T04:43:06.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.907+0000 7f1b4a200640 1 -- start start 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 0x7f1b440c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b440cee90 0x7f1b440c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440cb070 con 0x7f1b4415ab10 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440caef0 con 0x7f1b440ae6e0 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440cb1f0 con 0x7f1b440cee90 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 0x7f1b440c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b3aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b440cee90 0x7f1b440c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 0x7f1b440c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:39748/0 (socket says 172.21.15.164:39748) 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43400640 1 -- 172.21.15.164:0/2391567200 learned_addr learned my addr 172.21.15.164:0/2391567200 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:06.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40748/0 (socket says 172.21.15.164:40748) 2024-05-08T04:43:06.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 -- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b440cee90 msgr2=0x7f1b440c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b440cee90 0x7f1b440c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 -- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 msgr2=0x7f1b440c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:06.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 0x7f1b440c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:06.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 -- 172.21.15.164:0/2391567200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b441f34b0 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b43e00640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0ae000 tx=0x7f1b3c01b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0bd070 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c0b7790 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b441f3790 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0b7ba0 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b440c6d80 con 0x7f1b4415ab10 2024-05-08T04:43:06.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44072d30 con 0x7f1b4415ab10 2024-05-08T04:43:06.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.911+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f1b3c0c0070 con 0x7f1b4415ab10 2024-05-08T04:43:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.915+0000 7f1b3be00640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 0x7f1b20069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.915+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1b3c1222c0 con 0x7f1b4415ab10 2024-05-08T04:43:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.915+0000 7f1b43400640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 0x7f1b20069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.915+0000 7f1b43400640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 0x7f1b20069d20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b34003b90 tx=0x7f1b340098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:06.919+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b3c018820 con 0x7f1b4415ab10 2024-05-08T04:43:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.103+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1b4404f010 con 0x7f1b4415ab10 2024-05-08T04:43:07.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.299+0000 7f1b3be00640 1 -- 172.21.15.164:0/2391567200 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1b3c0f44e0 con 0x7f1b4415ab10 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 msgr2=0x7f1b20069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 0x7f1b20069d20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b34003b90 tx=0x7f1b340098d0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 msgr2=0x7f1b441f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0ae000 tx=0x7f1b3c01b910 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 shutdown_connections 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1b20067860 0x7f1b20069d20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b440cee90 0x7f1b440c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b4415ab10 0x7f1b441f2a00 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 --2- 172.21.15.164:0/2391567200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b440ae6e0 0x7f1b440c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 >> 172.21.15.164:0/2391567200 conn(0x7f1b440cf770 msgr2=0x7f1b440cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:07.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 shutdown_connections 2024-05-08T04:43:07.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.303+0000 7f1b4a200640 1 -- 172.21.15.164:0/2391567200 wait complete. 2024-05-08T04:43:07.323 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:43:07.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 Processor -- start 2024-05-08T04:43:07.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 -- start start 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff015ab60 0x7f9ff0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 0x7f9ff00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff0072640 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff00d0060 con 0x7f9ff015ab60 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9ff6200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff006c770 con 0x7f9ff00b26c0 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9fef400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff015ab60 0x7f9ff0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.459+0000 7f9fef400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff015ab60 0x7f9ff0072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40754/0 (socket says 172.21.15.164:40754) 2024-05-08T04:43:07.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 -- 172.21.15.164:0/454403809 learned_addr learned my addr 172.21.15.164:0/454403809 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 -- 172.21.15.164:0/454403809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 msgr2=0x7f9ff00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9feea00640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 0x7f9ff00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 0x7f9ff00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 -- 172.21.15.164:0/454403809 >> v1:172.21.15.172:6789/0 conn(0x7f9ff00cfbd0 legacy=0x7f9ff00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 -- 172.21.15.164:0/454403809 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff006ca50 con 0x7f9ff015ab60 2024-05-08T04:43:07.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9feea00640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 0x7f9ff00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fef400640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff015ab60 0x7f9ff0072100 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009870 tx=0x7f9fe001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e5c79b7f9192b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fee000640 1 -- 172.21.15.164:0/454403809 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00b2070 con 0x7f9ff015ab60 2024-05-08T04:43:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fee000640 1 -- 172.21.15.164:0/454403809 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe00a9070 con 0x7f9ff015ab60 2024-05-08T04:43:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.463+0000 7f9fee000640 1 -- 172.21.15.164:0/454403809 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00ad880 con 0x7f9ff015ab60 2024-05-08T04:43:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- 172.21.15.164:0/454403809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff015ab60 msgr2=0x7f9ff0072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff015ab60 0x7f9ff0072100 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009870 tx=0x7f9fe001c560 comp rx=0 tx=0).stop 2024-05-08T04:43:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- 172.21.15.164:0/454403809 shutdown_connections 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff00b26c0 0x7f9ff00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- 172.21.15.164:0/454403809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff015ab60 0x7f9ff0072100 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- 172.21.15.164:0/454403809 >> 172.21.15.164:0/454403809 conn(0x7f9ff006ac90 msgr2=0x7f9ff006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- 172.21.15.164:0/454403809 shutdown_connections 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- 172.21.15.164:0/454403809 wait complete. 2024-05-08T04:43:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 Processor -- start 2024-05-08T04:43:07.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- start start 2024-05-08T04:43:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff00b26c0 0x7f9ff015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff015ab60 0x7f9ff00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff006c550 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff006c3d0 con 0x7f9ff00b26c0 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9ff6200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00aed30 con 0x7f9ff015ab60 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40768/0 (socket says 172.21.15.164:40768) 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 -- 172.21.15.164:0/1611665048 learned_addr learned my addr 172.21.15.164:0/1611665048 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9fef400640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff00b26c0 0x7f9ff015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9fefe00640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff015ab60 0x7f9ff00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 -- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff015ab60 msgr2=0x7f9ff00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff015ab60 0x7f9ff00b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 -- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff00b26c0 msgr2=0x7f9ff015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff00b26c0 0x7f9ff015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 -- 172.21.15.164:0/1611665048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff014d8f0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.467+0000 7f9feea00640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c7f0 tx=0x7f9fdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc007a30 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc007bd0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0113f0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff014fee0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff01504a0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff0069d60 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f9fdc011590 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 0x7f9fd0069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f9fdc083aa0 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.471+0000 7f9fef400640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 0x7f9fd0069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.475+0000 7f9fef400640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 0x7f9fd0069d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00af000 tx=0x7f9fe0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.475+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fdc015020 con 0x7f9ff00cfbd0 2024-05-08T04:43:07.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.655+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ff006d9a0 con 0x7f9fd0067860 2024-05-08T04:43:07.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.655+0000 7f9fe7400640 1 -- 172.21.15.164:0/1611665048 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9ff006d9a0 con 0x7f9fd0067860 2024-05-08T04:43:07.658 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:43:07.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:07.658 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:43:07.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:07.658 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 msgr2=0x7f9fd0069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 0x7f9fd0069d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00af000 tx=0x7f9fe0008040 comp rx=0 tx=0).stop 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 msgr2=0x7f9ff0141260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c7f0 tx=0x7f9fdc00ccb0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 shutdown_connections 2024-05-08T04:43:07.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd0067860 0x7f9fd0069d20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff015ab60 0x7f9ff00b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00cfbd0 0x7f9ff0141260 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 --2- 172.21.15.164:0/1611665048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff00b26c0 0x7f9ff015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 >> 172.21.15.164:0/1611665048 conn(0x7f9ff006ac90 msgr2=0x7f9ff00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 shutdown_connections 2024-05-08T04:43:07.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.659+0000 7f9ff6200640 1 -- 172.21.15.164:0/1611665048 wait complete. 2024-05-08T04:43:07.679 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:43:07.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 Processor -- start 2024-05-08T04:43:07.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 -- start start 2024-05-08T04:43:07.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f5415a200 0x7f9f540aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9f5406dfa0 0x7f9f540ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f540caa90 con 0x7f9f540ca680 2024-05-08T04:43:07.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5406bca0 con 0x7f9f5406dfa0 2024-05-08T04:43:07.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f5b600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f540af150 con 0x7f9f5415a200 2024-05-08T04:43:07.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f53e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9f5406dfa0 0x7f9f540ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f59600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9f540ca680 0x7f9f5415de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37308/0 (socket says 172.21.15.164:37308) 2024-05-08T04:43:07.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.815+0000 7f9f59600640 1 -- 172.21.15.164:0/4088550239 learned_addr learned my addr 172.21.15.164:0/4088550239 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f58c00640 1 --2- 172.21.15.164:0/4088550239 >> v2:172.21.15.172:3300/0 conn(0x7f9f5415a200 0x7f9f540aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53e00640 1 -- 172.21.15.164:0/4088550239 >> v2:172.21.15.172:3300/0 conn(0x7f9f5415a200 msgr2=0x7f9f540aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53e00640 1 --2- 172.21.15.164:0/4088550239 >> v2:172.21.15.172:3300/0 conn(0x7f9f5415a200 0x7f9f540aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53e00640 1 -- 172.21.15.164:0/4088550239 >> v1:172.21.15.164:6789/0 conn(0x7f9f540ca680 legacy=0x7f9f5415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53e00640 1 -- 172.21.15.164:0/4088550239 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5406d150 con 0x7f9f5406dfa0 2024-05-08T04:43:07.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53400640 1 -- 172.21.15.164:0/4088550239 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795581313 0 0) 0x7f9f4c001120 con 0x7f9f540ca680 2024-05-08T04:43:07.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53e00640 1 --2- 172.21.15.164:0/4088550239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5406dfa0 0x7f9f540ca140 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cc150 tx=0x7f9f4001d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffde94a6dca67c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53400640 1 -- 172.21.15.164:0/4088550239 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400b3070 con 0x7f9f5406dfa0 2024-05-08T04:43:07.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53400640 1 -- 172.21.15.164:0/4088550239 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f400a67c0 con 0x7f9f5406dfa0 2024-05-08T04:43:07.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f53400640 1 -- 172.21.15.164:0/4088550239 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000b040 con 0x7f9f5406dfa0 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f5b600640 1 -- 172.21.15.164:0/4088550239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5406dfa0 msgr2=0x7f9f540ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f5b600640 1 --2- 172.21.15.164:0/4088550239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5406dfa0 0x7f9f540ca140 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cc150 tx=0x7f9f4001d790 comp rx=0 tx=0).stop 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f5b600640 1 -- 172.21.15.164:0/4088550239 shutdown_connections 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f5b600640 1 --2- 172.21.15.164:0/4088550239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5406dfa0 0x7f9f540ca140 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.819+0000 7f9f5b600640 1 --2- 172.21.15.164:0/4088550239 >> v2:172.21.15.172:3300/0 conn(0x7f9f5415a200 0x7f9f540aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- 172.21.15.164:0/4088550239 >> 172.21.15.164:0/4088550239 conn(0x7f9f540cf770 msgr2=0x7f9f540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:07.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- 172.21.15.164:0/4088550239 shutdown_connections 2024-05-08T04:43:07.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- 172.21.15.164:0/4088550239 wait complete. 2024-05-08T04:43:07.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 Processor -- start 2024-05-08T04:43:07.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- start start 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f5406dfa0 0x7f9f54145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f540ca680 0x7f9f541f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5415a760 con 0x7f9f5415a200 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5415a5e0 con 0x7f9f5406dfa0 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f5b600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5415e540 con 0x7f9f540ca680 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f540ca680 0x7f9f541f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40784/0 (socket says 172.21.15.164:40784) 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 -- 172.21.15.164:0/2397758214 learned_addr learned my addr 172.21.15.164:0/2397758214 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:07.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f58c00640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f5406dfa0 0x7f9f54145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 -- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f540ca680 msgr2=0x7f9f541f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f540ca680 0x7f9f541f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 -- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f5406dfa0 msgr2=0x7f9f54145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f5406dfa0 0x7f9f54145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 -- 172.21.15.164:0/2397758214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f541464c0 con 0x7f9f5415a200 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f59600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c780 tx=0x7f9f4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c018070 con 0x7f9f5415a200 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.823+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0070d0 con 0x7f9f5415a200 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541489f0 con 0x7f9f5415a200 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007500 con 0x7f9f5415a200 2024-05-08T04:43:07.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5415c480 con 0x7f9f5415a200 2024-05-08T04:43:07.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f54148f10 con 0x7f9f5415a200 2024-05-08T04:43:07.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f9f4c01b070 con 0x7f9f5415a200 2024-05-08T04:43:07.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f50c00640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 0x7f9f34069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:07.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f9f4c082aa0 con 0x7f9f5415a200 2024-05-08T04:43:07.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f58c00640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 0x7f9f34069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:07.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.827+0000 7f9f58c00640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 0x7f9f34069d20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cfcc0 tx=0x7f9f44009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:07.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.831+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f54148f10 con 0x7f9f5415a200 2024-05-08T04:43:07.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9f54072d30 con 0x7f9f5415a200 2024-05-08T04:43:07.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f50c00640 1 -- 172.21.15.164:0/2397758214 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f540caa90 con 0x7f9f5415a200 2024-05-08T04:43:07.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 msgr2=0x7f9f34069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 0x7f9f34069d20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cfcc0 tx=0x7f9f44009830 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 msgr2=0x7f9f54147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c780 tx=0x7f9f4c00cc40 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 shutdown_connections 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9f34067860 0x7f9f34069d20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f540ca680 0x7f9f541f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5415a200 0x7f9f54147fe0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 --2- 172.21.15.164:0/2397758214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f5406dfa0 0x7f9f54145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 >> 172.21.15.164:0/2397758214 conn(0x7f9f540cf770 msgr2=0x7f9f540cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 shutdown_connections 2024-05-08T04:43:07.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:07.987+0000 7f9f5b600640 1 -- 172.21.15.164:0/2397758214 wait complete. 2024-05-08T04:43:08.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol82380 2024-05-08T04:43:08.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 Processor -- start 2024-05-08T04:43:08.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 -- start start 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ff815ab10 0x7f9ff806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80cee90 0x7f9ff80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff815e870 con 0x7f9ff80ae6e0 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff806bb80 con 0x7f9ff815ab10 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ffde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff815eb50 con 0x7f9ff80cee90 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80cee90 0x7f9ff80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ff815ab10 0x7f9ff806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9fefe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9ff80ae6e0 0x7f9ff80ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46672/0 (socket says 172.21.15.164:46672) 2024-05-08T04:43:08.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9fefe00640 1 -- 172.21.15.164:0/1387895771 learned_addr learned my addr 172.21.15.164:0/1387895771 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:08.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff6a00640 1 -- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80cee90 msgr2=0x7f9ff80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:08.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff6a00640 1 --2- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80cee90 0x7f9ff80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff6a00640 1 -- 172.21.15.164:0/1387895771 >> v1:172.21.15.172:6789/0 conn(0x7f9ff80ae6e0 legacy=0x7f9ff80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.147+0000 7f9ff6a00640 1 -- 172.21.15.164:0/1387895771 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff815ee30 con 0x7f9ff815ab10 2024-05-08T04:43:08.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ff6a00640 1 --2- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff806cce0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097c0 tx=0x7f9fe801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de72daa9131b662a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ff6000640 1 -- 172.21.15.164:0/1387895771 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80b1070 con 0x7f9ff815ab10 2024-05-08T04:43:08.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ff6000640 1 -- 172.21.15.164:0/1387895771 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe80ac600 con 0x7f9ff815ab10 2024-05-08T04:43:08.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ff6000640 1 -- 172.21.15.164:0/1387895771 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80ac9d0 con 0x7f9ff815ab10 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 -- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 msgr2=0x7f9ff806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 --2- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff806cce0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097c0 tx=0x7f9fe801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 -- 172.21.15.164:0/1387895771 shutdown_connections 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 --2- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80cee90 0x7f9ff80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 --2- 172.21.15.164:0/1387895771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff806cce0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 -- 172.21.15.164:0/1387895771 >> 172.21.15.164:0/1387895771 conn(0x7f9ff80cf770 msgr2=0x7f9ff80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 -- 172.21.15.164:0/1387895771 shutdown_connections 2024-05-08T04:43:08.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.151+0000 7f9ffde00640 1 -- 172.21.15.164:0/1387895771 wait complete. 2024-05-08T04:43:08.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 Processor -- start 2024-05-08T04:43:08.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- start start 2024-05-08T04:43:08.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80ae6e0 0x7f9ff815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80cb070 con 0x7f9ff80cee90 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80caef0 con 0x7f9ff815ab10 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80cb1f0 con 0x7f9ff80ae6e0 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40786/0 (socket says 172.21.15.164:40786) 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 -- 172.21.15.164:0/2046043154 learned_addr learned my addr 172.21.15.164:0/2046043154 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ff6a00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80ae6e0 0x7f9ff815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ff7400640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 -- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80ae6e0 msgr2=0x7f9ff815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80ae6e0 0x7f9ff815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 -- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 msgr2=0x7f9ff80c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff80c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 -- 172.21.15.164:0/2046043154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff815df40 con 0x7f9ff80cee90 2024-05-08T04:43:08.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ff7400640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff80c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:08.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fefe00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c450 tx=0x7f9fe000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0017070 con 0x7f9ff80cee90 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe0005cf0 con 0x7f9ff80cee90 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff80c8540 con 0x7f9ff80cee90 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007290 con 0x7f9ff80cee90 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.155+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff80c8ac0 con 0x7f9ff80cee90 2024-05-08T04:43:08.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff8072d30 con 0x7f9ff80cee90 2024-05-08T04:43:08.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f9fe001a070 con 0x7f9ff80cee90 2024-05-08T04:43:08.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9fef400640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 0x7f9fd4069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f9fe0082b20 con 0x7f9ff80cee90 2024-05-08T04:43:08.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9ff6a00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 0x7f9fd4069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.159+0000 7f9ff6a00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 0x7f9fd4069d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80ae000 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.163+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fe0054f70 con 0x7f9ff80cee90 2024-05-08T04:43:08.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.343+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol82380", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ff80c9be0 con 0x7f9fd4067860 2024-05-08T04:43:08.350 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:08.347+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:08.351 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:08.347+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:08.351 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:08.347+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:08.351 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:08.347+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:08.351 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:08.347+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:08.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.379+0000 7f9fef400640 1 -- 172.21.15.164:0/2046043154 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9fe80ac5e0 con 0x7f9fd4067860 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 msgr2=0x7f9fd4069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 0x7f9fd4069d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80ae000 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 msgr2=0x7f9ff8159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c450 tx=0x7f9fe000c910 comp rx=0 tx=0).stop 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 shutdown_connections 2024-05-08T04:43:08.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9fd4067860 0x7f9fd4069d20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ff815ab10 0x7f9ff80c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80cee90 0x7f9ff8159390 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 --2- 172.21.15.164:0/2046043154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ff80ae6e0 0x7f9ff815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 >> 172.21.15.164:0/2046043154 conn(0x7f9ff80cf770 msgr2=0x7f9ff806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 shutdown_connections 2024-05-08T04:43:08.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.383+0000 7f9ffde00640 1 -- 172.21.15.164:0/2046043154 wait complete. 2024-05-08T04:43:08.404 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol82380 subvol_snap7130 2024-05-08T04:43:08.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 Processor -- start 2024-05-08T04:43:08.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 -- start start 2024-05-08T04:43:08.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7480b0b90 0x7fd74806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd74806ce60 0x7fd7480ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806eea0 con 0x7fd74806f6f0 2024-05-08T04:43:08.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74806c490 con 0x7fd7480b0b90 2024-05-08T04:43:08.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74806f120 con 0x7fd74806ce60 2024-05-08T04:43:08.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74cc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7480b0b90 0x7fd74806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74d600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd74806f6f0 0x7fd7480cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37310/0 (socket says 172.21.15.164:37310) 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74d600640 1 -- 172.21.15.164:0/3359135480 learned_addr learned my addr 172.21.15.164:0/3359135480 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd73fe00640 1 --2- 172.21.15.164:0/3359135480 >> v2:172.21.15.172:3300/0 conn(0x7fd74806ce60 0x7fd7480ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74cc00640 1 -- 172.21.15.164:0/3359135480 >> v2:172.21.15.172:3300/0 conn(0x7fd74806ce60 msgr2=0x7fd7480ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74cc00640 1 --2- 172.21.15.164:0/3359135480 >> v2:172.21.15.172:3300/0 conn(0x7fd74806ce60 0x7fd7480ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74cc00640 1 -- 172.21.15.164:0/3359135480 >> v1:172.21.15.164:6789/0 conn(0x7fd74806f6f0 legacy=0x7fd7480cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd74cc00640 1 -- 172.21.15.164:0/3359135480 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74806f400 con 0x7fd7480b0b90 2024-05-08T04:43:08.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.543+0000 7fd747e00640 1 -- 172.21.15.164:0/3359135480 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383381706 0 0) 0x7fd7400014a0 con 0x7fd74806f6f0 2024-05-08T04:43:08.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74cc00640 1 --2- 172.21.15.164:0/3359135480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd74806c920 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd74806fbf0 tx=0x7fd73801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6f4c53aa3b7e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd747e00640 1 -- 172.21.15.164:0/3359135480 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380b0070 con 0x7fd7480b0b90 2024-05-08T04:43:08.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd747e00640 1 -- 172.21.15.164:0/3359135480 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7380ab600 con 0x7fd7480b0b90 2024-05-08T04:43:08.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd747e00640 1 -- 172.21.15.164:0/3359135480 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380ab9d0 con 0x7fd7480b0b90 2024-05-08T04:43:08.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 -- 172.21.15.164:0/3359135480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 msgr2=0x7fd74806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 --2- 172.21.15.164:0/3359135480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd74806c920 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd74806fbf0 tx=0x7fd73801c3f0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 -- 172.21.15.164:0/3359135480 shutdown_connections 2024-05-08T04:43:08.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 --2- 172.21.15.164:0/3359135480 >> v2:172.21.15.172:3300/0 conn(0x7fd74806ce60 0x7fd7480ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 --2- 172.21.15.164:0/3359135480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd74806c920 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 -- 172.21.15.164:0/3359135480 >> 172.21.15.164:0/3359135480 conn(0x7fd7480b3370 msgr2=0x7fd7480b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:08.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 -- 172.21.15.164:0/3359135480 shutdown_connections 2024-05-08T04:43:08.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.547+0000 7fd74e000640 1 -- 172.21.15.164:0/3359135480 wait complete. 2024-05-08T04:43:08.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 Processor -- start 2024-05-08T04:43:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- start start 2024-05-08T04:43:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd74806ce60 0x7fd748158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd74806f6f0 0x7fd7481570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480ce8b0 con 0x7fd7480b0b90 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480ce730 con 0x7fd74806f6f0 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480cea30 con 0x7fd74806ce60 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40816/0 (socket says 172.21.15.164:40816) 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 -- 172.21.15.164:0/4106764844 learned_addr learned my addr 172.21.15.164:0/4106764844 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74cc00640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd74806ce60 0x7fd748158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd73fe00640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd74806f6f0 0x7fd7481570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 -- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd74806ce60 msgr2=0x7fd748158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd74806ce60 0x7fd748158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 -- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd74806f6f0 msgr2=0x7fd7481570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd74806f6f0 0x7fd7481570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 -- 172.21.15.164:0/4106764844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480b5c40 con 0x7fd7480b0b90 2024-05-08T04:43:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74d600640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c700 tx=0x7fd74000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740018070 con 0x7fd7480b0b90 2024-05-08T04:43:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74000ce10 con 0x7fd7480b0b90 2024-05-08T04:43:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000a860 con 0x7fd7480b0b90 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd748158040 con 0x7fd7480b0b90 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.551+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748158600 con 0x7fd7480b0b90 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74806b900 con 0x7fd7480b0b90 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7fd74000aa00 con 0x7fd7480b0b90 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd745600640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 0x7fd724069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fd740083360 con 0x7fd7480b0b90 2024-05-08T04:43:08.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd74cc00640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 0x7fd724069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.555+0000 7fd74cc00640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 0x7fd724069d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7380ad000 tx=0x7fd7380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.559+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd740021020 con 0x7fd7480b0b90 2024-05-08T04:43:08.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.739+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol82380", "snap_name": "subvol_snap7130", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7480b5e40 con 0x7fd724067860 2024-05-08T04:43:08.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd745600640 1 -- 172.21.15.164:0/4106764844 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd7380ab660 con 0x7fd724067860 2024-05-08T04:43:08.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 msgr2=0x7fd724069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 0x7fd724069d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7380ad000 tx=0x7fd7380a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:08.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 msgr2=0x7fd7481577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c700 tx=0x7fd74000cbc0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 shutdown_connections 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd724067860 0x7fd724069d20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7480b0b90 0x7fd7481577f0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd74806f6f0 0x7fd7481570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 --2- 172.21.15.164:0/4106764844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd74806ce60 0x7fd748158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 >> 172.21.15.164:0/4106764844 conn(0x7fd7480b3370 msgr2=0x7fd7480d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 shutdown_connections 2024-05-08T04:43:08.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.751+0000 7fd74e000640 1 -- 172.21.15.164:0/4106764844 wait complete. 2024-05-08T04:43:08.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol82380 --retain-snapshots 2024-05-08T04:43:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 Processor -- start 2024-05-08T04:43:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 -- start start 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 0x7f1c280ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c280addd0 0x7f1c280c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c280c5af0 con 0x7f1c2815dd90 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c280ae200 con 0x7f1c280addd0 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.911+0000 7f1c30000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c280caad0 con 0x7f1c2815e170 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2d600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1c2815dd90 0x7f1c280cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37322/0 (socket says 172.21.15.164:37322) 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2cc00640 1 --2- 172.21.15.164:0/392626946 >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 0x7f1c280ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2d600640 1 -- 172.21.15.164:0/392626946 learned_addr learned my addr 172.21.15.164:0/392626946 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 --2- 172.21.15.164:0/392626946 >> v2:172.21.15.164:3300/0 conn(0x7f1c280addd0 0x7f1c280c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 -- 172.21.15.164:0/392626946 >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 msgr2=0x7f1c280ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 --2- 172.21.15.164:0/392626946 >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 0x7f1c280ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 -- 172.21.15.164:0/392626946 >> v1:172.21.15.164:6789/0 conn(0x7f1c2815dd90 legacy=0x7f1c280cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 -- 172.21.15.164:0/392626946 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c280cadb0 con 0x7f1c280addd0 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c27e00640 1 -- 172.21.15.164:0/392626946 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435291405 0 0) 0x7f1c280c5af0 con 0x7f1c2815dd90 2024-05-08T04:43:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2cc00640 1 --2- 172.21.15.164:0/392626946 >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 0x7f1c280ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c2e000640 1 --2- 172.21.15.164:0/392626946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c280addd0 0x7f1c280c4e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f1c28074a20 tx=0x7f1c2001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9236972c3325983d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c27e00640 1 -- 172.21.15.164:0/392626946 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200b2070 con 0x7f1c280addd0 2024-05-08T04:43:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c27e00640 1 -- 172.21.15.164:0/392626946 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200a5900 con 0x7f1c280addd0 2024-05-08T04:43:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.915+0000 7f1c27e00640 1 -- 172.21.15.164:0/392626946 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200a5cd0 con 0x7f1c280addd0 2024-05-08T04:43:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- 172.21.15.164:0/392626946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c280addd0 msgr2=0x7f1c280c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- 172.21.15.164:0/392626946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c280addd0 0x7f1c280c4e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f1c28074a20 tx=0x7f1c2001c930 comp rx=0 tx=0).stop 2024-05-08T04:43:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- 172.21.15.164:0/392626946 shutdown_connections 2024-05-08T04:43:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- 172.21.15.164:0/392626946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c280addd0 0x7f1c280c4e70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- 172.21.15.164:0/392626946 >> v2:172.21.15.172:3300/0 conn(0x7f1c2815e170 0x7f1c280ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- 172.21.15.164:0/392626946 >> 172.21.15.164:0/392626946 conn(0x7f1c280cc1f0 msgr2=0x7f1c280cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- 172.21.15.164:0/392626946 shutdown_connections 2024-05-08T04:43:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- 172.21.15.164:0/392626946 wait complete. 2024-05-08T04:43:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 Processor -- start 2024-05-08T04:43:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- start start 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c280addd0 0x7f1c2814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c2815e170 0x7f1c2814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280ce000 con 0x7f1c2815dd90 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280cde80 con 0x7f1c280addd0 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c30000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280ce180 con 0x7f1c2815e170 2024-05-08T04:43:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40842/0 (socket says 172.21.15.164:40842) 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 -- 172.21.15.164:0/3796677961 learned_addr learned my addr 172.21.15.164:0/3796677961 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2e000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c2815e170 0x7f1c2814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2d600640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c280addd0 0x7f1c2814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 -- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c2815e170 msgr2=0x7f1c2814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c2815e170 0x7f1c2814f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 -- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c280addd0 msgr2=0x7f1c2814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c280addd0 0x7f1c2814b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.919+0000 7f1c2cc00640 1 -- 172.21.15.164:0/3796677961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c281557e0 con 0x7f1c2815dd90 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c2cc00640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ebe0 tx=0x7f1c1400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14017070 con 0x7f1c2815dd90 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400cc30 con 0x7f1c2815dd90 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14013470 con 0x7f1c2815dd90 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c28159100 con 0x7f1c2815dd90 2024-05-08T04:43:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c281596c0 con 0x7f1c2815dd90 2024-05-08T04:43:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c28072d30 con 0x7f1c2815dd90 2024-05-08T04:43:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f1c1401a070 con 0x7f1c2815dd90 2024-05-08T04:43:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 0x7f1c08069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1c14083a30 con 0x7f1c2815dd90 2024-05-08T04:43:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.923+0000 7f1c2d600640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 0x7f1c08069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.927+0000 7f1c2d600640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 0x7f1c08069d50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007fd0 tx=0x7f1c1c007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:08.927+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c14055bc0 con 0x7f1c2815dd90 2024-05-08T04:43:09.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.107+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol82380", "retain_snapshots": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1c2804f010 con 0x7f1c08067890 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.131+0000 7f1c25600640 1 -- 172.21.15.164:0/3796677961 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1c2804f010 con 0x7f1c08067890 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 msgr2=0x7f1c08069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 0x7f1c08069d50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007fd0 tx=0x7f1c1c007d60 comp rx=0 tx=0).stop 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 msgr2=0x7f1c2806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ebe0 tx=0x7f1c1400c520 comp rx=0 tx=0).stop 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 shutdown_connections 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c08067890 0x7f1c08069d50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c2815e170 0x7f1c2814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c2815dd90 0x7f1c2806cbe0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 --2- 172.21.15.164:0/3796677961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c280addd0 0x7f1c2814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 >> 172.21.15.164:0/3796677961 conn(0x7f1c280cc1f0 msgr2=0x7f1c280af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 shutdown_connections 2024-05-08T04:43:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.135+0000 7f1c30000640 1 -- 172.21.15.164:0/3796677961 wait complete. 2024-05-08T04:43:09.789 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.139+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.789 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.139+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.789 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.139+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.139+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.139+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.159+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.159+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.159+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.159+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.790 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.159+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol82380 --pool_layout invalid_pool 2024-05-08T04:43:09.792 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.167+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.792 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.167+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.792 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.167+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.792 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.167+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.792 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:09.167+0000 7ffa59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:09.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 Processor -- start 2024-05-08T04:43:09.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 -- start start 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 0x7f14580d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1458160300 0x7f14580d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14580cfc60 con 0x7f14580cf760 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14580d2120 con 0x7f1458160300 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145815f820 con 0x7f145815b9f0 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.931+0000 7f145e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1458160300 0x7f14580d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1458160300 0x7f14580d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52132/0 (socket says 172.21.15.164:52132) 2024-05-08T04:43:09.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 -- 172.21.15.164:0/3382623165 learned_addr learned my addr 172.21.15.164:0/3382623165 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145de00640 1 --2- 172.21.15.164:0/3382623165 >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 0x7f14580d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 -- 172.21.15.164:0/3382623165 >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 msgr2=0x7f14580d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 --2- 172.21.15.164:0/3382623165 >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 0x7f14580d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 -- 172.21.15.164:0/3382623165 >> v1:172.21.15.172:6789/0 conn(0x7f14580cf760 legacy=0x7f145806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 -- 172.21.15.164:0/3382623165 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145815fb00 con 0x7f1458160300 2024-05-08T04:43:09.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145de00640 1 --2- 172.21.15.164:0/3382623165 >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 0x7f14580d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:09.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145e800640 1 --2- 172.21.15.164:0/3382623165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580d1be0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f14580b4550 tx=0x7f145401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687e6a3ee4eaa6a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:09.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145ca00640 1 -- 172.21.15.164:0/3382623165 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540a56c0 con 0x7f1458160300 2024-05-08T04:43:09.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145ca00640 1 -- 172.21.15.164:0/3382623165 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14540a5860 con 0x7f1458160300 2024-05-08T04:43:09.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.935+0000 7f145ca00640 1 -- 172.21.15.164:0/3382623165 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540a5c30 con 0x7f1458160300 2024-05-08T04:43:09.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- 172.21.15.164:0/3382623165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 msgr2=0x7f14580d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- 172.21.15.164:0/3382623165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580d1be0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f14580b4550 tx=0x7f145401c9f0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- 172.21.15.164:0/3382623165 shutdown_connections 2024-05-08T04:43:09.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- 172.21.15.164:0/3382623165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580d1be0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- 172.21.15.164:0/3382623165 >> v2:172.21.15.172:3300/0 conn(0x7f145815b9f0 0x7f14580d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- 172.21.15.164:0/3382623165 >> 172.21.15.164:0/3382623165 conn(0x7f14580723e0 msgr2=0x7f14580727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:09.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- 172.21.15.164:0/3382623165 shutdown_connections 2024-05-08T04:43:09.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- 172.21.15.164:0/3382623165 wait complete. 2024-05-08T04:43:09.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 Processor -- start 2024-05-08T04:43:09.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- start start 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14580cf760 0x7f1458157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f145815b9f0 0x7f14580cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581689b0 con 0x7f1458160300 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580b0a60 con 0x7f14580cf760 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1458168b30 con 0x7f145815b9f0 2024-05-08T04:43:09.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52146/0 (socket says 172.21.15.164:52146) 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 -- 172.21.15.164:0/3074738645 learned_addr learned my addr 172.21.15.164:0/3074738645 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145de00640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14580cf760 0x7f1458157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145d400640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f145815b9f0 0x7f14580cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 -- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f145815b9f0 msgr2=0x7f14580cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f145815b9f0 0x7f14580cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 -- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14580cf760 msgr2=0x7f1458157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14580cf760 0x7f1458157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 -- 172.21.15.164:0/3074738645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14581667d0 con 0x7f1458160300 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145de00640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14580cf760 0x7f1458157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.939+0000 7f145e800640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458160300 0x7f14580ce0f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f14540af040 tx=0x7f1454019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:09.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540b8640 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14540bf070 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f145f200640 1 -- 172.21.15.164:0/3074738645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14580b2100 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540b8a40 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f145f200640 1 -- 172.21.15.164:0/3074738645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14581fabd0 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f145f200640 1 -- 172.21.15.164:0/3074738645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145804ef30 con 0x7f1458160300 2024-05-08T04:43:09.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f14540b8be0 con 0x7f1458160300 2024-05-08T04:43:09.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f143c067860 0x7f143c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:09.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f145411e640 con 0x7f1458160300 2024-05-08T04:43:09.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f145de00640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f143c067860 0x7f143c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:09.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.943+0000 7f145de00640 1 --2- 172.21.15.164:0/3074738645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f143c067860 0x7f143c069d20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f144c009970 tx=0x7f144c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:09.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:09.947+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14540165e0 con 0x7f1458160300 2024-05-08T04:43:10.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.127+0000 7f145f200640 1 -- 172.21.15.164:0/3074738645 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol82380", "pool_layout": "invalid_pool", "target": ["mon-mgr", ""]}) v1 -- 0x7f1458157c60 con 0x7f143c067860 2024-05-08T04:43:10.146 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:10.143+0000 7ffa56200640 -1 mgr.server reply reply (22) Invalid argument invalid pool layout 'invalid_pool'--need a valid data pool 2024-05-08T04:43:10.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.143+0000 7f1452000640 1 -- 172.21.15.164:0/3074738645 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -22 invalid pool layout 'invalid_pool'--need a valid data pool) v1 ==== 66+0+0 (secure 0 0 0) 0x7f1458157c60 con 0x7f143c067860 2024-05-08T04:43:10.147 INFO:teuthology.orchestra.run.smithi164.stderr:Error EINVAL: invalid pool layout 'invalid_pool'--need a valid data pool 2024-05-08T04:43:10.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T04:43:10.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol82380 2024-05-08T04:43:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 Processor -- start 2024-05-08T04:43:10.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 -- start start 2024-05-08T04:43:10.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f195c0cec40 0x7f195c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f195c0b1d40 0x7f195c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c15c900 con 0x7f195c0b1960 2024-05-08T04:43:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c15cec0 con 0x7f195c0cec40 2024-05-08T04:43:10.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.299+0000 7f1962a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c15cbe0 con 0x7f195c0b1d40 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f195c0cec40 0x7f195c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f195c0cec40 0x7f195c15bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52150/0 (socket says 172.21.15.164:52150) 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 -- 172.21.15.164:0/729223111 learned_addr learned my addr 172.21.15.164:0/729223111 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1962000640 1 --2- 172.21.15.164:0/729223111 >> v2:172.21.15.172:3300/0 conn(0x7f195c0b1d40 0x7f195c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 -- 172.21.15.164:0/729223111 >> v2:172.21.15.172:3300/0 conn(0x7f195c0b1d40 msgr2=0x7f195c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 --2- 172.21.15.164:0/729223111 >> v2:172.21.15.172:3300/0 conn(0x7f195c0b1d40 0x7f195c0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 -- 172.21.15.164:0/729223111 >> v1:172.21.15.172:6789/0 conn(0x7f195c0b1960 legacy=0x7f195c071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 -- 172.21.15.164:0/729223111 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0712a0 con 0x7f195c0cec40 2024-05-08T04:43:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f1960c00640 1 --2- 172.21.15.164:0/729223111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15bf90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195c072460 tx=0x7f194801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b12194fd631079c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f195be00640 1 -- 172.21.15.164:0/729223111 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480b1070 con 0x7f195c0cec40 2024-05-08T04:43:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f195be00640 1 -- 172.21.15.164:0/729223111 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19480ac640 con 0x7f195c0cec40 2024-05-08T04:43:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.303+0000 7f195be00640 1 -- 172.21.15.164:0/729223111 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480aca10 con 0x7f195c0cec40 2024-05-08T04:43:10.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- 172.21.15.164:0/729223111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 msgr2=0x7f195c15bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- 172.21.15.164:0/729223111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15bf90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195c072460 tx=0x7f194801c400 comp rx=0 tx=0).stop 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- 172.21.15.164:0/729223111 shutdown_connections 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- 172.21.15.164:0/729223111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15bf90 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- 172.21.15.164:0/729223111 >> v2:172.21.15.172:3300/0 conn(0x7f195c0b1d40 0x7f195c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- 172.21.15.164:0/729223111 >> 172.21.15.164:0/729223111 conn(0x7f195c06ac90 msgr2=0x7f195c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- 172.21.15.164:0/729223111 shutdown_connections 2024-05-08T04:43:10.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- 172.21.15.164:0/729223111 wait complete. 2024-05-08T04:43:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 Processor -- start 2024-05-08T04:43:10.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- start start 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195c0b1960 0x7f195c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f195c0b1d40 0x7f195c149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0b3bf0 con 0x7f195c0cec40 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15c6e0 con 0x7f195c0b1960 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0b3d70 con 0x7f195c0b1d40 2024-05-08T04:43:10.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1961600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195c0b1960 0x7f195c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1960c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f195c0b1d40 0x7f195c149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52156/0 (socket says 172.21.15.164:52156) 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.307+0000 7f1962000640 1 -- 172.21.15.164:0/1967847266 learned_addr learned my addr 172.21.15.164:0/1967847266 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 -- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f195c0b1d40 msgr2=0x7f195c149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f195c0b1d40 0x7f195c149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 -- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195c0b1960 msgr2=0x7f195c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195c0b1960 0x7f195c0afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 -- 172.21.15.164:0/1967847266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c149740 con 0x7f195c0cec40 2024-05-08T04:43:10.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962000640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1954009dd0 tx=0x7f195400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954018020 con 0x7f195c0cec40 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954014480 con 0x7f195c0cec40 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c0b0170 con 0x7f195c0cec40 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954014870 con 0x7f195c0cec40 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c0b0730 con 0x7f195c0cec40 2024-05-08T04:43:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.311+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c04ef30 con 0x7f195c0cec40 2024-05-08T04:43:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.315+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) v1 ==== 87543+0+0 (secure 0 0 0) 0x7f195401b020 con 0x7f195c0cec40 2024-05-08T04:43:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.315+0000 7f1959600640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 0x7f193c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.315+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1954084220 con 0x7f195c0cec40 2024-05-08T04:43:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.315+0000 7f1961600640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 0x7f193c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.315+0000 7f1961600640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 0x7f193c069d20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1950001810 tx=0x7f1950009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.319+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1954022070 con 0x7f195c0cec40 2024-05-08T04:43:10.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.323+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f1954012050 con 0x7f195c0cec40 2024-05-08T04:43:10.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.503+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol82380", "target": ["mon-mgr", ""]}) v1 -- 0x7f195c15b0c0 con 0x7f193c067860 2024-05-08T04:43:10.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.507+0000 7f1959600640 1 -- 172.21.15.164:0/1967847266 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+306 (secure 0 0 0) 0x7f195c15b0c0 con 0x7f193c067860 2024-05-08T04:43:10.510 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "features": [ 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-clone", 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-autoprotect", 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-retention" 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "flavor": 2, 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "mon_addrs": [ 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.164:6789", 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6789", 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6790" 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "snapshot-retained", 2024-05-08T04:43:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "subvolume" 2024-05-08T04:43:10.512 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 msgr2=0x7f193c069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 0x7f193c069d20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1950001810 tx=0x7f1950009040 comp rx=0 tx=0).stop 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 msgr2=0x7f195c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1954009dd0 tx=0x7f195400c490 comp rx=0 tx=0).stop 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 shutdown_connections 2024-05-08T04:43:10.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f193c067860 0x7f193c069d20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f195c0b1d40 0x7f195c149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f195c0cec40 0x7f195c15ab80 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 --2- 172.21.15.164:0/1967847266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195c0b1960 0x7f195c0afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 >> 172.21.15.164:0/1967847266 conn(0x7f195c06ac90 msgr2=0x7f195c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 shutdown_connections 2024-05-08T04:43:10.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.511+0000 7f1962a00640 1 -- 172.21.15.164:0/1967847266 wait complete. 2024-05-08T04:43:10.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol82380 2024-05-08T04:43:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 Processor -- start 2024-05-08T04:43:10.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 -- start start 2024-05-08T04:43:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340c6600 con 0x7f183415ba20 2024-05-08T04:43:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183415fbe0 con 0x7f183406e390 2024-05-08T04:43:10.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1839400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340c6040 con 0x7f183415ce10 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1833200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:52156/0 (socket says 172.21.15.164:52156) 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1832800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f183415ba20 0x7f1834160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:38096/0 (socket says 172.21.15.164:38096) 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 -- 172.21.15.164:0/2023028704 learned_addr learned my addr 172.21.15.164:0/2023028704 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1833200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:54198/0 (socket says 172.21.15.164:54198) 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 -- 172.21.15.164:0/2023028704 >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 msgr2=0x7f18340ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:10.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 --2- 172.21.15.164:0/2023028704 >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 -- 172.21.15.164:0/2023028704 >> v1:172.21.15.172:6789/0 conn(0x7f183415ba20 legacy=0x7f1834160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1831e00640 1 -- 172.21.15.164:0/2023028704 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340c6320 con 0x7f183415ce10 2024-05-08T04:43:10.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.671+0000 7f1833200640 1 --2- 172.21.15.164:0/2023028704 >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:10.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1831e00640 1 --2- 172.21.15.164:0/2023028704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1820009870 tx=0x7f182001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e126857fcb78186 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1831400640 1 -- 172.21.15.164:0/2023028704 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200b0070 con 0x7f183415ce10 2024-05-08T04:43:10.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1831400640 1 -- 172.21.15.164:0/2023028704 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18200ab600 con 0x7f183415ce10 2024-05-08T04:43:10.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1831400640 1 -- 172.21.15.164:0/2023028704 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200ab9d0 con 0x7f183415ce10 2024-05-08T04:43:10.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 -- 172.21.15.164:0/2023028704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 msgr2=0x7f183406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 --2- 172.21.15.164:0/2023028704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1820009870 tx=0x7f182001c4d0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 -- 172.21.15.164:0/2023028704 shutdown_connections 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 --2- 172.21.15.164:0/2023028704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ce10 0x7f183406de50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 --2- 172.21.15.164:0/2023028704 >> v2:172.21.15.172:3300/0 conn(0x7f183406e390 0x7f18340ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 -- 172.21.15.164:0/2023028704 >> 172.21.15.164:0/2023028704 conn(0x7f18340cf770 msgr2=0x7f18340cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 -- 172.21.15.164:0/2023028704 shutdown_connections 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.675+0000 7f1839400640 1 -- 172.21.15.164:0/2023028704 wait complete. 2024-05-08T04:43:10.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 Processor -- start 2024-05-08T04:43:10.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- start start 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183406e390 0x7f183414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ba20 0x7f18340b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183415ce10 0x7f18340ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834168b70 con 0x7f183406e390 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340ce090 con 0x7f183415ce10 2024-05-08T04:43:10.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1834168cf0 con 0x7f183415ba20 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183406e390 0x7f183414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183406e390 0x7f183414ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52166/0 (socket says 172.21.15.164:52166) 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 -- 172.21.15.164:0/3020399781 learned_addr learned my addr 172.21.15.164:0/3020399781 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1833200640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183415ce10 0x7f18340ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1831e00640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ba20 0x7f18340b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 -- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ba20 msgr2=0x7f18340b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f183415ba20 0x7f18340b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 -- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183415ce10 msgr2=0x7f18340ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183415ce10 0x7f18340ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 -- 172.21.15.164:0/3020399781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18341f7600 con 0x7f183406e390 2024-05-08T04:43:10.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1832800640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183406e390 0x7f183414ee60 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f182400c790 tx=0x7f182400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824018070 con 0x7f183406e390 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824004d70 con 0x7f183406e390 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- 172.21.15.164:0/3020399781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18341f78e0 con 0x7f183406e390 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400a840 con 0x7f183406e390 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- 172.21.15.164:0/3020399781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18341f7ea0 con 0x7f183406e390 2024-05-08T04:43:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.679+0000 7f1839400640 1 -- 172.21.15.164:0/3020399781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1834072d30 con 0x7f183406e390 2024-05-08T04:43:10.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.683+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f182401b070 con 0x7f183406e390 2024-05-08T04:43:10.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.683+0000 7f182aa00640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1814067ad0 0x7f1814069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:10.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.683+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1824083360 con 0x7f183406e390 2024-05-08T04:43:10.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.683+0000 7f1831e00640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1814067ad0 0x7f1814069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:10.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.683+0000 7f1831e00640 1 --2- 172.21.15.164:0/3020399781 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1814067ad0 0x7f1814069f90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18200ad000 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:10.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.687+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1824055520 con 0x7f183406e390 2024-05-08T04:43:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.863+0000 7f1839400640 1 -- 172.21.15.164:0/3020399781 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol82380", "target": ["mon-mgr", ""]}) v1 -- 0x7f18340b07f0 con 0x7f1814067ad0 2024-05-08T04:43:10.870 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:10.867+0000 7ffa56200640 -1 mgr.server reply reply (2) No such file or directory subvolume "subvol82380" is removed and has only snapshots retained 2024-05-08T04:43:10.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:10.867+0000 7f182aa00640 1 -- 172.21.15.164:0/3020399781 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -2 subvolume "subvol82380" is removed and has only snapshots retained) v1 ==== 74+0+0 (secure 0 0 0) 0x7f18340b07f0 con 0x7f1814067ad0 2024-05-08T04:43:10.871 INFO:teuthology.orchestra.run.smithi164.stderr:Error ENOENT: subvolume "subvol82380" is removed and has only snapshots retained 2024-05-08T04:43:10.880 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T04:43:10.880 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol82380 subvol_snap7130 2024-05-08T04:43:11.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 Processor -- start 2024-05-08T04:43:11.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 -- start start 2024-05-08T04:43:11.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f07d40ae6e0 0x7f07d40aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d415e870 con 0x7f07d40ce780 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d406bb80 con 0x7f07d40ae6e0 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d415eb50 con 0x7f07d415ab10 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f07d40ae6e0 0x7f07d40aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f07d40ce780 0x7f07d40c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49364/0 (socket says 172.21.15.164:49364) 2024-05-08T04:43:11.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d3400640 1 -- 172.21.15.164:0/2419164443 learned_addr learned my addr 172.21.15.164:0/2419164443 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2a00640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2000640 1 -- 172.21.15.164:0/2419164443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 msgr2=0x7f07d406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2000640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2000640 1 -- 172.21.15.164:0/2419164443 >> v1:172.21.15.164:6789/0 conn(0x7f07d40ce780 legacy=0x7f07d40c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d2000640 1 -- 172.21.15.164:0/2419164443 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d415ee30 con 0x7f07d40ae6e0 2024-05-08T04:43:11.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.023+0000 7f07d1600640 1 -- 172.21.15.164:0/2419164443 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388640067 0 0) 0x7f07d415e870 con 0x7f07d40ce780 2024-05-08T04:43:11.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d2000640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ae6e0 0x7f07d40aeb40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caec63751f3cff75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d1600640 1 -- 172.21.15.164:0/2419164443 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b1070 con 0x7f07d40ae6e0 2024-05-08T04:43:11.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d1600640 1 -- 172.21.15.164:0/2419164443 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00a8050 con 0x7f07d40ae6e0 2024-05-08T04:43:11.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d1600640 1 -- 172.21.15.164:0/2419164443 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00ac860 con 0x7f07d40ae6e0 2024-05-08T04:43:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- 172.21.15.164:0/2419164443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ae6e0 msgr2=0x7f07d40aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ae6e0 0x7f07d40aeb40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c001c3e0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- 172.21.15.164:0/2419164443 shutdown_connections 2024-05-08T04:43:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ae6e0 0x7f07d40aeb40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 --2- 172.21.15.164:0/2419164443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- 172.21.15.164:0/2419164443 >> 172.21.15.164:0/2419164443 conn(0x7f07d40cf770 msgr2=0x7f07d40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:11.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- 172.21.15.164:0/2419164443 shutdown_connections 2024-05-08T04:43:11.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- 172.21.15.164:0/2419164443 wait complete. 2024-05-08T04:43:11.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 Processor -- start 2024-05-08T04:43:11.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.027+0000 7f07d9c00640 1 -- start start 2024-05-08T04:43:11.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d40ae6e0 0x7f07d40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d4151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d4167ae0 con 0x7f07d40ce780 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40cefc0 con 0x7f07d40ae6e0 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d4167c60 con 0x7f07d415ab10 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52178/0 (socket says 172.21.15.164:52178) 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 -- 172.21.15.164:0/705489680 learned_addr learned my addr 172.21.15.164:0/705489680 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d3400640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d4151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2a00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d40ae6e0 0x7f07d40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 -- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 msgr2=0x7f07d4151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d4151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 -- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d40ae6e0 msgr2=0x7f07d40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d40ae6e0 0x7f07d40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 -- 172.21.15.164:0/705489680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40b42b0 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d2000640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f07c001a010 tx=0x7f07c001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b1040 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00bc070 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b7690 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d4152090 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d40b4830 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.031+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d4072d30 con 0x7f07d40ce780 2024-05-08T04:43:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.035+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f07c00b7830 con 0x7f07d40ce780 2024-05-08T04:43:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.035+0000 7f07caa00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 0x7f07b4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.035+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f07c011dfa0 con 0x7f07d40ce780 2024-05-08T04:43:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.035+0000 7f07d2a00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 0x7f07b4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.035+0000 7f07d2a00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 0x7f07b4069f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b930 tx=0x7f07c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.039+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07c0016090 con 0x7f07d40ce780 2024-05-08T04:43:11.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.219+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol82380", "snap_name": "subvol_snap7130", "target": ["mon-mgr", ""]}) v1 -- 0x7f07d404f010 con 0x7f07b4067aa0 2024-05-08T04:43:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07caa00640 1 -- 172.21.15.164:0/705489680 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f07d404f010 con 0x7f07b4067aa0 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 msgr2=0x7f07b4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 0x7f07b4069f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b930 tx=0x7f07c4008040 comp rx=0 tx=0).stop 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 msgr2=0x7f07d414bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f07c001a010 tx=0x7f07c001ac20 comp rx=0 tx=0).stop 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 shutdown_connections 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f07b4067aa0 0x7f07b4069f60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d415ab10 0x7f07d4151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07d40ce780 0x7f07d414bbd0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 --2- 172.21.15.164:0/705489680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d40ae6e0 0x7f07d40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 >> 172.21.15.164:0/705489680 conn(0x7f07d40cf770 msgr2=0x7f07d40b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:11.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 shutdown_connections 2024-05-08T04:43:11.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.259+0000 7f07d9c00640 1 -- 172.21.15.164:0/705489680 wait complete. 2024-05-08T04:43:11.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:11.290 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-08T04:43:11.304 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-05-08T04:43:11.305 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:43:11.306 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:11.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:11.520 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:43:11.520 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:11.520 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:11.529 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:11.529 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:11.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 Processor -- start 2024-05-08T04:43:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 -- start start 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6d006ac90 0x7fc6d006b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d006b5b0 con 0x7fc6d00add10 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d0072120 con 0x7fc6d00ad890 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6d4c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d0072400 con 0x7fc6d006ac90 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6ce800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc6d00add10 0x7fc6d006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49380/0 (socket says 172.21.15.164:49380) 2024-05-08T04:43:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6ce800640 1 -- 172.21.15.164:0/3229340658 learned_addr learned my addr 172.21.15.164:0/3229340658 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:11.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.755+0000 7fc6cde00640 1 --2- 172.21.15.164:0/3229340658 >> v2:172.21.15.172:3300/0 conn(0x7fc6d006ac90 0x7fc6d006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426257246 0 0) 0x7fc6d006b5b0 con 0x7fc6d00add10 2024-05-08T04:43:11.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b4003530 con 0x7fc6d00add10 2024-05-08T04:43:11.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336599475 0 0) 0x7fc6b4003530 con 0x7fc6d00add10 2024-05-08T04:43:11.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cde00640 1 -- 172.21.15.164:0/3229340658 >> v1:172.21.15.172:6789/0 conn(0x7fc6d00ad890 legacy=0x7fc6d0071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cde00640 1 -- 172.21.15.164:0/3229340658 >> v1:172.21.15.164:6789/0 conn(0x7fc6d00add10 legacy=0x7fc6d006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cde00640 1 -- 172.21.15.164:0/3229340658 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d00726e0 con 0x7fc6d006ac90 2024-05-08T04:43:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cde00640 1 --2- 172.21.15.164:0/3229340658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d006b070 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d90 tx=0x7fc6c001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223da80972cfd405 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00b2070 con 0x7fc6d006ac90 2024-05-08T04:43:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c00a55c0 con 0x7fc6d006ac90 2024-05-08T04:43:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.759+0000 7fc6cd400640 1 -- 172.21.15.164:0/3229340658 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8001680 con 0x7fc6d006ac90 2024-05-08T04:43:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- 172.21.15.164:0/3229340658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 msgr2=0x7fc6d006b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/3229340658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d006b070 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d90 tx=0x7fc6c001c890 comp rx=0 tx=0).stop 2024-05-08T04:43:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- 172.21.15.164:0/3229340658 shutdown_connections 2024-05-08T04:43:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/3229340658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d006b070 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- 172.21.15.164:0/3229340658 >> 172.21.15.164:0/3229340658 conn(0x7fc6d0066fa0 msgr2=0x7fc6d006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- 172.21.15.164:0/3229340658 shutdown_connections 2024-05-08T04:43:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- 172.21.15.164:0/3229340658 wait complete. 2024-05-08T04:43:11.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 Processor -- start 2024-05-08T04:43:11.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- start start 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d00d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 0x7fc6d0161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d016c700 con 0x7fc6d00add10 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d016c580 con 0x7fc6d006ac90 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6d4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d016c880 con 0x7fc6d00ad890 2024-05-08T04:43:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52184/0 (socket says 172.21.15.164:52184) 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 -- 172.21.15.164:0/212159883 learned_addr learned my addr 172.21.15.164:0/212159883 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6cde00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d00d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6c5400640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 0x7fc6d0161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 -- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 msgr2=0x7fc6d0161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 0x7fc6d0161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 -- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 msgr2=0x7fc6d00d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d00d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 -- 172.21.15.164:0/212159883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d01600e0 con 0x7fc6d00add10 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6c5400640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 0x7fc6d0161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.763+0000 7fc6ce800640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c600 tx=0x7fc6c800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8005cf0 con 0x7fc6d00add10 2024-05-08T04:43:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8007070 con 0x7fc6d00add10 2024-05-08T04:43:11.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d01603c0 con 0x7fc6d00add10 2024-05-08T04:43:11.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8007460 con 0x7fc6d00add10 2024-05-08T04:43:11.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d00d0710 con 0x7fc6d00add10 2024-05-08T04:43:11.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d0069d60 con 0x7fc6d00add10 2024-05-08T04:43:11.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fc6c8007600 con 0x7fc6d00add10 2024-05-08T04:43:11.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 0x7fc6ac069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:11.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fc6c80839b0 con 0x7fc6d00add10 2024-05-08T04:43:11.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.767+0000 7fc6cde00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 0x7fc6ac069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:11.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.771+0000 7fc6cde00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 0x7fc6ac069f60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00af000 tx=0x7fc6c00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:11.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.771+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6c80561c0 con 0x7fc6d00add10 2024-05-08T04:43:11.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.927+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6d00b55d0 con 0x7fc6d00add10 2024-05-08T04:43:11.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.927+0000 7fc6c6a00640 1 -- 172.21.15.164:0/212159883 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16915 (secure 0 0 0) 0x7fc6c8055ac0 con 0x7fc6d00add10 2024-05-08T04:43:11.930 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:11.930 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":113,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:11.321420+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.36","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 msgr2=0x7fc6ac069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 0x7fc6ac069f60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00af000 tx=0x7fc6c00a7070 comp rx=0 tx=0).stop 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 msgr2=0x7fc6d014e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c600 tx=0x7fc6c800cac0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 shutdown_connections 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc6ac067aa0 0x7fc6ac069f60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d00ad890 0x7fc6d0161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d00add10 0x7fc6d014e810 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 --2- 172.21.15.164:0/212159883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d006ac90 0x7fc6d00d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:11.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 >> 172.21.15.164:0/212159883 conn(0x7fc6d0066fa0 msgr2=0x7fc6d006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:11.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 shutdown_connections 2024-05-08T04:43:11.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:11.931+0000 7fc6d4c00640 1 -- 172.21.15.164:0/212159883 wait complete. 2024-05-08T04:43:11.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:51.994625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:52.615473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:11.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:12.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.087+0000 7f8de2200640 1 Processor -- start 2024-05-08T04:43:12.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.087+0000 7f8de2200640 1 -- start start 2024-05-08T04:43:12.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8de2200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8de2200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ddc0ca140 0x7f8ddc0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8de2200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0caa90 con 0x7f8ddc15a200 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8de2200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc06b3a0 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8de2200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc06bca0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52188/0 (socket says 172.21.15.164:52188) 2024-05-08T04:43:12.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 -- 172.21.15.164:0/481808257 learned_addr learned my addr 172.21.15.164:0/481808257 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:12.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 -- 172.21.15.164:0/481808257 >> v2:172.21.15.172:3300/0 conn(0x7f8ddc0ca140 msgr2=0x7f8ddc0ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 --2- 172.21.15.164:0/481808257 >> v2:172.21.15.172:3300/0 conn(0x7f8ddc0ca140 0x7f8ddc0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 -- 172.21.15.164:0/481808257 >> v1:172.21.15.172:6789/0 conn(0x7f8ddc15a200 legacy=0x7f8ddc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.091+0000 7f8ddb400640 1 -- 172.21.15.164:0/481808257 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc06d150 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8ddb400640 1 --2- 172.21.15.164:0/481808257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15dd70 tx=0x7f8dc401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b45e6f28765f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8ddaa00640 1 -- 172.21.15.164:0/481808257 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40b0070 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8ddaa00640 1 -- 172.21.15.164:0/481808257 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc40ab600 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8ddaa00640 1 -- 172.21.15.164:0/481808257 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40ab9d0 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- 172.21.15.164:0/481808257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc06dfa0 msgr2=0x7f8ddc0aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- 172.21.15.164:0/481808257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15dd70 tx=0x7f8dc401c270 comp rx=0 tx=0).stop 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- 172.21.15.164:0/481808257 shutdown_connections 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- 172.21.15.164:0/481808257 >> v2:172.21.15.172:3300/0 conn(0x7f8ddc0ca140 0x7f8ddc0ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- 172.21.15.164:0/481808257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc0aeac0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- 172.21.15.164:0/481808257 >> 172.21.15.164:0/481808257 conn(0x7f8ddc0cf770 msgr2=0x7f8ddc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- 172.21.15.164:0/481808257 shutdown_connections 2024-05-08T04:43:12.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- 172.21.15.164:0/481808257 wait complete. 2024-05-08T04:43:12.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 Processor -- start 2024-05-08T04:43:12.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- start start 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ddc15a200 0x7f8ddc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ce780 con 0x7f8ddc0ca140 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15a690 con 0x7f8ddc06dfa0 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8de2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ce900 con 0x7f8ddc15a200 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.095+0000 7f8dd2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52196/0 (socket says 172.21.15.164:52196) 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 -- 172.21.15.164:0/3100629632 learned_addr learned my addr 172.21.15.164:0/3100629632 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8ddb400640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8ddbe00640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ddc15a200 0x7f8ddc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 -- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ddc15a200 msgr2=0x7f8ddc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ddc15a200 0x7f8ddc0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 -- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc06dfa0 msgr2=0x7f8ddc1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 -- 172.21.15.164:0/3100629632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc1528f0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd2a00640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00c4c0 tx=0x7f8dcc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc017070 con 0x7f8ddc0ca140 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc003600 con 0x7f8ddc0ca140 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0039f0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc158cd0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc159290 con 0x7f8ddc0ca140 2024-05-08T04:43:12.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc072d30 con 0x7f8ddc0ca140 2024-05-08T04:43:12.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f8dcc01a070 con 0x7f8ddc0ca140 2024-05-08T04:43:12.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.099+0000 7f8dd3e00640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 0x7f8db8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.103+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f8dcc082fd0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.103+0000 7f8ddb400640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 0x7f8db8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.103+0000 7f8ddb400640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 0x7f8db8069f60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ad000 tx=0x7f8dc40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.103+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dcc0557e0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.259+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ddc04f010 con 0x7f8ddc0ca140 2024-05-08T04:43:12.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.259+0000 7f8dd3e00640 1 -- 172.21.15.164:0/3100629632 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dcc0550e0 con 0x7f8ddc0ca140 2024-05-08T04:43:12.262 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:12.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 msgr2=0x7f8db8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 0x7f8db8069f60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ad000 tx=0x7f8dc40a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:12.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 msgr2=0x7f8ddc0cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00c4c0 tx=0x7f8dcc001e80 comp rx=0 tx=0).stop 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 shutdown_connections 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8db8067aa0 0x7f8db8069f60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ddc15a200 0x7f8ddc0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ddc0ca140 0x7f8ddc0cdb60 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 --2- 172.21.15.164:0/3100629632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc06dfa0 0x7f8ddc1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 >> 172.21.15.164:0/3100629632 conn(0x7f8ddc0cf770 msgr2=0x7f8ddc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 shutdown_connections 2024-05-08T04:43:12.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.263+0000 7f8de2200640 1 -- 172.21.15.164:0/3100629632 wait complete. 2024-05-08T04:43:12.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:12.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 Processor -- start 2024-05-08T04:43:12.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 -- start start 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f902c15ba20 0x7f902c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c06de50 0x7f902c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c0c6600 con 0x7f902c0ae6e0 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c15fbe0 con 0x7f902c15ba20 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9031400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c6040 con 0x7f902c06de50 2024-05-08T04:43:12.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902b200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f902c0ae6e0 0x7f902c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49388/0 (socket says 172.21.15.164:49388) 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902b200640 1 -- 172.21.15.164:0/2765243323 learned_addr learned my addr 172.21.15.164:0/2765243323 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902a800640 1 --2- 172.21.15.164:0/2765243323 >> v2:172.21.15.172:3300/0 conn(0x7f902c15ba20 0x7f902c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9029e00640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c06de50 0x7f902c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9029400640 1 -- 172.21.15.164:0/2765243323 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290747205 0 0) 0x7f902c0c6600 con 0x7f902c0ae6e0 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f9029400640 1 -- 172.21.15.164:0/2765243323 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9010003530 con 0x7f902c0ae6e0 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902a800640 1 -- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c06de50 msgr2=0x7f902c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902a800640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c06de50 0x7f902c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902a800640 1 -- 172.21.15.164:0/2765243323 >> v1:172.21.15.164:6789/0 conn(0x7f902c0ae6e0 legacy=0x7f902c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.435+0000 7f902a800640 1 -- 172.21.15.164:0/2765243323 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0c6320 con 0x7f902c15ba20 2024-05-08T04:43:12.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f902a800640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c15ba20 0x7f902c160070 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f902c15fd60 tx=0x7f901c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e34db0e2760d2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9029400640 1 -- 172.21.15.164:0/2765243323 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c0b2070 con 0x7f902c15ba20 2024-05-08T04:43:12.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9029400640 1 -- 172.21.15.164:0/2765243323 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c0a5980 con 0x7f902c15ba20 2024-05-08T04:43:12.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9029400640 1 -- 172.21.15.164:0/2765243323 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c0a5d30 con 0x7f902c15ba20 2024-05-08T04:43:12.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 -- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c15ba20 msgr2=0x7f902c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c15ba20 0x7f902c160070 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f902c15fd60 tx=0x7f901c01c920 comp rx=0 tx=0).stop 2024-05-08T04:43:12.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 -- 172.21.15.164:0/2765243323 shutdown_connections 2024-05-08T04:43:12.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c06de50 0x7f902c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 --2- 172.21.15.164:0/2765243323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c15ba20 0x7f902c160070 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 -- 172.21.15.164:0/2765243323 >> 172.21.15.164:0/2765243323 conn(0x7f902c0cf770 msgr2=0x7f902c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 -- 172.21.15.164:0/2765243323 shutdown_connections 2024-05-08T04:43:12.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.439+0000 7f9031400640 1 -- 172.21.15.164:0/2765243323 wait complete. 2024-05-08T04:43:12.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 Processor -- start 2024-05-08T04:43:12.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- start start 2024-05-08T04:43:12.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c06de50 0x7f902c153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c0ae6e0 0x7f902c06aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0aec40 con 0x7f902c15ba20 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0aeac0 con 0x7f902c06de50 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0aedc0 con 0x7f902c0ae6e0 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52212/0 (socket says 172.21.15.164:52212) 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 -- 172.21.15.164:0/1479615169 learned_addr learned my addr 172.21.15.164:0/1479615169 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9029e00640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c0ae6e0 0x7f902c06aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902a800640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c06de50 0x7f902c153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 -- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c0ae6e0 msgr2=0x7f902c06aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c0ae6e0 0x7f902c06aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 -- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c06de50 msgr2=0x7f902c153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c06de50 0x7f902c153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 -- 172.21.15.164:0/1479615169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c14edb0 con 0x7f902c15ba20 2024-05-08T04:43:12.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f902b200640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f902400c450 tx=0x7f902400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024017070 con 0x7f902c15ba20 2024-05-08T04:43:12.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9024005cf0 con 0x7f902c15ba20 2024-05-08T04:43:12.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90240072b0 con 0x7f902c15ba20 2024-05-08T04:43:12.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c14f090 con 0x7f902c15ba20 2024-05-08T04:43:12.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.443+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c14f650 con 0x7f902c15ba20 2024-05-08T04:43:12.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c072d30 con 0x7f902c15ba20 2024-05-08T04:43:12.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f9024007450 con 0x7f902c15ba20 2024-05-08T04:43:12.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f9022a00640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 0x7f900c069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f9024082e10 con 0x7f902c15ba20 2024-05-08T04:43:12.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f902a800640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 0x7f900c069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.447+0000 7f902a800640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 0x7f900c069f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f901c0af000 tx=0x7f901c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.451+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9024007740 con 0x7f902c15ba20 2024-05-08T04:43:12.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.603+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f902c06e690 con 0x7f902c15ba20 2024-05-08T04:43:12.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9022a00640 1 -- 172.21.15.164:0/1479615169 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9024054fd0 con 0x7f902c15ba20 2024-05-08T04:43:12.607 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 msgr2=0x7f900c069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 0x7f900c069f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f901c0af000 tx=0x7f901c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 msgr2=0x7f902c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f902400c450 tx=0x7f902400c910 comp rx=0 tx=0).stop 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 shutdown_connections 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f900c067ad0 0x7f900c069f90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f902c0ae6e0 0x7f902c06aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c15ba20 0x7f902c15a740 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 --2- 172.21.15.164:0/1479615169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f902c06de50 0x7f902c153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 >> 172.21.15.164:0/1479615169 conn(0x7f902c0cf770 msgr2=0x7f902c1514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 shutdown_connections 2024-05-08T04:43:12.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.607+0000 7f9031400640 1 -- 172.21.15.164:0/1479615169 wait complete. 2024-05-08T04:43:12.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:12.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.763+0000 7f3489800640 1 Processor -- start 2024-05-08T04:43:12.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.763+0000 7f3489800640 1 -- start start 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3489800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f34840ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3489800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348415e870 con 0x7f34840cecc0 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3489800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348415eb50 con 0x7f348415ab10 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3489800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cf130 con 0x7f348406cce0 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f34840cecc0 0x7f34840ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49392/0 (socket says 172.21.15.164:49392) 2024-05-08T04:43:12.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 -- 172.21.15.164:0/3076107406 learned_addr learned my addr 172.21.15.164:0/3076107406 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:12.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f347a000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f34840ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773882473 0 0) 0x7f348415e870 con 0x7f34840cecc0 2024-05-08T04:43:12.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3468003530 con 0x7f34840cecc0 2024-05-08T04:43:12.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895766198 0 0) 0x7f348415eb50 con 0x7f348415ab10 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f348415e870 con 0x7f348415ab10 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066194917 0 0) 0x7f3468003530 con 0x7f34840cecc0 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 msgr2=0x7f34840ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f34840ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 >> v1:172.21.15.172:6789/0 conn(0x7f348415ab10 legacy=0x7f34840ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348415ee30 con 0x7f34840cecc0 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f347c001a00 con 0x7f34840cecc0 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 >> v1:172.21.15.164:6789/0 conn(0x7f34840cecc0 legacy=0x7f34840ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f3468003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34680040e0 0x7f3468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3468004d50 0x7f34680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cf130 con 0x7f3468004d50 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346800bb60 con 0x7f34680040e0 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346800bd30 con 0x7f348406cce0 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3468004d50 0x7f34680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f347a000640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34680040e0 0x7f3468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3482a00640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f3468003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 -- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 msgr2=0x7f3468003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348406cce0 0x7f3468003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 -- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34680040e0 msgr2=0x7f3468004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34680040e0 0x7f3468004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.767+0000 7f3483400640 1 -- 172.21.15.164:0/3076107406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468003530 con 0x7f3468004d50 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3483400640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3468004d50 0x7f34680051d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f347c00a800 tx=0x7f347c01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd052d884b2d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00a430 con 0x7f3468004d50 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c0ad650 con 0x7f3468004d50 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3482000640 1 -- 172.21.15.164:0/3076107406 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0ada80 con 0x7f3468004d50 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- 172.21.15.164:0/3076107406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3468004d50 msgr2=0x7f34680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 --2- 172.21.15.164:0/3076107406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3468004d50 0x7f34680051d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f347c00a800 tx=0x7f347c01d300 comp rx=0 tx=0).stop 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3483400640 1 -- 172.21.15.164:0/3076107406 reap_dead start 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- 172.21.15.164:0/3076107406 shutdown_connections 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- 172.21.15.164:0/3076107406 >> 172.21.15.164:0/3076107406 conn(0x7f34840cf770 msgr2=0x7f34840cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- 172.21.15.164:0/3076107406 shutdown_connections 2024-05-08T04:43:12.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- 172.21.15.164:0/3076107406 wait complete. 2024-05-08T04:43:12.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 Processor -- start 2024-05-08T04:43:12.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.771+0000 7f3489800640 1 -- start start 2024-05-08T04:43:12.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3468004d50 0x7f34841534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 0x7f3484157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348415f3f0 con 0x7f3484070ce0 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348406bb80 con 0x7f3484156bb0 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348415aef0 con 0x7f3468004d50 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52234/0 (socket says 172.21.15.164:52234) 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 -- 172.21.15.164:0/3424670777 learned_addr learned my addr 172.21.15.164:0/3424670777 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3482a00640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3468004d50 0x7f34841534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3483400640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 0x7f3484157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 -- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3468004d50 msgr2=0x7f34841534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3468004d50 0x7f34841534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 -- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 msgr2=0x7f3484157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 0x7f3484157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 -- 172.21.15.164:0/3424670777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840716a0 con 0x7f3484070ce0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3483400640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 0x7f3484157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347a000640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f346c011a30 tx=0x7f346c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00fbb0 con 0x7f3484070ce0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c00fd50 con 0x7f3484070ce0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c014680 con 0x7f3484070ce0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840c97b0 con 0x7f3484070ce0 2024-05-08T04:43:12.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840c68e0 con 0x7f3484070ce0 2024-05-08T04:43:12.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3484072d30 con 0x7f3484070ce0 2024-05-08T04:43:12.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.775+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f346c012040 con 0x7f3484070ce0 2024-05-08T04:43:12.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.779+0000 7f347b400640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 0x7f3460069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:12.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.779+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f346c0841c0 con 0x7f3484070ce0 2024-05-08T04:43:12.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.779+0000 7f3482a00640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 0x7f3460069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:12.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.779+0000 7f3482a00640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 0x7f3460069f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f348406e4b0 tx=0x7f34740014b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:12.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.783+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34840c97b0 con 0x7f3484070ce0 2024-05-08T04:43:12.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.935+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f348404f010 con 0x7f3484070ce0 2024-05-08T04:43:12.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.935+0000 7f347b400640 1 -- 172.21.15.164:0/3424670777 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f346c019030 con 0x7f3484070ce0 2024-05-08T04:43:12.938 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:12.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 msgr2=0x7f3460069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 0x7f3460069f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f348406e4b0 tx=0x7f34740014b0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 msgr2=0x7f3484071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f346c011a30 tx=0x7f346c011ef0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 shutdown_connections 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3460067ad0 0x7f3460069f90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3484156bb0 0x7f3484157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3484070ce0 0x7f3484071160 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 --2- 172.21.15.164:0/3424670777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3468004d50 0x7f34841534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 >> 172.21.15.164:0/3424670777 conn(0x7f34840cf770 msgr2=0x7f348406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 shutdown_connections 2024-05-08T04:43:12.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:12.939+0000 7f3489800640 1 -- 172.21.15.164:0/3424670777 wait complete. 2024-05-08T04:43:12.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:43:13.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.091+0000 7f3924400640 1 Processor -- start 2024-05-08T04:43:13.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3924400640 1 -- start start 2024-05-08T04:43:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3924400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f391c15b390 0x7f391c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3924400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c06e0e0 con 0x7f391c0aee30 2024-05-08T04:43:13.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3924400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c15ed80 con 0x7f391c0cf150 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3924400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c15e8f0 con 0x7f391c15b390 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f391c15b390 0x7f391c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f391c0aee30 0x7f391c0cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49400/0 (socket says 172.21.15.164:49400) 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921000640 1 -- 172.21.15.164:0/1320810896 learned_addr learned my addr 172.21.15.164:0/1320810896 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921a00640 1 -- 172.21.15.164:0/1320810896 >> v1:172.21.15.172:6789/0 conn(0x7f391c0cf150 legacy=0x7f391c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921a00640 1 -- 172.21.15.164:0/1320810896 >> v1:172.21.15.164:6789/0 conn(0x7f391c0aee30 legacy=0x7f391c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f391be00640 1 -- 172.21.15.164:0/1320810896 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833686113 0 0) 0x7f391c06e0e0 con 0x7f391c0aee30 2024-05-08T04:43:13.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.095+0000 7f3921a00640 1 -- 172.21.15.164:0/1320810896 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c15dfc0 con 0x7f391c15b390 2024-05-08T04:43:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3921a00640 1 --2- 172.21.15.164:0/1320810896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c06d8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f391c15ea70 tx=0x7f391001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e080ee6ce0176 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f391be00640 1 -- 172.21.15.164:0/1320810896 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100b0070 con 0x7f391c15b390 2024-05-08T04:43:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f391be00640 1 -- 172.21.15.164:0/1320810896 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39100ab600 con 0x7f391c15b390 2024-05-08T04:43:13.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f391be00640 1 -- 172.21.15.164:0/1320810896 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100ab9d0 con 0x7f391c15b390 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- 172.21.15.164:0/1320810896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 msgr2=0x7f391c06d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 --2- 172.21.15.164:0/1320810896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c06d8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f391c15ea70 tx=0x7f391001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- 172.21.15.164:0/1320810896 shutdown_connections 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 --2- 172.21.15.164:0/1320810896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c06d8b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- 172.21.15.164:0/1320810896 >> 172.21.15.164:0/1320810896 conn(0x7f391c06ac90 msgr2=0x7f391c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- 172.21.15.164:0/1320810896 shutdown_connections 2024-05-08T04:43:13.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- 172.21.15.164:0/1320810896 wait complete. 2024-05-08T04:43:13.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 Processor -- start 2024-05-08T04:43:13.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.099+0000 7f3924400640 1 -- start start 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f391c0aee30 0x7f391c1514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f391c0cf150 0x7f391c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c168260 con 0x7f391c15b390 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c15b770 con 0x7f391c0cf150 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1683e0 con 0x7f391c0aee30 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52254/0 (socket says 172.21.15.164:52254) 2024-05-08T04:43:13.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 -- 172.21.15.164:0/733480064 learned_addr learned my addr 172.21.15.164:0/733480064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3921a00640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f391c0aee30 0x7f391c1514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3921000640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f391c0cf150 0x7f391c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 -- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f391c0aee30 msgr2=0x7f391c1514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f391c0aee30 0x7f391c1514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 -- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f391c0cf150 msgr2=0x7f391c14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f391c0cf150 0x7f391c14ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 -- 172.21.15.164:0/733480064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c1fa470 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3922400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f391400c7a0 tx=0x7f391400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914003b80 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3914003d20 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400aa30 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c1fa750 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1fad10 con 0x7f391c15b390 2024-05-08T04:43:13.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.103+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c069d60 con 0x7f391c15b390 2024-05-08T04:43:13.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.107+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f391400abd0 con 0x7f391c15b390 2024-05-08T04:43:13.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.107+0000 7f3919600640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 0x7f38fc069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.107+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f3914083ad0 con 0x7f391c15b390 2024-05-08T04:43:13.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.107+0000 7f3921a00640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 0x7f38fc069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.107+0000 7f3921a00640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 0x7f38fc069f90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f39100ad000 tx=0x7f39100a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.111+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3914056280 con 0x7f391c15b390 2024-05-08T04:43:13.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.279+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f391c0d0580 con 0x7f391c15b390 2024-05-08T04:43:13.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.279+0000 7f3919600640 1 -- 172.21.15.164:0/733480064 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3914055b80 con 0x7f391c15b390 2024-05-08T04:43:13.281 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 msgr2=0x7f38fc069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 0x7f38fc069f90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f39100ad000 tx=0x7f39100a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 msgr2=0x7f391c0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f391400c7a0 tx=0x7f391400cc60 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 shutdown_connections 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f38fc067ad0 0x7f38fc069f90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f391c15b390 0x7f391c0ad890 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f391c0cf150 0x7f391c14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 --2- 172.21.15.164:0/733480064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f391c0aee30 0x7f391c1514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 >> 172.21.15.164:0/733480064 conn(0x7f391c06ac90 msgr2=0x7f391c0b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:13.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 shutdown_connections 2024-05-08T04:43:13.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.283+0000 7f3924400640 1 -- 172.21.15.164:0/733480064 wait complete. 2024-05-08T04:43:13.302 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:43:13.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:13.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:13.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:43:13.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:13.329 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:43:13.383 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:13.383 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:13.392 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:13.393 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:13.442 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.575+0000 7f3b98a00640 1 Processor -- start 2024-05-08T04:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 -- start start 2024-05-08T04:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3b9415b630 0x7f3b9415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 0x7f3b940b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b940b3cc0 con 0x7f3b9406ac90 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9415e730 con 0x7f3b9415b630 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b98a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9415ba10 con 0x7f3b940af540 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3b9415b630 0x7f3b9415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3b9406ac90 0x7f3b940d0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49414/0 (socket says 172.21.15.164:49414) 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93e00640 1 -- 172.21.15.164:0/3513975980 learned_addr learned my addr 172.21.15.164:0/3513975980 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b8aa00640 1 --2- 172.21.15.164:0/3513975980 >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 0x7f3b940b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93400640 1 -- 172.21.15.164:0/3513975980 >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 msgr2=0x7f3b940b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93400640 1 --2- 172.21.15.164:0/3513975980 >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 0x7f3b940b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93400640 1 -- 172.21.15.164:0/3513975980 >> v1:172.21.15.164:6789/0 conn(0x7f3b9406ac90 legacy=0x7f3b940d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b93400640 1 -- 172.21.15.164:0/3513975980 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940afbb0 con 0x7f3b9415b630 2024-05-08T04:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b92a00640 1 -- 172.21.15.164:0/3513975980 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719330602 0 0) 0x7f3b940b3cc0 con 0x7f3b9406ac90 2024-05-08T04:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.579+0000 7f3b8aa00640 1 --2- 172.21.15.164:0/3513975980 >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 0x7f3b940b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:13.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b93400640 1 --2- 172.21.15.164:0/3513975980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b9415b630 0x7f3b9415ebc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3b9406cb90 tx=0x7f3b8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e54992750c475 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b92a00640 1 -- 172.21.15.164:0/3513975980 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b840b0070 con 0x7f3b9415b630 2024-05-08T04:43:13.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b92a00640 1 -- 172.21.15.164:0/3513975980 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b840ab600 con 0x7f3b9415b630 2024-05-08T04:43:13.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b92a00640 1 -- 172.21.15.164:0/3513975980 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b840ab9d0 con 0x7f3b9415b630 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- 172.21.15.164:0/3513975980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b9415b630 msgr2=0x7f3b9415ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 --2- 172.21.15.164:0/3513975980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b9415b630 0x7f3b9415ebc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3b9406cb90 tx=0x7f3b8401c480 comp rx=0 tx=0).stop 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- 172.21.15.164:0/3513975980 shutdown_connections 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 --2- 172.21.15.164:0/3513975980 >> v2:172.21.15.172:3300/0 conn(0x7f3b940af540 0x7f3b940b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 --2- 172.21.15.164:0/3513975980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b9415b630 0x7f3b9415ebc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- 172.21.15.164:0/3513975980 >> 172.21.15.164:0/3513975980 conn(0x7f3b94066fa0 msgr2=0x7f3b9406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- 172.21.15.164:0/3513975980 shutdown_connections 2024-05-08T04:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- 172.21.15.164:0/3513975980 wait complete. 2024-05-08T04:43:13.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 Processor -- start 2024-05-08T04:43:13.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.583+0000 7f3b98a00640 1 -- start start 2024-05-08T04:43:13.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b9406ac90 0x7f3b941494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b9415b630 0x7f3b940ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9406b4b0 con 0x7f3b940af540 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9416a7f0 con 0x7f3b9406ac90 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b94149a20 con 0x7f3b9415b630 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52268/0 (socket says 172.21.15.164:52268) 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 -- 172.21.15.164:0/2190166400 learned_addr learned my addr 172.21.15.164:0/2190166400 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b93400640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b9406ac90 0x7f3b941494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b93e00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b9415b630 0x7f3b940ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 -- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b9415b630 msgr2=0x7f3b940ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b9415b630 0x7f3b940ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 -- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b9406ac90 msgr2=0x7f3b941494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b9406ac90 0x7f3b941494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 -- 172.21.15.164:0/2190166400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94162e60 con 0x7f3b940af540 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8aa00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00e840 tx=0x7f3b7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c017070 con 0x7f3b940af540 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00c870 con 0x7f3b940af540 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00f040 con 0x7f3b940af540 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b94163100 con 0x7f3b940af540 2024-05-08T04:43:13.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941636c0 con 0x7f3b940af540 2024-05-08T04:43:13.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9404ef30 con 0x7f3b940af540 2024-05-08T04:43:13.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.587+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f3b7c01a070 con 0x7f3b940af540 2024-05-08T04:43:13.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.591+0000 7f3b8be00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 0x7f3b70069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.591+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f3b7c083790 con 0x7f3b940af540 2024-05-08T04:43:13.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.591+0000 7f3b93400640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 0x7f3b70069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.591+0000 7f3b93400640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 0x7f3b70069f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3b840ad000 tx=0x7f3b840a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.595+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b7c055950 con 0x7f3b940af540 2024-05-08T04:43:13.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b940cfee0 con 0x7f3b940af540 2024-05-08T04:43:13.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b8be00640 1 -- 172.21.15.164:0/2190166400 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17027 (secure 0 0 0) 0x7f3b7c07ce20 con 0x7f3b940af540 2024-05-08T04:43:13.752 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:13.752 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":114,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:12.323259+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.36","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:13.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 msgr2=0x7f3b70069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 0x7f3b70069f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3b840ad000 tx=0x7f3b840a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 msgr2=0x7f3b94167f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00e840 tx=0x7f3b7c00ed00 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 shutdown_connections 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3b70067aa0 0x7f3b70069f60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b9415b630 0x7f3b940ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b940af540 0x7f3b94167f30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 --2- 172.21.15.164:0/2190166400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b9406ac90 0x7f3b941494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 >> 172.21.15.164:0/2190166400 conn(0x7f3b94066fa0 msgr2=0x7f3b9415bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 shutdown_connections 2024-05-08T04:43:13.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.751+0000 7f3b98a00640 1 -- 172.21.15.164:0/2190166400 wait complete. 2024-05-08T04:43:13.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:51.994625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:52.615473+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:13.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:13.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 Processor -- start 2024-05-08T04:43:13.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 -- start start 2024-05-08T04:43:13.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c15ab10 0x7fa96c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa96c0cee90 0x7fa96c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c15e870 con 0x7fa96c0ae6e0 2024-05-08T04:43:13.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c06bb80 con 0x7fa96c0cee90 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa973200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c15eb50 con 0x7fa96c15ab10 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa96c0cee90 0x7fa96c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa96c0cee90 0x7fa96c0aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52280/0 (socket says 172.21.15.164:52280) 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 -- 172.21.15.164:0/4224822231 learned_addr learned my addr 172.21.15.164:0/4224822231 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa96be00640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c15ab10 0x7fa96c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 -- 172.21.15.164:0/4224822231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c15ab10 msgr2=0x7fa96c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c15ab10 0x7fa96c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 -- 172.21.15.164:0/4224822231 >> v1:172.21.15.172:6789/0 conn(0x7fa96c0ae6e0 legacy=0x7fa96c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.911+0000 7fa971200640 1 -- 172.21.15.164:0/4224822231 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c15ee30 con 0x7fa96c0cee90 2024-05-08T04:43:13.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa971200640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c0cee90 0x7fa96c0aeb40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa964009870 tx=0x7fa96401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96530135ed9f6159 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa96b400640 1 -- 172.21.15.164:0/4224822231 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640ab660 con 0x7fa96c0cee90 2024-05-08T04:43:13.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa96b400640 1 -- 172.21.15.164:0/4224822231 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640b0070 con 0x7fa96c0cee90 2024-05-08T04:43:13.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa96b400640 1 -- 172.21.15.164:0/4224822231 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640aba20 con 0x7fa96c0cee90 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 -- 172.21.15.164:0/4224822231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c0cee90 msgr2=0x7fa96c0aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c0cee90 0x7fa96c0aeb40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa964009870 tx=0x7fa96401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 -- 172.21.15.164:0/4224822231 shutdown_connections 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c0cee90 0x7fa96c0aeb40 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 --2- 172.21.15.164:0/4224822231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c15ab10 0x7fa96c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 -- 172.21.15.164:0/4224822231 >> 172.21.15.164:0/4224822231 conn(0x7fa96c0cf770 msgr2=0x7fa96c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:13.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 -- 172.21.15.164:0/4224822231 shutdown_connections 2024-05-08T04:43:13.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.915+0000 7fa973200640 1 -- 172.21.15.164:0/4224822231 wait complete. 2024-05-08T04:43:13.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 Processor -- start 2024-05-08T04:43:13.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- start start 2024-05-08T04:43:13.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c0ae6e0 0x7fa96c0adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa96c0cee90 0x7fa96c14bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0cb070 con 0x7fa96c15ab10 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0caef0 con 0x7fa96c0cee90 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0cb1f0 con 0x7fa96c0ae6e0 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52292/0 (socket says 172.21.15.164:52292) 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 -- 172.21.15.164:0/936986818 learned_addr learned my addr 172.21.15.164:0/936986818 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa963400640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa96c0cee90 0x7fa96c14bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa96be00640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c0ae6e0 0x7fa96c0adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 -- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c0ae6e0 msgr2=0x7fa96c0adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c0ae6e0 0x7fa96c0adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 -- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa96c0cee90 msgr2=0x7fa96c14bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa96c0cee90 0x7fa96c14bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 -- 172.21.15.164:0/936986818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c06b7c0 con 0x7fa96c15ab10 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa971200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa9640ad040 tx=0x7fa96401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640be5f0 con 0x7fa96c15ab10 2024-05-08T04:43:13.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640be7b0 con 0x7fa96c15ab10 2024-05-08T04:43:13.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640beba0 con 0x7fa96c15ab10 2024-05-08T04:43:13.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c15bcb0 con 0x7fa96c15ab10 2024-05-08T04:43:13.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.919+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c15c270 con 0x7fa96c15ab10 2024-05-08T04:43:13.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c072d30 con 0x7fa96c15ab10 2024-05-08T04:43:13.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fa9640b43f0 con 0x7fa96c15ab10 2024-05-08T04:43:13.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa968c00640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 0x7fa948069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:13.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fa96411e3b0 con 0x7fa96c15ab10 2024-05-08T04:43:13.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa96be00640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 0x7fa948069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:13.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.923+0000 7fa96be00640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 0x7fa948069f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa95c011340 tx=0x7fa95c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:13.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:13.927+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa964016150 con 0x7fa96c15ab10 2024-05-08T04:43:14.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.079+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa96c04f010 con 0x7fa96c15ab10 2024-05-08T04:43:14.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa968c00640 1 -- 172.21.15.164:0/936986818 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9640f0940 con 0x7fa96c15ab10 2024-05-08T04:43:14.083 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 msgr2=0x7fa948069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 0x7fa948069f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa95c011340 tx=0x7fa95c00b040 comp rx=0 tx=0).stop 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 msgr2=0x7fa96c159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa9640ad040 tx=0x7fa96401ace0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 shutdown_connections 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa948067aa0 0x7fa948069f60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c15ab10 0x7fa96c159700 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa96c0cee90 0x7fa96c14bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 --2- 172.21.15.164:0/936986818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa96c0ae6e0 0x7fa96c0adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 >> 172.21.15.164:0/936986818 conn(0x7fa96c0cf770 msgr2=0x7fa96c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 shutdown_connections 2024-05-08T04:43:14.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.083+0000 7fa973200640 1 -- 172.21.15.164:0/936986818 wait complete. 2024-05-08T04:43:14.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:14.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 Processor -- start 2024-05-08T04:43:14.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 -- start start 2024-05-08T04:43:14.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa79015ab10 0x7fa79006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79015e870 con 0x7fa7900ae6e0 2024-05-08T04:43:14.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79015eb50 con 0x7fa7900cee90 2024-05-08T04:43:14.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa794a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79006bb80 con 0x7fa79015ab10 2024-05-08T04:43:14.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa79015ab10 0x7fa79006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78d400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa7900ae6e0 0x7fa7900ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49422/0 (socket says 172.21.15.164:49422) 2024-05-08T04:43:14.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78d400640 1 -- 172.21.15.164:0/3914664456 learned_addr learned my addr 172.21.15.164:0/3914664456 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:14.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78de00640 1 -- 172.21.15.164:0/3914664456 >> v1:172.21.15.172:6789/0 conn(0x7fa7900cee90 legacy=0x7fa7900ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78de00640 1 -- 172.21.15.164:0/3914664456 >> v1:172.21.15.164:6789/0 conn(0x7fa7900ae6e0 legacy=0x7fa7900ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.243+0000 7fa78de00640 1 -- 172.21.15.164:0/3914664456 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79015ee30 con 0x7fa79015ab10 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa78de00640 1 --2- 172.21.15.164:0/3914664456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79015ab10 0x7fa79006cce0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa780009d90 tx=0x7fa78001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f633bb15a3c7e40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa78ca00640 1 -- 172.21.15.164:0/3914664456 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800b2070 con 0x7fa79015ab10 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa78ca00640 1 -- 172.21.15.164:0/3914664456 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7800a5970 con 0x7fa79015ab10 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa78ca00640 1 -- 172.21.15.164:0/3914664456 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800a5d40 con 0x7fa79015ab10 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 -- 172.21.15.164:0/3914664456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79015ab10 msgr2=0x7fa79006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 --2- 172.21.15.164:0/3914664456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79015ab10 0x7fa79006cce0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa780009d90 tx=0x7fa78001c8b0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 -- 172.21.15.164:0/3914664456 shutdown_connections 2024-05-08T04:43:14.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 --2- 172.21.15.164:0/3914664456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79015ab10 0x7fa79006cce0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 -- 172.21.15.164:0/3914664456 >> 172.21.15.164:0/3914664456 conn(0x7fa7900cf770 msgr2=0x7fa7900cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 -- 172.21.15.164:0/3914664456 shutdown_connections 2024-05-08T04:43:14.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.247+0000 7fa794a00640 1 -- 172.21.15.164:0/3914664456 wait complete. 2024-05-08T04:43:14.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 Processor -- start 2024-05-08T04:43:14.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- start start 2024-05-08T04:43:14.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7900cee90 0x7fa790154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa79015ab10 0x7fa79015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790168af0 con 0x7fa7900ae6e0 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79015aef0 con 0x7fa7900cee90 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa790168c70 con 0x7fa79015ab10 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52310/0 (socket says 172.21.15.164:52310) 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 -- 172.21.15.164:0/1157285785 learned_addr learned my addr 172.21.15.164:0/1157285785 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78d400640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7900cee90 0x7fa790154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78e800640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa79015ab10 0x7fa79015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 -- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa79015ab10 msgr2=0x7fa79015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa79015ab10 0x7fa79015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 -- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7900cee90 msgr2=0x7fa790154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7900cee90 0x7fa790154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 -- 172.21.15.164:0/1157285785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7901552f0 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa78de00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa780009d90 tx=0x7fa780019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800bd020 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7800ad6c0 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800b2040 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7901501c0 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.251+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa790155010 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa790072d30 con 0x7fa7900ae6e0 2024-05-08T04:43:14.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fa7800c0020 con 0x7fa7900ae6e0 2024-05-08T04:43:14.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa786000640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 0x7fa770069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fa78011e9e0 con 0x7fa7900ae6e0 2024-05-08T04:43:14.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa78d400640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 0x7fa770069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.255+0000 7fa78d400640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 0x7fa770069f90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa79015d9b0 tx=0x7fa77c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.259+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7800c7070 con 0x7fa7900ae6e0 2024-05-08T04:43:14.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.419+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa79004f010 con 0x7fa7900ae6e0 2024-05-08T04:43:14.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.419+0000 7fa786000640 1 -- 172.21.15.164:0/1157285785 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa780016560 con 0x7fa7900ae6e0 2024-05-08T04:43:14.423 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 msgr2=0x7fa770069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 0x7fa770069f90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa79015d9b0 tx=0x7fa77c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 msgr2=0x7fa7901fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa780009d90 tx=0x7fa780019650 comp rx=0 tx=0).stop 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 shutdown_connections 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa770067ad0 0x7fa770069f90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa79015ab10 0x7fa79015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7900cee90 0x7fa790154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 --2- 172.21.15.164:0/1157285785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7900ae6e0 0x7fa7901fa270 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 >> 172.21.15.164:0/1157285785 conn(0x7fa7900cf770 msgr2=0x7fa7901561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 shutdown_connections 2024-05-08T04:43:14.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.423+0000 7fa794a00640 1 -- 172.21.15.164:0/1157285785 wait complete. 2024-05-08T04:43:14.443 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.579+0000 7fb401800640 1 Processor -- start 2024-05-08T04:43:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.579+0000 7fb401800640 1 -- start start 2024-05-08T04:43:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb401800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb3fc0b2ff0 0x7fb3fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb401800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc15e300 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb401800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc15e570 con 0x7fb3fc0adfa0 2024-05-08T04:43:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb401800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b5300 con 0x7fb3fc0b2ff0 2024-05-08T04:43:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3faa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb3fc0af6d0 0x7fb3fc0ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49434/0 (socket says 172.21.15.164:49434) 2024-05-08T04:43:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3faa00640 1 -- 172.21.15.164:0/659061906 learned_addr learned my addr 172.21.15.164:0/659061906 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799331 0 0) 0x7fb3fc15e300 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e0003530 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fb400640 1 --2- 172.21.15.164:0/659061906 >> v2:172.21.15.172:3300/0 conn(0x7fb3fc0b2ff0 0x7fb3fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092718460 0 0) 0x7fb3e0003530 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 >> v2:172.21.15.172:3300/0 conn(0x7fb3fc0b2ff0 msgr2=0x7fb3fc0b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 --2- 172.21.15.164:0/659061906 >> v2:172.21.15.172:3300/0 conn(0x7fb3fc0b2ff0 0x7fb3fc0b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 >> v1:172.21.15.172:6789/0 conn(0x7fb3fc0adfa0 legacy=0x7fb3fc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc15e850 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fb3ec001a00 con 0x7fb3fc0af6d0 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fb400640 1 --2- 172.21.15.164:0/659061906 >> v2:172.21.15.172:3300/0 conn(0x7fb3fc0b2ff0 0x7fb3fc0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 >> v1:172.21.15.164:6789/0 conn(0x7fb3fc0af6d0 legacy=0x7fb3fc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e00039c0 0x7fb3e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 0x7fb3e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e00052f0 0x7fb3e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b5300 con 0x7fb3e0004810 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000c1a0 con 0x7fb3e00052f0 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000c3a0 con 0x7fb3e00039c0 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 0x7fb3e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3faa00640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e00039c0 0x7fb3e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fb400640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e00052f0 0x7fb3e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 -- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e00039c0 msgr2=0x7fb3e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e00039c0 0x7fb3e0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 -- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e00052f0 msgr2=0x7fb3e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e00052f0 0x7fb3e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3f2000640 1 -- 172.21.15.164:0/659061906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0003530 con 0x7fb3e0004810 2024-05-08T04:43:14.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.583+0000 7fb3fb400640 1 -- 172.21.15.164:0/659061906 reap_dead start 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb3f2000640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 0x7fb3e0004c90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4007fd0 tx=0x7fb3e401c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8603490c6b56d81f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e40b0070 con 0x7fb3e0004810 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e40ab620 con 0x7fb3e0004810 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb3fa000640 1 -- 172.21.15.164:0/659061906 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e40039b0 con 0x7fb3e0004810 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 msgr2=0x7fb3e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 0x7fb3e0004c90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4007fd0 tx=0x7fb3e401c2e0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- 172.21.15.164:0/659061906 shutdown_connections 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 --2- 172.21.15.164:0/659061906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e0004810 0x7fb3e0004c90 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- 172.21.15.164:0/659061906 >> 172.21.15.164:0/659061906 conn(0x7fb3fc06cca0 msgr2=0x7fb3fc06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- 172.21.15.164:0/659061906 shutdown_connections 2024-05-08T04:43:14.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- 172.21.15.164:0/659061906 wait complete. 2024-05-08T04:43:14.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 Processor -- start 2024-05-08T04:43:14.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.587+0000 7fb401800640 1 -- start start 2024-05-08T04:43:14.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e0004810 0x7fb3fc1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 0x7fb3fc1ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc161ff0 con 0x7fb3fc157120 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b3400 con 0x7fb3fc1ea0f0 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc162170 con 0x7fb3e0004810 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52322/0 (socket says 172.21.15.164:52322) 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 -- 172.21.15.164:0/587443042 learned_addr learned my addr 172.21.15.164:0/587443042 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f2000640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 0x7fb3fc1ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3faa00640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e0004810 0x7fb3fc1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 -- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e0004810 msgr2=0x7fb3fc1f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e0004810 0x7fb3fc1f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 -- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 msgr2=0x7fb3fc1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 0x7fb3fc1ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 -- 172.21.15.164:0/587443042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc1eade0 con 0x7fb3fc157120 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f2000640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 0x7fb3fc1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3fb400640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc06f550 tx=0x7fb3f40108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400f040 con 0x7fb3fc157120 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb3fc157120 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc156360 con 0x7fb3fc157120 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4003620 con 0x7fb3fc157120 2024-05-08T04:43:14.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc156920 con 0x7fb3fc157120 2024-05-08T04:43:14.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc06ddf0 con 0x7fb3fc157120 2024-05-08T04:43:14.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.591+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fb3f40037e0 con 0x7fb3fc157120 2024-05-08T04:43:14.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.595+0000 7fb3f3400640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 0x7fb3d8069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.595+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fb3f4083610 con 0x7fb3fc157120 2024-05-08T04:43:14.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.595+0000 7fb3faa00640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 0x7fb3d8069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.595+0000 7fb3faa00640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 0x7fb3d8069f90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec002240 tx=0x7fb3ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.599+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3f40560d0 con 0x7fb3fc157120 2024-05-08T04:43:14.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.751+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3fc04f010 con 0x7fb3fc157120 2024-05-08T04:43:14.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.751+0000 7fb3f3400640 1 -- 172.21.15.164:0/587443042 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3f40559d0 con 0x7fb3fc157120 2024-05-08T04:43:14.754 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:14.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 msgr2=0x7fb3d8069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 0x7fb3d8069f90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec002240 tx=0x7fb3ec009040 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 msgr2=0x7fb3fc1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc06f550 tx=0x7fb3f40108b0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 shutdown_connections 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3d8067ad0 0x7fb3d8069f90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3fc157120 0x7fb3fc1575a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3fc1ea0f0 0x7fb3fc1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 --2- 172.21.15.164:0/587443042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3e0004810 0x7fb3fc1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 >> 172.21.15.164:0/587443042 conn(0x7fb3fc06cca0 msgr2=0x7fb3fc15f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 shutdown_connections 2024-05-08T04:43:14.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.755+0000 7fb401800640 1 -- 172.21.15.164:0/587443042 wait complete. 2024-05-08T04:43:14.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:43:14.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.907+0000 7fc2cf800640 1 Processor -- start 2024-05-08T04:43:14.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 -- start start 2024-05-08T04:43:14.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2c815ba70 0x7fc2c8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80cd9f0 con 0x7fc2c815be50 2024-05-08T04:43:14.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c815fe70 con 0x7fc2c815ba70 2024-05-08T04:43:14.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cf800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80cdcd0 con 0x7fc2c80cfda0 2024-05-08T04:43:14.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2c815ba70 0x7fc2c8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:52330/0 (socket says 172.21.15.164:52330) 2024-05-08T04:43:14.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 -- 172.21.15.164:0/118033097 learned_addr learned my addr 172.21.15.164:0/118033097 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:14.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 -- 172.21.15.164:0/118033097 >> v2:172.21.15.172:3300/0 conn(0x7fc2c815ba70 msgr2=0x7fc2c8160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:14.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 --2- 172.21.15.164:0/118033097 >> v2:172.21.15.172:3300/0 conn(0x7fc2c815ba70 0x7fc2c8160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 -- 172.21.15.164:0/118033097 >> v1:172.21.15.172:6789/0 conn(0x7fc2c815be50 legacy=0x7fc2c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.911+0000 7fc2cd800640 1 -- 172.21.15.164:0/118033097 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c815f820 con 0x7fc2c80cfda0 2024-05-08T04:43:14.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cd800640 1 --2- 172.21.15.164:0/118033097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845c18b16345d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2c7e00640 1 -- 172.21.15.164:0/118033097 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00a57e0 con 0x7fc2c80cfda0 2024-05-08T04:43:14.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2c7e00640 1 -- 172.21.15.164:0/118033097 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c00a5980 con 0x7fc2c80cfda0 2024-05-08T04:43:14.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2c7e00640 1 -- 172.21.15.164:0/118033097 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00a5d50 con 0x7fc2c80cfda0 2024-05-08T04:43:14.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 -- 172.21.15.164:0/118033097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 msgr2=0x7fc2c80b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 --2- 172.21.15.164:0/118033097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c001ca00 comp rx=0 tx=0).stop 2024-05-08T04:43:14.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 -- 172.21.15.164:0/118033097 shutdown_connections 2024-05-08T04:43:14.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 --2- 172.21.15.164:0/118033097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c80cfda0 0x7fc2c80b31a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 --2- 172.21.15.164:0/118033097 >> v2:172.21.15.172:3300/0 conn(0x7fc2c815ba70 0x7fc2c8160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 -- 172.21.15.164:0/118033097 >> 172.21.15.164:0/118033097 conn(0x7fc2c806ac90 msgr2=0x7fc2c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:14.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 -- 172.21.15.164:0/118033097 shutdown_connections 2024-05-08T04:43:14.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.915+0000 7fc2cf800640 1 -- 172.21.15.164:0/118033097 wait complete. 2024-05-08T04:43:14.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 Processor -- start 2024-05-08T04:43:14.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- start start 2024-05-08T04:43:14.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2c80cfda0 0x7fc2c80b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c815ba70 0x7fc2c815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8168d40 con 0x7fc2c815be50 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8168bc0 con 0x7fc2c80cfda0 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8168ec0 con 0x7fc2c815ba70 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52338/0 (socket says 172.21.15.164:52338) 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 -- 172.21.15.164:0/3195272511 learned_addr learned my addr 172.21.15.164:0/3195272511 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cce00640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2c80cfda0 0x7fc2c80b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2bfe00640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c815ba70 0x7fc2c815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 -- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c815ba70 msgr2=0x7fc2c815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c815ba70 0x7fc2c815ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 -- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2c80cfda0 msgr2=0x7fc2c80b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:14.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2c80cfda0 0x7fc2c80b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:14.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 -- 172.21.15.164:0/3195272511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80b0a20 con 0x7fc2c815be50 2024-05-08T04:43:14.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cd800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00af040 tx=0x7fc2c00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00b8670 con 0x7fc2c815be50 2024-05-08T04:43:14.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c00be070 con 0x7fc2c815be50 2024-05-08T04:43:14.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00b8a70 con 0x7fc2c815be50 2024-05-08T04:43:14.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c80b0d00 con 0x7fc2c815be50 2024-05-08T04:43:14.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c80717e0 con 0x7fc2c815be50 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.919+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c8069d60 con 0x7fc2c815be50 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.923+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fc2c00b8c10 con 0x7fc2c815be50 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.923+0000 7fc2c5600640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 0x7fc2a4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.923+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fc2c011ea50 con 0x7fc2c815be50 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.923+0000 7fc2cce00640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 0x7fc2a4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:14.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.923+0000 7fc2cce00640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 0x7fc2a4069f60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009b60 tx=0x7fc2b8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:14.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:14.927+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2c0016620 con 0x7fc2c815be50 2024-05-08T04:43:15.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.091+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc2c806e210 con 0x7fc2c815be50 2024-05-08T04:43:15.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.091+0000 7fc2c5600640 1 -- 172.21.15.164:0/3195272511 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc2c00f0e40 con 0x7fc2c815be50 2024-05-08T04:43:15.095 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 msgr2=0x7fc2a4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 0x7fc2a4069f60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009b60 tx=0x7fc2b8009190 comp rx=0 tx=0).stop 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 msgr2=0x7fc2c8071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00af040 tx=0x7fc2c00197a0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 shutdown_connections 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc2a4067aa0 0x7fc2a4069f60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2c815ba70 0x7fc2c815ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2c815be50 0x7fc2c8071ce0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 --2- 172.21.15.164:0/3195272511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2c80cfda0 0x7fc2c80b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 >> 172.21.15.164:0/3195272511 conn(0x7fc2c806ac90 msgr2=0x7fc2c806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 shutdown_connections 2024-05-08T04:43:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.095+0000 7fc2cf800640 1 -- 172.21.15.164:0/3195272511 wait complete. 2024-05-08T04:43:15.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:43:15.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:15.116 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:15.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:43:15.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:15.149 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:43:15.203 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:15.203 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:15.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:15.211 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:15.211 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:15.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:43:15.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.391+0000 7f1738800640 1 Processor -- start 2024-05-08T04:43:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.391+0000 7f1738800640 1 -- start start 2024-05-08T04:43:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1738800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1730144200 0x7f1730161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1738800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1730161fe0 con 0x7f173015ea20 2024-05-08T04:43:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1738800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17301622c0 con 0x7f17300cfd60 2024-05-08T04:43:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1738800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1730162880 con 0x7f1730144200 2024-05-08T04:43:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f173015ea20 0x7f17300ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49438/0 (socket says 172.21.15.164:49438) 2024-05-08T04:43:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 -- 172.21.15.164:0/1540938751 learned_addr learned my addr 172.21.15.164:0/1540938751 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1735e00640 1 --2- 172.21.15.164:0/1540938751 >> v2:172.21.15.172:3300/0 conn(0x7f1730144200 0x7f1730161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043454768 0 0) 0x7f1730161fe0 con 0x7f173015ea20 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1718003530 con 0x7f173015ea20 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053755536 0 0) 0x7f1718003530 con 0x7f173015ea20 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 >> v2:172.21.15.172:3300/0 conn(0x7f1730144200 msgr2=0x7f1730161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 --2- 172.21.15.164:0/1540938751 >> v2:172.21.15.172:3300/0 conn(0x7f1730144200 0x7f1730161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 >> v1:172.21.15.172:6789/0 conn(0x7f17300cfd60 legacy=0x7f17300cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17301625a0 con 0x7f173015ea20 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f172c001a00 con 0x7f173015ea20 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 >> v1:172.21.15.164:6789/0 conn(0x7f173015ea20 legacy=0x7f17300ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17180039c0 0x7f1718003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 0x7f171800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1730162880 con 0x7f17180051d0 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171800c170 con 0x7f17180039c0 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171800c370 con 0x7f1718004810 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 0x7f171800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1735400640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1735e00640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17180039c0 0x7f1718003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 -- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1718004810 msgr2=0x7f1718004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 -- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17180039c0 msgr2=0x7f1718003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17180039c0 0x7f1718003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.395+0000 7f1736800640 1 -- 172.21.15.164:0/1540938751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718003530 con 0x7f17180051d0 2024-05-08T04:43:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1736800640 1 -- 172.21.15.164:0/1540938751 reap_dead start 2024-05-08T04:43:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1736800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 0x7f171800b9d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f172c00a840 tx=0x7f172c01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b682e5abbde069 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00a460 con 0x7f17180051d0 2024-05-08T04:43:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c0aa710 con 0x7f17180051d0 2024-05-08T04:43:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1734a00640 1 -- 172.21.15.164:0/1540938751 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0aab00 con 0x7f17180051d0 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 -- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 msgr2=0x7f171800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 0x7f171800b9d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f172c00a840 tx=0x7f172c01fac0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 -- 172.21.15.164:0/1540938751 shutdown_connections 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 --2- 172.21.15.164:0/1540938751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17180051d0 0x7f171800b9d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 -- 172.21.15.164:0/1540938751 >> 172.21.15.164:0/1540938751 conn(0x7f173006ac90 msgr2=0x7f173006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 -- 172.21.15.164:0/1540938751 shutdown_connections 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 -- 172.21.15.164:0/1540938751 wait complete. 2024-05-08T04:43:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.399+0000 7f1738800640 1 Processor -- start 2024-05-08T04:43:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- start start 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 0x7f17300b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 0x7f17300b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173015ee50 con 0x7f17300c48e0 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17301445e0 con 0x7f17300b3be0 2024-05-08T04:43:15.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173015efd0 con 0x7f17180051d0 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52362/0 (socket says 172.21.15.164:52362) 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 -- 172.21.15.164:0/222865821 learned_addr learned my addr 172.21.15.164:0/222865821 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1736800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 0x7f17300b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735e00640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 0x7f17300b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 -- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 msgr2=0x7f17300b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 0x7f17300b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 -- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 msgr2=0x7f17300b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 0x7f17300b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 -- 172.21.15.164:0/222865821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300b45a0 con 0x7f17300c48e0 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1736800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 0x7f17300b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:15.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735e00640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 0x7f17300b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1735400640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f172000f5c0 tx=0x7f172000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17200052e0 con 0x7f17300c48e0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720018070 con 0x7f17300c48e0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720005480 con 0x7f17300c48e0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17300b2b40 con 0x7f17300c48e0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17300b3100 con 0x7f17300c48e0 2024-05-08T04:43:15.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.403+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1730069d60 con 0x7f17300c48e0 2024-05-08T04:43:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.407+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f1720003820 con 0x7f17300c48e0 2024-05-08T04:43:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.407+0000 7f172a000640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 0x7f1714069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.407+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f1720083720 con 0x7f17300c48e0 2024-05-08T04:43:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.407+0000 7f1735e00640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 0x7f1714069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.407+0000 7f1735e00640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 0x7f1714069f60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1724009b60 tx=0x7f1724008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.411+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1720056180 con 0x7f17300c48e0 2024-05-08T04:43:15.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17300af110 con 0x7f17300c48e0 2024-05-08T04:43:15.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f172a000640 1 -- 172.21.15.164:0/222865821 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1720055a80 con 0x7f17300c48e0 2024-05-08T04:43:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:15.601 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:43:15.601 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 73 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 msgr2=0x7f1714069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 0x7f1714069f60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1724009b60 tx=0x7f1724008040 comp rx=0 tx=0).stop 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 msgr2=0x7f173014fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f172000f5c0 tx=0x7f172000fa80 comp rx=0 tx=0).stop 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 shutdown_connections 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1714067aa0 0x7f1714069f60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17300b3be0 0x7f17300b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17300c48e0 0x7f173014fc60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.599+0000 7f1738800640 1 --2- 172.21.15.164:0/222865821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17180051d0 0x7f17300b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.603+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 >> 172.21.15.164:0/222865821 conn(0x7f173006ac90 msgr2=0x7f17300b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:15.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.603+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 shutdown_connections 2024-05-08T04:43:15.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.603+0000 7f1738800640 1 -- 172.21.15.164:0/222865821 wait complete. 2024-05-08T04:43:15.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:15.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.755+0000 7f134bc00640 1 Processor -- start 2024-05-08T04:43:15.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.755+0000 7f134bc00640 1 -- start start 2024-05-08T04:43:15.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f134bc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f134415ab60 0x7f1344072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f134bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 0x7f13440cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f134bc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1344072640 con 0x7f13440b26c0 2024-05-08T04:43:15.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f134bc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134406c770 con 0x7f134415ab60 2024-05-08T04:43:15.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f134bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134406ca50 con 0x7f13440cfda0 2024-05-08T04:43:15.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 0x7f13440cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f134415ab60 0x7f1344072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f133be00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f13440b26c0 0x7f13440cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:38174/0 (socket says 172.21.15.164:38174) 2024-05-08T04:43:15.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f133be00640 1 -- 172.21.15.164:0/2487842779 learned_addr learned my addr 172.21.15.164:0/2487842779 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349200640 1 -- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 msgr2=0x7f13440cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349200640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 0x7f13440cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349200640 1 -- 172.21.15.164:0/2487842779 >> v1:172.21.15.172:6789/0 conn(0x7f13440b26c0 legacy=0x7f13440cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349200640 1 -- 172.21.15.164:0/2487842779 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134406cd30 con 0x7f134415ab60 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1349c00640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 0x7f13440cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:15.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.759+0000 7f1343e00640 1 -- 172.21.15.164:0/2487842779 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700906808 0 0) 0x7f1344072640 con 0x7f13440b26c0 2024-05-08T04:43:15.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f1349200640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f134415ab60 0x7f1344072100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f13340097c0 tx=0x7f133401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e59d773173868679 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f1343e00640 1 -- 172.21.15.164:0/2487842779 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13340b0070 con 0x7f134415ab60 2024-05-08T04:43:15.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f1343e00640 1 -- 172.21.15.164:0/2487842779 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13340ab600 con 0x7f134415ab60 2024-05-08T04:43:15.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f1343e00640 1 -- 172.21.15.164:0/2487842779 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13340ab9d0 con 0x7f134415ab60 2024-05-08T04:43:15.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f134415ab60 msgr2=0x7f1344072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f134415ab60 0x7f1344072100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f13340097c0 tx=0x7f133401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- 172.21.15.164:0/2487842779 shutdown_connections 2024-05-08T04:43:15.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13440cfda0 0x7f13440cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 --2- 172.21.15.164:0/2487842779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f134415ab60 0x7f1344072100 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- 172.21.15.164:0/2487842779 >> 172.21.15.164:0/2487842779 conn(0x7f134406ac90 msgr2=0x7f134406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:15.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- 172.21.15.164:0/2487842779 shutdown_connections 2024-05-08T04:43:15.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- 172.21.15.164:0/2487842779 wait complete. 2024-05-08T04:43:15.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 Processor -- start 2024-05-08T04:43:15.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.763+0000 7f134bc00640 1 -- start start 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13440cfda0 0x7f134415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f134415ab60 0x7f1344152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440aeeb0 con 0x7f13440b26c0 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440aed30 con 0x7f13440cfda0 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1344167970 con 0x7f134415ab60 2024-05-08T04:43:15.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52378/0 (socket says 172.21.15.164:52378) 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 -- 172.21.15.164:0/660661730 learned_addr learned my addr 172.21.15.164:0/660661730 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f133be00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13440cfda0 0x7f134415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349c00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f134415ab60 0x7f1344152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 -- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f134415ab60 msgr2=0x7f1344152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f134415ab60 0x7f1344152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 -- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13440cfda0 msgr2=0x7f134415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13440cfda0 0x7f134415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 -- 172.21.15.164:0/660661730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134415d4f0 con 0x7f13440b26c0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1349200640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f133401a0d0 tx=0x7f133401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13340b0050 con 0x7f13440b26c0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13340bc070 con 0x7f13440b26c0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134415d7d0 con 0x7f13440b26c0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334006e20 con 0x7f13440b26c0 2024-05-08T04:43:15.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134415b330 con 0x7f13440b26c0 2024-05-08T04:43:15.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.767+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1344069d60 con 0x7f13440b26c0 2024-05-08T04:43:15.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f13340b5830 con 0x7f13440b26c0 2024-05-08T04:43:15.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f1341600640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 0x7f1320069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:15.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f133411eb10 con 0x7f13440b26c0 2024-05-08T04:43:15.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f133be00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 0x7f1320069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:15.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f133be00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 0x7f1320069f60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1344157540 tx=0x7f132c007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:15.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.771+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1334016090 con 0x7f13440b26c0 2024-05-08T04:43:15.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.927+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f134406d9a0 con 0x7f13440b26c0 2024-05-08T04:43:15.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f1341600640 1 -- 172.21.15.164:0/660661730 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17027 (secure 0 0 0) 0x7f13340f1570 con 0x7f13440b26c0 2024-05-08T04:43:15.931 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:15.932 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":114,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:12.323259+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.36","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:15.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 msgr2=0x7f1320069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 0x7f1320069f60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1344157540 tx=0x7f132c007d60 comp rx=0 tx=0).stop 2024-05-08T04:43:15.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 msgr2=0x7f134415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:15.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f133401a0d0 tx=0x7f133401ace0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 shutdown_connections 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1320067aa0 0x7f1320069f60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f134415ab60 0x7f1344152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13440cfda0 0x7f134415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 --2- 172.21.15.164:0/660661730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13440b26c0 0x7f134415cdf0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 >> 172.21.15.164:0/660661730 conn(0x7f134406ac90 msgr2=0x7f13440b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 shutdown_connections 2024-05-08T04:43:15.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:15.931+0000 7f134bc00640 1 -- 172.21.15.164:0/660661730 wait complete. 2024-05-08T04:43:15.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:42:51.994625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:42:52.615473+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:15.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:16.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.087+0000 7efe30200640 1 Processor -- start 2024-05-08T04:43:16.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe30200640 1 -- start start 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe30200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe280d0100 0x7efe280cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe30200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe280b2a80 con 0x7efe2815f200 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe30200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe280cd0c0 con 0x7efe280b2fb0 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe30200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe280cce70 con 0x7efe280d0100 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2d800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7efe2815f200 0x7efe280cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49444/0 (socket says 172.21.15.164:49444) 2024-05-08T04:43:16.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2d800640 1 -- 172.21.15.164:0/2279301450 learned_addr learned my addr 172.21.15.164:0/2279301450 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:16.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2ce00640 1 --2- 172.21.15.164:0/2279301450 >> v2:172.21.15.164:3300/0 conn(0x7efe280d0100 0x7efe280cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2ce00640 1 -- 172.21.15.164:0/2279301450 >> v1:172.21.15.172:6789/0 conn(0x7efe280b2fb0 legacy=0x7efe280ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:16.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2ce00640 1 -- 172.21.15.164:0/2279301450 >> v1:172.21.15.164:6789/0 conn(0x7efe2815f200 legacy=0x7efe280cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe2ce00640 1 -- 172.21.15.164:0/2279301450 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280cd3a0 con 0x7efe280d0100 2024-05-08T04:43:16.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.091+0000 7efe27e00640 1 -- 172.21.15.164:0/2279301450 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977128448 0 0) 0x7efe280b2a80 con 0x7efe2815f200 2024-05-08T04:43:16.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe2ce00640 1 --2- 172.21.15.164:0/2279301450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280d0100 0x7efe280cdba0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efe140097c0 tx=0x7efe280cf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b3339b6336ed64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe27e00640 1 -- 172.21.15.164:0/2279301450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140b1070 con 0x7efe280d0100 2024-05-08T04:43:16.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe27e00640 1 -- 172.21.15.164:0/2279301450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe140ac620 con 0x7efe280d0100 2024-05-08T04:43:16.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe27e00640 1 -- 172.21.15.164:0/2279301450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140ac9f0 con 0x7efe280d0100 2024-05-08T04:43:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- 172.21.15.164:0/2279301450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280d0100 msgr2=0x7efe280cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 --2- 172.21.15.164:0/2279301450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280d0100 0x7efe280cdba0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efe140097c0 tx=0x7efe280cf000 comp rx=0 tx=0).stop 2024-05-08T04:43:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- 172.21.15.164:0/2279301450 shutdown_connections 2024-05-08T04:43:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 --2- 172.21.15.164:0/2279301450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280d0100 0x7efe280cdba0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- 172.21.15.164:0/2279301450 >> 172.21.15.164:0/2279301450 conn(0x7efe280d1980 msgr2=0x7efe280d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- 172.21.15.164:0/2279301450 shutdown_connections 2024-05-08T04:43:16.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- 172.21.15.164:0/2279301450 wait complete. 2024-05-08T04:43:16.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 Processor -- start 2024-05-08T04:43:16.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.095+0000 7efe30200640 1 -- start start 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe280d0100 0x7efe280c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe2815f200 0x7efe280c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe28169ed0 con 0x7efe280b2fb0 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe28169d50 con 0x7efe280d0100 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2816a050 con 0x7efe2815f200 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52406/0 (socket says 172.21.15.164:52406) 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 -- 172.21.15.164:0/2518518552 learned_addr learned my addr 172.21.15.164:0/2518518552 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 -- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe2815f200 msgr2=0x7efe280c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe2815f200 0x7efe280c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 -- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe280d0100 msgr2=0x7efe280c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe280d0100 0x7efe280c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 -- 172.21.15.164:0/2518518552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280b7590 con 0x7efe280b2fb0 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe2d800640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c480 tx=0x7efe1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c018070 con 0x7efe280b2fb0 2024-05-08T04:43:16.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c007500 con 0x7efe280b2fb0 2024-05-08T04:43:16.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004a30 con 0x7efe280b2fb0 2024-05-08T04:43:16.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe280cadc0 con 0x7efe280b2fb0 2024-05-08T04:43:16.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe280b6ca0 con 0x7efe280b2fb0 2024-05-08T04:43:16.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2804f010 con 0x7efe280b2fb0 2024-05-08T04:43:16.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.099+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7efe1c01b070 con 0x7efe280b2fb0 2024-05-08T04:43:16.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.103+0000 7efe25600640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 0x7efe08069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.103+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7efe1c082f40 con 0x7efe280b2fb0 2024-05-08T04:43:16.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.103+0000 7efe2ce00640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 0x7efe08069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.103+0000 7efe2ce00640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 0x7efe08069f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efe140ae000 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.107+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe1c055750 con 0x7efe280b2fb0 2024-05-08T04:43:16.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.259+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe28072890 con 0x7efe280b2fb0 2024-05-08T04:43:16.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.259+0000 7efe25600640 1 -- 172.21.15.164:0/2518518552 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe1c055050 con 0x7efe280b2fb0 2024-05-08T04:43:16.261 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:16.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 msgr2=0x7efe08069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 0x7efe08069f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efe140ae000 tx=0x7efe14008040 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 msgr2=0x7efe28153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c480 tx=0x7efe1c00c940 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 shutdown_connections 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efe08067ad0 0x7efe08069f90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe2815f200 0x7efe280c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe280d0100 0x7efe280c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 --2- 172.21.15.164:0/2518518552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe280b2fb0 0x7efe28153020 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 >> 172.21.15.164:0/2518518552 conn(0x7efe280d1980 msgr2=0x7efe281571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 shutdown_connections 2024-05-08T04:43:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.263+0000 7efe30200640 1 -- 172.21.15.164:0/2518518552 wait complete. 2024-05-08T04:43:16.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.419+0000 7f2c7aa00640 1 Processor -- start 2024-05-08T04:43:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.419+0000 7f2c7aa00640 1 -- start start 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c7aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2c7406c910 0x7f2c740cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c7aa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740b2be0 con 0x7f2c7406bda0 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c7aa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740d0c40 con 0x7f2c7406c910 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740b2eb0 con 0x7f2c7415e000 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2c7406c910 0x7f2c740cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2c7406c910 0x7f2c740cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52408/0 (socket says 172.21.15.164:52408) 2024-05-08T04:43:16.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 -- 172.21.15.164:0/2523827623 learned_addr learned my addr 172.21.15.164:0/2523827623 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:16.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c73e00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 -- 172.21.15.164:0/2523827623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 msgr2=0x7f2c740b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:16.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 -- 172.21.15.164:0/2523827623 >> v1:172.21.15.172:6789/0 conn(0x7f2c7406bda0 legacy=0x7f2c7406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:16.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.423+0000 7f2c78a00640 1 -- 172.21.15.164:0/2523827623 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740b3190 con 0x7f2c7406c910 2024-05-08T04:43:16.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c78a00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406c910 0x7f2c740cc070 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009870 tx=0x7f2c6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e3b18fb7b265c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c73400640 1 -- 172.21.15.164:0/2523827623 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0ab620 con 0x7f2c7406c910 2024-05-08T04:43:16.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c73400640 1 -- 172.21.15.164:0/2523827623 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c0b0070 con 0x7f2c7406c910 2024-05-08T04:43:16.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c73400640 1 -- 172.21.15.164:0/2523827623 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0ab9e0 con 0x7f2c7406c910 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- 172.21.15.164:0/2523827623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406c910 msgr2=0x7f2c740cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406c910 0x7f2c740cc070 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009870 tx=0x7f2c6c01c460 comp rx=0 tx=0).stop 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- 172.21.15.164:0/2523827623 shutdown_connections 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406c910 0x7f2c740cc070 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/2523827623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- 172.21.15.164:0/2523827623 >> 172.21.15.164:0/2523827623 conn(0x7f2c740b34f0 msgr2=0x7f2c740b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- 172.21.15.164:0/2523827623 shutdown_connections 2024-05-08T04:43:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- 172.21.15.164:0/2523827623 wait complete. 2024-05-08T04:43:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 Processor -- start 2024-05-08T04:43:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.427+0000 7f2c7aa00640 1 -- start start 2024-05-08T04:43:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c7406c910 0x7f2c741f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740ccf40 con 0x7f2c7406bda0 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740ccdc0 con 0x7f2c7406c910 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740cd0c0 con 0x7f2c7415e000 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52414/0 (socket says 172.21.15.164:52414) 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 -- 172.21.15.164:0/1681049715 learned_addr learned my addr 172.21.15.164:0/1681049715 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c78a00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c6b400640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c7406c910 0x7f2c741f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 -- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 msgr2=0x7f2c740b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 -- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c7406c910 msgr2=0x7f2c741f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c7406c910 0x7f2c741f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 -- 172.21.15.164:0/1681049715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740b59d0 con 0x7f2c7406bda0 2024-05-08T04:43:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c78a00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c73e00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007e70 tx=0x7f2c6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014420 con 0x7f2c7406bda0 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64019070 con 0x7f2c7406bda0 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014800 con 0x7f2c7406bda0 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740b5cb0 con 0x7f2c7406bda0 2024-05-08T04:43:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c740b6270 con 0x7f2c7406bda0 2024-05-08T04:43:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7409fb50 con 0x7f2c7406bda0 2024-05-08T04:43:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.431+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f2c640149a0 con 0x7f2c7406bda0 2024-05-08T04:43:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.435+0000 7f2c70c00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 0x7f2c50069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.435+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f2c640841d0 con 0x7f2c7406bda0 2024-05-08T04:43:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.435+0000 7f2c6b400640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 0x7f2c50069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.435+0000 7f2c6b400640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 0x7f2c50069f60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c74168c90 tx=0x7f2c5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.439+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c6408c020 con 0x7f2c7406bda0 2024-05-08T04:43:16.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.591+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c740701d0 con 0x7f2c7406bda0 2024-05-08T04:43:16.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.591+0000 7f2c70c00640 1 -- 172.21.15.164:0/1681049715 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c64056c90 con 0x7f2c7406bda0 2024-05-08T04:43:16.595 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 msgr2=0x7f2c50069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 0x7f2c50069f60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c74168c90 tx=0x7f2c5c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 msgr2=0x7f2c74154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007e70 tx=0x7f2c6400e4b0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 shutdown_connections 2024-05-08T04:43:16.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2c50067aa0 0x7f2c50069f60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c7415e000 0x7f2c740b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c7406c910 0x7f2c741f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 --2- 172.21.15.164:0/1681049715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c7406bda0 0x7f2c74154cb0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 >> 172.21.15.164:0/1681049715 conn(0x7f2c740b34f0 msgr2=0x7f2c741572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 shutdown_connections 2024-05-08T04:43:16.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.595+0000 7f2c7aa00640 1 -- 172.21.15.164:0/1681049715 wait complete. 2024-05-08T04:43:16.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:16.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 Processor -- start 2024-05-08T04:43:16.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 -- start start 2024-05-08T04:43:16.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f91d815e000 0x7f91d80b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f91d806bda0 0x7f91d806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80b2be0 con 0x7f91d806c740 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80d0c40 con 0x7f91d806bda0 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.751+0000 7f91de600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d806cbf0 con 0x7f91d815e000 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f91d806bda0 0x7f91d806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f91d806bda0 0x7f91d806c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52416/0 (socket says 172.21.15.164:52416) 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 -- 172.21.15.164:0/3474719579 learned_addr learned my addr 172.21.15.164:0/3474719579 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91d7400640 1 --2- 172.21.15.164:0/3474719579 >> v2:172.21.15.172:3300/0 conn(0x7f91d815e000 0x7f91d80b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 -- 172.21.15.164:0/3474719579 >> v2:172.21.15.172:3300/0 conn(0x7f91d815e000 msgr2=0x7f91d80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 --2- 172.21.15.164:0/3474719579 >> v2:172.21.15.172:3300/0 conn(0x7f91d815e000 0x7f91d80b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 -- 172.21.15.164:0/3474719579 >> v1:172.21.15.172:6789/0 conn(0x7f91d806c740 legacy=0x7f91d80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:16.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 -- 172.21.15.164:0/3474719579 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80b2eb0 con 0x7f91d806bda0 2024-05-08T04:43:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91cfe00640 1 --2- 172.21.15.164:0/3474719579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d806bda0 0x7f91d806c200 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b7ab0 tx=0x7f91c001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18655392e70ebc2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91d6a00640 1 -- 172.21.15.164:0/3474719579 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00ab620 con 0x7f91d806bda0 2024-05-08T04:43:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91d6a00640 1 -- 172.21.15.164:0/3474719579 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c00b0070 con 0x7f91d806bda0 2024-05-08T04:43:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.755+0000 7f91d6a00640 1 -- 172.21.15.164:0/3474719579 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00ab9e0 con 0x7f91d806bda0 2024-05-08T04:43:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- 172.21.15.164:0/3474719579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d806bda0 msgr2=0x7f91d806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- 172.21.15.164:0/3474719579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d806bda0 0x7f91d806c200 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b7ab0 tx=0x7f91c001c420 comp rx=0 tx=0).stop 2024-05-08T04:43:16.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- 172.21.15.164:0/3474719579 shutdown_connections 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- 172.21.15.164:0/3474719579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d806bda0 0x7f91d806c200 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- 172.21.15.164:0/3474719579 >> v2:172.21.15.172:3300/0 conn(0x7f91d815e000 0x7f91d80b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- 172.21.15.164:0/3474719579 >> 172.21.15.164:0/3474719579 conn(0x7f91d80b34f0 msgr2=0x7f91d80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- 172.21.15.164:0/3474719579 shutdown_connections 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- 172.21.15.164:0/3474719579 wait complete. 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 Processor -- start 2024-05-08T04:43:16.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- start start 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91d806bda0 0x7f91d815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91d806c740 0x7f91d814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815e560 con 0x7f91d815e000 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815e3e0 con 0x7f91d806bda0 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91de600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d816acd0 con 0x7f91d806c740 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52428/0 (socket says 172.21.15.164:52428) 2024-05-08T04:43:16.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 -- 172.21.15.164:0/3160305637 learned_addr learned my addr 172.21.15.164:0/3160305637 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:16.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91cfe00640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91d806c740 0x7f91d814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7400640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91d806bda0 0x7f91d815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 -- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91d806c740 msgr2=0x7f91d814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91d806c740 0x7f91d814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 -- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91d806bda0 msgr2=0x7f91d815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91d806bda0 0x7f91d815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 -- 172.21.15.164:0/3160305637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d815bfa0 con 0x7f91d815e000 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91d7e00640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b970 tx=0x7f91d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0019070 con 0x7f91d815e000 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.759+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000cb40 con 0x7f91d815e000 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0014440 con 0x7f91d815e000 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d81587c0 con 0x7f91d815e000 2024-05-08T04:43:16.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d815c280 con 0x7f91d815e000 2024-05-08T04:43:16.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d809fb50 con 0x7f91d815e000 2024-05-08T04:43:16.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f91d001c070 con 0x7f91d815e000 2024-05-08T04:43:16.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91cf400640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 0x7f91b4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:16.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f91d00837e0 con 0x7f91d815e000 2024-05-08T04:43:16.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91d7400640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 0x7f91b4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:16.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.763+0000 7f91d7400640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 0x7f91b4069f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007a60 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:16.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.767+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91d81587c0 con 0x7f91d815e000 2024-05-08T04:43:16.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.919+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91d8158ef0 con 0x7f91d815e000 2024-05-08T04:43:16.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.919+0000 7f91cf400640 1 -- 172.21.15.164:0/3160305637 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91d00559a0 con 0x7f91d815e000 2024-05-08T04:43:16.922 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 msgr2=0x7f91b4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 0x7f91b4069f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007a60 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 msgr2=0x7f91d81509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b970 tx=0x7f91d000be30 comp rx=0 tx=0).stop 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 shutdown_connections 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f91b4067aa0 0x7f91b4069f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91d806c740 0x7f91d814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d815e000 0x7f91d81509a0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 --2- 172.21.15.164:0/3160305637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91d806bda0 0x7f91d815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:16.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 >> 172.21.15.164:0/3160305637 conn(0x7f91d80b34f0 msgr2=0x7f91d806fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:16.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 shutdown_connections 2024-05-08T04:43:16.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:16.923+0000 7f91de600640 1 -- 172.21.15.164:0/3160305637 wait complete. 2024-05-08T04:43:16.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:17.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.075+0000 7f9e7f800640 1 Processor -- start 2024-05-08T04:43:17.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.075+0000 7f9e7f800640 1 -- start start 2024-05-08T04:43:17.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7f800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9e7815ba70 0x7f9e78160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7f800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 0x7f9e780cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7f800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e780cd9f0 con 0x7f9e780aeda0 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7f800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7815fe70 con 0x7f9e7815ba70 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7f800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e780cdcd0 con 0x7f9e7815be50 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9e7815ba70 0x7f9e78160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9e7815ba70 0x7f9e78160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52444/0 (socket says 172.21.15.164:52444) 2024-05-08T04:43:17.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 -- 172.21.15.164:0/1452478130 learned_addr learned my addr 172.21.15.164:0/1452478130 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e77e00640 1 --2- 172.21.15.164:0/1452478130 >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 0x7f9e780cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 -- 172.21.15.164:0/1452478130 >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 msgr2=0x7f9e780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 --2- 172.21.15.164:0/1452478130 >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 0x7f9e780cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 -- 172.21.15.164:0/1452478130 >> v1:172.21.15.172:6789/0 conn(0x7f9e780aeda0 legacy=0x7f9e780b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e7ce00640 1 -- 172.21.15.164:0/1452478130 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7815f820 con 0x7f9e7815ba70 2024-05-08T04:43:17.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.079+0000 7f9e77e00640 1 --2- 172.21.15.164:0/1452478130 >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 0x7f9e780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:17.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7ce00640 1 --2- 172.21.15.164:0/1452478130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e7815ba70 0x7f9e78160300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815f780 tx=0x7f9e6c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8910712f08e852f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e77400640 1 -- 172.21.15.164:0/1452478130 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0b1070 con 0x7f9e7815ba70 2024-05-08T04:43:17.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e77400640 1 -- 172.21.15.164:0/1452478130 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c0ac620 con 0x7f9e7815ba70 2024-05-08T04:43:17.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e77400640 1 -- 172.21.15.164:0/1452478130 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0ac9f0 con 0x7f9e7815ba70 2024-05-08T04:43:17.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- 172.21.15.164:0/1452478130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e7815ba70 msgr2=0x7f9e78160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 --2- 172.21.15.164:0/1452478130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e7815ba70 0x7f9e78160300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815f780 tx=0x7f9e6c01c4b0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- 172.21.15.164:0/1452478130 shutdown_connections 2024-05-08T04:43:17.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 --2- 172.21.15.164:0/1452478130 >> v2:172.21.15.172:3300/0 conn(0x7f9e7815be50 0x7f9e780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 --2- 172.21.15.164:0/1452478130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e7815ba70 0x7f9e78160300 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- 172.21.15.164:0/1452478130 >> 172.21.15.164:0/1452478130 conn(0x7f9e7806ac90 msgr2=0x7f9e7806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- 172.21.15.164:0/1452478130 shutdown_connections 2024-05-08T04:43:17.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- 172.21.15.164:0/1452478130 wait complete. 2024-05-08T04:43:17.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 Processor -- start 2024-05-08T04:43:17.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.083+0000 7f9e7f800640 1 -- start start 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7815ba70 0x7f9e78151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e7815be50 0x7f9e781519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78168d40 con 0x7f9e780aeda0 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78168bc0 con 0x7f9e7815ba70 2024-05-08T04:43:17.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78168ec0 con 0x7f9e7815be50 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52446/0 (socket says 172.21.15.164:52446) 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 -- 172.21.15.164:0/189474234 learned_addr learned my addr 172.21.15.164:0/189474234 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e77e00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7815ba70 0x7f9e78151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7d800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e7815be50 0x7f9e781519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 -- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e7815be50 msgr2=0x7f9e781519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e7815be50 0x7f9e781519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 -- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7815ba70 msgr2=0x7f9e78151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7815ba70 0x7f9e78151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 -- 172.21.15.164:0/189474234 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e781563a0 con 0x7f9e780aeda0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7ce00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0af040 tx=0x7f9e6c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0b1040 con 0x7f9e780aeda0 2024-05-08T04:43:17.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c0bd070 con 0x7f9e780aeda0 2024-05-08T04:43:17.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0b7880 con 0x7f9e780aeda0 2024-05-08T04:43:17.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e78155b40 con 0x7f9e780aeda0 2024-05-08T04:43:17.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e78155de0 con 0x7f9e780aeda0 2024-05-08T04:43:17.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e78069d60 con 0x7f9e780aeda0 2024-05-08T04:43:17.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.087+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f9e6c0b7a20 con 0x7f9e780aeda0 2024-05-08T04:43:17.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.091+0000 7f9e74c00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 0x7f9e58069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.091+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f9e6c11de40 con 0x7f9e780aeda0 2024-05-08T04:43:17.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.091+0000 7f9e77e00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 0x7f9e58069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.091+0000 7f9e77e00640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 0x7f9e58069f60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9e781fab70 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.095+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e781563a0 con 0x7f9e780aeda0 2024-05-08T04:43:17.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.283+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e7806d970 con 0x7f9e780aeda0 2024-05-08T04:43:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.283+0000 7f9e74c00640 1 -- 172.21.15.164:0/189474234 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9e6c0160d0 con 0x7f9e780aeda0 2024-05-08T04:43:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:17.285 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:43:17.285 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 73 2024-05-08T04:43:17.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 msgr2=0x7f9e58069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 0x7f9e58069f60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9e781fab70 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2024-05-08T04:43:17.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 msgr2=0x7f9e7806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0af040 tx=0x7f9e6c01acf0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 shutdown_connections 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9e58067aa0 0x7f9e58069f60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e7815be50 0x7f9e781519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7815ba70 0x7f9e78151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 --2- 172.21.15.164:0/189474234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e780aeda0 0x7f9e7806db90 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 >> 172.21.15.164:0/189474234 conn(0x7f9e7806ac90 msgr2=0x7f9e780ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 shutdown_connections 2024-05-08T04:43:17.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.287+0000 7f9e7f800640 1 -- 172.21.15.164:0/189474234 wait complete. 2024-05-08T04:43:17.307 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:17.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 Processor -- start 2024-05-08T04:43:17.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 -- start start 2024-05-08T04:43:17.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd97c0cfda0 0x7fd97c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c06c770 con 0x7fd97c15ab60 2024-05-08T04:43:17.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c06ca50 con 0x7fd97c0b26c0 2024-05-08T04:43:17.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd982200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c06cd30 con 0x7fd97c0cfda0 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd97be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd97c0cfda0 0x7fd97c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd97b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd97c15ab60 0x7fd97c072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49456/0 (socket says 172.21.15.164:49456) 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.443+0000 7fd97b400640 1 -- 172.21.15.164:0/1384719989 learned_addr learned my addr 172.21.15.164:0/1384719989 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97be00640 1 -- 172.21.15.164:0/1384719989 >> v1:172.21.15.172:6789/0 conn(0x7fd97c0b26c0 legacy=0x7fd97c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97be00640 1 -- 172.21.15.164:0/1384719989 >> v1:172.21.15.164:6789/0 conn(0x7fd97c15ab60 legacy=0x7fd97c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97be00640 1 -- 172.21.15.164:0/1384719989 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c06d010 con 0x7fd97c0cfda0 2024-05-08T04:43:17.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97be00640 1 --2- 172.21.15.164:0/1384719989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0cfda0 0x7fd97c0cd910 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd974009d90 tx=0x7fd97401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b225be0ccf4caa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97aa00640 1 -- 172.21.15.164:0/1384719989 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740b2070 con 0x7fd97c0cfda0 2024-05-08T04:43:17.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97aa00640 1 -- 172.21.15.164:0/1384719989 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740a5830 con 0x7fd97c0cfda0 2024-05-08T04:43:17.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd97aa00640 1 -- 172.21.15.164:0/1384719989 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740a5c00 con 0x7fd97c0cfda0 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 -- 172.21.15.164:0/1384719989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0cfda0 msgr2=0x7fd97c0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 --2- 172.21.15.164:0/1384719989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0cfda0 0x7fd97c0cd910 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd974009d90 tx=0x7fd97401c970 comp rx=0 tx=0).stop 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 -- 172.21.15.164:0/1384719989 shutdown_connections 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 --2- 172.21.15.164:0/1384719989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0cfda0 0x7fd97c0cd910 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 -- 172.21.15.164:0/1384719989 >> 172.21.15.164:0/1384719989 conn(0x7fd97c06ac90 msgr2=0x7fd97c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 -- 172.21.15.164:0/1384719989 shutdown_connections 2024-05-08T04:43:17.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.447+0000 7fd982200640 1 -- 172.21.15.164:0/1384719989 wait complete. 2024-05-08T04:43:17.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 Processor -- start 2024-05-08T04:43:17.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- start start 2024-05-08T04:43:17.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd97c0b26c0 0x7fd97c1fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd97c0cfda0 0x7fd97c14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c168b00 con 0x7fd97c15ab60 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c168980 con 0x7fd97c0b26c0 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c168c80 con 0x7fd97c0cfda0 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52466/0 (socket says 172.21.15.164:52466) 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 -- 172.21.15.164:0/223699150 learned_addr learned my addr 172.21.15.164:0/223699150 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97b400640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd97c0b26c0 0x7fd97c1fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd972a00640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd97c0cfda0 0x7fd97c14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 -- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd97c0cfda0 msgr2=0x7fd97c14cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd97c0cfda0 0x7fd97c14cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 -- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd97c0b26c0 msgr2=0x7fd97c1fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd97c0b26c0 0x7fd97c1fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 -- 172.21.15.164:0/223699150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c159c20 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd97be00640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd974009d90 tx=0x7fd974019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740b2050 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740be070 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c1588b0 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740adb10 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.451+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c158e60 con 0x7fd97c15ab60 2024-05-08T04:43:17.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c069d60 con 0x7fd97c15ab60 2024-05-08T04:43:17.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fd9740adcb0 con 0x7fd97c15ab60 2024-05-08T04:43:17.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd973e00640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 0x7fd958069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fd97411eb40 con 0x7fd97c15ab60 2024-05-08T04:43:17.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd97b400640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 0x7fd958069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.455+0000 7fd97b400640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 0x7fd958069f60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005890 tx=0x7fd96c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.459+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd974016560 con 0x7fd97c15ab60 2024-05-08T04:43:17.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.615+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd97c06d9a0 con 0x7fd97c15ab60 2024-05-08T04:43:17.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd973e00640 1 -- 172.21.15.164:0/223699150 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17027 (secure 0 0 0) 0x7fd9740f0ff0 con 0x7fd97c15ab60 2024-05-08T04:43:17.620 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:17.620 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":114,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:12.323259+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.36","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 msgr2=0x7fd958069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 0x7fd958069f60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005890 tx=0x7fd96c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 msgr2=0x7fd97c158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd974009d90 tx=0x7fd974019710 comp rx=0 tx=0).stop 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 shutdown_connections 2024-05-08T04:43:17.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd958067aa0 0x7fd958069f60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd97c0cfda0 0x7fd97c14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c15ab60 0x7fd97c158370 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 --2- 172.21.15.164:0/223699150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd97c0b26c0 0x7fd97c1fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 >> 172.21.15.164:0/223699150 conn(0x7fd97c06ac90 msgr2=0x7fd97c157920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 shutdown_connections 2024-05-08T04:43:17.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.619+0000 7fd982200640 1 -- 172.21.15.164:0/223699150 wait complete. 2024-05-08T04:43:17.642 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:43:17.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:17.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 Processor -- start 2024-05-08T04:43:17.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 -- start start 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc03015ba70 0x7fc030160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0300cfda0 0x7fc0300b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0300cd9f0 con 0x7fc03015be50 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03015fe70 con 0x7fc03015ba70 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc038800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0300cdcd0 con 0x7fc0300cfda0 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc035400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc03015be50 0x7fc0300cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49458/0 (socket says 172.21.15.164:49458) 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc035400640 1 -- 172.21.15.164:0/3712783333 learned_addr learned my addr 172.21.15.164:0/3712783333 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.775+0000 7fc035e00640 1 --2- 172.21.15.164:0/3712783333 >> v2:172.21.15.164:3300/0 conn(0x7fc03015ba70 0x7fc030160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc036800640 1 --2- 172.21.15.164:0/3712783333 >> v2:172.21.15.172:3300/0 conn(0x7fc0300cfda0 0x7fc0300b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc035e00640 1 -- 172.21.15.164:0/3712783333 >> v2:172.21.15.172:3300/0 conn(0x7fc0300cfda0 msgr2=0x7fc0300b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:17.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc035e00640 1 --2- 172.21.15.164:0/3712783333 >> v2:172.21.15.172:3300/0 conn(0x7fc0300cfda0 0x7fc0300b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc034a00640 1 -- 172.21.15.164:0/3712783333 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657568307 0 0) 0x7fc0300cd9f0 con 0x7fc03015be50 2024-05-08T04:43:17.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc035e00640 1 -- 172.21.15.164:0/3712783333 >> v1:172.21.15.164:6789/0 conn(0x7fc03015be50 legacy=0x7fc0300cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc035e00640 1 -- 172.21.15.164:0/3712783333 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03015f820 con 0x7fc03015ba70 2024-05-08T04:43:17.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc035e00640 1 --2- 172.21.15.164:0/3712783333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030160300 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc03015fde0 tx=0x7fc02401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13cc0dbd2441d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc034a00640 1 -- 172.21.15.164:0/3712783333 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240b2070 con 0x7fc03015ba70 2024-05-08T04:43:17.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc034a00640 1 -- 172.21.15.164:0/3712783333 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0240a9070 con 0x7fc03015ba70 2024-05-08T04:43:17.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc034a00640 1 -- 172.21.15.164:0/3712783333 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240ad880 con 0x7fc03015ba70 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc038800640 1 -- 172.21.15.164:0/3712783333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 msgr2=0x7fc030160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.779+0000 7fc038800640 1 --2- 172.21.15.164:0/3712783333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030160300 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc03015fde0 tx=0x7fc02401c450 comp rx=0 tx=0).stop 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/3712783333 shutdown_connections 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 --2- 172.21.15.164:0/3712783333 >> v2:172.21.15.172:3300/0 conn(0x7fc0300cfda0 0x7fc0300b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 --2- 172.21.15.164:0/3712783333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030160300 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/3712783333 >> 172.21.15.164:0/3712783333 conn(0x7fc03006ac90 msgr2=0x7fc03006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/3712783333 shutdown_connections 2024-05-08T04:43:17.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/3712783333 wait complete. 2024-05-08T04:43:17.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 Processor -- start 2024-05-08T04:43:17.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- start start 2024-05-08T04:43:17.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 0x7fc030154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 0x7fc030071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030168d40 con 0x7fc03015ba70 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030168bc0 con 0x7fc03015be50 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc030168ec0 con 0x7fc0300cfda0 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52484/0 (socket says 172.21.15.164:52484) 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 -- 172.21.15.164:0/606829673 learned_addr learned my addr 172.21.15.164:0/606829673 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc036800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 0x7fc030071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035e00640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 0x7fc030154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 -- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 msgr2=0x7fc030154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 0x7fc030154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 -- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 msgr2=0x7fc030071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 0x7fc030071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 -- 172.21.15.164:0/606829673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0301515d0 con 0x7fc03015ba70 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035e00640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 0x7fc030154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc036800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 0x7fc030071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc035400640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c480 tx=0x7fc02000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020018070 con 0x7fc03015ba70 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020007500 con 0x7fc03015ba70 2024-05-08T04:43:17.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301518b0 con 0x7fc03015ba70 2024-05-08T04:43:17.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020004a30 con 0x7fc03015ba70 2024-05-08T04:43:17.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.783+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc030151e70 con 0x7fc03015ba70 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030069d60 con 0x7fc03015ba70 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fc02001b070 con 0x7fc03015ba70 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc02a000640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 0x7fc014069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fc020082ff0 con 0x7fc03015ba70 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc035e00640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 0x7fc014069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:17.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.787+0000 7fc035e00640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 0x7fc014069f60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0240af000 tx=0x7fc0240a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:17.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.791+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc020055800 con 0x7fc03015ba70 2024-05-08T04:43:17.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.975+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03006e210 con 0x7fc03015ba70 2024-05-08T04:43:17.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.975+0000 7fc02a000640 1 -- 172.21.15.164:0/606829673 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc020055100 con 0x7fc03015ba70 2024-05-08T04:43:17.978 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:17.978 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:43:17.979 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 73 2024-05-08T04:43:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 msgr2=0x7fc014069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 0x7fc014069f60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0240af000 tx=0x7fc0240a7000 comp rx=0 tx=0).stop 2024-05-08T04:43:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 msgr2=0x7fc030152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c480 tx=0x7fc02000c940 comp rx=0 tx=0).stop 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 shutdown_connections 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc014067aa0 0x7fc014069f60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03015be50 0x7fc030071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc03015ba70 0x7fc030152cf0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 --2- 172.21.15.164:0/606829673 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0300cfda0 0x7fc030154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 >> 172.21.15.164:0/606829673 conn(0x7fc03006ac90 msgr2=0x7fc0301588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 shutdown_connections 2024-05-08T04:43:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:17.979+0000 7fc038800640 1 -- 172.21.15.164:0/606829673 wait complete. 2024-05-08T04:43:17.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:18.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.131+0000 7f994d600640 1 Processor -- start 2024-05-08T04:43:18.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.131+0000 7f994d600640 1 -- start start 2024-05-08T04:43:18.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f994d600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f994815ba70 0x7f9948160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f994d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99480cfda0 0x7f99480b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f994d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99480cd9f0 con 0x7f994815be50 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f994d600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994815fe70 con 0x7f994815ba70 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f994d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99480cdcd0 con 0x7f99480cfda0 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f994815ba70 0x7f9948160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f994815ba70 0x7f9948160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52500/0 (socket says 172.21.15.164:52500) 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 -- 172.21.15.164:0/437712492 learned_addr learned my addr 172.21.15.164:0/437712492 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9947400640 1 --2- 172.21.15.164:0/437712492 >> v2:172.21.15.172:3300/0 conn(0x7f99480cfda0 0x7f99480b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 -- 172.21.15.164:0/437712492 >> v2:172.21.15.172:3300/0 conn(0x7f99480cfda0 msgr2=0x7f99480b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 --2- 172.21.15.164:0/437712492 >> v2:172.21.15.172:3300/0 conn(0x7f99480cfda0 0x7f99480b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 -- 172.21.15.164:0/437712492 >> v1:172.21.15.172:6789/0 conn(0x7f994815be50 legacy=0x7f99480cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.135+0000 7f9946a00640 1 -- 172.21.15.164:0/437712492 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994815f820 con 0x7f994815ba70 2024-05-08T04:43:18.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f9946a00640 1 --2- 172.21.15.164:0/437712492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f994815ba70 0x7f9948160300 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f994815fde0 tx=0x7f993801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4ea8a0db077e2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f9945600640 1 -- 172.21.15.164:0/437712492 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380ab600 con 0x7f994815ba70 2024-05-08T04:43:18.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f9945600640 1 -- 172.21.15.164:0/437712492 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380b0070 con 0x7f994815ba70 2024-05-08T04:43:18.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f9945600640 1 -- 172.21.15.164:0/437712492 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380ab9c0 con 0x7f994815ba70 2024-05-08T04:43:18.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- 172.21.15.164:0/437712492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f994815ba70 msgr2=0x7f9948160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 --2- 172.21.15.164:0/437712492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f994815ba70 0x7f9948160300 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f994815fde0 tx=0x7f993801c4f0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- 172.21.15.164:0/437712492 shutdown_connections 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 --2- 172.21.15.164:0/437712492 >> v2:172.21.15.172:3300/0 conn(0x7f99480cfda0 0x7f99480b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 --2- 172.21.15.164:0/437712492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f994815ba70 0x7f9948160300 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- 172.21.15.164:0/437712492 >> 172.21.15.164:0/437712492 conn(0x7f994806ac90 msgr2=0x7f994806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- 172.21.15.164:0/437712492 shutdown_connections 2024-05-08T04:43:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- 172.21.15.164:0/437712492 wait complete. 2024-05-08T04:43:18.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 Processor -- start 2024-05-08T04:43:18.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.139+0000 7f994d600640 1 -- start start 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f994815ba70 0x7f99480af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f994815be50 0x7f994815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948168d40 con 0x7f99480cfda0 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948168bc0 con 0x7f994815ba70 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9948168ec0 con 0x7f994815be50 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52504/0 (socket says 172.21.15.164:52504) 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 -- 172.21.15.164:0/2175012179 learned_addr learned my addr 172.21.15.164:0/2175012179 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946000640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f994815ba70 0x7f99480af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9947400640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f994815be50 0x7f994815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 -- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f994815be50 msgr2=0x7f994815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f994815be50 0x7f994815ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 -- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f994815ba70 msgr2=0x7f99480af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f994815ba70 0x7f99480af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 -- 172.21.15.164:0/2175012179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948150ef0 con 0x7f99480cfda0 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f9946a00640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f99380ad040 tx=0x7f993801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380be5f0 con 0x7f99480cfda0 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380be790 con 0x7f99480cfda0 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9948148180 con 0x7f99480cfda0 2024-05-08T04:43:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380beb80 con 0x7f99480cfda0 2024-05-08T04:43:18.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994815cb20 con 0x7f99480cfda0 2024-05-08T04:43:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9948069d60 con 0x7f99480cfda0 2024-05-08T04:43:18.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.143+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f99380aa070 con 0x7f99480cfda0 2024-05-08T04:43:18.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.147+0000 7f993ea00640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 0x7f9928069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.147+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f993811e000 con 0x7f99480cfda0 2024-05-08T04:43:18.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.147+0000 7f9946000640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 0x7f9928069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.147+0000 7f9946000640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 0x7f9928069f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f994814bb00 tx=0x7f9934009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.151+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9938016110 con 0x7f99480cfda0 2024-05-08T04:43:18.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f994806e210 con 0x7f99480cfda0 2024-05-08T04:43:18.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f993ea00640 1 -- 172.21.15.164:0/2175012179 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f99380f0470 con 0x7f99480cfda0 2024-05-08T04:43:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:18.345 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:42:53.623714+0000","modified":"2024-05-08T04:42:56.632348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/1496523549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":1496523549},{"type":"v1","addr":"172.21.15.164:6837","nonce":1496523549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-08T04:43:18.345 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 73 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 msgr2=0x7f9928069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 0x7f9928069f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f994814bb00 tx=0x7f9934009040 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 msgr2=0x7f99481509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f99380ad040 tx=0x7f993801ad30 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 shutdown_connections 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9928067aa0 0x7f9928069f60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f994815be50 0x7f994815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f994815ba70 0x7f99480af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 --2- 172.21.15.164:0/2175012179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99480cfda0 0x7f99481509b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 >> 172.21.15.164:0/2175012179 conn(0x7f994806ac90 msgr2=0x7f99480cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.343+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 shutdown_connections 2024-05-08T04:43:18.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.347+0000 7f994d600640 1 -- 172.21.15.164:0/2175012179 wait complete. 2024-05-08T04:43:18.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:18.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.499+0000 7f701ec00640 1 Processor -- start 2024-05-08T04:43:18.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.499+0000 7f701ec00640 1 -- start start 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f70180cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70180cfbd0 0x7f70180cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701ec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701806c770 con 0x7f701815ab60 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70180d0060 con 0x7f70180cfbd0 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701806ca50 con 0x7f70180b26c0 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f701815ab60 0x7f7018072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:49470/0 (socket says 172.21.15.164:49470) 2024-05-08T04:43:18.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017e00640 1 -- 172.21.15.164:0/998377652 learned_addr learned my addr 172.21.15.164:0/998377652 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701cc00640 1 --2- 172.21.15.164:0/998377652 >> v2:172.21.15.172:3300/0 conn(0x7f70180cfbd0 0x7f70180cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f70180cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254325358 0 0) 0x7f701806c770 con 0x7f701815ab60 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ffc003530 con 0x7f701815ab60 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2907502424 0 0) 0x7f6ffc003530 con 0x7f701815ab60 2024-05-08T04:43:18.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 msgr2=0x7f70180cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f70180cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 >> v2:172.21.15.172:3300/0 conn(0x7f70180cfbd0 msgr2=0x7f70180cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 --2- 172.21.15.164:0/998377652 >> v2:172.21.15.172:3300/0 conn(0x7f70180cfbd0 0x7f70180cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701806cd30 con 0x7f701815ab60 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f7008001680 con 0x7f701815ab60 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 >> v1:172.21.15.164:6789/0 conn(0x7f701815ab60 legacy=0x7f7018072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ffc0039c0 0x7f6ffc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ffc004810 0x7f6ffc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f6ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701806ca50 con 0x7f6ffc004810 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc00b9d0 con 0x7f6ffc0039c0 2024-05-08T04:43:18.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc00bb50 con 0x7f70180b26c0 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ffc004810 0x7f6ffc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f701cc00640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f6ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017e00640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ffc0039c0 0x7f6ffc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 -- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 msgr2=0x7f6ffc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70180b26c0 0x7f6ffc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 -- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ffc0039c0 msgr2=0x7f6ffc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ffc0039c0 0x7f6ffc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f700f400640 1 -- 172.21.15.164:0/998377652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc003530 con 0x7f6ffc004810 2024-05-08T04:43:18.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.503+0000 7f7017e00640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ffc0039c0 0x7f6ffc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:18.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f700f400640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ffc004810 0x7f6ffc004c90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ae0 tx=0x7f700001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a66c29a974cb05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000b2070 con 0x7f6ffc004810 2024-05-08T04:43:18.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000a5580 con 0x7f6ffc004810 2024-05-08T04:43:18.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f7017400640 1 -- 172.21.15.164:0/998377652 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000096b0 con 0x7f6ffc004810 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- 172.21.15.164:0/998377652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ffc004810 msgr2=0x7f6ffc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 --2- 172.21.15.164:0/998377652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ffc004810 0x7f6ffc004c90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ae0 tx=0x7f700001ca70 comp rx=0 tx=0).stop 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701cc00640 1 -- 172.21.15.164:0/998377652 reap_dead start 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- 172.21.15.164:0/998377652 shutdown_connections 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- 172.21.15.164:0/998377652 >> 172.21.15.164:0/998377652 conn(0x7f701806ac90 msgr2=0x7f701806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- 172.21.15.164:0/998377652 shutdown_connections 2024-05-08T04:43:18.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- 172.21.15.164:0/998377652 wait complete. 2024-05-08T04:43:18.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 Processor -- start 2024-05-08T04:43:18.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.507+0000 7f701ec00640 1 -- start start 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ffc004810 0x7f7018155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70181f23b0 0x7f70181f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701806c3d0 con 0x7f70181e6020 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701806d2f0 con 0x7f70181f23b0 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701806c550 con 0x7f6ffc004810 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52514/0 (socket says 172.21.15.164:52514) 2024-05-08T04:43:18.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 -- 172.21.15.164:0/4033293894 learned_addr learned my addr 172.21.15.164:0/4033293894 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f7017e00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ffc004810 0x7f7018155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701cc00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70181f23b0 0x7f70181f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 -- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ffc004810 msgr2=0x7f7018155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ffc004810 0x7f7018155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 -- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70181f23b0 msgr2=0x7f70181f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70181f23b0 0x7f70181f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 -- 172.21.15.164:0/4033293894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70181e6c00 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f700f400640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f70000b0040 tx=0x7f6ffc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000b2040 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000f000 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000096b0 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70181e6ee0 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70181f1e90 con 0x7f70181e6020 2024-05-08T04:43:18.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70181f19c0 con 0x7f70181e6020 2024-05-08T04:43:18.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.511+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f70000c0030 con 0x7f70181e6020 2024-05-08T04:43:18.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.515+0000 7f7014c00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 0x7f6ff4069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.515+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f700011d6b0 con 0x7f70181e6020 2024-05-08T04:43:18.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.515+0000 7f7017e00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 0x7f6ff4069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.515+0000 7f7017e00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 0x7f6ff4069f90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7008001000 tx=0x7f7008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.519+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70000c7060 con 0x7f70181e6020 2024-05-08T04:43:18.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.671+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7018069d60 con 0x7f70181e6020 2024-05-08T04:43:18.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.671+0000 7f7014c00640 1 -- 172.21.15.164:0/4033293894 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17027 (secure 0 0 0) 0x7f7000016780 con 0x7f70181e6020 2024-05-08T04:43:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:18.675 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":114,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:12.323259+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:42:51.994625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-08T04:42:52.615473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.36","mappings":[{"from":0,"to":5}]},{"pgid":"17.37","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 msgr2=0x7f6ff4069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 0x7f6ff4069f90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7008001000 tx=0x7f7008009040 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 msgr2=0x7f70181e64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f70000b0040 tx=0x7f6ffc005840 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 shutdown_connections 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6ff4067ad0 0x7f6ff4069f90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70181f23b0 0x7f70181f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70181e6020 0x7f70181e64a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 --2- 172.21.15.164:0/4033293894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ffc004810 0x7f7018155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 >> 172.21.15.164:0/4033293894 conn(0x7f701806ac90 msgr2=0x7f7018070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:18.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 shutdown_connections 2024-05-08T04:43:18.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.675+0000 7f701ec00640 1 -- 172.21.15.164:0/4033293894 wait complete. 2024-05-08T04:43:18.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:43:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 Processor -- start 2024-05-08T04:43:18.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 -- start start 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0dc15ba70 0x7ff0dc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff0dc0cfda0 0x7ff0dc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0cd9f0 con 0x7ff0dc15be50 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc15fe70 con 0x7ff0dc15ba70 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e4200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc0cdcd0 con 0x7ff0dc0cfda0 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e1800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0dc15ba70 0x7ff0dc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e1800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0dc15ba70 0x7ff0dc160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52522/0 (socket says 172.21.15.164:52522) 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.831+0000 7ff0e1800640 1 -- 172.21.15.164:0/1698523270 learned_addr learned my addr 172.21.15.164:0/1698523270 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:18.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e2200640 1 --2- 172.21.15.164:0/1698523270 >> v2:172.21.15.172:3300/0 conn(0x7ff0dc0cfda0 0x7ff0dc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e1800640 1 -- 172.21.15.164:0/1698523270 >> v2:172.21.15.172:3300/0 conn(0x7ff0dc0cfda0 msgr2=0x7ff0dc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e1800640 1 --2- 172.21.15.164:0/1698523270 >> v2:172.21.15.172:3300/0 conn(0x7ff0dc0cfda0 0x7ff0dc0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e1800640 1 -- 172.21.15.164:0/1698523270 >> v1:172.21.15.172:6789/0 conn(0x7ff0dc15be50 legacy=0x7ff0dc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:18.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e1800640 1 -- 172.21.15.164:0/1698523270 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc15f820 con 0x7ff0dc15ba70 2024-05-08T04:43:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0e1800640 1 --2- 172.21.15.164:0/1698523270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc160300 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc15fde0 tx=0x7ff0d001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182fc6d4f08ed308 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0dbe00640 1 -- 172.21.15.164:0/1698523270 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00a5790 con 0x7ff0dc15ba70 2024-05-08T04:43:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0dbe00640 1 -- 172.21.15.164:0/1698523270 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d00a5930 con 0x7ff0dc15ba70 2024-05-08T04:43:18.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.835+0000 7ff0dbe00640 1 -- 172.21.15.164:0/1698523270 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00a5d00 con 0x7ff0dc15ba70 2024-05-08T04:43:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- 172.21.15.164:0/1698523270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 msgr2=0x7ff0dc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- 172.21.15.164:0/1698523270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc160300 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc15fde0 tx=0x7ff0d001c9c0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- 172.21.15.164:0/1698523270 shutdown_connections 2024-05-08T04:43:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- 172.21.15.164:0/1698523270 >> v2:172.21.15.172:3300/0 conn(0x7ff0dc0cfda0 0x7ff0dc0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- 172.21.15.164:0/1698523270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc160300 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- 172.21.15.164:0/1698523270 >> 172.21.15.164:0/1698523270 conn(0x7ff0dc06ac90 msgr2=0x7ff0dc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- 172.21.15.164:0/1698523270 shutdown_connections 2024-05-08T04:43:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- 172.21.15.164:0/1698523270 wait complete. 2024-05-08T04:43:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 Processor -- start 2024-05-08T04:43:18.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- start start 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 0x7ff0dc15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc168d40 con 0x7ff0dc15ba70 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc168bc0 con 0x7ff0dc0cfda0 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e4200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc168ec0 con 0x7ff0dc15be50 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52526/0 (socket says 172.21.15.164:52526) 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 -- 172.21.15.164:0/2332717644 learned_addr learned my addr 172.21.15.164:0/2332717644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:18.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e2200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 0x7ff0dc15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e1800640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 -- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 msgr2=0x7ff0dc15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 0x7ff0dc15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 -- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0dc0cfda0 msgr2=0x7ff0dc1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 -- 172.21.15.164:0/2332717644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc150ef0 con 0x7ff0dc15ba70 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e2200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 0x7ff0dc15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.839+0000 7ff0e0e00640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c370 tx=0x7ff0c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80052d0 con 0x7ff0dc15ba70 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8005470 con 0x7ff0dc15ba70 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc148180 con 0x7ff0dc15ba70 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8007260 con 0x7ff0dc15ba70 2024-05-08T04:43:18.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc15cb20 con 0x7ff0dc15ba70 2024-05-08T04:43:18.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc069d60 con 0x7ff0dc15ba70 2024-05-08T04:43:18.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7ff0c8007400 con 0x7ff0dc15ba70 2024-05-08T04:43:18.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 0x7ff0bc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:18.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5907+0+0 (secure 0 0 0) 0x7ff0c8082fe0 con 0x7ff0dc15ba70 2024-05-08T04:43:18.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.843+0000 7ff0e1800640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 0x7ff0bc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:18.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.847+0000 7ff0e1800640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 0x7ff0bc069f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ae000 tx=0x7ff0d00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:18.847+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0c8087e20 con 0x7ff0dc15ba70 2024-05-08T04:43:19.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.035+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff0dc06e210 con 0x7ff0dc15ba70 2024-05-08T04:43:19.038 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:43:19.035+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:43:19.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0d9600640 1 -- 172.21.15.164:0/2332717644 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0c80551a0 con 0x7ff0dc15ba70 2024-05-08T04:43:19.040 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:43:19.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 msgr2=0x7ff0bc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 0x7ff0bc069f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ae000 tx=0x7ff0d00a7070 comp rx=0 tx=0).stop 2024-05-08T04:43:19.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 msgr2=0x7ff0dc0af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c370 tx=0x7ff0c800c830 comp rx=0 tx=0).stop 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 shutdown_connections 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff0bc067aa0 0x7ff0bc069f60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0dc15be50 0x7ff0dc15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0dc15ba70 0x7ff0dc0af210 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 --2- 172.21.15.164:0/2332717644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.039+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 >> 172.21.15.164:0/2332717644 conn(0x7ff0dc06ac90 msgr2=0x7ff0dc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:19.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.043+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 shutdown_connections 2024-05-08T04:43:19.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.043+0000 7ff0e4200640 1 -- 172.21.15.164:0/2332717644 wait complete. 2024-05-08T04:43:19.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:43:19.110 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:43:19.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 Processor -- start 2024-05-08T04:43:19.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 -- start start 2024-05-08T04:43:19.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9781451b0 0x7fe97806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9780b5890 0x7fe9780aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97806f820 con 0x7fe9780b4c70 2024-05-08T04:43:19.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97806fb00 con 0x7fe9780b5890 2024-05-08T04:43:19.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe97d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806fde0 con 0x7fe9781451b0 2024-05-08T04:43:19.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe976a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe9780b4c70 0x7fe9780b5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59658/0 (socket says 172.21.15.164:59658) 2024-05-08T04:43:19.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe976a00640 1 -- 172.21.15.164:0/3445859356 learned_addr learned my addr 172.21.15.164:0/3445859356 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:19.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> v2:172.21.15.172:3300/0 conn(0x7fe9780b5890 0x7fe9780aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe977400640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9781451b0 0x7fe97806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979493732 0 0) 0x7fe97806f820 con 0x7fe9780b4c70 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.207+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe95c003530 con 0x7fe9780b4c70 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599021566 0 0) 0x7fe95c003530 con 0x7fe9780b4c70 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9781451b0 msgr2=0x7fe97806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9781451b0 0x7fe97806eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 >> v2:172.21.15.172:3300/0 conn(0x7fe9780b5890 msgr2=0x7fe9780aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 --2- 172.21.15.164:0/3445859356 >> v2:172.21.15.172:3300/0 conn(0x7fe9780b5890 0x7fe9780aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780700c0 con 0x7fe9780b4c70 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fe964001680 con 0x7fe9780b4c70 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 >> v1:172.21.15.164:6789/0 conn(0x7fe9780b4c70 legacy=0x7fe9780b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe95c0039c0 0x7fe95c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe95c004810 0x7fe95c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe95c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806fb00 con 0x7fe95c0051d0 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806fde0 con 0x7fe95c0039c0 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c00bf10 con 0x7fe95c004810 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977400640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9781451b0 0x7fe97806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> v2:172.21.15.172:3300/0 conn(0x7fe9780b5890 0x7fe9780aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976a00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe95c004810 0x7fe95c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe95c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977400640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe95c0039c0 0x7fe95c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 -- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe95c004810 msgr2=0x7fe95c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe95c004810 0x7fe95c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 -- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe95c0039c0 msgr2=0x7fe95c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:19.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe95c0039c0 0x7fe95c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe977e00640 1 -- 172.21.15.164:0/3445859356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c003530 con 0x7fe95c0051d0 2024-05-08T04:43:19.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.211+0000 7fe976a00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe95c004810 0x7fe95c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:19.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe977e00640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe95c00b9d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe97000e380 tx=0x7fe97001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a715cdbb74972347 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:19.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970009dc0 con 0x7fe95c0051d0 2024-05-08T04:43:19.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9700a5e50 con 0x7fe95c0051d0 2024-05-08T04:43:19.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe976000640 1 -- 172.21.15.164:0/3445859356 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970003ba0 con 0x7fe95c0051d0 2024-05-08T04:43:19.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- 172.21.15.164:0/3445859356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 msgr2=0x7fe95c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 --2- 172.21.15.164:0/3445859356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe95c00b9d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe97000e380 tx=0x7fe97001ce40 comp rx=0 tx=0).stop 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe977e00640 1 -- 172.21.15.164:0/3445859356 reap_dead start 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- 172.21.15.164:0/3445859356 shutdown_connections 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- 172.21.15.164:0/3445859356 >> 172.21.15.164:0/3445859356 conn(0x7fe97809aa20 msgr2=0x7fe9780b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- 172.21.15.164:0/3445859356 shutdown_connections 2024-05-08T04:43:19.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- 172.21.15.164:0/3445859356 wait complete. 2024-05-08T04:43:19.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 Processor -- start 2024-05-08T04:43:19.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.215+0000 7fe97d200640 1 -- start start 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 0x7fe97813ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97813ef80 0x7fe978142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978145710 con 0x7fe95c0051d0 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978145590 con 0x7fe9780d13e0 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806eed0 con 0x7fe97813ef80 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58952/0 (socket says 172.21.15.164:58952) 2024-05-08T04:43:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 -- 172.21.15.164:0/890401579 learned_addr learned my addr 172.21.15.164:0/890401579 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977e00640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97813ef80 0x7fe978142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe976a00640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 0x7fe97813ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 -- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97813ef80 msgr2=0x7fe978142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97813ef80 0x7fe978142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 -- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 msgr2=0x7fe97813ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 0x7fe97813ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 -- 172.21.15.164:0/890401579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780d1860 con 0x7fe95c0051d0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe976a00640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 0x7fe97813ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe977400640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe9680119d0 tx=0x7fe968011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800fbb0 con 0x7fe95c0051d0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96800fd50 con 0x7fe95c0051d0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968014660 con 0x7fe95c0051d0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9780d1b40 con 0x7fe95c0051d0 2024-05-08T04:43:19.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978141d00 con 0x7fe95c0051d0 2024-05-08T04:43:19.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97804ef30 con 0x7fe95c0051d0 2024-05-08T04:43:19.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.219+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fe968012040 con 0x7fe95c0051d0 2024-05-08T04:43:19.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.223+0000 7fe96f400640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 0x7fe958069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:19.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.223+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5965+0+0 (secure 0 0 0) 0x7fe968084290 con 0x7fe95c0051d0 2024-05-08T04:43:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.223+0000 7fe976a00640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 0x7fe958069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.223+0000 7fe976a00640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 0x7fe958069f90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe964001020 tx=0x7fe964009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:19.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.227+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe968019030 con 0x7fe95c0051d0 2024-05-08T04:43:19.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:19.419+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe97809af00 con 0x7fe95c0051d0 2024-05-08T04:43:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.039+0000 7fe96f400640 1 -- 172.21.15.164:0/890401579 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe968056b00 con 0x7fe95c0051d0 2024-05-08T04:43:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 msgr2=0x7fe958069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 0x7fe958069f90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe964001020 tx=0x7fe964009040 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 msgr2=0x7fe9781417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe9680119d0 tx=0x7fe968011e90 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 shutdown_connections 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe958067ad0 0x7fe958069f90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe97813ef80 0x7fe978142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9780d13e0 0x7fe97813ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 --2- 172.21.15.164:0/890401579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95c0051d0 0x7fe9781417c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 >> 172.21.15.164:0/890401579 conn(0x7fe97809aa20 msgr2=0x7fe9780b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 shutdown_connections 2024-05-08T04:43:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.043+0000 7fe97d200640 1 -- 172.21.15.164:0/890401579 wait complete. 2024-05-08T04:43:20.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:43:20.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.251+0000 7f1ec5800640 1 Processor -- start 2024-05-08T04:43:20.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.251+0000 7f1ec5800640 1 -- start start 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 0x7f1ec006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ec5800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ec00cee90 0x7f1ec00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ec5800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec015e870 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ec5800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec006bb80 con 0x7f1ec00cee90 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ec5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec015eb50 con 0x7f1ec015ab10 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ec00cee90 0x7f1ec00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:20.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1eb6000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1ec00ae6e0 0x7f1ec00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59664/0 (socket says 172.21.15.164:59664) 2024-05-08T04:43:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1eb6000640 1 -- 172.21.15.164:0/2381115869 learned_addr learned my addr 172.21.15.164:0/2381115869 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebea00640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 0x7f1ec006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 -- 172.21.15.164:0/2381115869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 msgr2=0x7f1ec006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 0x7f1ec006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 -- 172.21.15.164:0/2381115869 >> v1:172.21.15.164:6789/0 conn(0x7f1ec00ae6e0 legacy=0x7f1ec00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebe000640 1 -- 172.21.15.164:0/2381115869 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140177957 0 0) 0x7f1ec015e870 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 -- 172.21.15.164:0/2381115869 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec015ee30 con 0x7f1ec00cee90 2024-05-08T04:43:20.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebea00640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 0x7f1ec006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:20.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.255+0000 7f1ebf400640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00cee90 0x7f1ec00aeb40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097c0 tx=0x7f1eb801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb05898a8515d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:20.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebe000640 1 -- 172.21.15.164:0/2381115869 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80b1070 con 0x7f1ec00cee90 2024-05-08T04:43:20.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebe000640 1 -- 172.21.15.164:0/2381115869 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb80ac600 con 0x7f1ec00cee90 2024-05-08T04:43:20.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebe000640 1 -- 172.21.15.164:0/2381115869 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80ac9d0 con 0x7f1ec00cee90 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- 172.21.15.164:0/2381115869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00cee90 msgr2=0x7f1ec00aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00cee90 0x7f1ec00aeb40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097c0 tx=0x7f1eb801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- 172.21.15.164:0/2381115869 shutdown_connections 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00cee90 0x7f1ec00aeb40 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2381115869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015ab10 0x7f1ec006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- 172.21.15.164:0/2381115869 >> 172.21.15.164:0/2381115869 conn(0x7f1ec00cf770 msgr2=0x7f1ec00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- 172.21.15.164:0/2381115869 shutdown_connections 2024-05-08T04:43:20.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- 172.21.15.164:0/2381115869 wait complete. 2024-05-08T04:43:20.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 Processor -- start 2024-05-08T04:43:20.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- start start 2024-05-08T04:43:20.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec00cee90 0x7f1ec014faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 0x7f1ec00713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00cb070 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00caef0 con 0x7f1ec015ab10 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ec5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00cb1f0 con 0x7f1ec00cee90 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58968/0 (socket says 172.21.15.164:58968) 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 -- 172.21.15.164:0/2341217330 learned_addr learned my addr 172.21.15.164:0/2341217330 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 -- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec00cee90 msgr2=0x7f1ec014faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec00cee90 0x7f1ec014faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebea00640 1 -- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 msgr2=0x7f1ec00713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.259+0000 7f1ebf400640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 0x7f1ec00713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ebea00640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 0x7f1ec00713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ebea00640 1 -- 172.21.15.164:0/2341217330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec01fa840 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ebf400640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 0x7f1ec00713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ebea00640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000ce30 tx=0x7f1eb000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000f890 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000fa30 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01fab20 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000fe00 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec0155440 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0072d30 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f1eb0018400 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 0x7f1e9c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5965+0+0 (secure 0 0 0) 0x7f1eb0084780 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.263+0000 7f1eb6000640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 0x7f1e9c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.267+0000 7f1eb6000640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 0x7f1e9c069f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80011f0 tx=0x7f1ea8007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.267+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1eb0014030 con 0x7f1ec00ae6e0 2024-05-08T04:43:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:20.455+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ec0072300 con 0x7f1ec00ae6e0 2024-05-08T04:43:21.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1eb7400640 1 -- 172.21.15.164:0/2341217330 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1eb000ae20 con 0x7f1ec00ae6e0 2024-05-08T04:43:21.380 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 msgr2=0x7f1e9c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 0x7f1e9c069f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80011f0 tx=0x7f1ea8007930 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 msgr2=0x7f1ec01fa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000ce30 tx=0x7f1eb000a650 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 shutdown_connections 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e9c067aa0 0x7f1e9c069f60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec00cee90 0x7f1ec014faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec015ab10 0x7f1ec00713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 --2- 172.21.15.164:0/2341217330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec00ae6e0 0x7f1ec01fa260 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 >> 172.21.15.164:0/2341217330 conn(0x7f1ec00cf770 msgr2=0x7f1ec00ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 shutdown_connections 2024-05-08T04:43:21.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.379+0000 7f1ec5800640 1 -- 172.21.15.164:0/2341217330 wait complete. 2024-05-08T04:43:21.411 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:43:21.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 Processor -- start 2024-05-08T04:43:21.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 -- start start 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f295015bbf0 0x7f29500cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f295006f100 0x7f29500ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295015bfd0 con 0x7f29500d0280 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29500d06e0 con 0x7f295015bbf0 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f2955400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29500b0ef0 con 0x7f295006f100 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f295015bbf0 0x7f29500cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f295015bbf0 0x7f29500cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58978/0 (socket says 172.21.15.164:58978) 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 -- 172.21.15.164:0/1230103970 learned_addr learned my addr 172.21.15.164:0/1230103970 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294f200640 1 --2- 172.21.15.164:0/1230103970 >> v2:172.21.15.172:3300/0 conn(0x7f295006f100 0x7f29500ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 -- 172.21.15.164:0/1230103970 >> v2:172.21.15.172:3300/0 conn(0x7f295006f100 msgr2=0x7f29500ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 --2- 172.21.15.164:0/1230103970 >> v2:172.21.15.172:3300/0 conn(0x7f295006f100 0x7f29500ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 -- 172.21.15.164:0/1230103970 >> v1:172.21.15.172:6789/0 conn(0x7f29500d0280 legacy=0x7f295006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:21.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.555+0000 7f294e800640 1 -- 172.21.15.164:0/1230103970 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295006f5d0 con 0x7f295015bbf0 2024-05-08T04:43:21.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f294e800640 1 --2- 172.21.15.164:0/1230103970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500cfd40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f29500d24c0 tx=0x7f294001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e96842cdd483ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:21.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f294d400640 1 -- 172.21.15.164:0/1230103970 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400b0070 con 0x7f295015bbf0 2024-05-08T04:43:21.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f294d400640 1 -- 172.21.15.164:0/1230103970 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29400ab600 con 0x7f295015bbf0 2024-05-08T04:43:21.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f294d400640 1 -- 172.21.15.164:0/1230103970 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400ab9d0 con 0x7f295015bbf0 2024-05-08T04:43:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 -- 172.21.15.164:0/1230103970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 msgr2=0x7f29500cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 --2- 172.21.15.164:0/1230103970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500cfd40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f29500d24c0 tx=0x7f294001c450 comp rx=0 tx=0).stop 2024-05-08T04:43:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 -- 172.21.15.164:0/1230103970 shutdown_connections 2024-05-08T04:43:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 --2- 172.21.15.164:0/1230103970 >> v2:172.21.15.172:3300/0 conn(0x7f295006f100 0x7f29500ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 --2- 172.21.15.164:0/1230103970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500cfd40 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 -- 172.21.15.164:0/1230103970 >> 172.21.15.164:0/1230103970 conn(0x7f295009aa20 msgr2=0x7f295009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:21.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 -- 172.21.15.164:0/1230103970 shutdown_connections 2024-05-08T04:43:21.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.559+0000 7f2955400640 1 -- 172.21.15.164:0/1230103970 wait complete. 2024-05-08T04:43:21.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 Processor -- start 2024-05-08T04:43:21.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- start start 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f295006f100 0x7f29500c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29500d0280 0x7f295006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500af530 con 0x7f295015bbf0 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500af3b0 con 0x7f29500d0280 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500af6b0 con 0x7f295006f100 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58988/0 (socket says 172.21.15.164:58988) 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 -- 172.21.15.164:0/2734994258 learned_addr learned my addr 172.21.15.164:0/2734994258 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:21.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294de00640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29500d0280 0x7f295006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294e800640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f295006f100 0x7f29500c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 -- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f295006f100 msgr2=0x7f29500c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f295006f100 0x7f29500c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 -- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29500d0280 msgr2=0x7f295006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29500d0280 0x7f295006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 -- 172.21.15.164:0/2734994258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295006d920 con 0x7f295015bbf0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f294f200640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2948007e10 tx=0x7f294800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948014620 con 0x7f295015bbf0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948019070 con 0x7f295015bbf0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295006c340 con 0x7f295015bbf0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948014a20 con 0x7f295015bbf0 2024-05-08T04:43:21.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.563+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295006c900 con 0x7f295015bbf0 2024-05-08T04:43:21.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295009f9e0 con 0x7f295015bbf0 2024-05-08T04:43:21.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f2948014bc0 con 0x7f295015bbf0 2024-05-08T04:43:21.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f2946a00640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 0x7f2930069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:21.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5499+0+0 (secure 0 0 0) 0x7f2948084280 con 0x7f295015bbf0 2024-05-08T04:43:21.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f294e800640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 0x7f2930069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:21.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.567+0000 7f294e800640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 0x7f2930069f90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f29500d14a0 tx=0x7f2940009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:21.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.571+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2948011260 con 0x7f295015bbf0 2024-05-08T04:43:21.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:21.723+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f295004f010 con 0x7f295015bbf0 2024-05-08T04:43:22.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.391+0000 7f2946a00640 1 -- 172.21.15.164:0/2734994258 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2948056cb0 con 0x7f295015bbf0 2024-05-08T04:43:22.395 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:43:22.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 msgr2=0x7f2930069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 0x7f2930069f90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f29500d14a0 tx=0x7f2940009040 comp rx=0 tx=0).stop 2024-05-08T04:43:22.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 msgr2=0x7f29500b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2948007e10 tx=0x7f294800e4b0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 shutdown_connections 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2930067ad0 0x7f2930069f90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f295015bbf0 0x7f29500b6dc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29500d0280 0x7f295006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 --2- 172.21.15.164:0/2734994258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f295006f100 0x7f29500c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 >> 172.21.15.164:0/2734994258 conn(0x7f295009aa20 msgr2=0x7f29500afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 shutdown_connections 2024-05-08T04:43:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.399+0000 7f2955400640 1 -- 172.21.15.164:0/2734994258 wait complete. 2024-05-08T04:43:22.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:43:22.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.571+0000 7f9ce5400640 1 Processor -- start 2024-05-08T04:43:22.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.571+0000 7f9ce5400640 1 -- start start 2024-05-08T04:43:22.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9ce5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9ce5400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce00c6600 con 0x7f9ce00aedf0 2024-05-08T04:43:22.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9ce5400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce00c6040 con 0x7f9ce006de50 2024-05-08T04:43:22.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9ce5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce015fbe0 con 0x7f9ce015ba20 2024-05-08T04:43:22.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdf200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9ce00aedf0 0x7f9ce00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59666/0 (socket says 172.21.15.164:59666) 2024-05-08T04:43:22.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdf200640 1 -- 172.21.15.164:0/3611013700 learned_addr learned my addr 172.21.15.164:0/3611013700 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:22.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cde800640 1 --2- 172.21.15.164:0/3611013700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001910044 0 0) 0x7f9ce00c6600 con 0x7f9ce00aedf0 2024-05-08T04:43:22.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc4003530 con 0x7f9ce00aedf0 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038243927 0 0) 0x7f9ce00c6040 con 0x7f9ce006de50 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce00c6600 con 0x7f9ce006de50 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2890440500 0 0) 0x7f9cc4003530 con 0x7f9ce00aedf0 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cde800640 1 -- 172.21.15.164:0/3611013700 >> v1:172.21.15.172:6789/0 conn(0x7f9ce006de50 legacy=0x7f9ce00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cde800640 1 -- 172.21.15.164:0/3611013700 >> v1:172.21.15.164:6789/0 conn(0x7f9ce00aedf0 legacy=0x7f9ce00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.575+0000 7f9cde800640 1 -- 172.21.15.164:0/3611013700 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00c6320 con 0x7f9ce015ba20 2024-05-08T04:43:22.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cde800640 1 --2- 172.21.15.164:0/3611013700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0160070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ce015fd60 tx=0x7f9cd001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5054610a660f912 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00b0070 con 0x7f9ce015ba20 2024-05-08T04:43:22.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd00ab400 con 0x7f9ce015ba20 2024-05-08T04:43:22.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cdd400640 1 -- 172.21.15.164:0/3611013700 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8001a00 con 0x7f9ce015ba20 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- 172.21.15.164:0/3611013700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 msgr2=0x7f9ce0160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 --2- 172.21.15.164:0/3611013700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0160070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ce015fd60 tx=0x7f9cd001c3b0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- 172.21.15.164:0/3611013700 shutdown_connections 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 --2- 172.21.15.164:0/3611013700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0160070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- 172.21.15.164:0/3611013700 >> 172.21.15.164:0/3611013700 conn(0x7f9ce00cf770 msgr2=0x7f9ce00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- 172.21.15.164:0/3611013700 shutdown_connections 2024-05-08T04:43:22.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- 172.21.15.164:0/3611013700 wait complete. 2024-05-08T04:43:22.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 Processor -- start 2024-05-08T04:43:22.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- start start 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ce00aedf0 0x7f9ce01528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce006e3b0 con 0x7f9ce006de50 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce006e230 con 0x7f9ce00aedf0 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9ce5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce006e530 con 0x7f9ce015ba20 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cde800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cde800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58994/0 (socket says 172.21.15.164:58994) 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.579+0000 7f9cde800640 1 -- 172.21.15.164:0/465483761 learned_addr learned my addr 172.21.15.164:0/465483761 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cdf200640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cdde00640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ce00aedf0 0x7f9ce01528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 -- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 msgr2=0x7f9ce0152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 -- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ce00aedf0 msgr2=0x7f9ce01528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ce00aedf0 0x7f9ce01528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 -- 172.21.15.164:0/465483761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00b0d90 con 0x7f9ce006de50 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cde800640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0019fe0 tx=0x7f9cd0019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00bd020 con 0x7f9ce006de50 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce00b1070 con 0x7f9ce006de50 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd0004020 con 0x7f9ce006de50 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00b0040 con 0x7f9ce006de50 2024-05-08T04:43:22.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce00ca0a0 con 0x7f9ce006de50 2024-05-08T04:43:22.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce0072d30 con 0x7f9ce006de50 2024-05-08T04:43:22.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.583+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f9cd00c0070 con 0x7f9ce006de50 2024-05-08T04:43:22.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.587+0000 7f9cd6a00640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 0x7f9cc0069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.587+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f9cd011d550 con 0x7f9ce006de50 2024-05-08T04:43:22.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.587+0000 7f9cdde00640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 0x7f9cc0069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.587+0000 7f9cdde00640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 0x7f9cc0069f90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9ce01fab30 tx=0x7f9ccc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.591+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cd0015fd0 con 0x7f9ce006de50 2024-05-08T04:43:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.747+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9ce004f010 con 0x7f9ce006de50 2024-05-08T04:43:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9cd6a00640 1 -- 172.21.15.164:0/465483761 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9cd00c0360 con 0x7f9ce006de50 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 msgr2=0x7f9cc0069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 0x7f9cc0069f90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9ce01fab30 tx=0x7f9ccc009040 comp rx=0 tx=0).stop 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 msgr2=0x7f9ce015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0019fe0 tx=0x7f9cd0019150 comp rx=0 tx=0).stop 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 shutdown_connections 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f9cc0067ad0 0x7f9cc0069f90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ce015ba20 0x7f9ce0152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ce00aedf0 0x7f9ce01528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 --2- 172.21.15.164:0/465483761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ce006de50 0x7f9ce015d5e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 >> 172.21.15.164:0/465483761 conn(0x7f9ce00cf770 msgr2=0x7f9ce01533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 shutdown_connections 2024-05-08T04:43:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.751+0000 7f9ce5400640 1 -- 172.21.15.164:0/465483761 wait complete. 2024-05-08T04:43:22.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate' 2024-05-08T04:43:22.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 Processor -- start 2024-05-08T04:43:22.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 -- start start 2024-05-08T04:43:22.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f092c15df50 0x7f092c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c15e3e0 con 0x7f092c0b4c40 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c15e680 con 0x7f092c0b3a90 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f092b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c15e960 con 0x7f092c15df50 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f092c15df50 0x7f092c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f092c0b4c40 0x7f092c161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59682/0 (socket says 172.21.15.164:59682) 2024-05-08T04:43:22.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929e00640 1 -- 172.21.15.164:0/1078822928 learned_addr learned my addr 172.21.15.164:0/1078822928 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:22.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929400640 1 -- 172.21.15.164:0/1078822928 >> v1:172.21.15.172:6789/0 conn(0x7f092c0b3a90 legacy=0x7f092c0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:22.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929400640 1 -- 172.21.15.164:0/1078822928 >> v1:172.21.15.164:6789/0 conn(0x7f092c0b4c40 legacy=0x7f092c161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.911+0000 7f0929400640 1 -- 172.21.15.164:0/1078822928 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c15ec40 con 0x7f092c15df50 2024-05-08T04:43:22.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f0929400640 1 --2- 172.21.15.164:0/1078822928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c15df50 0x7f092c0afc90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f09180097c0 tx=0x7f091801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74d03bc652f5333 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f0928a00640 1 -- 172.21.15.164:0/1078822928 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180b0070 con 0x7f092c15df50 2024-05-08T04:43:22.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f0928a00640 1 -- 172.21.15.164:0/1078822928 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180ab600 con 0x7f092c15df50 2024-05-08T04:43:22.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f0928a00640 1 -- 172.21.15.164:0/1078822928 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180ab9d0 con 0x7f092c15df50 2024-05-08T04:43:22.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 -- 172.21.15.164:0/1078822928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c15df50 msgr2=0x7f092c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 --2- 172.21.15.164:0/1078822928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c15df50 0x7f092c0afc90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f09180097c0 tx=0x7f091801c460 comp rx=0 tx=0).stop 2024-05-08T04:43:22.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 -- 172.21.15.164:0/1078822928 shutdown_connections 2024-05-08T04:43:22.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 --2- 172.21.15.164:0/1078822928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c15df50 0x7f092c0afc90 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 -- 172.21.15.164:0/1078822928 >> 172.21.15.164:0/1078822928 conn(0x7f092c06bf70 msgr2=0x7f092c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:22.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 -- 172.21.15.164:0/1078822928 shutdown_connections 2024-05-08T04:43:22.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.915+0000 7f092b200640 1 -- 172.21.15.164:0/1078822928 wait complete. 2024-05-08T04:43:22.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 Processor -- start 2024-05-08T04:43:22.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- start start 2024-05-08T04:43:22.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f092c0b3a90 0x7f092c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f092c15df50 0x7f092c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0d1180 con 0x7f092c0b4c40 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0d1000 con 0x7f092c0b3a90 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0b3ea0 con 0x7f092c15df50 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59010/0 (socket says 172.21.15.164:59010) 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 -- 172.21.15.164:0/3358886876 learned_addr learned my addr 172.21.15.164:0/3358886876 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929e00640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f092c0b3a90 0x7f092c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092a800640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f092c15df50 0x7f092c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 -- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f092c15df50 msgr2=0x7f092c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f092c15df50 0x7f092c0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 -- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f092c0b3a90 msgr2=0x7f092c156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f092c0b3a90 0x7f092c156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 -- 172.21.15.164:0/3358886876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c1585d0 con 0x7f092c0b4c40 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0929400640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0918019070 tx=0x7f0918019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180b8020 con 0x7f092c0b4c40 2024-05-08T04:43:22.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180b4610 con 0x7f092c0b4c40 2024-05-08T04:43:22.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c1588b0 con 0x7f092c0b4c40 2024-05-08T04:43:22.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180b0040 con 0x7f092c0b4c40 2024-05-08T04:43:22.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.919+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0ceb90 con 0x7f092c0b4c40 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c04ef30 con 0x7f092c0b4c40 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f09180bb020 con 0x7f092c0b4c40 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f0922000640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 0x7f090c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f091811d4b0 con 0x7f092c0b4c40 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f0929e00640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 0x7f090c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:22.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.923+0000 7f0929e00640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 0x7f090c069f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f091c0012f0 tx=0x7f091c001a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:22.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:22.927+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09180c2030 con 0x7f092c0b4c40 2024-05-08T04:43:23.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.135+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate"]} v 0) v1 -- 0x7f092c067850 con 0x7f092c0b4c40 2024-05-08T04:43:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f0922000640 1 -- 172.21.15.164:0/3358886876 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_invalid_recreate"]}]=0 v306) v1 ==== 175+0+0 (secure 0 0 0) 0x7f0918016050 con 0x7f092c0b4c40 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 msgr2=0x7f090c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 0x7f090c069f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f091c0012f0 tx=0x7f091c001a40 comp rx=0 tx=0).stop 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 msgr2=0x7f092c15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0918019070 tx=0x7f0918019200 comp rx=0 tx=0).stop 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 shutdown_connections 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f090c067aa0 0x7f090c069f60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f092c15df50 0x7f092c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f092c0b4c40 0x7f092c15a140 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 --2- 172.21.15.164:0/3358886876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f092c0b3a90 0x7f092c156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.395+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 >> 172.21.15.164:0/3358886876 conn(0x7f092c06bf70 msgr2=0x7f092c161c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:23.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.399+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 shutdown_connections 2024-05-08T04:43:23.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.399+0000 7f092b200640 1 -- 172.21.15.164:0/3358886876 wait complete. 2024-05-08T04:43:23.419 INFO:tasks.cephfs_test_runner:ensure retained subvolume recreate does not leave any incarnations in the subvolume and trash ... ok 2024-05-08T04:43:23.419 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ensure a retained subvolume can be recreated and further snapshotted 2024-05-08T04:43:23.419 INFO:tasks.cephfs_test_runner:test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:43:23.420 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:23.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 Processor -- start 2024-05-08T04:43:23.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 -- start start 2024-05-08T04:43:23.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f005415ab60 0x7f0054072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 0x7f00540cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0054072640 con 0x7f00540b26c0 2024-05-08T04:43:23.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005406c770 con 0x7f005415ab60 2024-05-08T04:43:23.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f005ba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005406ca50 con 0x7f00540cfda0 2024-05-08T04:43:23.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f0053e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f00540b26c0 0x7f00540cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59686/0 (socket says 172.21.15.164:59686) 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f0053e00640 1 -- 172.21.15.164:0/1956770933 learned_addr learned my addr 172.21.15.164:0/1956770933 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f0059000640 1 --2- 172.21.15.164:0/1956770933 >> v2:172.21.15.172:3300/0 conn(0x7f005415ab60 0x7f0054072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.555+0000 7f0059a00640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 0x7f00540cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916094639 0 0) 0x7f0054072640 con 0x7f00540b26c0 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0038003530 con 0x7f00540b26c0 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790905316 0 0) 0x7f0038003530 con 0x7f00540b26c0 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059000640 1 -- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 msgr2=0x7f00540cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059000640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 0x7f00540cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059000640 1 -- 172.21.15.164:0/1956770933 >> v1:172.21.15.164:6789/0 conn(0x7f00540b26c0 legacy=0x7f00540cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059000640 1 -- 172.21.15.164:0/1956770933 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005406cd30 con 0x7f005415ab60 2024-05-08T04:43:23.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059a00640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 0x7f00540cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:23.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0059000640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f0054072100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0044009d90 tx=0x7f004401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8111a1c454e25e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440ac070 con 0x7f005415ab60 2024-05-08T04:43:23.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00440a64a0 con 0x7f005415ab60 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.559+0000 7f0053400640 1 -- 172.21.15.164:0/1956770933 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040001680 con 0x7f005415ab60 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 msgr2=0x7f0054072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f0054072100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0044009d90 tx=0x7f004401c890 comp rx=0 tx=0).stop 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- 172.21.15.164:0/1956770933 shutdown_connections 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540cfda0 0x7f00540cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- 172.21.15.164:0/1956770933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f0054072100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- 172.21.15.164:0/1956770933 >> 172.21.15.164:0/1956770933 conn(0x7f005406ac90 msgr2=0x7f005406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:23.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- 172.21.15.164:0/1956770933 shutdown_connections 2024-05-08T04:43:23.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- 172.21.15.164:0/1956770933 wait complete. 2024-05-08T04:43:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 Processor -- start 2024-05-08T04:43:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- start start 2024-05-08T04:43:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540b26c0 0x7f0054148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f00541605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540aeeb0 con 0x7f00540cfda0 2024-05-08T04:43:23.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540aed30 con 0x7f005415ab60 2024-05-08T04:43:23.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f005ba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0054167970 con 0x7f00540b26c0 2024-05-08T04:43:23.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59024/0 (socket says 172.21.15.164:59024) 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 -- 172.21.15.164:0/1341191933 learned_addr learned my addr 172.21.15.164:0/1341191933 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0059a00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f00541605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0059000640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540b26c0 0x7f0054148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 -- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540b26c0 msgr2=0x7f0054148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540b26c0 0x7f0054148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 -- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 msgr2=0x7f00541605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f00541605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.563+0000 7f0053e00640 1 -- 172.21.15.164:0/1341191933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054156d70 con 0x7f00540cfda0 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0053e00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f004000c750 tx=0x7f004000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040005ae0 con 0x7f00540cfda0 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040005ca0 con 0x7f00540cfda0 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400072b0 con 0x7f00540cfda0 2024-05-08T04:43:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0054157050 con 0x7f00540cfda0 2024-05-08T04:43:23.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0054157610 con 0x7f00540cfda0 2024-05-08T04:43:23.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0054069d60 con 0x7f00540cfda0 2024-05-08T04:43:23.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f0040007450 con 0x7f00540cfda0 2024-05-08T04:43:23.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 0x7f0034069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0040086400 con 0x7f00540cfda0 2024-05-08T04:43:23.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.567+0000 7f0059000640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 0x7f0034069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.571+0000 7f0059000640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 0x7f0034069f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f00440ae000 tx=0x7f00440b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.571+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0040056240 con 0x7f00540cfda0 2024-05-08T04:43:23.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.735+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f005406d9a0 con 0x7f00540cfda0 2024-05-08T04:43:23.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f0050c00640 1 -- 172.21.15.164:0/1341191933 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+13226 (secure 0 0 0) 0x7f0040055b40 con 0x7f00540cfda0 2024-05-08T04:43:23.739 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:23.740 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":118,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:22.389830+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/1496523549":"2024-05-09T04:43:19.038933+0000","172.21.15.164:6835/1496523549":"2024-05-09T04:43:19.038933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:23.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 msgr2=0x7f0034069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 0x7f0034069f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f00440ae000 tx=0x7f00440b4040 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 msgr2=0x7f0054160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f004000c750 tx=0x7f004000cb50 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 shutdown_connections 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0034067aa0 0x7f0034069f60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005415ab60 0x7f00541605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00540cfda0 0x7f0054160080 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 --2- 172.21.15.164:0/1341191933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00540b26c0 0x7f0054148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 >> 172.21.15.164:0/1341191933 conn(0x7f005406ac90 msgr2=0x7f00540cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 shutdown_connections 2024-05-08T04:43:23.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.739+0000 7f005ba00640 1 -- 172.21.15.164:0/1341191933 wait complete. 2024-05-08T04:43:23.760 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:23.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:23.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 Processor -- start 2024-05-08T04:43:23.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 -- start start 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 0x7f8bd406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd40aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8bd40ce780 0x7f8bd40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40cf130 con 0x7f8bd40ce780 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd406bb80 con 0x7f8bd415ab10 2024-05-08T04:43:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bdaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd415e870 con 0x7f8bd40ae6e0 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8bd40ce780 0x7f8bd40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8bd40ce780 0x7f8bd40cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59026/0 (socket says 172.21.15.164:59026) 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 -- 172.21.15.164:0/4243006536 learned_addr learned my addr 172.21.15.164:0/4243006536 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd3e00640 1 --2- 172.21.15.164:0/4243006536 >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 0x7f8bd406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bcbe00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd40aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 -- 172.21.15.164:0/4243006536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 msgr2=0x7f8bd40aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd40aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 -- 172.21.15.164:0/4243006536 >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 msgr2=0x7f8bd406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 --2- 172.21.15.164:0/4243006536 >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 0x7f8bd406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd8a00640 1 -- 172.21.15.164:0/4243006536 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd415eb50 con 0x7f8bd40ce780 2024-05-08T04:43:23.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.899+0000 7f8bd3e00640 1 --2- 172.21.15.164:0/4243006536 >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 0x7f8bd406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bd8a00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40cebf0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40c4920 tx=0x7f8bcc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8257889a787677 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bd3400640 1 -- 172.21.15.164:0/4243006536 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0a57b0 con 0x7f8bd40ce780 2024-05-08T04:43:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bd3400640 1 -- 172.21.15.164:0/4243006536 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0a5950 con 0x7f8bd40ce780 2024-05-08T04:43:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bd3400640 1 -- 172.21.15.164:0/4243006536 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0a5d20 con 0x7f8bd40ce780 2024-05-08T04:43:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 -- 172.21.15.164:0/4243006536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 msgr2=0x7f8bd40cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40cebf0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40c4920 tx=0x7f8bcc01c930 comp rx=0 tx=0).stop 2024-05-08T04:43:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 -- 172.21.15.164:0/4243006536 shutdown_connections 2024-05-08T04:43:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40cebf0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/4243006536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd40aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/4243006536 >> v2:172.21.15.172:3300/0 conn(0x7f8bd415ab10 0x7f8bd406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 -- 172.21.15.164:0/4243006536 >> 172.21.15.164:0/4243006536 conn(0x7f8bd40cf770 msgr2=0x7f8bd40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 -- 172.21.15.164:0/4243006536 shutdown_connections 2024-05-08T04:43:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.903+0000 7f8bdaa00640 1 -- 172.21.15.164:0/4243006536 wait complete. 2024-05-08T04:43:23.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 Processor -- start 2024-05-08T04:43:23.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- start start 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd4159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 0x7f8bd41535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40c5b70 con 0x7f8bd40ce780 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd415aef0 con 0x7f8bd415ab10 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40c5cf0 con 0x7f8bd40ae6e0 2024-05-08T04:43:23.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59028/0 (socket says 172.21.15.164:59028) 2024-05-08T04:43:23.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 -- 172.21.15.164:0/3527559685 learned_addr learned my addr 172.21.15.164:0/3527559685 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:23.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd8a00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 0x7f8bd41535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 -- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 msgr2=0x7f8bd4159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd4159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 -- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 msgr2=0x7f8bd41535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:23.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 0x7f8bd41535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:23.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 -- 172.21.15.164:0/3527559685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd41581b0 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd3e00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd4159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd8a00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 0x7f8bd41535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bcbe00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00d830 tx=0x7f8bbc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00b6c0 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc019070 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4158490 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00ba60 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd4158a50 con 0x7f8bd40ce780 2024-05-08T04:43:23.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.907+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4072d30 con 0x7f8bd40ce780 2024-05-08T04:43:23.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.911+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f8bbc01e460 con 0x7f8bd40ce780 2024-05-08T04:43:23.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.911+0000 7f8bd0c00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 0x7f8bb0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:23.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.911+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f8bbc083f80 con 0x7f8bd40ce780 2024-05-08T04:43:23.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.911+0000 7f8bd3e00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 0x7f8bb0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:23.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.911+0000 7f8bd3e00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 0x7f8bb0069f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006e90 tx=0x7f8bc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:23.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:23.915+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bbc056390 con 0x7f8bd40ce780 2024-05-08T04:43:24.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.067+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bd404f010 con 0x7f8bd40ce780 2024-05-08T04:43:24.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.067+0000 7f8bd0c00640 1 -- 172.21.15.164:0/3527559685 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bbc05a430 con 0x7f8bd40ce780 2024-05-08T04:43:24.069 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:24.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 msgr2=0x7f8bb0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 0x7f8bb0069f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006e90 tx=0x7f8bc4008040 comp rx=0 tx=0).stop 2024-05-08T04:43:24.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 msgr2=0x7f8bd40715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00d830 tx=0x7f8bbc00dcf0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 shutdown_connections 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8bb0067aa0 0x7f8bb0069f60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd415ab10 0x7f8bd41535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd40ce780 0x7f8bd40715d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 --2- 172.21.15.164:0/3527559685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40ae6e0 0x7f8bd4159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 >> 172.21.15.164:0/3527559685 conn(0x7f8bd40cf770 msgr2=0x7f8bd40ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 shutdown_connections 2024-05-08T04:43:24.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.071+0000 7f8bdaa00640 1 -- 172.21.15.164:0/3527559685 wait complete. 2024-05-08T04:43:24.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume' 2024-05-08T04:43:24.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 Processor -- start 2024-05-08T04:43:24.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 -- start start 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f2174160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740cd9f0 con 0x7f217415ba70 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740cdcd0 con 0x7f21740cfda0 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2179400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217415fe70 con 0x7f217415be50 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2172800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f217415ba70 0x7f21740cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59702/0 (socket says 172.21.15.164:59702) 2024-05-08T04:43:24.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2172800640 1 -- 172.21.15.164:0/3638912582 learned_addr learned my addr 172.21.15.164:0/3638912582 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:24.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2173200640 1 --2- 172.21.15.164:0/3638912582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f2174160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488708373 0 0) 0x7f21740cd9f0 con 0x7f217415ba70 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2158003530 con 0x7f217415ba70 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718351314 0 0) 0x7f21740cdcd0 con 0x7f21740cfda0 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2173200640 1 -- 172.21.15.164:0/3638912582 >> v1:172.21.15.172:6789/0 conn(0x7f21740cfda0 legacy=0x7f21740b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2173200640 1 -- 172.21.15.164:0/3638912582 >> v1:172.21.15.164:6789/0 conn(0x7f217415ba70 legacy=0x7f21740cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.227+0000 7f2173200640 1 -- 172.21.15.164:0/3638912582 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217415f820 con 0x7f217415be50 2024-05-08T04:43:24.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2173200640 1 --2- 172.21.15.164:0/3638912582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f2174160300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f217415fff0 tx=0x7f216c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3552fdcee25916 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:24.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0b2070 con 0x7f217415be50 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0a5930 con 0x7f217415be50 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2171e00640 1 -- 172.21.15.164:0/3638912582 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0a5d00 con 0x7f217415be50 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 -- 172.21.15.164:0/3638912582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 msgr2=0x7f2174160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 --2- 172.21.15.164:0/3638912582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f2174160300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f217415fff0 tx=0x7f216c01c960 comp rx=0 tx=0).stop 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 -- 172.21.15.164:0/3638912582 shutdown_connections 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 --2- 172.21.15.164:0/3638912582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f2174160300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 -- 172.21.15.164:0/3638912582 >> 172.21.15.164:0/3638912582 conn(0x7f217406ac90 msgr2=0x7f217406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 -- 172.21.15.164:0/3638912582 shutdown_connections 2024-05-08T04:43:24.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.231+0000 7f2179400640 1 -- 172.21.15.164:0/3638912582 wait complete. 2024-05-08T04:43:24.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 Processor -- start 2024-05-08T04:43:24.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- start start 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f217415ba70 0x7f21740ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f21740cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174169f60 con 0x7f21740cfda0 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174169de0 con 0x7f217415ba70 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217416a0e0 con 0x7f217415be50 2024-05-08T04:43:24.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59038/0 (socket says 172.21.15.164:59038) 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 -- 172.21.15.164:0/1492188261 learned_addr learned my addr 172.21.15.164:0/1492188261 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2169e00640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f217415ba70 0x7f21740ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2173200640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f21740cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 -- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 msgr2=0x7f21740cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f21740cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 -- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f217415ba70 msgr2=0x7f21740ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f217415ba70 0x7f21740ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 -- 172.21.15.164:0/1492188261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217414d780 con 0x7f21740cfda0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2172800640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f216400c480 tx=0x7f216400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164017070 con 0x7f21740cfda0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164005cf0 con 0x7f21740cfda0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740b05e0 con 0x7f21740cfda0 2024-05-08T04:43:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164007290 con 0x7f21740cfda0 2024-05-08T04:43:24.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21740b0ba0 con 0x7f21740cfda0 2024-05-08T04:43:24.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.235+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2174069d60 con 0x7f21740cfda0 2024-05-08T04:43:24.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.239+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f216401a070 con 0x7f21740cfda0 2024-05-08T04:43:24.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.239+0000 7f216b400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 0x7f2150069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:24.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.239+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f21640830b0 con 0x7f21740cfda0 2024-05-08T04:43:24.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.239+0000 7f2169e00640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 0x7f2150069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:24.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.239+0000 7f2169e00640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 0x7f2150069f60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f217415f2a0 tx=0x7f215c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.243+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21640558a0 con 0x7f21740cfda0 2024-05-08T04:43:24.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:24.451+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume"]} v 0) v1 -- 0x7f217406e210 con 0x7f21740cfda0 2024-05-08T04:43:25.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f216b400640 1 -- 172.21.15.164:0/1492188261 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume"]}]=0 v308) v1 ==== 180+0+0 (secure 0 0 0) 0x7f21640551a0 con 0x7f21740cfda0 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 msgr2=0x7f2150069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 0x7f2150069f60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f217415f2a0 tx=0x7f215c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 msgr2=0x7f21741533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f216400c480 tx=0x7f216400c940 comp rx=0 tx=0).stop 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 shutdown_connections 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2150067aa0 0x7f2150069f60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f217415be50 0x7f21740cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f217415ba70 0x7f21740ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 --2- 172.21.15.164:0/1492188261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21740cfda0 0x7f21741533b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.403+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 >> 172.21.15.164:0/1492188261 conn(0x7f217406ac90 msgr2=0x7f217406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:25.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.407+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 shutdown_connections 2024-05-08T04:43:25.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.407+0000 7f2179400640 1 -- 172.21.15.164:0/1492188261 wait complete. 2024-05-08T04:43:25.431 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:43:25.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.571+0000 7f0b2f400640 1 Processor -- start 2024-05-08T04:43:25.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.571+0000 7f0b2f400640 1 -- start start 2024-05-08T04:43:25.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b2815ab10 0x7f0b2806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b280aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2f400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2815e870 con 0x7f0b280ae6e0 2024-05-08T04:43:25.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2f400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2806bb80 con 0x7f0b2815ab10 2024-05-08T04:43:25.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815eb50 con 0x7f0b280cee90 2024-05-08T04:43:25.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0b280ae6e0 0x7f0b280ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59718/0 (socket says 172.21.15.164:59718) 2024-05-08T04:43:25.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 -- 172.21.15.164:0/1620578248 learned_addr learned my addr 172.21.15.164:0/1620578248 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:25.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2ca00640 1 --2- 172.21.15.164:0/1620578248 >> v2:172.21.15.172:3300/0 conn(0x7f0b2815ab10 0x7f0b2806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2d400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b280aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794572721 0 0) 0x7f0b2815e870 con 0x7f0b280ae6e0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c003510 con 0x7f0b280ae6e0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686913799 0 0) 0x7f0b0c003510 con 0x7f0b280ae6e0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 msgr2=0x7f0b280aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b280aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 >> v2:172.21.15.172:3300/0 conn(0x7f0b2815ab10 msgr2=0x7f0b2806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 --2- 172.21.15.164:0/1620578248 >> v2:172.21.15.172:3300/0 conn(0x7f0b2815ab10 0x7f0b2806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2815ee30 con 0x7f0b280ae6e0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f0b10001680 con 0x7f0b280ae6e0 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 >> v1:172.21.15.164:6789/0 conn(0x7f0b280ae6e0 legacy=0x7f0b280ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b0c004b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b0c003790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c003d40 0x7f0b0c0041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2806bb80 con 0x7f0b0c005100 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815eb50 con 0x7f0b0c003d40 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c00ba10 con 0x7f0b280cee90 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b0c003790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2ca00640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b0c004b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2d400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c003d40 0x7f0b0c0041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 -- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 msgr2=0x7f0b0c004b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b280cee90 0x7f0b0c004b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 -- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c003d40 msgr2=0x7f0b0c0041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:25.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c003d40 0x7f0b0c0041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b27e00640 1 -- 172.21.15.164:0/1620578248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c003510 con 0x7f0b0c005100 2024-05-08T04:43:25.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.575+0000 7f0b2d400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b0c003d40 0x7f0b0c0041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b0c003790 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c240 tx=0x7f0b1001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96123172b0cab2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007d60 con 0x7f0b0c005100 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100a9020 con 0x7f0b0c005100 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b27400640 1 -- 172.21.15.164:0/1620578248 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100a5ba0 con 0x7f0b0c005100 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 -- 172.21.15.164:0/1620578248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 msgr2=0x7f0b0c003790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1620578248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b0c003790 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c240 tx=0x7f0b1001cc40 comp rx=0 tx=0).stop 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2d400640 1 -- 172.21.15.164:0/1620578248 reap_dead start 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 -- 172.21.15.164:0/1620578248 shutdown_connections 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 -- 172.21.15.164:0/1620578248 >> 172.21.15.164:0/1620578248 conn(0x7f0b280cf770 msgr2=0x7f0b280cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 -- 172.21.15.164:0/1620578248 shutdown_connections 2024-05-08T04:43:25.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.579+0000 7f0b2f400640 1 -- 172.21.15.164:0/1620578248 wait complete. 2024-05-08T04:43:25.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 Processor -- start 2024-05-08T04:43:25.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- start start 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 0x7f0b28152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 0x7f0b28141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280caef0 con 0x7f0b0c005100 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815aef0 con 0x7f0b28151150 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280cb070 con 0x7f0b281530b0 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59060/0 (socket says 172.21.15.164:59060) 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 -- 172.21.15.164:0/1907443584 learned_addr learned my addr 172.21.15.164:0/1907443584 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 0x7f0b28152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2d400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 0x7f0b28141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 -- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 msgr2=0x7f0b28141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 0x7f0b28141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 -- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 msgr2=0x7f0b28152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 0x7f0b28152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 -- 172.21.15.164:0/1907443584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28153770 con 0x7f0b0c005100 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 0x7f0b28152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2d400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 0x7f0b28141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2ca00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0b18015d30 tx=0x7f0b18013580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801d020 con 0x7f0b0c005100 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18013e90 con 0x7f0b0c005100 2024-05-08T04:43:25.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28153a50 con 0x7f0b0c005100 2024-05-08T04:43:25.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004250 con 0x7f0b0c005100 2024-05-08T04:43:25.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b280b2af0 con 0x7f0b0c005100 2024-05-08T04:43:25.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.583+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b28072d30 con 0x7f0b0c005100 2024-05-08T04:43:25.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.587+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f0b18020020 con 0x7f0b0c005100 2024-05-08T04:43:25.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.587+0000 7f0b24c00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 0x7f0b04069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:25.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.587+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0b18088640 con 0x7f0b0c005100 2024-05-08T04:43:25.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.587+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 0x7f0b04069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:25.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.587+0000 7f0b27e00640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 0x7f0b04069f60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0b100aa000 tx=0x7f0b10001ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:25.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.591+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b18027020 con 0x7f0b0c005100 2024-05-08T04:43:25.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.747+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b281534c0 con 0x7f0b0c005100 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b24c00640 1 -- 172.21.15.164:0/1907443584 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b18014040 con 0x7f0b0c005100 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 msgr2=0x7f0b04069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 0x7f0b04069f60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0b100aa000 tx=0x7f0b10001ac0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 msgr2=0x7f0b28150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0b18015d30 tx=0x7f0b18013580 comp rx=0 tx=0).stop 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 shutdown_connections 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0b04067aa0 0x7f0b04069f60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b281530b0 0x7f0b28141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b28151150 0x7f0b28152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 --2- 172.21.15.164:0/1907443584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b0c005100 0x7f0b28150c10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:25.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 >> 172.21.15.164:0/1907443584 conn(0x7f0b280cf770 msgr2=0x7f0b2806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:25.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 shutdown_connections 2024-05-08T04:43:25.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:25.751+0000 7f0b2f400640 1 -- 172.21.15.164:0/1907443584 wait complete. 2024-05-08T04:43:25.772 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:43:25.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:25.834 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:25.912 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:43:25.912 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:25.912 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:25.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:25.921 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:25.921 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:25.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:43:25.932 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:25.932 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:25.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:25.971 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:25.971 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:25.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:43:25.986 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T04:43:25.986 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 Processor -- start 2024-05-08T04:43:26.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 -- start start 2024-05-08T04:43:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6480b26c0 0x7fa6480cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64806c770 con 0x7fa6480cfbd0 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64806ca50 con 0x7fa64815ab60 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa64dc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6480d0060 con 0x7fa6480b26c0 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa646000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa6480b26c0 0x7fa6480cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa647400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa6480cfbd0 0x7fa6480cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59720/0 (socket says 172.21.15.164:59720) 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.147+0000 7fa647400640 1 -- 172.21.15.164:0/1432453882 learned_addr learned my addr 172.21.15.164:0/1432453882 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:26.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa645600640 1 -- 172.21.15.164:0/1432453882 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569895361 0 0) 0x7fa64806c770 con 0x7fa6480cfbd0 2024-05-08T04:43:26.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa646000640 1 -- 172.21.15.164:0/1432453882 >> v1:172.21.15.172:6789/0 conn(0x7fa64815ab60 legacy=0x7fa648072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa646000640 1 -- 172.21.15.164:0/1432453882 >> v1:172.21.15.164:6789/0 conn(0x7fa6480cfbd0 legacy=0x7fa6480cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa646000640 1 -- 172.21.15.164:0/1432453882 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64806cd30 con 0x7fa6480b26c0 2024-05-08T04:43:26.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa646000640 1 --2- 172.21.15.164:0/1432453882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6480b26c0 0x7fa6480cf690 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097c0 tx=0x7fa63001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b685bb1933bc1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa645600640 1 -- 172.21.15.164:0/1432453882 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300b0070 con 0x7fa6480b26c0 2024-05-08T04:43:26.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa645600640 1 -- 172.21.15.164:0/1432453882 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6300ab600 con 0x7fa6480b26c0 2024-05-08T04:43:26.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa645600640 1 -- 172.21.15.164:0/1432453882 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300ab9d0 con 0x7fa6480b26c0 2024-05-08T04:43:26.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa64dc00640 1 -- 172.21.15.164:0/1432453882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6480b26c0 msgr2=0x7fa6480cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.151+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1432453882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6480b26c0 0x7fa6480cf690 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097c0 tx=0x7fa63001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- 172.21.15.164:0/1432453882 shutdown_connections 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1432453882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6480b26c0 0x7fa6480cf690 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- 172.21.15.164:0/1432453882 >> 172.21.15.164:0/1432453882 conn(0x7fa64806ac90 msgr2=0x7fa64806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- 172.21.15.164:0/1432453882 shutdown_connections 2024-05-08T04:43:26.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- 172.21.15.164:0/1432453882 wait complete. 2024-05-08T04:43:26.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 Processor -- start 2024-05-08T04:43:26.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- start start 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6480b26c0 0x7fa64813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 0x7fa648150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480aeeb0 con 0x7fa64815ab60 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480aed30 con 0x7fa6480cfbd0 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa64dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa648168980 con 0x7fa6480b26c0 2024-05-08T04:43:26.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59088/0 (socket says 172.21.15.164:59088) 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 -- 172.21.15.164:0/1608248048 learned_addr learned my addr 172.21.15.164:0/1608248048 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa646000640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 0x7fa648150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa646a00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6480b26c0 0x7fa64813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 -- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6480b26c0 msgr2=0x7fa64813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6480b26c0 0x7fa64813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 -- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 msgr2=0x7fa648150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 0x7fa648150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 -- 172.21.15.164:0/1608248048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648152ae0 con 0x7fa64815ab60 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa646000640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 0x7fa648150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa647400640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa640001570 tx=0x7fa64000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640018070 con 0x7fa64815ab60 2024-05-08T04:43:26.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.155+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64000a640 con 0x7fa64815ab60 2024-05-08T04:43:26.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64000aa30 con 0x7fa64815ab60 2024-05-08T04:43:26.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa648151380 con 0x7fa64815ab60 2024-05-08T04:43:26.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa648151620 con 0x7fa64815ab60 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa648069d60 con 0x7fa64815ab60 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fa64001b070 con 0x7fa64815ab60 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa63ea00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 0x7fa624069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fa6400832c0 con 0x7fa64815ab60 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa646a00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 0x7fa624069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.159+0000 7fa646a00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 0x7fa624069f60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa638000f70 tx=0x7fa638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.163+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa640055b30 con 0x7fa64815ab60 2024-05-08T04:43:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.351+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6480741e0 con 0x7fa64815ab60 2024-05-08T04:43:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.351+0000 7fa63ea00640 1 -- 172.21.15.164:0/1608248048 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fa640055430 con 0x7fa64815ab60 2024-05-08T04:43:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:26.354 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6701,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1897246782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1897246782},{"type":"v1","addr":"172.21.15.172:6835","nonce":1897246782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/208572957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":208572957},{"type":"v1","addr":"172.21.15.172:6833","nonce":208572957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/1476310273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1476310273},{"type":"v1","addr":"172.21.15.164:6836","nonce":1476310273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8316,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2980038264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2980038264},{"type":"v1","addr":"172.21.15.164:6837","nonce":2980038264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[]} 2024-05-08T04:43:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 75 2024-05-08T04:43:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 msgr2=0x7fa624069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 0x7fa624069f60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa638000f70 tx=0x7fa638009040 comp rx=0 tx=0).stop 2024-05-08T04:43:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 msgr2=0x7fa64813b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa640001570 tx=0x7fa64000cca0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 shutdown_connections 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa624067aa0 0x7fa624069f60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64815ab60 0x7fa64813b420 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6480cfbd0 0x7fa648150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 --2- 172.21.15.164:0/1608248048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6480b26c0 0x7fa64813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 >> 172.21.15.164:0/1608248048 conn(0x7fa64806ac90 msgr2=0x7fa6481543c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 shutdown_connections 2024-05-08T04:43:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.355+0000 7fa64dc00640 1 -- 172.21.15.164:0/1608248048 wait complete. 2024-05-08T04:43:26.375 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T04:43:26.375 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T04:43:26.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:43:26.375 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:43:26.376 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T04:43:26.376 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T04:43:26.376 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:43:26.376 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:43:26.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T04:43:26.376 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T04:43:26.376 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:43:26.377 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:43:26.377 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T04:43:26.377 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T04:43:26.377 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:43:26.377 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:43:26.389 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:43:26.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T04:43:26.390 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:43:26.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T04:43:26.392 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:43:26.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T04:43:26.393 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:43:26.393 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T04:43:26.436 INFO:tasks.ceph.mds.b:Started 2024-05-08T04:43:26.436 DEBUG:teuthology.parallel:result is None 2024-05-08T04:43:26.437 INFO:tasks.ceph.mds.a:Started 2024-05-08T04:43:26.437 DEBUG:teuthology.parallel:result is None 2024-05-08T04:43:26.437 INFO:tasks.ceph.mds.d:Started 2024-05-08T04:43:26.438 DEBUG:teuthology.parallel:result is None 2024-05-08T04:43:26.438 INFO:tasks.ceph.mds.c:Started 2024-05-08T04:43:26.439 DEBUG:teuthology.parallel:result is None 2024-05-08T04:43:26.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T04:43:26.523 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-05-08T04:43:26.524 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:43:26.525 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-05-08T04:43:26.526 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-05-08T04:43:26.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 Processor -- start 2024-05-08T04:43:26.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 -- start start 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e6815ba70 0x7f1e68160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e680cd9f0 con 0x7f1e6815be50 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6815fe70 con 0x7f1e6815ba70 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e6d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680cdcd0 con 0x7f1e680cfda0 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e66a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e6815ba70 0x7f1e68160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42046/0 (socket says 172.21.15.164:42046) 2024-05-08T04:43:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 -- 172.21.15.164:0/825425598 learned_addr learned my addr 172.21.15.164:0/825425598 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 -- 172.21.15.164:0/825425598 >> v2:172.21.15.164:3300/0 conn(0x7f1e6815ba70 msgr2=0x7f1e68160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 --2- 172.21.15.164:0/825425598 >> v2:172.21.15.164:3300/0 conn(0x7f1e6815ba70 0x7f1e68160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 -- 172.21.15.164:0/825425598 >> v1:172.21.15.172:6789/0 conn(0x7f1e6815be50 legacy=0x7f1e680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e66000640 1 -- 172.21.15.164:0/825425598 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270494396 0 0) 0x7f1e680cd9f0 con 0x7f1e6815be50 2024-05-08T04:43:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.583+0000 7f1e67400640 1 -- 172.21.15.164:0/825425598 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6815f820 con 0x7f1e680cfda0 2024-05-08T04:43:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e67400640 1 --2- 172.21.15.164:0/825425598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009870 tx=0x7f1e6001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38afa37ac36df000 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e66000640 1 -- 172.21.15.164:0/825425598 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600ab640 con 0x7f1e680cfda0 2024-05-08T04:43:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e66000640 1 -- 172.21.15.164:0/825425598 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e600b0070 con 0x7f1e680cfda0 2024-05-08T04:43:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e66000640 1 -- 172.21.15.164:0/825425598 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600aba00 con 0x7f1e680cfda0 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 -- 172.21.15.164:0/825425598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 msgr2=0x7f1e680b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 --2- 172.21.15.164:0/825425598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009870 tx=0x7f1e6001c530 comp rx=0 tx=0).stop 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 -- 172.21.15.164:0/825425598 shutdown_connections 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 --2- 172.21.15.164:0/825425598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e680b31a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 --2- 172.21.15.164:0/825425598 >> v2:172.21.15.164:3300/0 conn(0x7f1e6815ba70 0x7f1e68160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 -- 172.21.15.164:0/825425598 >> 172.21.15.164:0/825425598 conn(0x7f1e6806ac90 msgr2=0x7f1e6806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 -- 172.21.15.164:0/825425598 shutdown_connections 2024-05-08T04:43:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.587+0000 7f1e6d800640 1 -- 172.21.15.164:0/825425598 wait complete. 2024-05-08T04:43:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 Processor -- start 2024-05-08T04:43:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- start start 2024-05-08T04:43:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e6814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e6815be50 0x7f1e6814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68168d40 con 0x7f1e6815ba70 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68168bc0 con 0x7f1e6815be50 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68168ec0 con 0x7f1e680cfda0 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59114/0 (socket says 172.21.15.164:59114) 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 -- 172.21.15.164:0/791916277 learned_addr learned my addr 172.21.15.164:0/791916277 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e66a00640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e6814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e67400640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e6815be50 0x7f1e6814a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 -- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 msgr2=0x7f1e6814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e6814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 -- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e6815be50 msgr2=0x7f1e6814a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e6815be50 0x7f1e6814a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 -- 172.21.15.164:0/791916277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680710a0 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5e000640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000c450 tx=0x7f1e5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50017070 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50005480 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50003840 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e68071380 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e68071940 con 0x7f1e6815ba70 2024-05-08T04:43:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.591+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e68069d60 con 0x7f1e6815ba70 2024-05-08T04:43:26.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.595+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f1e5001a070 con 0x7f1e6815ba70 2024-05-08T04:43:26.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.595+0000 7f1e5f400640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 0x7f1e44069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:26.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.595+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f1e50082e90 con 0x7f1e6815ba70 2024-05-08T04:43:26.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.595+0000 7f1e66a00640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 0x7f1e44069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:26.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.595+0000 7f1e66a00640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 0x7f1e44069f60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e580113e0 tx=0x7f1e5800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:26.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.599+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e50055050 con 0x7f1e6815ba70 2024-05-08T04:43:26.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:26.763+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1e6806e210 con 0x7f1e6815ba70 2024-05-08T04:43:27.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.423+0000 7f1e5f400640 1 -- 172.21.15.164:0/791916277 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1e500590f0 con 0x7f1e6815ba70 2024-05-08T04:43:27.424 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-05-08T04:43:27.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 msgr2=0x7f1e44069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 0x7f1e44069f60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e580113e0 tx=0x7f1e5800b040 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 msgr2=0x7f1e680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000c450 tx=0x7f1e5000c910 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 shutdown_connections 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1e44067aa0 0x7f1e44069f60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e6815be50 0x7f1e6814a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ba70 0x7f1e680706c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 --2- 172.21.15.164:0/791916277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e680cfda0 0x7f1e6814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 >> 172.21.15.164:0/791916277 conn(0x7f1e6806ac90 msgr2=0x7f1e680b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 shutdown_connections 2024-05-08T04:43:27.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.427+0000 7f1e6d800640 1 -- 172.21.15.164:0/791916277 wait complete. 2024-05-08T04:43:27.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T04:43:27.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 Processor -- start 2024-05-08T04:43:27.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 -- start start 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe68015bb60 0x7fe68015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6801603d0 con 0x7fe6800cf850 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800cdf60 con 0x7fe6800cd2d0 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe685000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68015d000 con 0x7fe68015bb60 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe68015bb60 0x7fe68015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67ea00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe6800cf850 0x7fe68015f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59736/0 (socket says 172.21.15.164:59736) 2024-05-08T04:43:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67ea00640 1 -- 172.21.15.164:0/390971881 learned_addr learned my addr 172.21.15.164:0/390971881 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67f400640 1 -- 172.21.15.164:0/390971881 >> v1:172.21.15.172:6789/0 conn(0x7fe6800cd2d0 legacy=0x7fe68015fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67f400640 1 -- 172.21.15.164:0/390971881 >> v1:172.21.15.164:6789/0 conn(0x7fe6800cf850 legacy=0x7fe68015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.591+0000 7fe67f400640 1 -- 172.21.15.164:0/390971881 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6800ce240 con 0x7fe68015bb60 2024-05-08T04:43:27.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe67f400640 1 --2- 172.21.15.164:0/390971881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe68015bb60 0x7fe68015cac0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe6700097c0 tx=0x7fe67001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ed7d4569c233b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe67e000640 1 -- 172.21.15.164:0/390971881 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b0070 con 0x7fe68015bb60 2024-05-08T04:43:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe67e000640 1 -- 172.21.15.164:0/390971881 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670004230 con 0x7fe68015bb60 2024-05-08T04:43:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe67e000640 1 -- 172.21.15.164:0/390971881 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670003620 con 0x7fe68015bb60 2024-05-08T04:43:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- 172.21.15.164:0/390971881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe68015bb60 msgr2=0x7fe68015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 --2- 172.21.15.164:0/390971881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe68015bb60 0x7fe68015cac0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe6700097c0 tx=0x7fe67001c460 comp rx=0 tx=0).stop 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- 172.21.15.164:0/390971881 shutdown_connections 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 --2- 172.21.15.164:0/390971881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe68015bb60 0x7fe68015cac0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- 172.21.15.164:0/390971881 >> 172.21.15.164:0/390971881 conn(0x7fe6800b32e0 msgr2=0x7fe6800b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- 172.21.15.164:0/390971881 shutdown_connections 2024-05-08T04:43:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- 172.21.15.164:0/390971881 wait complete. 2024-05-08T04:43:27.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 Processor -- start 2024-05-08T04:43:27.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.595+0000 7fe685000640 1 -- start start 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6800cf850 0x7fe680152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe68015bb60 0x7fe680152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800d0070 con 0x7fe6800cd2d0 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800cedc0 con 0x7fe6800cf850 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68006c6d0 con 0x7fe68015bb60 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59118/0 (socket says 172.21.15.164:59118) 2024-05-08T04:43:27.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 -- 172.21.15.164:0/765788091 learned_addr learned my addr 172.21.15.164:0/765788091 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67ea00640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6800cf850 0x7fe680152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67fe00640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe68015bb60 0x7fe680152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 -- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe68015bb60 msgr2=0x7fe680152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe68015bb60 0x7fe680152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 -- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6800cf850 msgr2=0x7fe680152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6800cf850 0x7fe680152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 -- 172.21.15.164:0/765788091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680156580 con 0x7fe6800cd2d0 2024-05-08T04:43:27.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe67f400640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe670019070 tx=0x7fe670019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b9070 con 0x7fe6800cd2d0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700043e0 con 0x7fe6800cd2d0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b0040 con 0x7fe6800cd2d0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68006bda0 con 0x7fe6800cd2d0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68006c360 con 0x7fe6800cd2d0 2024-05-08T04:43:27.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68004ef30 con 0x7fe6800cd2d0 2024-05-08T04:43:27.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.599+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fe6700bc070 con 0x7fe6800cd2d0 2024-05-08T04:43:27.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.603+0000 7fe677400640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 0x7fe660069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.603+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fe67011dff0 con 0x7fe6800cd2d0 2024-05-08T04:43:27.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.603+0000 7fe67ea00640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 0x7fe660069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.603+0000 7fe67ea00640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 0x7fe660069f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe68006e9c0 tx=0x7fe66c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.607+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe670016050 con 0x7fe6800cd2d0 2024-05-08T04:43:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.811+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe6800cbec0 con 0x7fe6800cd2d0 2024-05-08T04:43:27.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.811+0000 7fe677400640 1 -- 172.21.15.164:0/765788091 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe6700f0920 con 0x7fe6800cd2d0 2024-05-08T04:43:27.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:27.813 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmPCXABBAA6ct16hdKvV7SDdniH3Q84A==", 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm422KEBAAdXU0cUal+6RGxjUSkKmF2A==", 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.815 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.c", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmyE/BCxAAK3+zz1WiLAOoQuTIfL34iQ==", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.d", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtml2apFxAAgocSr9BPwDTKtNtt3QndaA==", 2024-05-08T04:43:27.816 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCNATtm30EuGxAAttKsh7YMSyKuSLj4c37glQ==", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.817 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCQATtmYYMdJxAAqX6Er67o9cqfGKklPNK0CA==", 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCTATtmgY5bLxAApqw6Lxg5EBgYkS/mYYI+FQ==", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCWATtmgeELORAAAMv3UDk3V+VGj5lw5/v7Vw==", 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.819 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCcATtmUcK+MRAArk3TLvARtcx6/jLt+wBpjg==", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCgATtmeu7nARAAgkggsO7ykY1MEIrx0Zm0tA==", 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.821 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCjATtm8dCwDBAAVG4bftJdmpZEXe6EYLacAA==", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCmATtmTCMDFhAA2texpko2cd5leGmP61xAxw==", 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.822 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ==", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow r", 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T04:43:27.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.1", 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm44fvJBAAnUQkwwxtE93utDaNp10pxg==", 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.824 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.2", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtmxwRiIRAAbV8uyGHJ+5kDXpRGnXP9Kg==", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.3", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCKATtm65PbKhAAR3dQmpW8S2uMNEKBBEM8yQ==", 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.825 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmEz82JBAAAcrwCzRZ7BPnYO+NjLc2qA==", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-05-08T04:43:27.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmko6GOxAAuzzfRh7JSl10eNKNtT+J3Q==", 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCJATtmem/JNxAAErIef4+OFPMdkhCjBsq1QQ==", 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-08T04:43:27.828 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:27.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 msgr2=0x7fe660069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 0x7fe660069f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe68006e9c0 tx=0x7fe66c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 msgr2=0x7fe680150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe670019070 tx=0x7fe670019200 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 shutdown_connections 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe660067ad0 0x7fe660069f90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe68015bb60 0x7fe680152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6800cf850 0x7fe680152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 --2- 172.21.15.164:0/765788091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6800cd2d0 0x7fe680150c10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 >> 172.21.15.164:0/765788091 conn(0x7fe6800b32e0 msgr2=0x7fe68006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:27.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 shutdown_connections 2024-05-08T04:43:27.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.815+0000 7fe685000640 1 -- 172.21.15.164:0/765788091 wait complete. 2024-05-08T04:43:27.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:27.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.967+0000 7fc656a00640 1 Processor -- start 2024-05-08T04:43:27.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 -- start start 2024-05-08T04:43:27.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6500c4420 0x7fc65015f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc65015fb60 0x7fc65015d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500c49e0 con 0x7fc650159760 2024-05-08T04:43:27.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65015ffd0 con 0x7fc6500c4420 2024-05-08T04:43:27.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500c4cc0 con 0x7fc65015fb60 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc654c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6500c4420 0x7fc65015f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc655600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc650159760 0x7fc6500c5e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59750/0 (socket says 172.21.15.164:59750) 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc655600640 1 -- 172.21.15.164:0/3953101468 learned_addr learned my addr 172.21.15.164:0/3953101468 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc656000640 1 --2- 172.21.15.164:0/3953101468 >> v2:172.21.15.172:3300/0 conn(0x7fc65015fb60 0x7fc65015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc654c00640 1 -- 172.21.15.164:0/3953101468 >> v2:172.21.15.172:3300/0 conn(0x7fc65015fb60 msgr2=0x7fc65015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc654c00640 1 --2- 172.21.15.164:0/3953101468 >> v2:172.21.15.172:3300/0 conn(0x7fc65015fb60 0x7fc65015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc654c00640 1 -- 172.21.15.164:0/3953101468 >> v1:172.21.15.164:6789/0 conn(0x7fc650159760 legacy=0x7fc6500c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc654c00640 1 -- 172.21.15.164:0/3953101468 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500c4fa0 con 0x7fc6500c4420 2024-05-08T04:43:27.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.971+0000 7fc64fe00640 1 -- 172.21.15.164:0/3953101468 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366212321 0 0) 0x7fc6500c49e0 con 0x7fc650159760 2024-05-08T04:43:27.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc654c00640 1 --2- 172.21.15.164:0/3953101468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6500c4420 0x7fc65015f620 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b1140 tx=0x7fc63c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f455339e3c5369 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc64fe00640 1 -- 172.21.15.164:0/3953101468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0b0070 con 0x7fc6500c4420 2024-05-08T04:43:27.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc64fe00640 1 -- 172.21.15.164:0/3953101468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c0ab600 con 0x7fc6500c4420 2024-05-08T04:43:27.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc64fe00640 1 -- 172.21.15.164:0/3953101468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0ab9d0 con 0x7fc6500c4420 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- 172.21.15.164:0/3953101468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6500c4420 msgr2=0x7fc65015f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 --2- 172.21.15.164:0/3953101468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6500c4420 0x7fc65015f620 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b1140 tx=0x7fc63c01c3f0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- 172.21.15.164:0/3953101468 shutdown_connections 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 --2- 172.21.15.164:0/3953101468 >> v2:172.21.15.172:3300/0 conn(0x7fc65015fb60 0x7fc65015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 --2- 172.21.15.164:0/3953101468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6500c4420 0x7fc65015f620 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- 172.21.15.164:0/3953101468 >> 172.21.15.164:0/3953101468 conn(0x7fc65006b670 msgr2=0x7fc6500b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- 172.21.15.164:0/3953101468 shutdown_connections 2024-05-08T04:43:27.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- 172.21.15.164:0/3953101468 wait complete. 2024-05-08T04:43:27.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 Processor -- start 2024-05-08T04:43:27.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.975+0000 7fc656a00640 1 -- start start 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6500c4420 0x7fc6500c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc65015fb60 0x7fc6501f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015e7f0 con 0x7fc650159760 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015e670 con 0x7fc65015fb60 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015e970 con 0x7fc6500c4420 2024-05-08T04:43:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59136/0 (socket says 172.21.15.164:59136) 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 -- 172.21.15.164:0/2380617862 learned_addr learned my addr 172.21.15.164:0/2380617862 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 -- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6500c4420 msgr2=0x7fc6500c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6500c4420 0x7fc6500c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 -- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc65015fb60 msgr2=0x7fc6501f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc65015fb60 0x7fc6501f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 -- 172.21.15.164:0/2380617862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6501f8f80 con 0x7fc650159760 2024-05-08T04:43:27.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc654c00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0ae040 tx=0x7fc63c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0b8020 con 0x7fc650159760 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c0b4410 con 0x7fc650159760 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0b0040 con 0x7fc650159760 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6500c8f50 con 0x7fc650159760 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6501f8240 con 0x7fc650159760 2024-05-08T04:43:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6500c9470 con 0x7fc650159760 2024-05-08T04:43:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.979+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fc63c0bb020 con 0x7fc650159760 2024-05-08T04:43:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.983+0000 7fc64d600640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 0x7fc630069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.983+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fc63c11d500 con 0x7fc650159760 2024-05-08T04:43:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.983+0000 7fc655600640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 0x7fc630069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.983+0000 7fc655600640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 0x7fc630069f90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc6440015a0 tx=0x7fc644007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:27.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:27.987+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc63c0c2030 con 0x7fc650159760 2024-05-08T04:43:28.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.139+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc65004ef30 con 0x7fc650159760 2024-05-08T04:43:28.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc64d600640 1 -- 172.21.15.164:0/2380617862 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13095 (secure 0 0 0) 0x7fc63c016070 con 0x7fc650159760 2024-05-08T04:43:28.143 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:28.143 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":122,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:27.433044+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 msgr2=0x7fc630069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 0x7fc630069f90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc6440015a0 tx=0x7fc644007970 comp rx=0 tx=0).stop 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 msgr2=0x7fc65014c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0ae040 tx=0x7fc63c019190 comp rx=0 tx=0).stop 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 shutdown_connections 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc630067ad0 0x7fc630069f90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc65015fb60 0x7fc6501f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc650159760 0x7fc65014c9d0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 --2- 172.21.15.164:0/2380617862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6500c4420 0x7fc6500c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 >> 172.21.15.164:0/2380617862 conn(0x7fc65006b670 msgr2=0x7fc65014f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 shutdown_connections 2024-05-08T04:43:28.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.143+0000 7fc656a00640 1 -- 172.21.15.164:0/2380617862 wait complete. 2024-05-08T04:43:28.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:28.165 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:28.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 Processor -- start 2024-05-08T04:43:28.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 -- start start 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbba40ae6e0 0x7fbba40aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40c6600 con 0x7fbba415ba20 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40c6040 con 0x7fbba415ce10 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbac600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba415fbe0 con 0x7fbba40ae6e0 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba9c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbba415ba20 0x7fbba4160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59766/0 (socket says 172.21.15.164:59766) 2024-05-08T04:43:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba9c00640 1 -- 172.21.15.164:0/1947048844 learned_addr learned my addr 172.21.15.164:0/1947048844 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbaa600640 1 --2- 172.21.15.164:0/1947048844 >> v2:172.21.15.172:3300/0 conn(0x7fbba40ae6e0 0x7fbba40aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710089642 0 0) 0x7fbba40c6600 con 0x7fbba415ba20 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb8c003530 con 0x7fbba415ba20 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148570054 0 0) 0x7fbb8c003530 con 0x7fbba415ba20 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 >> v2:172.21.15.172:3300/0 conn(0x7fbba40ae6e0 msgr2=0x7fbba40aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 --2- 172.21.15.164:0/1947048844 >> v2:172.21.15.172:3300/0 conn(0x7fbba40ae6e0 0x7fbba40aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 >> v1:172.21.15.172:6789/0 conn(0x7fbba415ce10 legacy=0x7fbba406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40c6320 con 0x7fbba415ba20 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fbb98001a00 con 0x7fbba415ba20 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 >> v1:172.21.15.164:6789/0 conn(0x7fbba415ba20 legacy=0x7fbba4160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 0x7fbb8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8c004810 0x7fbb8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0051d0 0x7fbb8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba415fbe0 con 0x7fbb8c0039c0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c00c070 con 0x7fbb8c004810 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c00c270 con 0x7fbb8c0051d0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 0x7fbb8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbba9200640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8c004810 0x7fbb8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.303+0000 7fbbaa600640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0051d0 0x7fbb8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 -- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0051d0 msgr2=0x7fbb8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0051d0 0x7fbb8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 -- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8c004810 msgr2=0x7fbb8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8c004810 0x7fbb8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 -- 172.21.15.164:0/1947048844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c003530 con 0x7fbb8c0039c0 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9200640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8c004810 0x7fbb8c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbaa600640 1 -- 172.21.15.164:0/1947048844 reap_dead start 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 0x7fbb8c003de0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800a840 tx=0x7fbb9801d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63199d12c0c98cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800a460 con 0x7fbb8c0039c0 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb980ad650 con 0x7fbb8c0039c0 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbba3e00640 1 -- 172.21.15.164:0/1947048844 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980ada80 con 0x7fbb8c0039c0 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 -- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 msgr2=0x7fbb8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 0x7fbb8c003de0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800a840 tx=0x7fbb9801d2b0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 -- 172.21.15.164:0/1947048844 shutdown_connections 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 --2- 172.21.15.164:0/1947048844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb8c0039c0 0x7fbb8c003de0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 -- 172.21.15.164:0/1947048844 >> 172.21.15.164:0/1947048844 conn(0x7fbba40cf770 msgr2=0x7fbba40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 -- 172.21.15.164:0/1947048844 shutdown_connections 2024-05-08T04:43:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.307+0000 7fbbac600640 1 -- 172.21.15.164:0/1947048844 wait complete. 2024-05-08T04:43:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 Processor -- start 2024-05-08T04:43:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- start start 2024-05-08T04:43:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 0x7fbba40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbba40c92f0 0x7fbba40c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40af050 con 0x7fbba41e9330 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40aeed0 con 0x7fbba40c92f0 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4160050 con 0x7fbb8c0039c0 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59162/0 (socket says 172.21.15.164:59162) 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 -- 172.21.15.164:0/1789435892 learned_addr learned my addr 172.21.15.164:0/1789435892 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbaa600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbba40c92f0 0x7fbba40c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 0x7fbba40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 -- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 msgr2=0x7fbba40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 0x7fbba40b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 -- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbba40c92f0 msgr2=0x7fbba40c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbba40c92f0 0x7fbba40c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 -- 172.21.15.164:0/1789435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba41e99c0 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 0x7fbba40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba9200640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e4e0 tx=0x7fbb9000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb900052e0 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90018070 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90005480 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba40ca4c0 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.311+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba40c9770 con 0x7fbba41e9330 2024-05-08T04:43:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba4072d30 con 0x7fbba41e9330 2024-05-08T04:43:28.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fbb90003820 con 0x7fbba41e9330 2024-05-08T04:43:28.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbba1600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 0x7fbb84069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fbb90083560 con 0x7fbba41e9330 2024-05-08T04:43:28.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 0x7fbb84069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.315+0000 7fbba9c00640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 0x7fbb84069f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbb980ad000 tx=0x7fbba40c6210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.319+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb90055fc0 con 0x7fbba41e9330 2024-05-08T04:43:28.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbba40c4e20 con 0x7fbba41e9330 2024-05-08T04:43:28.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbba1600640 1 -- 172.21.15.164:0/1789435892 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb900558c0 con 0x7fbba41e9330 2024-05-08T04:43:28.480 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 msgr2=0x7fbb84069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 0x7fbb84069f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbb980ad000 tx=0x7fbba40c6210 comp rx=0 tx=0).stop 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 msgr2=0x7fbba41f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e4e0 tx=0x7fbb9000e9a0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 shutdown_connections 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbb84067aa0 0x7fbb84069f60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbba40c92f0 0x7fbba40c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba41e9330 0x7fbba41f6ed0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 --2- 172.21.15.164:0/1789435892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb8c0039c0 0x7fbba40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 >> 172.21.15.164:0/1789435892 conn(0x7fbba40cf770 msgr2=0x7fbba40c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 shutdown_connections 2024-05-08T04:43:28.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.479+0000 7fbbac600640 1 -- 172.21.15.164:0/1789435892 wait complete. 2024-05-08T04:43:28.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T04:43:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 Processor -- start 2024-05-08T04:43:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 -- start start 2024-05-08T04:43:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f681415be50 0x7f68140cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140cd9f0 con 0x7f68140aeda0 2024-05-08T04:43:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681415fe70 con 0x7f681415be50 2024-05-08T04:43:28.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f681b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140cdcd0 con 0x7f681415ba70 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f6813e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f6813e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:42126/0 (socket says 172.21.15.164:42126) 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f6813e00640 1 -- 172.21.15.164:0/667342650 learned_addr learned my addr 172.21.15.164:0/667342650 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.635+0000 7f680b400640 1 --2- 172.21.15.164:0/667342650 >> v2:172.21.15.172:3300/0 conn(0x7f681415be50 0x7f68140cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813e00640 1 -- 172.21.15.164:0/667342650 >> v2:172.21.15.172:3300/0 conn(0x7f681415be50 msgr2=0x7f68140cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813e00640 1 --2- 172.21.15.164:0/667342650 >> v2:172.21.15.172:3300/0 conn(0x7f681415be50 0x7f68140cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813e00640 1 -- 172.21.15.164:0/667342650 >> v1:172.21.15.172:6789/0 conn(0x7f68140aeda0 legacy=0x7f68140b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813e00640 1 -- 172.21.15.164:0/667342650 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681415f820 con 0x7f681415ba70 2024-05-08T04:43:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813e00640 1 --2- 172.21.15.164:0/667342650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6804009870 tx=0x7f680401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b5f43daa9451939 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813400640 1 -- 172.21.15.164:0/667342650 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040ab640 con 0x7f681415ba70 2024-05-08T04:43:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813400640 1 -- 172.21.15.164:0/667342650 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68040b0070 con 0x7f681415ba70 2024-05-08T04:43:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f6813400640 1 -- 172.21.15.164:0/667342650 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040aba00 con 0x7f681415ba70 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 -- 172.21.15.164:0/667342650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 msgr2=0x7f6814160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 --2- 172.21.15.164:0/667342650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6804009870 tx=0x7f680401c530 comp rx=0 tx=0).stop 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 -- 172.21.15.164:0/667342650 shutdown_connections 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 --2- 172.21.15.164:0/667342650 >> v2:172.21.15.172:3300/0 conn(0x7f681415be50 0x7f68140cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 --2- 172.21.15.164:0/667342650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415ba70 0x7f6814160300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.639+0000 7f681b200640 1 -- 172.21.15.164:0/667342650 >> 172.21.15.164:0/667342650 conn(0x7f681406ac90 msgr2=0x7f681406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- 172.21.15.164:0/667342650 shutdown_connections 2024-05-08T04:43:28.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- 172.21.15.164:0/667342650 wait complete. 2024-05-08T04:43:28.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 Processor -- start 2024-05-08T04:43:28.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- start start 2024-05-08T04:43:28.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f681415ba70 0x7f6814153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415be50 0x7f6814150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814168d40 con 0x7f68140aeda0 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814168bc0 con 0x7f681415ba70 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f681b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6814168ec0 con 0x7f681415be50 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59172/0 (socket says 172.21.15.164:59172) 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 -- 172.21.15.164:0/3342615521 learned_addr learned my addr 172.21.15.164:0/3342615521 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f680b400640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f681415ba70 0x7f6814153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6819200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415be50 0x7f6814150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 -- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415be50 msgr2=0x7f6814150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415be50 0x7f6814150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 -- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f681415ba70 msgr2=0x7f6814153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f681415ba70 0x7f6814153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 -- 172.21.15.164:0/3342615521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814154260 con 0x7f68140aeda0 2024-05-08T04:43:28.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.643+0000 7f6813e00640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f68040ad040 tx=0x7f680401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040be3f0 con 0x7f68140aeda0 2024-05-08T04:43:28.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68040be5b0 con 0x7f68140aeda0 2024-05-08T04:43:28.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040be9a0 con 0x7f68140aeda0 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6814155ad0 con 0x7f68140aeda0 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6814156090 con 0x7f68140aeda0 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6814069d60 con 0x7f68140aeda0 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f68040b43f0 con 0x7f68140aeda0 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 0x7f67f0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f680411db40 con 0x7f68140aeda0 2024-05-08T04:43:28.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f680b400640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 0x7f67f0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.647+0000 7f680b400640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 0x7f67f0069f60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f68140ae680 tx=0x7f67fc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.651+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6804016150 con 0x7f68140aeda0 2024-05-08T04:43:28.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.807+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f681406e210 con 0x7f68140aeda0 2024-05-08T04:43:28.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.807+0000 7f6810c00640 1 -- 172.21.15.164:0/3342615521 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68040eff50 con 0x7f68140aeda0 2024-05-08T04:43:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:28.809 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:43:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-05-08T04:43:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-05-08T04:43:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:28.810 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:43:28.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 msgr2=0x7f67f0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 0x7f67f0069f60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f68140ae680 tx=0x7f67fc009190 comp rx=0 tx=0).stop 2024-05-08T04:43:28.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 msgr2=0x7f681414efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f68040ad040 tx=0x7f680401ad70 comp rx=0 tx=0).stop 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 shutdown_connections 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f67f0067aa0 0x7f67f0069f60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f681415be50 0x7f6814150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f681415ba70 0x7f6814153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 --2- 172.21.15.164:0/3342615521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68140aeda0 0x7f681414efd0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 >> 172.21.15.164:0/3342615521 conn(0x7f681406ac90 msgr2=0x7f68140cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 shutdown_connections 2024-05-08T04:43:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.811+0000 7f681b200640 1 -- 172.21.15.164:0/3342615521 wait complete. 2024-05-08T04:43:28.830 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T04:43:28.830 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T04:43:28.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 Processor -- start 2024-05-08T04:43:28.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 -- start start 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd640aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640c6600 con 0x7fbd6415ce10 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640c6040 con 0x7fbd6415ba20 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd6be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6415fbe0 con 0x7fbd640ae6e0 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd68a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbd6415ce10 0x7fbd6406de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59770/0 (socket says 172.21.15.164:59770) 2024-05-08T04:43:28.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd68a00640 1 -- 172.21.15.164:0/3904237159 learned_addr learned my addr 172.21.15.164:0/3904237159 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:28.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd69e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd640aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361918437 0 0) 0x7fbd640c6600 con 0x7fbd6415ce10 2024-05-08T04:43:28.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.967+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd4c003530 con 0x7fbd6415ce10 2024-05-08T04:43:28.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 435743542 0 0) 0x7fbd4c003530 con 0x7fbd6415ce10 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 msgr2=0x7fbd640aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd640aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 >> v1:172.21.15.172:6789/0 conn(0x7fbd6415ba20 legacy=0x7fbd64160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640c6320 con 0x7fbd6415ce10 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fbd50001680 con 0x7fbd6415ce10 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 >> v1:172.21.15.164:6789/0 conn(0x7fbd6415ce10 legacy=0x7fbd6406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c0039f0 0x7fbd4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c004810 0x7fbd4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6415fbe0 con 0x7fbd4c004810 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00ba80 con 0x7fbd4c0039f0 2024-05-08T04:43:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00bc50 con 0x7fbd640ae6e0 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c004810 0x7fbd4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd69400640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c0039f0 0x7fbd4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd69e00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 -- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 msgr2=0x7fbd4c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd640ae6e0 0x7fbd4c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 -- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c0039f0 msgr2=0x7fbd4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c0039f0 0x7fbd4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 -- 172.21.15.164:0/3904237159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c003530 con 0x7fbd4c004810 2024-05-08T04:43:28.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.971+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c004810 0x7fbd4c004c90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c0e0 tx=0x7fbd5001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9164b787110982b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50007c50 con 0x7fbd4c004810 2024-05-08T04:43:28.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd500a9020 con 0x7fbd4c004810 2024-05-08T04:43:28.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd63e00640 1 -- 172.21.15.164:0/3904237159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50005b00 con 0x7fbd4c004810 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- 172.21.15.164:0/3904237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c004810 msgr2=0x7fbd4c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3904237159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c004810 0x7fbd4c004c90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c0e0 tx=0x7fbd5001cb40 comp rx=0 tx=0).stop 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd69e00640 1 -- 172.21.15.164:0/3904237159 reap_dead start 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- 172.21.15.164:0/3904237159 shutdown_connections 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- 172.21.15.164:0/3904237159 >> 172.21.15.164:0/3904237159 conn(0x7fbd640cf770 msgr2=0x7fbd640cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- 172.21.15.164:0/3904237159 shutdown_connections 2024-05-08T04:43:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- 172.21.15.164:0/3904237159 wait complete. 2024-05-08T04:43:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 Processor -- start 2024-05-08T04:43:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- start start 2024-05-08T04:43:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c004810 0x7fbd640cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd64153470 0x7fbd640c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6406e560 con 0x7fbd640c6840 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640cf410 con 0x7fbd4c004810 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd6be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6406e6e0 con 0x7fbd64153470 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.975+0000 7fbd68a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59204/0 (socket says 172.21.15.164:59204) 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 -- 172.21.15.164:0/3070031645 learned_addr learned my addr 172.21.15.164:0/3070031645 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd69e00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd64153470 0x7fbd640c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd69400640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c004810 0x7fbd640cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 -- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd64153470 msgr2=0x7fbd640c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd64153470 0x7fbd640c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 -- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c004810 msgr2=0x7fbd640cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c004810 0x7fbd640cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 -- 172.21.15.164:0/3070031645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd641f3a70 con 0x7fbd640c6840 2024-05-08T04:43:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd68a00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c0e0 tx=0x7fbd4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50001b90 con 0x7fbd640c6840 2024-05-08T04:43:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd500a5840 con 0x7fbd640c6840 2024-05-08T04:43:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd500a5bf0 con 0x7fbd640c6840 2024-05-08T04:43:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd641f3d50 con 0x7fbd640c6840 2024-05-08T04:43:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641f45f0 con 0x7fbd640c6840 2024-05-08T04:43:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd64072d30 con 0x7fbd640c6840 2024-05-08T04:43:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.979+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fbd500ae430 con 0x7fbd640c6840 2024-05-08T04:43:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.983+0000 7fbd61600640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 0x7fbd44069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.983+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fbd5011d6e0 con 0x7fbd640c6840 2024-05-08T04:43:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.983+0000 7fbd69400640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 0x7fbd44069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.983+0000 7fbd69400640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 0x7fbd44069f90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007fd0 tx=0x7fbd580050f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:28.987+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd50016850 con 0x7fbd640c6840 2024-05-08T04:43:29.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:43:29.071 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:43:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.143+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbd6404f010 con 0x7fbd640c6840 2024-05-08T04:43:29.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.439+0000 7fbd61600640 1 -- 172.21.15.164:0/3070031645 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd500eff50 con 0x7fbd640c6840 2024-05-08T04:43:29.441 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-05-08T04:43:29.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 msgr2=0x7fbd44069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 0x7fbd44069f90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007fd0 tx=0x7fbd580050f0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 msgr2=0x7fbd64152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c0e0 tx=0x7fbd4c005840 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 shutdown_connections 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fbd44067ad0 0x7fbd44069f90 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd64153470 0x7fbd640c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd640c6840 0x7fbd64152f30 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 --2- 172.21.15.164:0/3070031645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd4c004810 0x7fbd640cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 >> 172.21.15.164:0/3070031645 conn(0x7fbd640cf770 msgr2=0x7fbd6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 shutdown_connections 2024-05-08T04:43:29.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.443+0000 7fbd6be00640 1 -- 172.21.15.164:0/3070031645 wait complete. 2024-05-08T04:43:29.468 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T04:43:29.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 Processor -- start 2024-05-08T04:43:29.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 -- start start 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c06de50 0x7f5d9c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0c6600 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c15fbe0 con 0x7f5d9c06de50 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.655+0000 7f5da3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c6040 con 0x7f5d9c15ba20 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c06de50 0x7f5d9c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5da1600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5d9c0ae6e0 0x7f5d9c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39680/0 (socket says 172.21.15.164:39680) 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5da1600640 1 -- 172.21.15.164:0/3356447465 learned_addr learned my addr 172.21.15.164:0/3356447465 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:29.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5da0c00640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 -- 172.21.15.164:0/3356447465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 msgr2=0x7f5d9c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:29.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 -- 172.21.15.164:0/3356447465 >> v1:172.21.15.164:6789/0 conn(0x7f5d9c0ae6e0 legacy=0x7f5d9c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 -- 172.21.15.164:0/3356447465 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0c6320 con 0x7f5d9c06de50 2024-05-08T04:43:29.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9b400640 1 -- 172.21.15.164:0/3356447465 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762114695 0 0) 0x7f5d9c0c6600 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9be00640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c06de50 0x7f5d9c06e290 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c15fd60 tx=0x7f5d8801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5ce2ad66d90dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:29.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9b400640 1 -- 172.21.15.164:0/3356447465 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880ab610 con 0x7f5d9c06de50 2024-05-08T04:43:29.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9b400640 1 -- 172.21.15.164:0/3356447465 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d880b0070 con 0x7f5d9c06de50 2024-05-08T04:43:29.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.659+0000 7f5d9b400640 1 -- 172.21.15.164:0/3356447465 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880ab9d0 con 0x7f5d9c06de50 2024-05-08T04:43:29.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- 172.21.15.164:0/3356447465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c06de50 msgr2=0x7f5d9c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c06de50 0x7f5d9c06e290 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c15fd60 tx=0x7f5d8801c470 comp rx=0 tx=0).stop 2024-05-08T04:43:29.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- 172.21.15.164:0/3356447465 shutdown_connections 2024-05-08T04:43:29.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c06de50 0x7f5d9c06e290 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- 172.21.15.164:0/3356447465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- 172.21.15.164:0/3356447465 >> 172.21.15.164:0/3356447465 conn(0x7f5d9c0cf770 msgr2=0x7f5d9c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:29.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- 172.21.15.164:0/3356447465 shutdown_connections 2024-05-08T04:43:29.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- 172.21.15.164:0/3356447465 wait complete. 2024-05-08T04:43:29.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 Processor -- start 2024-05-08T04:43:29.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- start start 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 0x7f5d9c147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0aec60 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0aeae0 con 0x7f5d9c06de50 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0aede0 con 0x7f5d9c15ba20 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59806/0 (socket says 172.21.15.164:59806) 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 -- 172.21.15.164:0/284203047 learned_addr learned my addr 172.21.15.164:0/284203047 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da0c00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 0x7f5d9c147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da1600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 -- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 msgr2=0x7f5d9c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 -- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 msgr2=0x7f5d9c147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 0x7f5d9c147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5d9be00640 1 -- 172.21.15.164:0/284203047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0b5240 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.663+0000 7f5da0c00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 0x7f5d9c147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:29.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d9be00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5d880ad040 tx=0x7f5d8801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88004020 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d880041c0 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88003620 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0af3b0 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0c6d80 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c072d30 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f5d880037c0 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 0x7f5d7c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:29.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5311+0+0 (secure 0 0 0) 0x7f5d8811e0e0 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.667+0000 7f5da0c00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 0x7f5d7c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:29.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.671+0000 7f5da0c00640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 0x7f5d7c069f60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c1f88e0 tx=0x7f5d90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:29.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.675+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d880a8050 con 0x7f5d9c0ae6e0 2024-05-08T04:43:29.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:29.827+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5d9c166940 con 0x7f5d9c0ae6e0 2024-05-08T04:43:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.447+0000 7f5d98c00640 1 -- 172.21.15.164:0/284203047 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5d88016090 con 0x7f5d9c0ae6e0 2024-05-08T04:43:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 msgr2=0x7f5d7c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 0x7f5d7c069f60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c1f88e0 tx=0x7f5d90008040 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 msgr2=0x7f5d9c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5d880ad040 tx=0x7f5d8801acb0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 shutdown_connections 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5d7c067aa0 0x7f5d7c069f60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d9c15ba20 0x7f5d9c0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0ae6e0 0x7f5d9c0c6840 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 --2- 172.21.15.164:0/284203047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d9c06de50 0x7f5d9c147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 >> 172.21.15.164:0/284203047 conn(0x7f5d9c0cf770 msgr2=0x7f5d9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 shutdown_connections 2024-05-08T04:43:30.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.451+0000 7f5da3600640 1 -- 172.21.15.164:0/284203047 wait complete. 2024-05-08T04:43:30.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T04:43:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 Processor -- start 2024-05-08T04:43:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 -- start start 2024-05-08T04:43:30.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c7415ba20 0x7f1c74160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7406de50 0x7f1c7406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740c6600 con 0x7f1c740ae6e0 2024-05-08T04:43:30.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7415fbe0 con 0x7f1c7415ba20 2024-05-08T04:43:30.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c7b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c6040 con 0x7f1c7406de50 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c73e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1c7415ba20 0x7f1c74160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c79000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1c740ae6e0 0x7f1c740cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39682/0 (socket says 172.21.15.164:39682) 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c79000640 1 -- 172.21.15.164:0/2389159349 learned_addr learned my addr 172.21.15.164:0/2389159349 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.627+0000 7f1c6b400640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7406de50 0x7f1c7406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73e00640 1 -- 172.21.15.164:0/2389159349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7406de50 msgr2=0x7f1c7406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73e00640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7406de50 0x7f1c7406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73e00640 1 -- 172.21.15.164:0/2389159349 >> v1:172.21.15.164:6789/0 conn(0x7f1c740ae6e0 legacy=0x7f1c740cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73e00640 1 -- 172.21.15.164:0/2389159349 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740c6320 con 0x7f1c7415ba20 2024-05-08T04:43:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73400640 1 -- 172.21.15.164:0/2389159349 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454882860 0 0) 0x7f1c740c6600 con 0x7f1c740ae6e0 2024-05-08T04:43:30.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73e00640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74160070 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1c7415fd60 tx=0x7f1c6401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e2a49397bc6cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:30.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73400640 1 -- 172.21.15.164:0/2389159349 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640ab610 con 0x7f1c7415ba20 2024-05-08T04:43:30.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73400640 1 -- 172.21.15.164:0/2389159349 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640b0070 con 0x7f1c7415ba20 2024-05-08T04:43:30.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c73400640 1 -- 172.21.15.164:0/2389159349 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640ab9d0 con 0x7f1c7415ba20 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c7b000640 1 -- 172.21.15.164:0/2389159349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 msgr2=0x7f1c74160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74160070 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1c7415fd60 tx=0x7f1c6401c470 comp rx=0 tx=0).stop 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c7b000640 1 -- 172.21.15.164:0/2389159349 shutdown_connections 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7406de50 0x7f1c7406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.631+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2389159349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74160070 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- 172.21.15.164:0/2389159349 >> 172.21.15.164:0/2389159349 conn(0x7f1c740cf770 msgr2=0x7f1c740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- 172.21.15.164:0/2389159349 shutdown_connections 2024-05-08T04:43:30.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- 172.21.15.164:0/2389159349 wait complete. 2024-05-08T04:43:30.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 Processor -- start 2024-05-08T04:43:30.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- start start 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7406de50 0x7f1c74151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c740ae6e0 0x7f1c74153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740aec60 con 0x7f1c7415ba20 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740aeae0 con 0x7f1c7406de50 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c7b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740aede0 con 0x7f1c740ae6e0 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59822/0 (socket says 172.21.15.164:59822) 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 -- 172.21.15.164:0/2357821782 learned_addr learned my addr 172.21.15.164:0/2357821782 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c6b400640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c740ae6e0 0x7f1c74153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c73e00640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7406de50 0x7f1c74151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 -- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c740ae6e0 msgr2=0x7f1c74153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c740ae6e0 0x7f1c74153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 -- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7406de50 msgr2=0x7f1c74151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7406de50 0x7f1c74151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 -- 172.21.15.164:0/2357821782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740b03f0 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c79000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c7a0 tx=0x7f1c6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.635+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c018070 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00a640 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740b06d0 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00aa70 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740b0c90 con 0x7f1c7415ba20 2024-05-08T04:43:30.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74072d30 con 0x7f1c7415ba20 2024-05-08T04:43:30.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.639+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f1c6c01b070 con 0x7f1c7415ba20 2024-05-08T04:43:30.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.643+0000 7f1c70c00640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 0x7f1c50069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:30.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.643+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5693+0+0 (secure 0 0 0) 0x7f1c6c0835c0 con 0x7f1c7415ba20 2024-05-08T04:43:30.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.643+0000 7f1c73e00640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 0x7f1c50069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:30.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.643+0000 7f1c73e00640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 0x7f1c50069f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1c640af000 tx=0x7f1c640a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:30.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.643+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c6c055730 con 0x7f1c7415ba20 2024-05-08T04:43:30.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:30.831+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1c74166940 con 0x7f1c7415ba20 2024-05-08T04:43:31.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.451+0000 7f1c70c00640 1 -- 172.21.15.164:0/2357821782 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1c6c0597d0 con 0x7f1c7415ba20 2024-05-08T04:43:31.452 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 18 and data pool 19 2024-05-08T04:43:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 msgr2=0x7f1c50069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 0x7f1c50069f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1c640af000 tx=0x7f1c640a7000 comp rx=0 tx=0).stop 2024-05-08T04:43:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 msgr2=0x7f1c74153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c7a0 tx=0x7f1c6c00cc60 comp rx=0 tx=0).stop 2024-05-08T04:43:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 shutdown_connections 2024-05-08T04:43:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1c50067aa0 0x7f1c50069f60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c740ae6e0 0x7f1c74153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c7415ba20 0x7f1c74153b80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 --2- 172.21.15.164:0/2357821782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7406de50 0x7f1c74151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 >> 172.21.15.164:0/2357821782 conn(0x7f1c740cf770 msgr2=0x7f1c740adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:31.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 shutdown_connections 2024-05-08T04:43:31.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.455+0000 7f1c7b000640 1 -- 172.21.15.164:0/2357821782 wait complete. 2024-05-08T04:43:31.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:31.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.643+0000 7f2f8c400640 1 Processor -- start 2024-05-08T04:43:31.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.643+0000 7f2f8c400640 1 -- start start 2024-05-08T04:43:31.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8c400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f8415b390 0x7f2f8406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8c400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 0x7f2f8406c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8c400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8406e0e0 con 0x7f2f840aee30 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8c400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8415e8f0 con 0x7f2f8415b390 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8c400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8415a8c0 con 0x7f2f840cf150 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f8415b390 0x7f2f8406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2f840aee30 0x7f2f840cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39692/0 (socket says 172.21.15.164:39692) 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89000640 1 -- 172.21.15.164:0/1791054859 learned_addr learned my addr 172.21.15.164:0/1791054859 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8a400640 1 --2- 172.21.15.164:0/1791054859 >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 0x7f2f8406c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89a00640 1 -- 172.21.15.164:0/1791054859 >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 msgr2=0x7f2f8406c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:31.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89a00640 1 --2- 172.21.15.164:0/1791054859 >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 0x7f2f8406c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89a00640 1 -- 172.21.15.164:0/1791054859 >> v1:172.21.15.164:6789/0 conn(0x7f2f840aee30 legacy=0x7f2f840cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f89a00640 1 -- 172.21.15.164:0/1791054859 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8415ed80 con 0x7f2f8415b390 2024-05-08T04:43:31.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f83e00640 1 -- 172.21.15.164:0/1791054859 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572687491 0 0) 0x7f2f8406e0e0 con 0x7f2f840aee30 2024-05-08T04:43:31.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.647+0000 7f2f8a400640 1 --2- 172.21.15.164:0/1791054859 >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 0x7f2f8406c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f89a00640 1 --2- 172.21.15.164:0/1791054859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f8406d8b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2f840aeb60 tx=0x7f2f7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8545027cd67225d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f83e00640 1 -- 172.21.15.164:0/1791054859 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f780b0070 con 0x7f2f8415b390 2024-05-08T04:43:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f83e00640 1 -- 172.21.15.164:0/1791054859 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f780ab600 con 0x7f2f8415b390 2024-05-08T04:43:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f83e00640 1 -- 172.21.15.164:0/1791054859 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f780ab9d0 con 0x7f2f8415b390 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- 172.21.15.164:0/1791054859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 msgr2=0x7f2f8406d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- 172.21.15.164:0/1791054859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f8406d8b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2f840aeb60 tx=0x7f2f7801c480 comp rx=0 tx=0).stop 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- 172.21.15.164:0/1791054859 shutdown_connections 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- 172.21.15.164:0/1791054859 >> v2:172.21.15.172:3300/0 conn(0x7f2f840cf150 0x7f2f8406c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- 172.21.15.164:0/1791054859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f8406d8b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- 172.21.15.164:0/1791054859 >> 172.21.15.164:0/1791054859 conn(0x7f2f8406ac90 msgr2=0x7f2f8406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- 172.21.15.164:0/1791054859 shutdown_connections 2024-05-08T04:43:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- 172.21.15.164:0/1791054859 wait complete. 2024-05-08T04:43:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 Processor -- start 2024-05-08T04:43:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- start start 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f840aee30 0x7f2f8414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f840cf150 0x7f2f8413b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415b770 con 0x7f2f8415b390 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8406ddf0 con 0x7f2f840cf150 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8406c910 con 0x7f2f840aee30 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59844/0 (socket says 172.21.15.164:59844) 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.651+0000 7f2f8a400640 1 -- 172.21.15.164:0/4199189303 learned_addr learned my addr 172.21.15.164:0/4199189303 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f89000640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f840cf150 0x7f2f8413b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f89a00640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f840aee30 0x7f2f8414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 -- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f840aee30 msgr2=0x7f2f8414bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f840aee30 0x7f2f8414bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 -- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f840cf150 msgr2=0x7f2f8413b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f840cf150 0x7f2f8413b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 -- 172.21.15.164:0/4199189303 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f841f6740 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8a400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007ef0 tx=0x7f2f7c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c01a070 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c003ba0 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00de80 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8414cc40 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f841f6ca0 con 0x7f2f8415b390 2024-05-08T04:43:31.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84069d60 con 0x7f2f8415b390 2024-05-08T04:43:31.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f2f7c01d070 con 0x7f2f8415b390 2024-05-08T04:43:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.655+0000 7f2f81600640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 0x7f2f64069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.659+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2f7c083b70 con 0x7f2f8415b390 2024-05-08T04:43:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.659+0000 7f2f89a00640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 0x7f2f64069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.659+0000 7f2f89a00640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 0x7f2f64069f60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ad000 tx=0x7f2f78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.663+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f841f6740 con 0x7f2f8415b390 2024-05-08T04:43:31.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.815+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f840cca70 con 0x7f2f8415b390 2024-05-08T04:43:31.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.815+0000 7f2f81600640 1 -- 172.21.15.164:0/4199189303 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f2f7c010070 con 0x7f2f8415b390 2024-05-08T04:43:31.818 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:31.819 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":125,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:30.835643+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 msgr2=0x7f2f64069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 0x7f2f64069f60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2f780ad000 tx=0x7f2f78008040 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 msgr2=0x7f2f840af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007ef0 tx=0x7f2f7c00d4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 shutdown_connections 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2f64067aa0 0x7f2f64069f60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8415b390 0x7f2f840af2e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f840cf150 0x7f2f8413b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 --2- 172.21.15.164:0/4199189303 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f840aee30 0x7f2f8414bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 >> 172.21.15.164:0/4199189303 conn(0x7f2f8406ac90 msgr2=0x7f2f8406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 shutdown_connections 2024-05-08T04:43:31.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.819+0000 7f2f8c400640 1 -- 172.21.15.164:0/4199189303 wait complete. 2024-05-08T04:43:31.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.971+0000 7f4029e00640 1 Processor -- start 2024-05-08T04:43:31.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.971+0000 7f4029e00640 1 -- start start 2024-05-08T04:43:31.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4029e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4029e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 0x7f40240cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4029e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4024072640 con 0x7f40240cfbd0 2024-05-08T04:43:31.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4029e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40240d0060 con 0x7f40240b26c0 2024-05-08T04:43:31.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4029e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402406c770 con 0x7f402415ab60 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f401a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 0x7f40240cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f401a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 0x7f40240cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42684/0 (socket says 172.21.15.164:42684) 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f401a000640 1 -- 172.21.15.164:0/730109181 learned_addr learned my addr 172.21.15.164:0/730109181 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 --2- 172.21.15.164:0/730109181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49038/0 (socket says 172.21.15.164:49038) 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 -- 172.21.15.164:0/730109181 >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 msgr2=0x7f40240cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:31.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 --2- 172.21.15.164:0/730109181 >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 0x7f40240cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 -- 172.21.15.164:0/730109181 >> v1:172.21.15.172:6789/0 conn(0x7f40240cfbd0 legacy=0x7f40240cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.975+0000 7f4022a00640 1 -- 172.21.15.164:0/730109181 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402406ca50 con 0x7f402415ab60 2024-05-08T04:43:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4022a00640 1 --2- 172.21.15.164:0/730109181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40140097c0 tx=0x7f401401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7cdfe52eab9613 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4022000640 1 -- 172.21.15.164:0/730109181 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140b0070 con 0x7f402415ab60 2024-05-08T04:43:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4022000640 1 -- 172.21.15.164:0/730109181 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40140ab600 con 0x7f402415ab60 2024-05-08T04:43:31.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4022000640 1 -- 172.21.15.164:0/730109181 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140ab9d0 con 0x7f402415ab60 2024-05-08T04:43:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- 172.21.15.164:0/730109181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 msgr2=0x7f4024072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 --2- 172.21.15.164:0/730109181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40140097c0 tx=0x7f401401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- 172.21.15.164:0/730109181 shutdown_connections 2024-05-08T04:43:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 --2- 172.21.15.164:0/730109181 >> v2:172.21.15.172:3300/0 conn(0x7f40240b26c0 0x7f40240cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 --2- 172.21.15.164:0/730109181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f402415ab60 0x7f4024072100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- 172.21.15.164:0/730109181 >> 172.21.15.164:0/730109181 conn(0x7f402406ac90 msgr2=0x7f402406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- 172.21.15.164:0/730109181 shutdown_connections 2024-05-08T04:43:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- 172.21.15.164:0/730109181 wait complete. 2024-05-08T04:43:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 Processor -- start 2024-05-08T04:43:31.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.979+0000 7f4029e00640 1 -- start start 2024-05-08T04:43:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40240cfbd0 0x7f402414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 0x7f402415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402406c550 con 0x7f402415ab60 2024-05-08T04:43:31.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402406c3d0 con 0x7f40240cfbd0 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40240aed30 con 0x7f40240b26c0 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4022a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 0x7f402415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4022a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49050/0 (socket says 172.21.15.164:49050) 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4022a00640 1 -- 172.21.15.164:0/2558622644 learned_addr learned my addr 172.21.15.164:0/2558622644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f401a000640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40240cfbd0 0x7f402414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 -- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 msgr2=0x7f40241faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 -- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40240cfbd0 msgr2=0x7f402414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:31.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40240cfbd0 0x7f402414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 -- 172.21.15.164:0/2558622644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40241fb1e0 con 0x7f402415ab60 2024-05-08T04:43:31.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4022a00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4023400640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 0x7f402415ba60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c700 tx=0x7f401c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c003b80 con 0x7f402415ab60 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c003d20 con 0x7f402415ab60 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402415b180 con 0x7f402415ab60 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c00aa30 con 0x7f402415ab60 2024-05-08T04:43:31.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40241fa4a0 con 0x7f402415ab60 2024-05-08T04:43:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.983+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4024069d60 con 0x7f402415ab60 2024-05-08T04:43:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.987+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f401c00abd0 con 0x7f402415ab60 2024-05-08T04:43:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.987+0000 7f401be00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 0x7f4000069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.987+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f401c084420 con 0x7f402415ab60 2024-05-08T04:43:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.987+0000 7f4022a00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 0x7f4000069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:31.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.987+0000 7f4022a00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 0x7f4000069f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f401401ab50 tx=0x7f4014008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:31.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:31.991+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f401c0565e0 con 0x7f402415ab60 2024-05-08T04:43:32.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.143+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40240cf950 con 0x7f402415ab60 2024-05-08T04:43:32.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f401be00640 1 -- 172.21.15.164:0/2558622644 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f401c05a680 con 0x7f402415ab60 2024-05-08T04:43:32.148 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:32.148 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":125,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:30.835643+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 msgr2=0x7f4000069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 0x7f4000069f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f401401ab50 tx=0x7f4014008040 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 msgr2=0x7f402415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 0x7f402415ba60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c700 tx=0x7f401c00cbc0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 shutdown_connections 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4000067ad0 0x7f4000069f90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402415ab60 0x7f402415ba60 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40240cfbd0 0x7f402414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 --2- 172.21.15.164:0/2558622644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40240b26c0 0x7f40241faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 >> 172.21.15.164:0/2558622644 conn(0x7f402406ac90 msgr2=0x7f40240b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:32.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 shutdown_connections 2024-05-08T04:43:32.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.147+0000 7f4029e00640 1 -- 172.21.15.164:0/2558622644 wait complete. 2024-05-08T04:43:32.169 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T04:43:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 Processor -- start 2024-05-08T04:43:32.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 -- start start 2024-05-08T04:43:32.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb284160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb28406de50 0x7fb28406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2840ae6e0 0x7fb2840aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28415fbe0 con 0x7fb2840ae6e0 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840af060 con 0x7fb28406de50 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb289400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840c6600 con 0x7fb28415ba20 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2840ae6e0 0x7fb2840aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2840ae6e0 0x7fb2840aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59856/0 (socket says 172.21.15.164:59856) 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 -- 172.21.15.164:0/485992329 learned_addr learned my addr 172.21.15.164:0/485992329 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb279e00640 1 --2- 172.21.15.164:0/485992329 >> v2:172.21.15.172:3300/0 conn(0x7fb28406de50 0x7fb28406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb282800640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb284160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 -- 172.21.15.164:0/485992329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 msgr2=0x7fb284160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:32.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb284160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 -- 172.21.15.164:0/485992329 >> v2:172.21.15.172:3300/0 conn(0x7fb28406de50 msgr2=0x7fb28406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:32.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 --2- 172.21.15.164:0/485992329 >> v2:172.21.15.172:3300/0 conn(0x7fb28406de50 0x7fb28406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb283200640 1 -- 172.21.15.164:0/485992329 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840c6040 con 0x7fb2840ae6e0 2024-05-08T04:43:32.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.307+0000 7fb282800640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb284160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:32.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb283200640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2840aeb20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb2840cd2f0 tx=0x7fb27c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d992f3b11df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:32.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb281e00640 1 -- 172.21.15.164:0/485992329 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0aa610 con 0x7fb2840ae6e0 2024-05-08T04:43:32.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb281e00640 1 -- 172.21.15.164:0/485992329 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c0af070 con 0x7fb2840ae6e0 2024-05-08T04:43:32.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb281e00640 1 -- 172.21.15.164:0/485992329 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0aa9d0 con 0x7fb2840ae6e0 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 -- 172.21.15.164:0/485992329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 msgr2=0x7fb2840aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2840aeb20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb2840cd2f0 tx=0x7fb27c01c4f0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 -- 172.21.15.164:0/485992329 shutdown_connections 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2840aeb20 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 --2- 172.21.15.164:0/485992329 >> v2:172.21.15.172:3300/0 conn(0x7fb28406de50 0x7fb28406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 --2- 172.21.15.164:0/485992329 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb284160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 -- 172.21.15.164:0/485992329 >> 172.21.15.164:0/485992329 conn(0x7fb2840cf770 msgr2=0x7fb2840cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 -- 172.21.15.164:0/485992329 shutdown_connections 2024-05-08T04:43:32.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.311+0000 7fb289400640 1 -- 172.21.15.164:0/485992329 wait complete. 2024-05-08T04:43:32.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 Processor -- start 2024-05-08T04:43:32.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- start start 2024-05-08T04:43:32.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb28406de50 0x7fb2840b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb2840b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cde80 con 0x7fb2840ae6e0 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2841605b0 con 0x7fb28406de50 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ce000 con 0x7fb28415ba20 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59860/0 (socket says 172.21.15.164:59860) 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 -- 172.21.15.164:0/1631840796 learned_addr learned my addr 172.21.15.164:0/1631840796 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb282800640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb28406de50 0x7fb2840b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb283200640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb2840b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 -- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 msgr2=0x7fb2840b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb2840b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 -- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb28406de50 msgr2=0x7fb2840b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb28406de50 0x7fb2840b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 -- 172.21.15.164:0/1631840796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28406d1f0 con 0x7fb2840ae6e0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb279e00640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00d800 tx=0x7fb26c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c018020 con 0x7fb2840ae6e0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0096a0 con 0x7fb2840ae6e0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c009a50 con 0x7fb2840ae6e0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840c6c40 con 0x7fb2840ae6e0 2024-05-08T04:43:32.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.315+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840c7200 con 0x7fb2840ae6e0 2024-05-08T04:43:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284072d30 con 0x7fb2840ae6e0 2024-05-08T04:43:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fb26c01b020 con 0x7fb2840ae6e0 2024-05-08T04:43:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb27b400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 0x7fb260069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb26c083db0 con 0x7fb2840ae6e0 2024-05-08T04:43:32.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb282800640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 0x7fb260069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:32.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.319+0000 7fb282800640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 0x7fb260069f60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb27400f500 tx=0x7fb274007770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:32.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.323+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb26c022020 con 0x7fb2840ae6e0 2024-05-08T04:43:32.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:32.519+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb284166940 con 0x7fb2840ae6e0 2024-05-08T04:43:33.431 INFO:tasks.ceph.mds.c.smithi164.stderr:2024-05-08T04:43:33.431+0000 7f0c77800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T04:43:33.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb27b400640 1 -- 172.21.15.164:0/1631840796 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb26c056190 con 0x7fb2840ae6e0 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 msgr2=0x7fb260069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 0x7fb260069f60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb27400f500 tx=0x7fb274007770 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 msgr2=0x7fb2841450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00d800 tx=0x7fb26c00dcc0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 shutdown_connections 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb260067aa0 0x7fb260069f60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb28415ba20 0x7fb2840b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2840ae6e0 0x7fb2841450b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 --2- 172.21.15.164:0/1631840796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb28406de50 0x7fb2840b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.471+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 >> 172.21.15.164:0/1631840796 conn(0x7fb2840cf770 msgr2=0x7fb28415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.475+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 shutdown_connections 2024-05-08T04:43:33.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.475+0000 7fb289400640 1 -- 172.21.15.164:0/1631840796 wait complete. 2024-05-08T04:43:33.498 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T04:43:33.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T04:43:33.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 Processor -- start 2024-05-08T04:43:33.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 -- start start 2024-05-08T04:43:33.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 0x7f16d40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d406d150 con 0x7f16d415a200 2024-05-08T04:43:33.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d406bca0 con 0x7f16d406dfa0 2024-05-08T04:43:33.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16dc400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40af150 con 0x7f16d40ca850 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16da400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 0x7f16d40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f16d415a200 0x7f16d40ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:57454/0 (socket says 172.21.15.164:57454) 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49074/0 (socket says 172.21.15.164:49074) 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16da400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 0x7f16d40cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42710/0 (socket says 172.21.15.164:42710) 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9a00640 1 -- 172.21.15.164:0/3665462150 learned_addr learned my addr 172.21.15.164:0/3665462150 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d3e00640 1 -- 172.21.15.164:0/3665462150 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333520425 0 0) 0x7f16d406d150 con 0x7f16d415a200 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 -- 172.21.15.164:0/3665462150 >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 msgr2=0x7f16d40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 --2- 172.21.15.164:0/3665462150 >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 0x7f16d40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 -- 172.21.15.164:0/3665462150 >> v1:172.21.15.172:6789/0 conn(0x7f16d415a200 legacy=0x7f16d40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.639+0000 7f16d9000640 1 -- 172.21.15.164:0/3665462150 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d406d430 con 0x7f16d40ca850 2024-05-08T04:43:33.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16d9000640 1 --2- 172.21.15.164:0/3665462150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16c00097c0 tx=0x7f16c001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9993f6e91c214513 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:33.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16d3e00640 1 -- 172.21.15.164:0/3665462150 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00b0070 con 0x7f16d40ca850 2024-05-08T04:43:33.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16d3e00640 1 -- 172.21.15.164:0/3665462150 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00ab600 con 0x7f16d40ca850 2024-05-08T04:43:33.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16d3e00640 1 -- 172.21.15.164:0/3665462150 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00ab9d0 con 0x7f16d40ca850 2024-05-08T04:43:33.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 -- 172.21.15.164:0/3665462150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 msgr2=0x7f16d40aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 --2- 172.21.15.164:0/3665462150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16c00097c0 tx=0x7f16c001c350 comp rx=0 tx=0).stop 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 -- 172.21.15.164:0/3665462150 shutdown_connections 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 --2- 172.21.15.164:0/3665462150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d40aeac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 --2- 172.21.15.164:0/3665462150 >> v2:172.21.15.172:3300/0 conn(0x7f16d406dfa0 0x7f16d40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 -- 172.21.15.164:0/3665462150 >> 172.21.15.164:0/3665462150 conn(0x7f16d40cf770 msgr2=0x7f16d40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 -- 172.21.15.164:0/3665462150 shutdown_connections 2024-05-08T04:43:33.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.643+0000 7f16dc400640 1 -- 172.21.15.164:0/3665462150 wait complete. 2024-05-08T04:43:33.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 Processor -- start 2024-05-08T04:43:33.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- start start 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d4148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d415a200 0x7f16d4156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40cdd30 con 0x7f16d406dfa0 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40cdbb0 con 0x7f16d415a200 2024-05-08T04:43:33.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40cdeb0 con 0x7f16d40ca850 2024-05-08T04:43:33.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59872/0 (socket says 172.21.15.164:59872) 2024-05-08T04:43:33.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 -- 172.21.15.164:0/3864625826 learned_addr learned my addr 172.21.15.164:0/3864625826 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:33.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16da400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d415a200 0x7f16d4156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9000640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d4148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 -- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 msgr2=0x7f16d4148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d4148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 -- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d415a200 msgr2=0x7f16d4156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d415a200 0x7f16d4156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 -- 172.21.15.164:0/3864625826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4070bb0 con 0x7f16d406dfa0 2024-05-08T04:43:33.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d9a00640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c480 tx=0x7f16c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8018070 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8007500 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d4070e70 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004a30 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.647+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d4071430 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d4072d30 con 0x7f16d406dfa0 2024-05-08T04:43:33.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f16c801b070 con 0x7f16d406dfa0 2024-05-08T04:43:33.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16d1600640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 0x7f16b4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:33.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f16c8082f50 con 0x7f16d406dfa0 2024-05-08T04:43:33.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16d9000640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 0x7f16b4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:33.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.651+0000 7f16d9000640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 0x7f16b4069f60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f16c00ad000 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:33.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.655+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16c8055110 con 0x7f16d406dfa0 2024-05-08T04:43:33.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:33.843+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f16d40cfcf0 con 0x7f16d406dfa0 2024-05-08T04:43:34.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.483+0000 7f16d1600640 1 -- 172.21.15.164:0/3864625826 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16c80591b0 con 0x7f16d406dfa0 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 msgr2=0x7f16b4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 0x7f16b4069f60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f16c00ad000 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 msgr2=0x7f16d40705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c480 tx=0x7f16c800c940 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 shutdown_connections 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f16b4067aa0 0x7f16b4069f60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40ca850 0x7f16d4148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d415a200 0x7f16d4156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 --2- 172.21.15.164:0/3864625826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d406dfa0 0x7f16d40705c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 >> 172.21.15.164:0/3864625826 conn(0x7f16d40cf770 msgr2=0x7f16d40cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:34.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 shutdown_connections 2024-05-08T04:43:34.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.487+0000 7f16dc400640 1 -- 172.21.15.164:0/3864625826 wait complete. 2024-05-08T04:43:34.510 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T04:43:34.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:34.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.643+0000 7f8652800640 1 Processor -- start 2024-05-08T04:43:34.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.643+0000 7f8652800640 1 -- start start 2024-05-08T04:43:34.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f8652800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f864c0cfda0 0x7f864c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:34.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f8652800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c06c770 con 0x7f864c15ab60 2024-05-08T04:43:34.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f8652800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c06ca50 con 0x7f864c0b26c0 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f8652800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c06cd30 con 0x7f864c0cfda0 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f864c0cfda0 0x7f864c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f864c0cfda0 0x7f864c0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59888/0 (socket says 172.21.15.164:59888) 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f864c15ab60 0x7f864c072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39708/0 (socket says 172.21.15.164:39708) 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 -- 172.21.15.164:0/1661066381 learned_addr learned my addr 172.21.15.164:0/1661066381 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 -- 172.21.15.164:0/1661066381 >> v1:172.21.15.172:6789/0 conn(0x7f864c0b26c0 legacy=0x7f864c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:34.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 -- 172.21.15.164:0/1661066381 >> v1:172.21.15.164:6789/0 conn(0x7f864c15ab60 legacy=0x7f864c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 -- 172.21.15.164:0/1661066381 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c06d010 con 0x7f864c0cfda0 2024-05-08T04:43:34.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864a000640 1 -- 172.21.15.164:0/1661066381 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525568073 0 0) 0x7f864c06c770 con 0x7f864c15ab60 2024-05-08T04:43:34.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.647+0000 7f864be00640 1 --2- 172.21.15.164:0/1661066381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c0cfda0 0x7f864c0cd910 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad1993f62052b63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:34.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864a000640 1 -- 172.21.15.164:0/1661066381 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440b0070 con 0x7f864c0cfda0 2024-05-08T04:43:34.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864a000640 1 -- 172.21.15.164:0/1661066381 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86440ab600 con 0x7f864c0cfda0 2024-05-08T04:43:34.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864a000640 1 -- 172.21.15.164:0/1661066381 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440ab9d0 con 0x7f864c0cfda0 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- 172.21.15.164:0/1661066381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c0cfda0 msgr2=0x7f864c0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 --2- 172.21.15.164:0/1661066381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c0cfda0 0x7f864c0cd910 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c520 comp rx=0 tx=0).stop 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- 172.21.15.164:0/1661066381 shutdown_connections 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 --2- 172.21.15.164:0/1661066381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c0cfda0 0x7f864c0cd910 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- 172.21.15.164:0/1661066381 >> 172.21.15.164:0/1661066381 conn(0x7f864c06ac90 msgr2=0x7f864c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- 172.21.15.164:0/1661066381 shutdown_connections 2024-05-08T04:43:34.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- 172.21.15.164:0/1661066381 wait complete. 2024-05-08T04:43:34.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 Processor -- start 2024-05-08T04:43:34.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- start start 2024-05-08T04:43:34.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f864c0b26c0 0x7f864c157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 0x7f864c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c168b00 con 0x7f864c15ab60 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c168980 con 0x7f864c0cfda0 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f8652800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c168c80 con 0x7f864c0b26c0 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f864c0b26c0 0x7f864c157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.651+0000 7f864be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59894/0 (socket says 172.21.15.164:59894) 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 -- 172.21.15.164:0/1055772070 learned_addr learned my addr 172.21.15.164:0/1055772070 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864aa00640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 0x7f864c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 -- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f864c0b26c0 msgr2=0x7f864c157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f864c0b26c0 0x7f864c157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 -- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 msgr2=0x7f864c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 0x7f864c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 -- 172.21.15.164:0/1055772070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c15deb0 con 0x7f864c15ab60 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864aa00640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 0x7f864c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864be00640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae040 tx=0x7f86440192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440b8070 con 0x7f864c15ab60 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86440b4610 con 0x7f864c15ab60 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440b0040 con 0x7f864c15ab60 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c15bf10 con 0x7f864c15ab60 2024-05-08T04:43:34.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c15dbd0 con 0x7f864c15ab60 2024-05-08T04:43:34.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c069d60 con 0x7f864c15ab60 2024-05-08T04:43:34.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f86440bb070 con 0x7f864c15ab60 2024-05-08T04:43:34.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 0x7f862c069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:34.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f864411d830 con 0x7f864c15ab60 2024-05-08T04:43:34.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.655+0000 7f864b400640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 0x7f862c069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:34.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.659+0000 7f864b400640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 0x7f862c069f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f863c0012f0 tx=0x7f863c001690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:34.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.663+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8644016110 con 0x7f864c15ab60 2024-05-08T04:43:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.847+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f864c06d9a0 con 0x7f864c15ab60 2024-05-08T04:43:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.847+0000 7f8643400640 1 -- 172.21.15.164:0/1055772070 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f86440efa50 con 0x7f864c15ab60 2024-05-08T04:43:34.850 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:34.850 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 82 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 msgr2=0x7f862c069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 0x7f862c069f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f863c0012f0 tx=0x7f863c001690 comp rx=0 tx=0).stop 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 msgr2=0x7f864c14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae040 tx=0x7f86440192c0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 shutdown_connections 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f862c067ad0 0x7f862c069f90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f864c15ab60 0x7f864c14a180 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f864c0cfda0 0x7f864c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 --2- 172.21.15.164:0/1055772070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f864c0b26c0 0x7f864c157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:34.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 >> 172.21.15.164:0/1055772070 conn(0x7f864c06ac90 msgr2=0x7f864c151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:34.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 shutdown_connections 2024-05-08T04:43:34.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:34.851+0000 7f8652800640 1 -- 172.21.15.164:0/1055772070 wait complete. 2024-05-08T04:43:34.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:35.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.003+0000 7f6379000640 1 Processor -- start 2024-05-08T04:43:35.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.003+0000 7f6379000640 1 -- start start 2024-05-08T04:43:35.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6379000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637415da80 0x7f63740b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6379000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63740ae4a0 0x7f63740727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6379000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637415de60 con 0x7f63740af290 2024-05-08T04:43:35.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6379000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63740af720 con 0x7f63740ae4a0 2024-05-08T04:43:35.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6379000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6374070c00 con 0x7f637415da80 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63740ae4a0 0x7f63740727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f63740ae4a0 0x7f63740727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59900/0 (socket says 172.21.15.164:59900) 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 -- 172.21.15.164:0/2509449609 learned_addr learned my addr 172.21.15.164:0/2509449609 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373400640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637415da80 0x7f63740b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 -- 172.21.15.164:0/2509449609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637415da80 msgr2=0x7f63740b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637415da80 0x7f63740b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 -- 172.21.15.164:0/2509449609 >> v1:172.21.15.172:6789/0 conn(0x7f63740af290 legacy=0x7f63740ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.007+0000 7f6373e00640 1 -- 172.21.15.164:0/2509449609 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374070ee0 con 0x7f63740ae4a0 2024-05-08T04:43:35.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6373e00640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63740ae4a0 0x7f63740727a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d43fda1a0724b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6372000640 1 -- 172.21.15.164:0/2509449609 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0a5810 con 0x7f63740ae4a0 2024-05-08T04:43:35.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6372000640 1 -- 172.21.15.164:0/2509449609 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c0a59b0 con 0x7f63740ae4a0 2024-05-08T04:43:35.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6372000640 1 -- 172.21.15.164:0/2509449609 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0a5d80 con 0x7f63740ae4a0 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- 172.21.15.164:0/2509449609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63740ae4a0 msgr2=0x7f63740727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63740ae4a0 0x7f63740727a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c01ca30 comp rx=0 tx=0).stop 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- 172.21.15.164:0/2509449609 shutdown_connections 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63740ae4a0 0x7f63740727a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- 172.21.15.164:0/2509449609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637415da80 0x7f63740b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- 172.21.15.164:0/2509449609 >> 172.21.15.164:0/2509449609 conn(0x7f63740b21a0 msgr2=0x7f637406cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- 172.21.15.164:0/2509449609 shutdown_connections 2024-05-08T04:43:35.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- 172.21.15.164:0/2509449609 wait complete. 2024-05-08T04:43:35.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 Processor -- start 2024-05-08T04:43:35.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 -- start start 2024-05-08T04:43:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 0x7f6374151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637415da80 0x7f63741581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.011+0000 7f6379000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637406c500 con 0x7f6374158720 2024-05-08T04:43:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637406c380 con 0x7f637415da80 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637416a790 con 0x7f63740ae4a0 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 0x7f6374151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59912/0 (socket says 172.21.15.164:59912) 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 0x7f6374151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:49108/0 (socket says 172.21.15.164:49108) 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 -- 172.21.15.164:0/635231196 learned_addr learned my addr 172.21.15.164:0/635231196 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6372a00640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637415da80 0x7f63741581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 -- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 msgr2=0x7f6374151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 0x7f6374151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 -- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637415da80 msgr2=0x7f63741581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637415da80 0x7f63741581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 -- 172.21.15.164:0/635231196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374158d50 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6373e00640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0bc020 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c0b8470 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6374159030 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c009510 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63741595f0 con 0x7f6374158720 2024-05-08T04:43:35.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637404ef30 con 0x7f6374158720 2024-05-08T04:43:35.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.015+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f636c0bf020 con 0x7f6374158720 2024-05-08T04:43:35.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.019+0000 7f636b400640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 0x7f6354069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.019+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f636c11e700 con 0x7f6374158720 2024-05-08T04:43:35.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.019+0000 7f6373400640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 0x7f6354069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.019+0000 7f6373400640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 0x7f6354069f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6364009b60 tx=0x7f6364009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.023+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f636c0c6030 con 0x7f6374158720 2024-05-08T04:43:35.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.175+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63740b3d10 con 0x7f6374158720 2024-05-08T04:43:35.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.175+0000 7f636b400640 1 -- 172.21.15.164:0/635231196 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f636c016620 con 0x7f6374158720 2024-05-08T04:43:35.178 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:35.178 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 msgr2=0x7f6354069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 0x7f6354069f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6364009b60 tx=0x7f6364009190 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 msgr2=0x7f63741552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c0197d0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 shutdown_connections 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6354067aa0 0x7f6354069f60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6374158720 0x7f63741552a0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637415da80 0x7f63741581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 --2- 172.21.15.164:0/635231196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63740ae4a0 0x7f6374151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 >> 172.21.15.164:0/635231196 conn(0x7f63740b21a0 msgr2=0x7f63740b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 shutdown_connections 2024-05-08T04:43:35.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.179+0000 7f6379000640 1 -- 172.21.15.164:0/635231196 wait complete. 2024-05-08T04:43:35.200 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T04:43:35.201 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:43:35.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 Processor -- start 2024-05-08T04:43:35.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 -- start start 2024-05-08T04:43:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3a90161950 0x7f3a900cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a9015ea20 0x7f3a9015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a90161fe0 con 0x7f3a90144200 2024-05-08T04:43:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a90162880 con 0x7f3a90161950 2024-05-08T04:43:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a97000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a901622c0 con 0x7f3a9015ea20 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3a90144200 0x7f3a900cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39712/0 (socket says 172.21.15.164:39712) 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8fe00640 1 -- 172.21.15.164:0/3763384620 learned_addr learned my addr 172.21.15.164:0/3763384620 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8f400640 1 --2- 172.21.15.164:0/3763384620 >> v2:172.21.15.164:3300/0 conn(0x7f3a90161950 0x7f3a900cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8ea00640 1 -- 172.21.15.164:0/3763384620 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267676243 0 0) 0x7f3a90161fe0 con 0x7f3a90144200 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8ea00640 1 -- 172.21.15.164:0/3763384620 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a74003530 con 0x7f3a90144200 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a95000640 1 --2- 172.21.15.164:0/3763384620 >> v2:172.21.15.172:3300/0 conn(0x7f3a9015ea20 0x7f3a9015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8f400640 1 -- 172.21.15.164:0/3763384620 >> v2:172.21.15.172:3300/0 conn(0x7f3a9015ea20 msgr2=0x7f3a9015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8f400640 1 --2- 172.21.15.164:0/3763384620 >> v2:172.21.15.172:3300/0 conn(0x7f3a9015ea20 0x7f3a9015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8f400640 1 -- 172.21.15.164:0/3763384620 >> v1:172.21.15.164:6789/0 conn(0x7f3a90144200 legacy=0x7f3a900cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.335+0000 7f3a8f400640 1 -- 172.21.15.164:0/3763384620 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a901625a0 con 0x7f3a90161950 2024-05-08T04:43:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a8f400640 1 --2- 172.21.15.164:0/3763384620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a900cfd60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3a90067400 tx=0x7f3a7c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35a66f81ad67771 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a8ea00640 1 -- 172.21.15.164:0/3763384620 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b0070 con 0x7f3a90161950 2024-05-08T04:43:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a8ea00640 1 -- 172.21.15.164:0/3763384620 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0ab600 con 0x7f3a90161950 2024-05-08T04:43:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a8ea00640 1 -- 172.21.15.164:0/3763384620 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0ab9d0 con 0x7f3a90161950 2024-05-08T04:43:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 -- 172.21.15.164:0/3763384620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 msgr2=0x7f3a900cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 --2- 172.21.15.164:0/3763384620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a900cfd60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3a90067400 tx=0x7f3a7c01c3f0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 -- 172.21.15.164:0/3763384620 shutdown_connections 2024-05-08T04:43:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 --2- 172.21.15.164:0/3763384620 >> v2:172.21.15.172:3300/0 conn(0x7f3a9015ea20 0x7f3a9015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 --2- 172.21.15.164:0/3763384620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a900cfd60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 -- 172.21.15.164:0/3763384620 >> 172.21.15.164:0/3763384620 conn(0x7f3a9006ac90 msgr2=0x7f3a9006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 -- 172.21.15.164:0/3763384620 shutdown_connections 2024-05-08T04:43:35.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 -- 172.21.15.164:0/3763384620 wait complete. 2024-05-08T04:43:35.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.339+0000 7f3a97000640 1 Processor -- start 2024-05-08T04:43:35.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- start start 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a90144200 0x7f3a9013bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a9015ea20 0x7f3a9015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900cf370 con 0x7f3a90161950 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900cf1f0 con 0x7f3a9015ea20 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9016acf0 con 0x7f3a90144200 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59932/0 (socket says 172.21.15.164:59932) 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 -- 172.21.15.164:0/2730888001 learned_addr learned my addr 172.21.15.164:0/2730888001 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a8f400640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a9015ea20 0x7f3a9015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a8fe00640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a90144200 0x7f3a9013bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 -- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a90144200 msgr2=0x7f3a9013bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a90144200 0x7f3a9013bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 -- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a9015ea20 msgr2=0x7f3a9015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a9015ea20 0x7f3a9015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 -- 172.21.15.164:0/2730888001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9015fe90 con 0x7f3a90161950 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a95000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007e40 tx=0x7f3a8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014620 con 0x7f3a90161950 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88019070 con 0x7f3a90161950 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9015bce0 con 0x7f3a90161950 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014a20 con 0x7f3a90161950 2024-05-08T04:43:35.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9015d030 con 0x7f3a90161950 2024-05-08T04:43:35.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.343+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90069d60 con 0x7f3a90161950 2024-05-08T04:43:35.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.347+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f3a88014bc0 con 0x7f3a90161950 2024-05-08T04:43:35.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.347+0000 7f3a87e00640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 0x7f3a70069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.347+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3a88084130 con 0x7f3a90161950 2024-05-08T04:43:35.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.347+0000 7f3a8fe00640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 0x7f3a70069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.347+0000 7f3a8fe00640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 0x7f3a70069f60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3a80007fd0 tx=0x7f3a80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.351+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a88056540 con 0x7f3a90161950 2024-05-08T04:43:35.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.503+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a90071610 con 0x7f3a70067aa0 2024-05-08T04:43:35.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.507+0000 7f3a87e00640 1 -- 172.21.15.164:0/2730888001 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252444 (secure 0 0 0) 0x7f3a90071610 con 0x7f3a70067aa0 2024-05-08T04:43:35.512 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:35.513 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 msgr2=0x7f3a70069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 0x7f3a70069f60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3a80007fd0 tx=0x7f3a80009040 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 msgr2=0x7f3a9013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007e40 tx=0x7f3a8800e4b0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 shutdown_connections 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3a70067aa0 0x7f3a70069f60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a90161950 0x7f3a9013e850 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a9015ea20 0x7f3a9015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 --2- 172.21.15.164:0/2730888001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a90144200 0x7f3a9013bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 >> 172.21.15.164:0/2730888001 conn(0x7f3a9006ac90 msgr2=0x7f3a900af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 shutdown_connections 2024-05-08T04:43:35.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.511+0000 7f3a97000640 1 -- 172.21.15.164:0/2730888001 wait complete. 2024-05-08T04:43:35.532 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-05-08T04:43:35.491937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262892,"kb_used_data":13412,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711828,"statfs":{"total":773094113280,"available":772824911872,"internally_reserved":0,"allocated":13733888,"data_stored":6656698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001296"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457762+0000","last_change":"2024-05-08T04:43:31.486018+0000","last_active":"2024-05-08T04:43:32.457762+0000","last_peered":"2024-05-08T04:43:32.457762+0000","last_clean":"2024-05-08T04:43:32.457762+0000","last_became_active":"2024-05-08T04:43:31.485177+0000","last_became_peered":"2024-05-08T04:43:31.485177+0000","last_unstale":"2024-05-08T04:43:32.457762+0000","last_undegraded":"2024-05-08T04:43:32.457762+0000","last_fullsized":"2024-05-08T04:43:32.457762+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492906+0000","last_change":"2024-05-08T04:43:31.486210+0000","last_active":"2024-05-08T04:43:31.492906+0000","last_peered":"2024-05-08T04:43:31.492906+0000","last_clean":"2024-05-08T04:43:31.492906+0000","last_became_active":"2024-05-08T04:43:31.485711+0000","last_became_peered":"2024-05-08T04:43:31.485711+0000","last_unstale":"2024-05-08T04:43:31.492906+0000","last_undegraded":"2024-05-08T04:43:31.492906+0000","last_fullsized":"2024-05-08T04:43:31.492906+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:05.408804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460022+0000","last_change":"2024-05-08T04:43:31.477799+0000","last_active":"2024-05-08T04:43:32.460022+0000","last_peered":"2024-05-08T04:43:32.460022+0000","last_clean":"2024-05-08T04:43:32.460022+0000","last_became_active":"2024-05-08T04:43:31.477275+0000","last_became_peered":"2024-05-08T04:43:31.477275+0000","last_unstale":"2024-05-08T04:43:32.460022+0000","last_undegraded":"2024-05-08T04:43:32.460022+0000","last_fullsized":"2024-05-08T04:43:32.460022+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:55.658607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457515+0000","last_change":"2024-05-08T04:43:31.479594+0000","last_active":"2024-05-08T04:43:32.457515+0000","last_peered":"2024-05-08T04:43:32.457515+0000","last_clean":"2024-05-08T04:43:32.457515+0000","last_became_active":"2024-05-08T04:43:31.476126+0000","last_became_peered":"2024-05-08T04:43:31.476126+0000","last_unstale":"2024-05-08T04:43:32.457515+0000","last_undegraded":"2024-05-08T04:43:32.457515+0000","last_fullsized":"2024-05-08T04:43:32.457515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:17.307833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459112+0000","last_change":"2024-05-08T04:43:31.468081+0000","last_active":"2024-05-08T04:43:32.459112+0000","last_peered":"2024-05-08T04:43:32.459112+0000","last_clean":"2024-05-08T04:43:32.459112+0000","last_became_active":"2024-05-08T04:43:31.467837+0000","last_became_peered":"2024-05-08T04:43:31.467837+0000","last_unstale":"2024-05-08T04:43:32.459112+0000","last_undegraded":"2024-05-08T04:43:32.459112+0000","last_fullsized":"2024-05-08T04:43:32.459112+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:14.582994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.480538+0000","last_change":"2024-05-08T04:43:31.474245+0000","last_active":"2024-05-08T04:43:31.480538+0000","last_peered":"2024-05-08T04:43:31.480538+0000","last_clean":"2024-05-08T04:43:31.480538+0000","last_became_active":"2024-05-08T04:43:31.473970+0000","last_became_peered":"2024-05-08T04:43:31.473970+0000","last_unstale":"2024-05-08T04:43:31.480538+0000","last_undegraded":"2024-05-08T04:43:31.480538+0000","last_fullsized":"2024-05-08T04:43:31.480538+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:27.165879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.486453+0000","last_change":"2024-05-08T04:43:31.484686+0000","last_active":"2024-05-08T04:43:31.486453+0000","last_peered":"2024-05-08T04:43:31.486453+0000","last_clean":"2024-05-08T04:43:31.486453+0000","last_became_active":"2024-05-08T04:43:31.483615+0000","last_became_peered":"2024-05-08T04:43:31.483615+0000","last_unstale":"2024-05-08T04:43:31.486453+0000","last_undegraded":"2024-05-08T04:43:31.486453+0000","last_fullsized":"2024-05-08T04:43:31.486453+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:25.878002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459364+0000","last_change":"2024-05-08T04:43:31.482895+0000","last_active":"2024-05-08T04:43:32.459364+0000","last_peered":"2024-05-08T04:43:32.459364+0000","last_clean":"2024-05-08T04:43:32.459364+0000","last_became_active":"2024-05-08T04:43:31.481120+0000","last_became_peered":"2024-05-08T04:43:31.481120+0000","last_unstale":"2024-05-08T04:43:32.459364+0000","last_undegraded":"2024-05-08T04:43:32.459364+0000","last_fullsized":"2024-05-08T04:43:32.459364+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:22.734802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460271+0000","last_change":"2024-05-08T04:43:31.484206+0000","last_active":"2024-05-08T04:43:32.460271+0000","last_peered":"2024-05-08T04:43:32.460271+0000","last_clean":"2024-05-08T04:43:32.460271+0000","last_became_active":"2024-05-08T04:43:31.481803+0000","last_became_peered":"2024-05-08T04:43:31.481803+0000","last_unstale":"2024-05-08T04:43:32.460271+0000","last_undegraded":"2024-05-08T04:43:32.460271+0000","last_fullsized":"2024-05-08T04:43:32.460271+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:21.833733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492959+0000","last_change":"2024-05-08T04:43:31.486292+0000","last_active":"2024-05-08T04:43:31.492959+0000","last_peered":"2024-05-08T04:43:31.492959+0000","last_clean":"2024-05-08T04:43:31.492959+0000","last_became_active":"2024-05-08T04:43:31.486048+0000","last_became_peered":"2024-05-08T04:43:31.486048+0000","last_unstale":"2024-05-08T04:43:31.492959+0000","last_undegraded":"2024-05-08T04:43:31.492959+0000","last_fullsized":"2024-05-08T04:43:31.492959+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:29.154753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492062+0000","last_change":"2024-05-08T04:43:31.483425+0000","last_active":"2024-05-08T04:43:31.492062+0000","last_peered":"2024-05-08T04:43:31.492062+0000","last_clean":"2024-05-08T04:43:31.492062+0000","last_became_active":"2024-05-08T04:43:31.483127+0000","last_became_peered":"2024-05-08T04:43:31.483127+0000","last_unstale":"2024-05-08T04:43:31.492062+0000","last_undegraded":"2024-05-08T04:43:31.492062+0000","last_fullsized":"2024-05-08T04:43:31.492062+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:23.346207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458766+0000","last_change":"2024-05-08T04:43:31.483525+0000","last_active":"2024-05-08T04:43:32.458766+0000","last_peered":"2024-05-08T04:43:32.458766+0000","last_clean":"2024-05-08T04:43:32.458766+0000","last_became_active":"2024-05-08T04:43:31.480800+0000","last_became_peered":"2024-05-08T04:43:31.480800+0000","last_unstale":"2024-05-08T04:43:32.458766+0000","last_undegraded":"2024-05-08T04:43:32.458766+0000","last_fullsized":"2024-05-08T04:43:32.458766+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:53.310699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457397+0000","last_change":"2024-05-08T04:43:31.473654+0000","last_active":"2024-05-08T04:43:32.457397+0000","last_peered":"2024-05-08T04:43:32.457397+0000","last_clean":"2024-05-08T04:43:32.457397+0000","last_became_active":"2024-05-08T04:43:31.473317+0000","last_became_peered":"2024-05-08T04:43:31.473317+0000","last_unstale":"2024-05-08T04:43:32.457397+0000","last_undegraded":"2024-05-08T04:43:32.457397+0000","last_fullsized":"2024-05-08T04:43:32.457397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:40:26.873496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.493406+0000","last_change":"2024-05-08T04:43:31.487238+0000","last_active":"2024-05-08T04:43:31.493406+0000","last_peered":"2024-05-08T04:43:31.493406+0000","last_clean":"2024-05-08T04:43:31.493406+0000","last_became_active":"2024-05-08T04:43:31.486847+0000","last_became_peered":"2024-05-08T04:43:31.486847+0000","last_unstale":"2024-05-08T04:43:31.493406+0000","last_undegraded":"2024-05-08T04:43:31.493406+0000","last_fullsized":"2024-05-08T04:43:31.493406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:15.911273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457237+0000","last_change":"2024-05-08T04:43:31.487515+0000","last_active":"2024-05-08T04:43:32.457237+0000","last_peered":"2024-05-08T04:43:32.457237+0000","last_clean":"2024-05-08T04:43:32.457237+0000","last_became_active":"2024-05-08T04:43:31.484905+0000","last_became_peered":"2024-05-08T04:43:31.484905+0000","last_unstale":"2024-05-08T04:43:32.457237+0000","last_undegraded":"2024-05-08T04:43:32.457237+0000","last_fullsized":"2024-05-08T04:43:32.457237+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:15.662981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.487178+0000","last_change":"2024-05-08T04:43:31.481883+0000","last_active":"2024-05-08T04:43:31.487178+0000","last_peered":"2024-05-08T04:43:31.487178+0000","last_clean":"2024-05-08T04:43:31.487178+0000","last_became_active":"2024-05-08T04:43:31.481616+0000","last_became_peered":"2024-05-08T04:43:31.481616+0000","last_unstale":"2024-05-08T04:43:31.487178+0000","last_undegraded":"2024-05-08T04:43:31.487178+0000","last_fullsized":"2024-05-08T04:43:31.487178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:28.462327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.474690+0000","last_change":"2024-05-08T04:43:31.470256+0000","last_active":"2024-05-08T04:43:31.474690+0000","last_peered":"2024-05-08T04:43:31.474690+0000","last_clean":"2024-05-08T04:43:31.474690+0000","last_became_active":"2024-05-08T04:43:31.468022+0000","last_became_peered":"2024-05-08T04:43:31.468022+0000","last_unstale":"2024-05-08T04:43:31.474690+0000","last_undegraded":"2024-05-08T04:43:31.474690+0000","last_fullsized":"2024-05-08T04:43:31.474690+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:34.681198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479681+0000","last_change":"2024-05-08T04:43:31.476413+0000","last_active":"2024-05-08T04:43:31.479681+0000","last_peered":"2024-05-08T04:43:31.479681+0000","last_clean":"2024-05-08T04:43:31.479681+0000","last_became_active":"2024-05-08T04:43:31.475775+0000","last_became_peered":"2024-05-08T04:43:31.475775+0000","last_unstale":"2024-05-08T04:43:31.479681+0000","last_undegraded":"2024-05-08T04:43:31.479681+0000","last_fullsized":"2024-05-08T04:43:31.479681+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:45:46.603370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467328+0000","last_change":"2024-05-08T04:43:31.465156+0000","last_active":"2024-05-08T04:43:31.467328+0000","last_peered":"2024-05-08T04:43:31.467328+0000","last_clean":"2024-05-08T04:43:31.467328+0000","last_became_active":"2024-05-08T04:43:31.464852+0000","last_became_peered":"2024-05-08T04:43:31.464852+0000","last_unstale":"2024-05-08T04:43:31.467328+0000","last_undegraded":"2024-05-08T04:43:31.467328+0000","last_fullsized":"2024-05-08T04:43:31.467328+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:15.808802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490583+0000","last_change":"2024-05-08T04:43:31.486305+0000","last_active":"2024-05-08T04:43:31.490583+0000","last_peered":"2024-05-08T04:43:31.490583+0000","last_clean":"2024-05-08T04:43:31.490583+0000","last_became_active":"2024-05-08T04:43:31.486038+0000","last_became_peered":"2024-05-08T04:43:31.486038+0000","last_unstale":"2024-05-08T04:43:31.490583+0000","last_undegraded":"2024-05-08T04:43:31.490583+0000","last_fullsized":"2024-05-08T04:43:31.490583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:00.822358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458966+0000","last_change":"2024-05-08T04:43:31.467043+0000","last_active":"2024-05-08T04:43:32.458966+0000","last_peered":"2024-05-08T04:43:32.458966+0000","last_clean":"2024-05-08T04:43:32.458966+0000","last_became_active":"2024-05-08T04:43:31.466718+0000","last_became_peered":"2024-05-08T04:43:31.466718+0000","last_unstale":"2024-05-08T04:43:32.458966+0000","last_undegraded":"2024-05-08T04:43:32.458966+0000","last_fullsized":"2024-05-08T04:43:32.458966+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:04.139555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.482376+0000","last_change":"2024-05-08T04:43:31.477397+0000","last_active":"2024-05-08T04:43:31.482376+0000","last_peered":"2024-05-08T04:43:31.482376+0000","last_clean":"2024-05-08T04:43:31.482376+0000","last_became_active":"2024-05-08T04:43:31.477133+0000","last_became_peered":"2024-05-08T04:43:31.477133+0000","last_unstale":"2024-05-08T04:43:31.482376+0000","last_undegraded":"2024-05-08T04:43:31.482376+0000","last_fullsized":"2024-05-08T04:43:31.482376+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:48.844368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459400+0000","last_change":"2024-05-08T04:43:31.471141+0000","last_active":"2024-05-08T04:43:32.459400+0000","last_peered":"2024-05-08T04:43:32.459400+0000","last_clean":"2024-05-08T04:43:32.459400+0000","last_became_active":"2024-05-08T04:43:31.470704+0000","last_became_peered":"2024-05-08T04:43:31.470704+0000","last_unstale":"2024-05-08T04:43:32.459400+0000","last_undegraded":"2024-05-08T04:43:32.459400+0000","last_fullsized":"2024-05-08T04:43:32.459400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:40.175901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479774+0000","last_change":"2024-05-08T04:43:31.473031+0000","last_active":"2024-05-08T04:43:31.479774+0000","last_peered":"2024-05-08T04:43:31.479774+0000","last_clean":"2024-05-08T04:43:31.479774+0000","last_became_active":"2024-05-08T04:43:31.468864+0000","last_became_peered":"2024-05-08T04:43:31.468864+0000","last_unstale":"2024-05-08T04:43:31.479774+0000","last_undegraded":"2024-05-08T04:43:31.479774+0000","last_fullsized":"2024-05-08T04:43:31.479774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:10.018406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.489313+0000","last_change":"2024-05-08T04:43:31.483288+0000","last_active":"2024-05-08T04:43:31.489313+0000","last_peered":"2024-05-08T04:43:31.489313+0000","last_clean":"2024-05-08T04:43:31.489313+0000","last_became_active":"2024-05-08T04:43:31.483022+0000","last_became_peered":"2024-05-08T04:43:31.483022+0000","last_unstale":"2024-05-08T04:43:31.489313+0000","last_undegraded":"2024-05-08T04:43:31.489313+0000","last_fullsized":"2024-05-08T04:43:31.489313+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:56.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479702+0000","last_change":"2024-05-08T04:43:31.477853+0000","last_active":"2024-05-08T04:43:31.479702+0000","last_peered":"2024-05-08T04:43:31.479702+0000","last_clean":"2024-05-08T04:43:31.479702+0000","last_became_active":"2024-05-08T04:43:31.476151+0000","last_became_peered":"2024-05-08T04:43:31.476151+0000","last_unstale":"2024-05-08T04:43:31.479702+0000","last_undegraded":"2024-05-08T04:43:31.479702+0000","last_fullsized":"2024-05-08T04:43:31.479702+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:06.761771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457424+0000","last_change":"2024-05-08T04:43:31.476653+0000","last_active":"2024-05-08T04:43:32.457424+0000","last_peered":"2024-05-08T04:43:32.457424+0000","last_clean":"2024-05-08T04:43:32.457424+0000","last_became_active":"2024-05-08T04:43:31.474249+0000","last_became_peered":"2024-05-08T04:43:31.474249+0000","last_unstale":"2024-05-08T04:43:32.457424+0000","last_undegraded":"2024-05-08T04:43:32.457424+0000","last_fullsized":"2024-05-08T04:43:32.457424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:45.930400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.488090+0000","last_change":"2024-05-08T04:43:31.482185+0000","last_active":"2024-05-08T04:43:31.488090+0000","last_peered":"2024-05-08T04:43:31.488090+0000","last_clean":"2024-05-08T04:43:31.488090+0000","last_became_active":"2024-05-08T04:43:31.481865+0000","last_became_peered":"2024-05-08T04:43:31.481865+0000","last_unstale":"2024-05-08T04:43:31.488090+0000","last_undegraded":"2024-05-08T04:43:31.488090+0000","last_fullsized":"2024-05-08T04:43:31.488090+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:56.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457445+0000","last_change":"2024-05-08T04:43:31.482794+0000","last_active":"2024-05-08T04:43:32.457445+0000","last_peered":"2024-05-08T04:43:32.457445+0000","last_clean":"2024-05-08T04:43:32.457445+0000","last_became_active":"2024-05-08T04:43:31.482509+0000","last_became_peered":"2024-05-08T04:43:31.482509+0000","last_unstale":"2024-05-08T04:43:32.457445+0000","last_undegraded":"2024-05-08T04:43:32.457445+0000","last_fullsized":"2024-05-08T04:43:32.457445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:54.049101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457274+0000","last_change":"2024-05-08T04:43:31.477091+0000","last_active":"2024-05-08T04:43:32.457274+0000","last_peered":"2024-05-08T04:43:32.457274+0000","last_clean":"2024-05-08T04:43:32.457274+0000","last_became_active":"2024-05-08T04:43:31.472252+0000","last_became_peered":"2024-05-08T04:43:31.472252+0000","last_unstale":"2024-05-08T04:43:32.457274+0000","last_undegraded":"2024-05-08T04:43:32.457274+0000","last_fullsized":"2024-05-08T04:43:32.457274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:45.871755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458851+0000","last_change":"2024-05-08T04:43:31.481585+0000","last_active":"2024-05-08T04:43:32.458851+0000","last_peered":"2024-05-08T04:43:32.458851+0000","last_clean":"2024-05-08T04:43:32.458851+0000","last_became_active":"2024-05-08T04:43:31.480859+0000","last_became_peered":"2024-05-08T04:43:31.480859+0000","last_unstale":"2024-05-08T04:43:32.458851+0000","last_undegraded":"2024-05-08T04:43:32.458851+0000","last_fullsized":"2024-05-08T04:43:32.458851+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:17.859746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.476262+0000","last_change":"2024-05-08T04:43:31.473494+0000","last_active":"2024-05-08T04:43:31.476262+0000","last_peered":"2024-05-08T04:43:31.476262+0000","last_clean":"2024-05-08T04:43:31.476262+0000","last_became_active":"2024-05-08T04:43:31.473238+0000","last_became_peered":"2024-05-08T04:43:31.473238+0000","last_unstale":"2024-05-08T04:43:31.476262+0000","last_undegraded":"2024-05-08T04:43:31.476262+0000","last_fullsized":"2024-05-08T04:43:31.476262+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:20:00.752055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479512+0000","last_change":"2024-05-08T04:43:31.471881+0000","last_active":"2024-05-08T04:43:31.479512+0000","last_peered":"2024-05-08T04:43:31.479512+0000","last_clean":"2024-05-08T04:43:31.479512+0000","last_became_active":"2024-05-08T04:43:31.471628+0000","last_became_peered":"2024-05-08T04:43:31.471628+0000","last_unstale":"2024-05-08T04:43:31.479512+0000","last_undegraded":"2024-05-08T04:43:31.479512+0000","last_fullsized":"2024-05-08T04:43:31.479512+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:51:28.548705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459838+0000","last_change":"2024-05-08T04:43:30.471425+0000","last_active":"2024-05-08T04:43:32.459838+0000","last_peered":"2024-05-08T04:43:32.459838+0000","last_clean":"2024-05-08T04:43:32.459838+0000","last_became_active":"2024-05-08T04:43:30.470439+0000","last_became_peered":"2024-05-08T04:43:30.470439+0000","last_unstale":"2024-05-08T04:43:32.459838+0000","last_undegraded":"2024-05-08T04:43:32.459838+0000","last_fullsized":"2024-05-08T04:43:32.459838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:33:59.817602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460352+0000","last_change":"2024-05-08T04:43:31.485079+0000","last_active":"2024-05-08T04:43:32.460352+0000","last_peered":"2024-05-08T04:43:32.460352+0000","last_clean":"2024-05-08T04:43:32.460352+0000","last_became_active":"2024-05-08T04:43:31.484713+0000","last_became_peered":"2024-05-08T04:43:31.484713+0000","last_unstale":"2024-05-08T04:43:32.460352+0000","last_undegraded":"2024-05-08T04:43:32.460352+0000","last_fullsized":"2024-05-08T04:43:32.460352+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:02.801555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.483499+0000","last_change":"2024-05-08T04:43:30.476351+0000","last_active":"2024-05-08T04:43:31.483499+0000","last_peered":"2024-05-08T04:43:31.483499+0000","last_clean":"2024-05-08T04:43:31.483499+0000","last_became_active":"2024-05-08T04:43:30.476056+0000","last_became_peered":"2024-05-08T04:43:30.476056+0000","last_unstale":"2024-05-08T04:43:31.483499+0000","last_undegraded":"2024-05-08T04:43:31.483499+0000","last_fullsized":"2024-05-08T04:43:31.483499+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:44.925644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459311+0000","last_change":"2024-05-08T04:43:31.474313+0000","last_active":"2024-05-08T04:43:32.459311+0000","last_peered":"2024-05-08T04:43:32.459311+0000","last_clean":"2024-05-08T04:43:32.459311+0000","last_became_active":"2024-05-08T04:43:31.473952+0000","last_became_peered":"2024-05-08T04:43:31.473952+0000","last_unstale":"2024-05-08T04:43:32.459311+0000","last_undegraded":"2024-05-08T04:43:32.459311+0000","last_fullsized":"2024-05-08T04:43:32.459311+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:15.808802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492164+0000","last_change":"2024-05-08T04:43:30.478483+0000","last_active":"2024-05-08T04:43:31.492164+0000","last_peered":"2024-05-08T04:43:31.492164+0000","last_clean":"2024-05-08T04:43:31.492164+0000","last_became_active":"2024-05-08T04:43:30.478178+0000","last_became_peered":"2024-05-08T04:43:30.478178+0000","last_unstale":"2024-05-08T04:43:31.492164+0000","last_undegraded":"2024-05-08T04:43:31.492164+0000","last_fullsized":"2024-05-08T04:43:31.492164+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:02.752884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459000+0000","last_change":"2024-05-08T04:43:31.476742+0000","last_active":"2024-05-08T04:43:32.459000+0000","last_peered":"2024-05-08T04:43:32.459000+0000","last_clean":"2024-05-08T04:43:32.459000+0000","last_became_active":"2024-05-08T04:43:31.476524+0000","last_became_peered":"2024-05-08T04:43:31.476524+0000","last_unstale":"2024-05-08T04:43:32.459000+0000","last_undegraded":"2024-05-08T04:43:32.459000+0000","last_fullsized":"2024-05-08T04:43:32.459000+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:10.533184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.484514+0000","last_change":"2024-05-08T04:43:30.476756+0000","last_active":"2024-05-08T04:43:31.484514+0000","last_peered":"2024-05-08T04:43:31.484514+0000","last_clean":"2024-05-08T04:43:31.484514+0000","last_became_active":"2024-05-08T04:43:30.476498+0000","last_became_peered":"2024-05-08T04:43:30.476498+0000","last_unstale":"2024-05-08T04:43:31.484514+0000","last_undegraded":"2024-05-08T04:43:31.484514+0000","last_fullsized":"2024-05-08T04:43:31.484514+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:57.836878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479601+0000","last_change":"2024-05-08T04:43:31.474844+0000","last_active":"2024-05-08T04:43:31.479601+0000","last_peered":"2024-05-08T04:43:31.479601+0000","last_clean":"2024-05-08T04:43:31.479601+0000","last_became_active":"2024-05-08T04:43:31.474544+0000","last_became_peered":"2024-05-08T04:43:31.474544+0000","last_unstale":"2024-05-08T04:43:31.479601+0000","last_undegraded":"2024-05-08T04:43:31.479601+0000","last_fullsized":"2024-05-08T04:43:31.479601+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:22.810223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.483612+0000","last_change":"2024-05-08T04:43:30.479947+0000","last_active":"2024-05-08T04:43:31.483612+0000","last_peered":"2024-05-08T04:43:31.483612+0000","last_clean":"2024-05-08T04:43:31.483612+0000","last_became_active":"2024-05-08T04:43:30.479683+0000","last_became_peered":"2024-05-08T04:43:30.479683+0000","last_unstale":"2024-05-08T04:43:31.483612+0000","last_undegraded":"2024-05-08T04:43:31.483612+0000","last_fullsized":"2024-05-08T04:43:31.483612+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:44.925644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.470865+0000","last_change":"2024-05-08T04:43:31.464745+0000","last_active":"2024-05-08T04:43:31.470865+0000","last_peered":"2024-05-08T04:43:31.470865+0000","last_clean":"2024-05-08T04:43:31.470865+0000","last_became_active":"2024-05-08T04:43:31.464447+0000","last_became_peered":"2024-05-08T04:43:31.464447+0000","last_unstale":"2024-05-08T04:43:31.470865+0000","last_undegraded":"2024-05-08T04:43:31.470865+0000","last_fullsized":"2024-05-08T04:43:31.470865+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:21.833733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490338+0000","last_change":"2024-05-08T04:43:30.481314+0000","last_active":"2024-05-08T04:43:31.490338+0000","last_peered":"2024-05-08T04:43:31.490338+0000","last_clean":"2024-05-08T04:43:31.490338+0000","last_became_active":"2024-05-08T04:43:30.481035+0000","last_became_peered":"2024-05-08T04:43:30.481035+0000","last_unstale":"2024-05-08T04:43:31.490338+0000","last_undegraded":"2024-05-08T04:43:31.490338+0000","last_fullsized":"2024-05-08T04:43:31.490338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:29.055940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459524+0000","last_change":"2024-05-08T04:43:31.483737+0000","last_active":"2024-05-08T04:43:32.459524+0000","last_peered":"2024-05-08T04:43:32.459524+0000","last_clean":"2024-05-08T04:43:32.459524+0000","last_became_active":"2024-05-08T04:43:31.483248+0000","last_became_peered":"2024-05-08T04:43:31.483248+0000","last_unstale":"2024-05-08T04:43:32.459524+0000","last_undegraded":"2024-05-08T04:43:32.459524+0000","last_fullsized":"2024-05-08T04:43:32.459524+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:17.189675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.457323+0000","last_change":"2024-05-08T04:43:30.474778+0000","last_active":"2024-05-08T04:43:31.457323+0000","last_peered":"2024-05-08T04:43:31.457323+0000","last_clean":"2024-05-08T04:43:31.457323+0000","last_became_active":"2024-05-08T04:43:30.474495+0000","last_became_peered":"2024-05-08T04:43:30.474495+0000","last_unstale":"2024-05-08T04:43:31.457323+0000","last_undegraded":"2024-05-08T04:43:31.457323+0000","last_fullsized":"2024-05-08T04:43:31.457323+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:54.653851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492773+0000","last_change":"2024-05-08T04:43:31.484507+0000","last_active":"2024-05-08T04:43:31.492773+0000","last_peered":"2024-05-08T04:43:31.492773+0000","last_clean":"2024-05-08T04:43:31.492773+0000","last_became_active":"2024-05-08T04:43:31.483969+0000","last_became_peered":"2024-05-08T04:43:31.483969+0000","last_unstale":"2024-05-08T04:43:31.492773+0000","last_undegraded":"2024-05-08T04:43:31.492773+0000","last_fullsized":"2024-05-08T04:43:31.492773+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:54.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467641+0000","last_change":"2024-05-08T04:43:30.469079+0000","last_active":"2024-05-08T04:43:31.467641+0000","last_peered":"2024-05-08T04:43:31.467641+0000","last_clean":"2024-05-08T04:43:31.467641+0000","last_became_active":"2024-05-08T04:43:30.468828+0000","last_became_peered":"2024-05-08T04:43:30.468828+0000","last_unstale":"2024-05-08T04:43:31.467641+0000","last_undegraded":"2024-05-08T04:43:31.467641+0000","last_fullsized":"2024-05-08T04:43:31.467641+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:16.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457552+0000","last_change":"2024-05-08T04:43:31.476308+0000","last_active":"2024-05-08T04:43:32.457552+0000","last_peered":"2024-05-08T04:43:32.457552+0000","last_clean":"2024-05-08T04:43:32.457552+0000","last_became_active":"2024-05-08T04:43:31.475940+0000","last_became_peered":"2024-05-08T04:43:31.475940+0000","last_unstale":"2024-05-08T04:43:32.457552+0000","last_undegraded":"2024-05-08T04:43:32.457552+0000","last_fullsized":"2024-05-08T04:43:32.457552+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:03.366436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.458120+0000","last_change":"2024-05-08T04:43:30.472372+0000","last_active":"2024-05-08T04:43:31.458120+0000","last_peered":"2024-05-08T04:43:31.458120+0000","last_clean":"2024-05-08T04:43:31.458120+0000","last_became_active":"2024-05-08T04:43:30.472084+0000","last_became_peered":"2024-05-08T04:43:30.472084+0000","last_unstale":"2024-05-08T04:43:31.458120+0000","last_undegraded":"2024-05-08T04:43:31.458120+0000","last_fullsized":"2024-05-08T04:43:31.458120+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:53.044345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490416+0000","last_change":"2024-05-08T04:43:31.482283+0000","last_active":"2024-05-08T04:43:31.490416+0000","last_peered":"2024-05-08T04:43:31.490416+0000","last_clean":"2024-05-08T04:43:31.490416+0000","last_became_active":"2024-05-08T04:43:31.481954+0000","last_became_peered":"2024-05-08T04:43:31.481954+0000","last_unstale":"2024-05-08T04:43:31.490416+0000","last_undegraded":"2024-05-08T04:43:31.490416+0000","last_fullsized":"2024-05-08T04:43:31.490416+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:09.360952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.463577+0000","last_change":"2024-05-08T04:43:30.474689+0000","last_active":"2024-05-08T04:43:31.463577+0000","last_peered":"2024-05-08T04:43:31.463577+0000","last_clean":"2024-05-08T04:43:31.463577+0000","last_became_active":"2024-05-08T04:43:30.474393+0000","last_became_peered":"2024-05-08T04:43:30.474393+0000","last_unstale":"2024-05-08T04:43:31.463577+0000","last_undegraded":"2024-05-08T04:43:31.463577+0000","last_fullsized":"2024-05-08T04:43:31.463577+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:36.975260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458774+0000","last_change":"2024-05-08T04:43:31.478897+0000","last_active":"2024-05-08T04:43:32.458774+0000","last_peered":"2024-05-08T04:43:32.458774+0000","last_clean":"2024-05-08T04:43:32.458774+0000","last_became_active":"2024-05-08T04:43:31.477660+0000","last_became_peered":"2024-05-08T04:43:31.477660+0000","last_unstale":"2024-05-08T04:43:32.458774+0000","last_undegraded":"2024-05-08T04:43:32.458774+0000","last_fullsized":"2024-05-08T04:43:32.458774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:40.223740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.491868+0000","last_change":"2024-05-08T04:43:30.477574+0000","last_active":"2024-05-08T04:43:31.491868+0000","last_peered":"2024-05-08T04:43:31.491868+0000","last_clean":"2024-05-08T04:43:31.491868+0000","last_became_active":"2024-05-08T04:43:30.477208+0000","last_became_peered":"2024-05-08T04:43:30.477208+0000","last_unstale":"2024-05-08T04:43:31.491868+0000","last_undegraded":"2024-05-08T04:43:31.491868+0000","last_fullsized":"2024-05-08T04:43:31.491868+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:57.836878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459224+0000","last_change":"2024-05-08T04:43:31.479073+0000","last_active":"2024-05-08T04:43:32.459224+0000","last_peered":"2024-05-08T04:43:32.459224+0000","last_clean":"2024-05-08T04:43:32.459224+0000","last_became_active":"2024-05-08T04:43:31.478708+0000","last_became_peered":"2024-05-08T04:43:31.478708+0000","last_unstale":"2024-05-08T04:43:32.459224+0000","last_undegraded":"2024-05-08T04:43:32.459224+0000","last_fullsized":"2024-05-08T04:43:32.459224+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:03.757640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458898+0000","last_change":"2024-05-08T04:43:30.475273+0000","last_active":"2024-05-08T04:43:32.458898+0000","last_peered":"2024-05-08T04:43:32.458898+0000","last_clean":"2024-05-08T04:43:32.458898+0000","last_became_active":"2024-05-08T04:43:30.474818+0000","last_became_peered":"2024-05-08T04:43:30.474818+0000","last_unstale":"2024-05-08T04:43:32.458898+0000","last_undegraded":"2024-05-08T04:43:32.458898+0000","last_fullsized":"2024-05-08T04:43:32.458898+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:13.389423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457548+0000","last_change":"2024-05-08T04:43:31.470983+0000","last_active":"2024-05-08T04:43:32.457548+0000","last_peered":"2024-05-08T04:43:32.457548+0000","last_clean":"2024-05-08T04:43:32.457548+0000","last_became_active":"2024-05-08T04:43:31.470687+0000","last_became_peered":"2024-05-08T04:43:31.470687+0000","last_unstale":"2024-05-08T04:43:32.457548+0000","last_undegraded":"2024-05-08T04:43:32.457548+0000","last_fullsized":"2024-05-08T04:43:32.457548+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:01.300356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457392+0000","last_change":"2024-05-08T04:43:30.469508+0000","last_active":"2024-05-08T04:43:32.457392+0000","last_peered":"2024-05-08T04:43:32.457392+0000","last_clean":"2024-05-08T04:43:32.457392+0000","last_became_active":"2024-05-08T04:43:30.469254+0000","last_became_peered":"2024-05-08T04:43:30.469254+0000","last_unstale":"2024-05-08T04:43:32.457392+0000","last_undegraded":"2024-05-08T04:43:32.457392+0000","last_fullsized":"2024-05-08T04:43:32.457392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:56.733449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.484895+0000","last_change":"2024-05-08T04:43:31.478638+0000","last_active":"2024-05-08T04:43:31.484895+0000","last_peered":"2024-05-08T04:43:31.484895+0000","last_clean":"2024-05-08T04:43:31.484895+0000","last_became_active":"2024-05-08T04:43:31.478335+0000","last_became_peered":"2024-05-08T04:43:31.478335+0000","last_unstale":"2024-05-08T04:43:31.484895+0000","last_undegraded":"2024-05-08T04:43:31.484895+0000","last_fullsized":"2024-05-08T04:43:31.484895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.470461+0000","last_change":"2024-05-08T04:43:30.466280+0000","last_active":"2024-05-08T04:43:31.470461+0000","last_peered":"2024-05-08T04:43:31.470461+0000","last_clean":"2024-05-08T04:43:31.470461+0000","last_became_active":"2024-05-08T04:43:30.466019+0000","last_became_peered":"2024-05-08T04:43:30.466019+0000","last_unstale":"2024-05-08T04:43:31.470461+0000","last_undegraded":"2024-05-08T04:43:31.470461+0000","last_fullsized":"2024-05-08T04:43:31.470461+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:02.361679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459156+0000","last_change":"2024-05-08T04:43:31.475677+0000","last_active":"2024-05-08T04:43:32.459156+0000","last_peered":"2024-05-08T04:43:32.459156+0000","last_clean":"2024-05-08T04:43:32.459156+0000","last_became_active":"2024-05-08T04:43:31.475274+0000","last_became_peered":"2024-05-08T04:43:31.475274+0000","last_unstale":"2024-05-08T04:43:32.459156+0000","last_undegraded":"2024-05-08T04:43:32.459156+0000","last_fullsized":"2024-05-08T04:43:32.459156+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:46.331910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459618+0000","last_change":"2024-05-08T04:43:30.478500+0000","last_active":"2024-05-08T04:43:32.459618+0000","last_peered":"2024-05-08T04:43:32.459618+0000","last_clean":"2024-05-08T04:43:32.459618+0000","last_became_active":"2024-05-08T04:43:30.478153+0000","last_became_peered":"2024-05-08T04:43:30.478153+0000","last_unstale":"2024-05-08T04:43:32.459618+0000","last_undegraded":"2024-05-08T04:43:32.459618+0000","last_fullsized":"2024-05-08T04:43:32.459618+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:52.913803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460188+0000","last_change":"2024-05-08T04:43:31.488981+0000","last_active":"2024-05-08T04:43:32.460188+0000","last_peered":"2024-05-08T04:43:32.460188+0000","last_clean":"2024-05-08T04:43:32.460188+0000","last_became_active":"2024-05-08T04:43:31.486691+0000","last_became_peered":"2024-05-08T04:43:31.486691+0000","last_unstale":"2024-05-08T04:43:32.460188+0000","last_undegraded":"2024-05-08T04:43:32.460188+0000","last_fullsized":"2024-05-08T04:43:32.460188+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:08.574905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.485533+0000","last_change":"2024-05-08T04:43:30.466252+0000","last_active":"2024-05-08T04:43:31.485533+0000","last_peered":"2024-05-08T04:43:31.485533+0000","last_clean":"2024-05-08T04:43:31.485533+0000","last_became_active":"2024-05-08T04:43:30.465955+0000","last_became_peered":"2024-05-08T04:43:30.465955+0000","last_unstale":"2024-05-08T04:43:31.485533+0000","last_undegraded":"2024-05-08T04:43:31.485533+0000","last_fullsized":"2024-05-08T04:43:31.485533+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.475023+0000","last_change":"2024-05-08T04:43:31.472728+0000","last_active":"2024-05-08T04:43:31.475023+0000","last_peered":"2024-05-08T04:43:31.475023+0000","last_clean":"2024-05-08T04:43:31.475023+0000","last_became_active":"2024-05-08T04:43:31.472418+0000","last_became_peered":"2024-05-08T04:43:31.472418+0000","last_unstale":"2024-05-08T04:43:31.475023+0000","last_undegraded":"2024-05-08T04:43:31.475023+0000","last_fullsized":"2024-05-08T04:43:31.475023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492551+0000","last_change":"2024-05-08T04:43:30.478007+0000","last_active":"2024-05-08T04:43:31.492551+0000","last_peered":"2024-05-08T04:43:31.492551+0000","last_clean":"2024-05-08T04:43:31.492551+0000","last_became_active":"2024-05-08T04:43:30.477742+0000","last_became_peered":"2024-05-08T04:43:30.477742+0000","last_unstale":"2024-05-08T04:43:31.492551+0000","last_undegraded":"2024-05-08T04:43:31.492551+0000","last_fullsized":"2024-05-08T04:43:31.492551+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:09.528428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492727+0000","last_change":"2024-05-08T04:43:31.490092+0000","last_active":"2024-05-08T04:43:31.492727+0000","last_peered":"2024-05-08T04:43:31.492727+0000","last_clean":"2024-05-08T04:43:31.492727+0000","last_became_active":"2024-05-08T04:43:31.489489+0000","last_became_peered":"2024-05-08T04:43:31.489489+0000","last_unstale":"2024-05-08T04:43:31.492727+0000","last_undegraded":"2024-05-08T04:43:31.492727+0000","last_fullsized":"2024-05-08T04:43:31.492727+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:15.841643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492568+0000","last_change":"2024-05-08T04:43:30.480111+0000","last_active":"2024-05-08T04:43:31.492568+0000","last_peered":"2024-05-08T04:43:31.492568+0000","last_clean":"2024-05-08T04:43:31.492568+0000","last_became_active":"2024-05-08T04:43:30.479714+0000","last_became_peered":"2024-05-08T04:43:30.479714+0000","last_unstale":"2024-05-08T04:43:31.492568+0000","last_undegraded":"2024-05-08T04:43:31.492568+0000","last_fullsized":"2024-05-08T04:43:31.492568+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:07.570149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.493473+0000","last_change":"2024-05-08T04:43:31.489681+0000","last_active":"2024-05-08T04:43:31.493473+0000","last_peered":"2024-05-08T04:43:31.493473+0000","last_clean":"2024-05-08T04:43:31.493473+0000","last_became_active":"2024-05-08T04:43:31.489349+0000","last_became_peered":"2024-05-08T04:43:31.489349+0000","last_unstale":"2024-05-08T04:43:31.493473+0000","last_undegraded":"2024-05-08T04:43:31.493473+0000","last_fullsized":"2024-05-08T04:43:31.493473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:32.704399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.458454+0000","last_change":"2024-05-08T04:43:30.474577+0000","last_active":"2024-05-08T04:43:31.458454+0000","last_peered":"2024-05-08T04:43:31.458454+0000","last_clean":"2024-05-08T04:43:31.458454+0000","last_became_active":"2024-05-08T04:43:30.473767+0000","last_became_peered":"2024-05-08T04:43:30.473767+0000","last_unstale":"2024-05-08T04:43:31.458454+0000","last_undegraded":"2024-05-08T04:43:31.458454+0000","last_fullsized":"2024-05-08T04:43:31.458454+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:02.361679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457238+0000","last_change":"2024-05-08T04:43:31.474426+0000","last_active":"2024-05-08T04:43:32.457238+0000","last_peered":"2024-05-08T04:43:32.457238+0000","last_clean":"2024-05-08T04:43:32.457238+0000","last_became_active":"2024-05-08T04:43:31.473622+0000","last_became_peered":"2024-05-08T04:43:31.473622+0000","last_unstale":"2024-05-08T04:43:32.457238+0000","last_undegraded":"2024-05-08T04:43:32.457238+0000","last_fullsized":"2024-05-08T04:43:32.457238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:35:02.611988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490221+0000","last_change":"2024-05-08T04:43:30.470959+0000","last_active":"2024-05-08T04:43:31.490221+0000","last_peered":"2024-05-08T04:43:31.490221+0000","last_clean":"2024-05-08T04:43:31.490221+0000","last_became_active":"2024-05-08T04:43:30.470651+0000","last_became_peered":"2024-05-08T04:43:30.470651+0000","last_unstale":"2024-05-08T04:43:31.490221+0000","last_undegraded":"2024-05-08T04:43:31.490221+0000","last_fullsized":"2024-05-08T04:43:31.490221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:16.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.485726+0000","last_change":"2024-05-08T04:43:31.484048+0000","last_active":"2024-05-08T04:43:31.485726+0000","last_peered":"2024-05-08T04:43:31.485726+0000","last_clean":"2024-05-08T04:43:31.485726+0000","last_became_active":"2024-05-08T04:43:31.483130+0000","last_became_peered":"2024-05-08T04:43:31.483130+0000","last_unstale":"2024-05-08T04:43:31.485726+0000","last_undegraded":"2024-05-08T04:43:31.485726+0000","last_fullsized":"2024-05-08T04:43:31.485726+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.718248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462707+0000","last_change":"2024-05-08T04:43:30.478103+0000","last_active":"2024-05-08T04:43:31.462707+0000","last_peered":"2024-05-08T04:43:31.462707+0000","last_clean":"2024-05-08T04:43:31.462707+0000","last_became_active":"2024-05-08T04:43:30.477814+0000","last_became_peered":"2024-05-08T04:43:30.477814+0000","last_unstale":"2024-05-08T04:43:31.462707+0000","last_undegraded":"2024-05-08T04:43:31.462707+0000","last_fullsized":"2024-05-08T04:43:31.462707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:34.713492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.489192+0000","last_change":"2024-05-08T04:43:31.481488+0000","last_active":"2024-05-08T04:43:31.489192+0000","last_peered":"2024-05-08T04:43:31.489192+0000","last_clean":"2024-05-08T04:43:31.489192+0000","last_became_active":"2024-05-08T04:43:31.481230+0000","last_became_peered":"2024-05-08T04:43:31.481230+0000","last_unstale":"2024-05-08T04:43:31.489192+0000","last_undegraded":"2024-05-08T04:43:31.489192+0000","last_fullsized":"2024-05-08T04:43:31.489192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:37.585579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462820+0000","last_change":"2024-05-08T04:43:30.477308+0000","last_active":"2024-05-08T04:43:31.462820+0000","last_peered":"2024-05-08T04:43:31.462820+0000","last_clean":"2024-05-08T04:43:31.462820+0000","last_became_active":"2024-05-08T04:43:30.477067+0000","last_became_peered":"2024-05-08T04:43:30.477067+0000","last_unstale":"2024-05-08T04:43:31.462820+0000","last_undegraded":"2024-05-08T04:43:31.462820+0000","last_fullsized":"2024-05-08T04:43:31.462820+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:24.873246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458767+0000","last_change":"2024-05-08T04:43:31.477148+0000","last_active":"2024-05-08T04:43:32.458767+0000","last_peered":"2024-05-08T04:43:32.458767+0000","last_clean":"2024-05-08T04:43:32.458767+0000","last_became_active":"2024-05-08T04:43:31.476880+0000","last_became_peered":"2024-05-08T04:43:31.476880+0000","last_unstale":"2024-05-08T04:43:32.458767+0000","last_undegraded":"2024-05-08T04:43:32.458767+0000","last_fullsized":"2024-05-08T04:43:32.458767+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.900967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.455593+0000","last_change":"2024-05-08T04:43:30.474183+0000","last_active":"2024-05-08T04:43:31.455593+0000","last_peered":"2024-05-08T04:43:31.455593+0000","last_clean":"2024-05-08T04:43:31.455593+0000","last_became_active":"2024-05-08T04:43:30.473930+0000","last_became_peered":"2024-05-08T04:43:30.473930+0000","last_unstale":"2024-05-08T04:43:31.455593+0000","last_undegraded":"2024-05-08T04:43:31.455593+0000","last_fullsized":"2024-05-08T04:43:31.455593+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:16.854989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.472448+0000","last_change":"2024-05-08T04:43:31.467882+0000","last_active":"2024-05-08T04:43:31.472448+0000","last_peered":"2024-05-08T04:43:31.472448+0000","last_clean":"2024-05-08T04:43:31.472448+0000","last_became_active":"2024-05-08T04:43:31.467573+0000","last_became_peered":"2024-05-08T04:43:31.467573+0000","last_unstale":"2024-05-08T04:43:31.472448+0000","last_undegraded":"2024-05-08T04:43:31.472448+0000","last_fullsized":"2024-05-08T04:43:31.472448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:02.801555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457855+0000","last_change":"2024-05-08T04:43:30.468636+0000","last_active":"2024-05-08T04:43:32.457855+0000","last_peered":"2024-05-08T04:43:32.457855+0000","last_clean":"2024-05-08T04:43:32.457855+0000","last_became_active":"2024-05-08T04:43:30.468358+0000","last_became_peered":"2024-05-08T04:43:30.468358+0000","last_unstale":"2024-05-08T04:43:32.457855+0000","last_undegraded":"2024-05-08T04:43:32.457855+0000","last_fullsized":"2024-05-08T04:43:32.457855+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:50.652015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457637+0000","last_change":"2024-05-08T04:43:31.479917+0000","last_active":"2024-05-08T04:43:32.457637+0000","last_peered":"2024-05-08T04:43:32.457637+0000","last_clean":"2024-05-08T04:43:32.457637+0000","last_became_active":"2024-05-08T04:43:31.479619+0000","last_became_peered":"2024-05-08T04:43:31.479619+0000","last_unstale":"2024-05-08T04:43:32.457637+0000","last_undegraded":"2024-05-08T04:43:32.457637+0000","last_fullsized":"2024-05-08T04:43:32.457637+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:58.841635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459392+0000","last_change":"2024-05-08T04:43:30.470703+0000","last_active":"2024-05-08T04:43:32.459392+0000","last_peered":"2024-05-08T04:43:32.459392+0000","last_clean":"2024-05-08T04:43:32.459392+0000","last_became_active":"2024-05-08T04:43:30.470326+0000","last_became_peered":"2024-05-08T04:43:30.470326+0000","last_unstale":"2024-05-08T04:43:32.459392+0000","last_undegraded":"2024-05-08T04:43:32.459392+0000","last_fullsized":"2024-05-08T04:43:32.459392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:27.457570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.491692+0000","last_change":"2024-05-08T04:43:31.484669+0000","last_active":"2024-05-08T04:43:31.491692+0000","last_peered":"2024-05-08T04:43:31.491692+0000","last_clean":"2024-05-08T04:43:31.491692+0000","last_became_active":"2024-05-08T04:43:31.484135+0000","last_became_peered":"2024-05-08T04:43:31.484135+0000","last_unstale":"2024-05-08T04:43:31.491692+0000","last_undegraded":"2024-05-08T04:43:31.491692+0000","last_fullsized":"2024-05-08T04:43:31.491692+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:17:25.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.469721+0000","last_change":"2024-05-08T04:43:30.474283+0000","last_active":"2024-05-08T04:43:31.469721+0000","last_peered":"2024-05-08T04:43:31.469721+0000","last_clean":"2024-05-08T04:43:31.469721+0000","last_became_active":"2024-05-08T04:43:30.473788+0000","last_became_peered":"2024-05-08T04:43:30.473788+0000","last_unstale":"2024-05-08T04:43:31.469721+0000","last_undegraded":"2024-05-08T04:43:31.469721+0000","last_fullsized":"2024-05-08T04:43:31.469721+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.480284+0000","last_change":"2024-05-08T04:43:31.475545+0000","last_active":"2024-05-08T04:43:31.480284+0000","last_peered":"2024-05-08T04:43:31.480284+0000","last_clean":"2024-05-08T04:43:31.480284+0000","last_became_active":"2024-05-08T04:43:31.474573+0000","last_became_peered":"2024-05-08T04:43:31.474573+0000","last_unstale":"2024-05-08T04:43:31.480284+0000","last_undegraded":"2024-05-08T04:43:31.480284+0000","last_fullsized":"2024-05-08T04:43:31.480284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:30.060696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.456171+0000","last_change":"2024-05-08T04:43:30.474802+0000","last_active":"2024-05-08T04:43:31.456171+0000","last_peered":"2024-05-08T04:43:31.456171+0000","last_clean":"2024-05-08T04:43:31.456171+0000","last_became_active":"2024-05-08T04:43:30.473013+0000","last_became_peered":"2024-05-08T04:43:30.473013+0000","last_unstale":"2024-05-08T04:43:31.456171+0000","last_undegraded":"2024-05-08T04:43:31.456171+0000","last_fullsized":"2024-05-08T04:43:31.456171+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.578238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458623+0000","last_change":"2024-05-08T04:43:31.473086+0000","last_active":"2024-05-08T04:43:32.458623+0000","last_peered":"2024-05-08T04:43:32.458623+0000","last_clean":"2024-05-08T04:43:32.458623+0000","last_became_active":"2024-05-08T04:43:31.472787+0000","last_became_peered":"2024-05-08T04:43:31.472787+0000","last_unstale":"2024-05-08T04:43:32.458623+0000","last_undegraded":"2024-05-08T04:43:32.458623+0000","last_fullsized":"2024-05-08T04:43:32.458623+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:55:45.500801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467264+0000","last_change":"2024-05-08T04:43:30.465868+0000","last_active":"2024-05-08T04:43:31.467264+0000","last_peered":"2024-05-08T04:43:31.467264+0000","last_clean":"2024-05-08T04:43:31.467264+0000","last_became_active":"2024-05-08T04:43:30.465518+0000","last_became_peered":"2024-05-08T04:43:30.465518+0000","last_unstale":"2024-05-08T04:43:31.467264+0000","last_undegraded":"2024-05-08T04:43:31.467264+0000","last_fullsized":"2024-05-08T04:43:31.467264+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:53.044345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479271+0000","last_change":"2024-05-08T04:43:31.475699+0000","last_active":"2024-05-08T04:43:31.479271+0000","last_peered":"2024-05-08T04:43:31.479271+0000","last_clean":"2024-05-08T04:43:31.479271+0000","last_became_active":"2024-05-08T04:43:31.475371+0000","last_became_peered":"2024-05-08T04:43:31.475371+0000","last_unstale":"2024-05-08T04:43:31.479271+0000","last_undegraded":"2024-05-08T04:43:31.479271+0000","last_fullsized":"2024-05-08T04:43:31.479271+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:37.585579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459388+0000","last_change":"2024-05-08T04:43:30.478759+0000","last_active":"2024-05-08T04:43:32.459388+0000","last_peered":"2024-05-08T04:43:32.459388+0000","last_clean":"2024-05-08T04:43:32.459388+0000","last_became_active":"2024-05-08T04:43:30.478461+0000","last_became_peered":"2024-05-08T04:43:30.478461+0000","last_unstale":"2024-05-08T04:43:32.459388+0000","last_undegraded":"2024-05-08T04:43:32.459388+0000","last_fullsized":"2024-05-08T04:43:32.459388+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:05.757015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479688+0000","last_change":"2024-05-08T04:43:31.475609+0000","last_active":"2024-05-08T04:43:31.479688+0000","last_peered":"2024-05-08T04:43:31.479688+0000","last_clean":"2024-05-08T04:43:31.479688+0000","last_became_active":"2024-05-08T04:43:31.475312+0000","last_became_peered":"2024-05-08T04:43:31.475312+0000","last_unstale":"2024-05-08T04:43:31.479688+0000","last_undegraded":"2024-05-08T04:43:31.479688+0000","last_fullsized":"2024-05-08T04:43:31.479688+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:13:01.455965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462981+0000","last_change":"2024-05-08T04:43:30.476809+0000","last_active":"2024-05-08T04:43:31.462981+0000","last_peered":"2024-05-08T04:43:31.462981+0000","last_clean":"2024-05-08T04:43:31.462981+0000","last_became_active":"2024-05-08T04:43:30.476543+0000","last_became_peered":"2024-05-08T04:43:30.476543+0000","last_unstale":"2024-05-08T04:43:31.462981+0000","last_undegraded":"2024-05-08T04:43:31.462981+0000","last_fullsized":"2024-05-08T04:43:31.462981+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:14.524602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492750+0000","last_change":"2024-05-08T04:43:31.484279+0000","last_active":"2024-05-08T04:43:31.492750+0000","last_peered":"2024-05-08T04:43:31.492750+0000","last_clean":"2024-05-08T04:43:31.492750+0000","last_became_active":"2024-05-08T04:43:31.483667+0000","last_became_peered":"2024-05-08T04:43:31.483667+0000","last_unstale":"2024-05-08T04:43:31.492750+0000","last_undegraded":"2024-05-08T04:43:31.492750+0000","last_fullsized":"2024-05-08T04:43:31.492750+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:09.360952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.455799+0000","last_change":"2024-05-08T04:43:30.471608+0000","last_active":"2024-05-08T04:43:31.455799+0000","last_peered":"2024-05-08T04:43:31.455799+0000","last_clean":"2024-05-08T04:43:31.455799+0000","last_became_active":"2024-05-08T04:43:30.471356+0000","last_became_peered":"2024-05-08T04:43:30.471356+0000","last_unstale":"2024-05-08T04:43:31.455799+0000","last_undegraded":"2024-05-08T04:43:31.455799+0000","last_fullsized":"2024-05-08T04:43:31.455799+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:03.134798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458442+0000","last_change":"2024-05-08T04:43:31.472449+0000","last_active":"2024-05-08T04:43:32.458442+0000","last_peered":"2024-05-08T04:43:32.458442+0000","last_clean":"2024-05-08T04:43:32.458442+0000","last_became_active":"2024-05-08T04:43:31.472164+0000","last_became_peered":"2024-05-08T04:43:31.472164+0000","last_unstale":"2024-05-08T04:43:32.458442+0000","last_undegraded":"2024-05-08T04:43:32.458442+0000","last_fullsized":"2024-05-08T04:43:32.458442+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:48.832617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459227+0000","last_change":"2024-05-08T04:43:30.472241+0000","last_active":"2024-05-08T04:43:32.459227+0000","last_peered":"2024-05-08T04:43:32.459227+0000","last_clean":"2024-05-08T04:43:32.459227+0000","last_became_active":"2024-05-08T04:43:30.470213+0000","last_became_peered":"2024-05-08T04:43:30.470213+0000","last_unstale":"2024-05-08T04:43:32.459227+0000","last_undegraded":"2024-05-08T04:43:32.459227+0000","last_fullsized":"2024-05-08T04:43:32.459227+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:14.836887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458267+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:43:32.458267+0000","last_peered":"2024-05-08T04:43:32.458267+0000","last_clean":"2024-05-08T04:43:32.458267+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:43:32.458267+0000","last_undegraded":"2024-05-08T04:43:32.458267+0000","last_fullsized":"2024-05-08T04:43:32.458267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":1276522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":685516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":683926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":683974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1568768,"data_stored":684064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":684520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":683904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":1274272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:43:35.537 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:43:35.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.711+0000 7f0b21600640 1 Processor -- start 2024-05-08T04:43:35.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b21600640 1 -- start start 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b21600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b1c06c7b0 0x7f0b1c1620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b21600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c06cc70 con 0x7f0b1c1625f0 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b21600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c06cf50 con 0x7f0b1c15dd70 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b21600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b1570 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b13e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b1c06c7b0 0x7f0b1c1620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b1b400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0b1c1625f0 0x7f0b1c0b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39716/0 (socket says 172.21.15.164:39716) 2024-05-08T04:43:35.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b1b400640 1 -- 172.21.15.164:0/624585744 learned_addr learned my addr 172.21.15.164:0/624585744 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:35.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b13e00640 1 -- 172.21.15.164:0/624585744 >> v1:172.21.15.172:6789/0 conn(0x7f0b1c15dd70 legacy=0x7f0b1c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b13e00640 1 -- 172.21.15.164:0/624585744 >> v1:172.21.15.164:6789/0 conn(0x7f0b1c1625f0 legacy=0x7f0b1c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b13e00640 1 -- 172.21.15.164:0/624585744 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0b1850 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.715+0000 7f0b1a000640 1 -- 172.21.15.164:0/624585744 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983495742 0 0) 0x7f0b1c06cc70 con 0x7f0b1c1625f0 2024-05-08T04:43:35.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b13e00640 1 --2- 172.21.15.164:0/624585744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c1620b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304472107a281f14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b1a000640 1 -- 172.21.15.164:0/624585744 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040b0070 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b1a000640 1 -- 172.21.15.164:0/624585744 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b040ab600 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b1a000640 1 -- 172.21.15.164:0/624585744 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040ab9d0 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- 172.21.15.164:0/624585744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c06c7b0 msgr2=0x7f0b1c1620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 --2- 172.21.15.164:0/624585744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c1620b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097c0 tx=0x7f0b0401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- 172.21.15.164:0/624585744 shutdown_connections 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 --2- 172.21.15.164:0/624585744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c1620b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- 172.21.15.164:0/624585744 >> 172.21.15.164:0/624585744 conn(0x7f0b1c0b34f0 msgr2=0x7f0b1c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- 172.21.15.164:0/624585744 shutdown_connections 2024-05-08T04:43:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- 172.21.15.164:0/624585744 wait complete. 2024-05-08T04:43:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 Processor -- start 2024-05-08T04:43:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.719+0000 7f0b21600640 1 -- start start 2024-05-08T04:43:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b1c15dd70 0x7f0b1c0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b3060 con 0x7f0b1c1625f0 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b2ee0 con 0x7f0b1c06c7b0 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c16bcf0 con 0x7f0b1c15dd70 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59960/0 (socket says 172.21.15.164:59960) 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 -- 172.21.15.164:0/3075046821 learned_addr learned my addr 172.21.15.164:0/3075046821 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b13e00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b1c15dd70 0x7f0b1c0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1aa00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 -- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b1c15dd70 msgr2=0x7f0b1c0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b1c15dd70 0x7f0b1c0cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 -- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 msgr2=0x7f0b1c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 -- 172.21.15.164:0/3075046821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c15d570 con 0x7f0b1c1625f0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1aa00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b1b400640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c7d0 tx=0x7f0b1400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14018070 con 0x7f0b1c1625f0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1400a640 con 0x7f0b1c1625f0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c15d290 con 0x7f0b1c1625f0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400aa70 con 0x7f0b1c1625f0 2024-05-08T04:43:35.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c159a50 con 0x7f0b1c1625f0 2024-05-08T04:43:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c069d60 con 0x7f0b1c1625f0 2024-05-08T04:43:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.723+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f0b1401b070 con 0x7f0b1c1625f0 2024-05-08T04:43:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.727+0000 7f0b12a00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 0x7f0af8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.727+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0b14083140 con 0x7f0b1c1625f0 2024-05-08T04:43:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.727+0000 7f0b1aa00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 0x7f0af8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.727+0000 7f0b1aa00640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 0x7f0af8069f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c156330 tx=0x7f0b0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:35.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.731+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b14055300 con 0x7f0b1c1625f0 2024-05-08T04:43:35.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.879+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b1c0c6720 con 0x7f0af8067aa0 2024-05-08T04:43:35.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.887+0000 7f0b12a00640 1 -- 172.21.15.164:0/3075046821 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252444 (secure 0 0 0) 0x7f0b1c0c6720 con 0x7f0af8067aa0 2024-05-08T04:43:35.892 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:35.893 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:43:35.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 msgr2=0x7f0af8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 0x7f0af8069f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c156330 tx=0x7f0b0c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 msgr2=0x7f0b1c0d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c7d0 tx=0x7f0b1400cc90 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 shutdown_connections 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0af8067aa0 0x7f0af8069f60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b1c15dd70 0x7f0b1c0cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b1c1625f0 0x7f0b1c0d0100 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 --2- 172.21.15.164:0/3075046821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b1c06c7b0 0x7f0b1c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 >> 172.21.15.164:0/3075046821 conn(0x7f0b1c0b34f0 msgr2=0x7f0b1c06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 shutdown_connections 2024-05-08T04:43:35.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:35.891+0000 7f0b21600640 1 -- 172.21.15.164:0/3075046821 wait complete. 2024-05-08T04:43:35.912 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-05-08T04:43:35.491937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262892,"kb_used_data":13412,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711828,"statfs":{"total":773094113280,"available":772824911872,"internally_reserved":0,"allocated":13733888,"data_stored":6656698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001296"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457762+0000","last_change":"2024-05-08T04:43:31.486018+0000","last_active":"2024-05-08T04:43:32.457762+0000","last_peered":"2024-05-08T04:43:32.457762+0000","last_clean":"2024-05-08T04:43:32.457762+0000","last_became_active":"2024-05-08T04:43:31.485177+0000","last_became_peered":"2024-05-08T04:43:31.485177+0000","last_unstale":"2024-05-08T04:43:32.457762+0000","last_undegraded":"2024-05-08T04:43:32.457762+0000","last_fullsized":"2024-05-08T04:43:32.457762+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492906+0000","last_change":"2024-05-08T04:43:31.486210+0000","last_active":"2024-05-08T04:43:31.492906+0000","last_peered":"2024-05-08T04:43:31.492906+0000","last_clean":"2024-05-08T04:43:31.492906+0000","last_became_active":"2024-05-08T04:43:31.485711+0000","last_became_peered":"2024-05-08T04:43:31.485711+0000","last_unstale":"2024-05-08T04:43:31.492906+0000","last_undegraded":"2024-05-08T04:43:31.492906+0000","last_fullsized":"2024-05-08T04:43:31.492906+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:05.408804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460022+0000","last_change":"2024-05-08T04:43:31.477799+0000","last_active":"2024-05-08T04:43:32.460022+0000","last_peered":"2024-05-08T04:43:32.460022+0000","last_clean":"2024-05-08T04:43:32.460022+0000","last_became_active":"2024-05-08T04:43:31.477275+0000","last_became_peered":"2024-05-08T04:43:31.477275+0000","last_unstale":"2024-05-08T04:43:32.460022+0000","last_undegraded":"2024-05-08T04:43:32.460022+0000","last_fullsized":"2024-05-08T04:43:32.460022+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:55.658607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457515+0000","last_change":"2024-05-08T04:43:31.479594+0000","last_active":"2024-05-08T04:43:32.457515+0000","last_peered":"2024-05-08T04:43:32.457515+0000","last_clean":"2024-05-08T04:43:32.457515+0000","last_became_active":"2024-05-08T04:43:31.476126+0000","last_became_peered":"2024-05-08T04:43:31.476126+0000","last_unstale":"2024-05-08T04:43:32.457515+0000","last_undegraded":"2024-05-08T04:43:32.457515+0000","last_fullsized":"2024-05-08T04:43:32.457515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:17.307833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459112+0000","last_change":"2024-05-08T04:43:31.468081+0000","last_active":"2024-05-08T04:43:32.459112+0000","last_peered":"2024-05-08T04:43:32.459112+0000","last_clean":"2024-05-08T04:43:32.459112+0000","last_became_active":"2024-05-08T04:43:31.467837+0000","last_became_peered":"2024-05-08T04:43:31.467837+0000","last_unstale":"2024-05-08T04:43:32.459112+0000","last_undegraded":"2024-05-08T04:43:32.459112+0000","last_fullsized":"2024-05-08T04:43:32.459112+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:14.582994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.480538+0000","last_change":"2024-05-08T04:43:31.474245+0000","last_active":"2024-05-08T04:43:31.480538+0000","last_peered":"2024-05-08T04:43:31.480538+0000","last_clean":"2024-05-08T04:43:31.480538+0000","last_became_active":"2024-05-08T04:43:31.473970+0000","last_became_peered":"2024-05-08T04:43:31.473970+0000","last_unstale":"2024-05-08T04:43:31.480538+0000","last_undegraded":"2024-05-08T04:43:31.480538+0000","last_fullsized":"2024-05-08T04:43:31.480538+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:27.165879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.486453+0000","last_change":"2024-05-08T04:43:31.484686+0000","last_active":"2024-05-08T04:43:31.486453+0000","last_peered":"2024-05-08T04:43:31.486453+0000","last_clean":"2024-05-08T04:43:31.486453+0000","last_became_active":"2024-05-08T04:43:31.483615+0000","last_became_peered":"2024-05-08T04:43:31.483615+0000","last_unstale":"2024-05-08T04:43:31.486453+0000","last_undegraded":"2024-05-08T04:43:31.486453+0000","last_fullsized":"2024-05-08T04:43:31.486453+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:25.878002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459364+0000","last_change":"2024-05-08T04:43:31.482895+0000","last_active":"2024-05-08T04:43:32.459364+0000","last_peered":"2024-05-08T04:43:32.459364+0000","last_clean":"2024-05-08T04:43:32.459364+0000","last_became_active":"2024-05-08T04:43:31.481120+0000","last_became_peered":"2024-05-08T04:43:31.481120+0000","last_unstale":"2024-05-08T04:43:32.459364+0000","last_undegraded":"2024-05-08T04:43:32.459364+0000","last_fullsized":"2024-05-08T04:43:32.459364+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:22.734802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460271+0000","last_change":"2024-05-08T04:43:31.484206+0000","last_active":"2024-05-08T04:43:32.460271+0000","last_peered":"2024-05-08T04:43:32.460271+0000","last_clean":"2024-05-08T04:43:32.460271+0000","last_became_active":"2024-05-08T04:43:31.481803+0000","last_became_peered":"2024-05-08T04:43:31.481803+0000","last_unstale":"2024-05-08T04:43:32.460271+0000","last_undegraded":"2024-05-08T04:43:32.460271+0000","last_fullsized":"2024-05-08T04:43:32.460271+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:21.833733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492959+0000","last_change":"2024-05-08T04:43:31.486292+0000","last_active":"2024-05-08T04:43:31.492959+0000","last_peered":"2024-05-08T04:43:31.492959+0000","last_clean":"2024-05-08T04:43:31.492959+0000","last_became_active":"2024-05-08T04:43:31.486048+0000","last_became_peered":"2024-05-08T04:43:31.486048+0000","last_unstale":"2024-05-08T04:43:31.492959+0000","last_undegraded":"2024-05-08T04:43:31.492959+0000","last_fullsized":"2024-05-08T04:43:31.492959+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:29.154753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492062+0000","last_change":"2024-05-08T04:43:31.483425+0000","last_active":"2024-05-08T04:43:31.492062+0000","last_peered":"2024-05-08T04:43:31.492062+0000","last_clean":"2024-05-08T04:43:31.492062+0000","last_became_active":"2024-05-08T04:43:31.483127+0000","last_became_peered":"2024-05-08T04:43:31.483127+0000","last_unstale":"2024-05-08T04:43:31.492062+0000","last_undegraded":"2024-05-08T04:43:31.492062+0000","last_fullsized":"2024-05-08T04:43:31.492062+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:46:23.346207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458766+0000","last_change":"2024-05-08T04:43:31.483525+0000","last_active":"2024-05-08T04:43:32.458766+0000","last_peered":"2024-05-08T04:43:32.458766+0000","last_clean":"2024-05-08T04:43:32.458766+0000","last_became_active":"2024-05-08T04:43:31.480800+0000","last_became_peered":"2024-05-08T04:43:31.480800+0000","last_unstale":"2024-05-08T04:43:32.458766+0000","last_undegraded":"2024-05-08T04:43:32.458766+0000","last_fullsized":"2024-05-08T04:43:32.458766+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:53.310699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457397+0000","last_change":"2024-05-08T04:43:31.473654+0000","last_active":"2024-05-08T04:43:32.457397+0000","last_peered":"2024-05-08T04:43:32.457397+0000","last_clean":"2024-05-08T04:43:32.457397+0000","last_became_active":"2024-05-08T04:43:31.473317+0000","last_became_peered":"2024-05-08T04:43:31.473317+0000","last_unstale":"2024-05-08T04:43:32.457397+0000","last_undegraded":"2024-05-08T04:43:32.457397+0000","last_fullsized":"2024-05-08T04:43:32.457397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:40:26.873496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.493406+0000","last_change":"2024-05-08T04:43:31.487238+0000","last_active":"2024-05-08T04:43:31.493406+0000","last_peered":"2024-05-08T04:43:31.493406+0000","last_clean":"2024-05-08T04:43:31.493406+0000","last_became_active":"2024-05-08T04:43:31.486847+0000","last_became_peered":"2024-05-08T04:43:31.486847+0000","last_unstale":"2024-05-08T04:43:31.493406+0000","last_undegraded":"2024-05-08T04:43:31.493406+0000","last_fullsized":"2024-05-08T04:43:31.493406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:15.911273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457237+0000","last_change":"2024-05-08T04:43:31.487515+0000","last_active":"2024-05-08T04:43:32.457237+0000","last_peered":"2024-05-08T04:43:32.457237+0000","last_clean":"2024-05-08T04:43:32.457237+0000","last_became_active":"2024-05-08T04:43:31.484905+0000","last_became_peered":"2024-05-08T04:43:31.484905+0000","last_unstale":"2024-05-08T04:43:32.457237+0000","last_undegraded":"2024-05-08T04:43:32.457237+0000","last_fullsized":"2024-05-08T04:43:32.457237+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:15.662981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.487178+0000","last_change":"2024-05-08T04:43:31.481883+0000","last_active":"2024-05-08T04:43:31.487178+0000","last_peered":"2024-05-08T04:43:31.487178+0000","last_clean":"2024-05-08T04:43:31.487178+0000","last_became_active":"2024-05-08T04:43:31.481616+0000","last_became_peered":"2024-05-08T04:43:31.481616+0000","last_unstale":"2024-05-08T04:43:31.487178+0000","last_undegraded":"2024-05-08T04:43:31.487178+0000","last_fullsized":"2024-05-08T04:43:31.487178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:28.462327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.474690+0000","last_change":"2024-05-08T04:43:31.470256+0000","last_active":"2024-05-08T04:43:31.474690+0000","last_peered":"2024-05-08T04:43:31.474690+0000","last_clean":"2024-05-08T04:43:31.474690+0000","last_became_active":"2024-05-08T04:43:31.468022+0000","last_became_peered":"2024-05-08T04:43:31.468022+0000","last_unstale":"2024-05-08T04:43:31.474690+0000","last_undegraded":"2024-05-08T04:43:31.474690+0000","last_fullsized":"2024-05-08T04:43:31.474690+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:34.681198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479681+0000","last_change":"2024-05-08T04:43:31.476413+0000","last_active":"2024-05-08T04:43:31.479681+0000","last_peered":"2024-05-08T04:43:31.479681+0000","last_clean":"2024-05-08T04:43:31.479681+0000","last_became_active":"2024-05-08T04:43:31.475775+0000","last_became_peered":"2024-05-08T04:43:31.475775+0000","last_unstale":"2024-05-08T04:43:31.479681+0000","last_undegraded":"2024-05-08T04:43:31.479681+0000","last_fullsized":"2024-05-08T04:43:31.479681+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:45:46.603370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467328+0000","last_change":"2024-05-08T04:43:31.465156+0000","last_active":"2024-05-08T04:43:31.467328+0000","last_peered":"2024-05-08T04:43:31.467328+0000","last_clean":"2024-05-08T04:43:31.467328+0000","last_became_active":"2024-05-08T04:43:31.464852+0000","last_became_peered":"2024-05-08T04:43:31.464852+0000","last_unstale":"2024-05-08T04:43:31.467328+0000","last_undegraded":"2024-05-08T04:43:31.467328+0000","last_fullsized":"2024-05-08T04:43:31.467328+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:15.808802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490583+0000","last_change":"2024-05-08T04:43:31.486305+0000","last_active":"2024-05-08T04:43:31.490583+0000","last_peered":"2024-05-08T04:43:31.490583+0000","last_clean":"2024-05-08T04:43:31.490583+0000","last_became_active":"2024-05-08T04:43:31.486038+0000","last_became_peered":"2024-05-08T04:43:31.486038+0000","last_unstale":"2024-05-08T04:43:31.490583+0000","last_undegraded":"2024-05-08T04:43:31.490583+0000","last_fullsized":"2024-05-08T04:43:31.490583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:00.822358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458966+0000","last_change":"2024-05-08T04:43:31.467043+0000","last_active":"2024-05-08T04:43:32.458966+0000","last_peered":"2024-05-08T04:43:32.458966+0000","last_clean":"2024-05-08T04:43:32.458966+0000","last_became_active":"2024-05-08T04:43:31.466718+0000","last_became_peered":"2024-05-08T04:43:31.466718+0000","last_unstale":"2024-05-08T04:43:32.458966+0000","last_undegraded":"2024-05-08T04:43:32.458966+0000","last_fullsized":"2024-05-08T04:43:32.458966+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:04.139555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.482376+0000","last_change":"2024-05-08T04:43:31.477397+0000","last_active":"2024-05-08T04:43:31.482376+0000","last_peered":"2024-05-08T04:43:31.482376+0000","last_clean":"2024-05-08T04:43:31.482376+0000","last_became_active":"2024-05-08T04:43:31.477133+0000","last_became_peered":"2024-05-08T04:43:31.477133+0000","last_unstale":"2024-05-08T04:43:31.482376+0000","last_undegraded":"2024-05-08T04:43:31.482376+0000","last_fullsized":"2024-05-08T04:43:31.482376+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:48.844368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459400+0000","last_change":"2024-05-08T04:43:31.471141+0000","last_active":"2024-05-08T04:43:32.459400+0000","last_peered":"2024-05-08T04:43:32.459400+0000","last_clean":"2024-05-08T04:43:32.459400+0000","last_became_active":"2024-05-08T04:43:31.470704+0000","last_became_peered":"2024-05-08T04:43:31.470704+0000","last_unstale":"2024-05-08T04:43:32.459400+0000","last_undegraded":"2024-05-08T04:43:32.459400+0000","last_fullsized":"2024-05-08T04:43:32.459400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:40.175901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479774+0000","last_change":"2024-05-08T04:43:31.473031+0000","last_active":"2024-05-08T04:43:31.479774+0000","last_peered":"2024-05-08T04:43:31.479774+0000","last_clean":"2024-05-08T04:43:31.479774+0000","last_became_active":"2024-05-08T04:43:31.468864+0000","last_became_peered":"2024-05-08T04:43:31.468864+0000","last_unstale":"2024-05-08T04:43:31.479774+0000","last_undegraded":"2024-05-08T04:43:31.479774+0000","last_fullsized":"2024-05-08T04:43:31.479774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:10.018406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.489313+0000","last_change":"2024-05-08T04:43:31.483288+0000","last_active":"2024-05-08T04:43:31.489313+0000","last_peered":"2024-05-08T04:43:31.489313+0000","last_clean":"2024-05-08T04:43:31.489313+0000","last_became_active":"2024-05-08T04:43:31.483022+0000","last_became_peered":"2024-05-08T04:43:31.483022+0000","last_unstale":"2024-05-08T04:43:31.489313+0000","last_undegraded":"2024-05-08T04:43:31.489313+0000","last_fullsized":"2024-05-08T04:43:31.489313+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:56.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479702+0000","last_change":"2024-05-08T04:43:31.477853+0000","last_active":"2024-05-08T04:43:31.479702+0000","last_peered":"2024-05-08T04:43:31.479702+0000","last_clean":"2024-05-08T04:43:31.479702+0000","last_became_active":"2024-05-08T04:43:31.476151+0000","last_became_peered":"2024-05-08T04:43:31.476151+0000","last_unstale":"2024-05-08T04:43:31.479702+0000","last_undegraded":"2024-05-08T04:43:31.479702+0000","last_fullsized":"2024-05-08T04:43:31.479702+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:06.761771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457424+0000","last_change":"2024-05-08T04:43:31.476653+0000","last_active":"2024-05-08T04:43:32.457424+0000","last_peered":"2024-05-08T04:43:32.457424+0000","last_clean":"2024-05-08T04:43:32.457424+0000","last_became_active":"2024-05-08T04:43:31.474249+0000","last_became_peered":"2024-05-08T04:43:31.474249+0000","last_unstale":"2024-05-08T04:43:32.457424+0000","last_undegraded":"2024-05-08T04:43:32.457424+0000","last_fullsized":"2024-05-08T04:43:32.457424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:45.930400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.488090+0000","last_change":"2024-05-08T04:43:31.482185+0000","last_active":"2024-05-08T04:43:31.488090+0000","last_peered":"2024-05-08T04:43:31.488090+0000","last_clean":"2024-05-08T04:43:31.488090+0000","last_became_active":"2024-05-08T04:43:31.481865+0000","last_became_peered":"2024-05-08T04:43:31.481865+0000","last_unstale":"2024-05-08T04:43:31.488090+0000","last_undegraded":"2024-05-08T04:43:31.488090+0000","last_fullsized":"2024-05-08T04:43:31.488090+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:56.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457445+0000","last_change":"2024-05-08T04:43:31.482794+0000","last_active":"2024-05-08T04:43:32.457445+0000","last_peered":"2024-05-08T04:43:32.457445+0000","last_clean":"2024-05-08T04:43:32.457445+0000","last_became_active":"2024-05-08T04:43:31.482509+0000","last_became_peered":"2024-05-08T04:43:31.482509+0000","last_unstale":"2024-05-08T04:43:32.457445+0000","last_undegraded":"2024-05-08T04:43:32.457445+0000","last_fullsized":"2024-05-08T04:43:32.457445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:54.049101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457274+0000","last_change":"2024-05-08T04:43:31.477091+0000","last_active":"2024-05-08T04:43:32.457274+0000","last_peered":"2024-05-08T04:43:32.457274+0000","last_clean":"2024-05-08T04:43:32.457274+0000","last_became_active":"2024-05-08T04:43:31.472252+0000","last_became_peered":"2024-05-08T04:43:31.472252+0000","last_unstale":"2024-05-08T04:43:32.457274+0000","last_undegraded":"2024-05-08T04:43:32.457274+0000","last_fullsized":"2024-05-08T04:43:32.457274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:45.871755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458851+0000","last_change":"2024-05-08T04:43:31.481585+0000","last_active":"2024-05-08T04:43:32.458851+0000","last_peered":"2024-05-08T04:43:32.458851+0000","last_clean":"2024-05-08T04:43:32.458851+0000","last_became_active":"2024-05-08T04:43:31.480859+0000","last_became_peered":"2024-05-08T04:43:31.480859+0000","last_unstale":"2024-05-08T04:43:32.458851+0000","last_undegraded":"2024-05-08T04:43:32.458851+0000","last_fullsized":"2024-05-08T04:43:32.458851+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:17.859746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.476262+0000","last_change":"2024-05-08T04:43:31.473494+0000","last_active":"2024-05-08T04:43:31.476262+0000","last_peered":"2024-05-08T04:43:31.476262+0000","last_clean":"2024-05-08T04:43:31.476262+0000","last_became_active":"2024-05-08T04:43:31.473238+0000","last_became_peered":"2024-05-08T04:43:31.473238+0000","last_unstale":"2024-05-08T04:43:31.476262+0000","last_undegraded":"2024-05-08T04:43:31.476262+0000","last_fullsized":"2024-05-08T04:43:31.476262+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:20:00.752055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479512+0000","last_change":"2024-05-08T04:43:31.471881+0000","last_active":"2024-05-08T04:43:31.479512+0000","last_peered":"2024-05-08T04:43:31.479512+0000","last_clean":"2024-05-08T04:43:31.479512+0000","last_became_active":"2024-05-08T04:43:31.471628+0000","last_became_peered":"2024-05-08T04:43:31.471628+0000","last_unstale":"2024-05-08T04:43:31.479512+0000","last_undegraded":"2024-05-08T04:43:31.479512+0000","last_fullsized":"2024-05-08T04:43:31.479512+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:51:28.548705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459838+0000","last_change":"2024-05-08T04:43:30.471425+0000","last_active":"2024-05-08T04:43:32.459838+0000","last_peered":"2024-05-08T04:43:32.459838+0000","last_clean":"2024-05-08T04:43:32.459838+0000","last_became_active":"2024-05-08T04:43:30.470439+0000","last_became_peered":"2024-05-08T04:43:30.470439+0000","last_unstale":"2024-05-08T04:43:32.459838+0000","last_undegraded":"2024-05-08T04:43:32.459838+0000","last_fullsized":"2024-05-08T04:43:32.459838+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:33:59.817602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460352+0000","last_change":"2024-05-08T04:43:31.485079+0000","last_active":"2024-05-08T04:43:32.460352+0000","last_peered":"2024-05-08T04:43:32.460352+0000","last_clean":"2024-05-08T04:43:32.460352+0000","last_became_active":"2024-05-08T04:43:31.484713+0000","last_became_peered":"2024-05-08T04:43:31.484713+0000","last_unstale":"2024-05-08T04:43:32.460352+0000","last_undegraded":"2024-05-08T04:43:32.460352+0000","last_fullsized":"2024-05-08T04:43:32.460352+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:02.801555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.483499+0000","last_change":"2024-05-08T04:43:30.476351+0000","last_active":"2024-05-08T04:43:31.483499+0000","last_peered":"2024-05-08T04:43:31.483499+0000","last_clean":"2024-05-08T04:43:31.483499+0000","last_became_active":"2024-05-08T04:43:30.476056+0000","last_became_peered":"2024-05-08T04:43:30.476056+0000","last_unstale":"2024-05-08T04:43:31.483499+0000","last_undegraded":"2024-05-08T04:43:31.483499+0000","last_fullsized":"2024-05-08T04:43:31.483499+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:44.925644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459311+0000","last_change":"2024-05-08T04:43:31.474313+0000","last_active":"2024-05-08T04:43:32.459311+0000","last_peered":"2024-05-08T04:43:32.459311+0000","last_clean":"2024-05-08T04:43:32.459311+0000","last_became_active":"2024-05-08T04:43:31.473952+0000","last_became_peered":"2024-05-08T04:43:31.473952+0000","last_unstale":"2024-05-08T04:43:32.459311+0000","last_undegraded":"2024-05-08T04:43:32.459311+0000","last_fullsized":"2024-05-08T04:43:32.459311+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:15.808802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492164+0000","last_change":"2024-05-08T04:43:30.478483+0000","last_active":"2024-05-08T04:43:31.492164+0000","last_peered":"2024-05-08T04:43:31.492164+0000","last_clean":"2024-05-08T04:43:31.492164+0000","last_became_active":"2024-05-08T04:43:30.478178+0000","last_became_peered":"2024-05-08T04:43:30.478178+0000","last_unstale":"2024-05-08T04:43:31.492164+0000","last_undegraded":"2024-05-08T04:43:31.492164+0000","last_fullsized":"2024-05-08T04:43:31.492164+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:02.752884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459000+0000","last_change":"2024-05-08T04:43:31.476742+0000","last_active":"2024-05-08T04:43:32.459000+0000","last_peered":"2024-05-08T04:43:32.459000+0000","last_clean":"2024-05-08T04:43:32.459000+0000","last_became_active":"2024-05-08T04:43:31.476524+0000","last_became_peered":"2024-05-08T04:43:31.476524+0000","last_unstale":"2024-05-08T04:43:32.459000+0000","last_undegraded":"2024-05-08T04:43:32.459000+0000","last_fullsized":"2024-05-08T04:43:32.459000+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:10.533184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.484514+0000","last_change":"2024-05-08T04:43:30.476756+0000","last_active":"2024-05-08T04:43:31.484514+0000","last_peered":"2024-05-08T04:43:31.484514+0000","last_clean":"2024-05-08T04:43:31.484514+0000","last_became_active":"2024-05-08T04:43:30.476498+0000","last_became_peered":"2024-05-08T04:43:30.476498+0000","last_unstale":"2024-05-08T04:43:31.484514+0000","last_undegraded":"2024-05-08T04:43:31.484514+0000","last_fullsized":"2024-05-08T04:43:31.484514+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:57.836878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479601+0000","last_change":"2024-05-08T04:43:31.474844+0000","last_active":"2024-05-08T04:43:31.479601+0000","last_peered":"2024-05-08T04:43:31.479601+0000","last_clean":"2024-05-08T04:43:31.479601+0000","last_became_active":"2024-05-08T04:43:31.474544+0000","last_became_peered":"2024-05-08T04:43:31.474544+0000","last_unstale":"2024-05-08T04:43:31.479601+0000","last_undegraded":"2024-05-08T04:43:31.479601+0000","last_fullsized":"2024-05-08T04:43:31.479601+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:22.810223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.483612+0000","last_change":"2024-05-08T04:43:30.479947+0000","last_active":"2024-05-08T04:43:31.483612+0000","last_peered":"2024-05-08T04:43:31.483612+0000","last_clean":"2024-05-08T04:43:31.483612+0000","last_became_active":"2024-05-08T04:43:30.479683+0000","last_became_peered":"2024-05-08T04:43:30.479683+0000","last_unstale":"2024-05-08T04:43:31.483612+0000","last_undegraded":"2024-05-08T04:43:31.483612+0000","last_fullsized":"2024-05-08T04:43:31.483612+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:44.925644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.470865+0000","last_change":"2024-05-08T04:43:31.464745+0000","last_active":"2024-05-08T04:43:31.470865+0000","last_peered":"2024-05-08T04:43:31.470865+0000","last_clean":"2024-05-08T04:43:31.470865+0000","last_became_active":"2024-05-08T04:43:31.464447+0000","last_became_peered":"2024-05-08T04:43:31.464447+0000","last_unstale":"2024-05-08T04:43:31.470865+0000","last_undegraded":"2024-05-08T04:43:31.470865+0000","last_fullsized":"2024-05-08T04:43:31.470865+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:21.833733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490338+0000","last_change":"2024-05-08T04:43:30.481314+0000","last_active":"2024-05-08T04:43:31.490338+0000","last_peered":"2024-05-08T04:43:31.490338+0000","last_clean":"2024-05-08T04:43:31.490338+0000","last_became_active":"2024-05-08T04:43:30.481035+0000","last_became_peered":"2024-05-08T04:43:30.481035+0000","last_unstale":"2024-05-08T04:43:31.490338+0000","last_undegraded":"2024-05-08T04:43:31.490338+0000","last_fullsized":"2024-05-08T04:43:31.490338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:29.055940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459524+0000","last_change":"2024-05-08T04:43:31.483737+0000","last_active":"2024-05-08T04:43:32.459524+0000","last_peered":"2024-05-08T04:43:32.459524+0000","last_clean":"2024-05-08T04:43:32.459524+0000","last_became_active":"2024-05-08T04:43:31.483248+0000","last_became_peered":"2024-05-08T04:43:31.483248+0000","last_unstale":"2024-05-08T04:43:32.459524+0000","last_undegraded":"2024-05-08T04:43:32.459524+0000","last_fullsized":"2024-05-08T04:43:32.459524+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:17.189675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.457323+0000","last_change":"2024-05-08T04:43:30.474778+0000","last_active":"2024-05-08T04:43:31.457323+0000","last_peered":"2024-05-08T04:43:31.457323+0000","last_clean":"2024-05-08T04:43:31.457323+0000","last_became_active":"2024-05-08T04:43:30.474495+0000","last_became_peered":"2024-05-08T04:43:30.474495+0000","last_unstale":"2024-05-08T04:43:31.457323+0000","last_undegraded":"2024-05-08T04:43:31.457323+0000","last_fullsized":"2024-05-08T04:43:31.457323+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:54.653851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492773+0000","last_change":"2024-05-08T04:43:31.484507+0000","last_active":"2024-05-08T04:43:31.492773+0000","last_peered":"2024-05-08T04:43:31.492773+0000","last_clean":"2024-05-08T04:43:31.492773+0000","last_became_active":"2024-05-08T04:43:31.483969+0000","last_became_peered":"2024-05-08T04:43:31.483969+0000","last_unstale":"2024-05-08T04:43:31.492773+0000","last_undegraded":"2024-05-08T04:43:31.492773+0000","last_fullsized":"2024-05-08T04:43:31.492773+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:54.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467641+0000","last_change":"2024-05-08T04:43:30.469079+0000","last_active":"2024-05-08T04:43:31.467641+0000","last_peered":"2024-05-08T04:43:31.467641+0000","last_clean":"2024-05-08T04:43:31.467641+0000","last_became_active":"2024-05-08T04:43:30.468828+0000","last_became_peered":"2024-05-08T04:43:30.468828+0000","last_unstale":"2024-05-08T04:43:31.467641+0000","last_undegraded":"2024-05-08T04:43:31.467641+0000","last_fullsized":"2024-05-08T04:43:31.467641+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:16.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457552+0000","last_change":"2024-05-08T04:43:31.476308+0000","last_active":"2024-05-08T04:43:32.457552+0000","last_peered":"2024-05-08T04:43:32.457552+0000","last_clean":"2024-05-08T04:43:32.457552+0000","last_became_active":"2024-05-08T04:43:31.475940+0000","last_became_peered":"2024-05-08T04:43:31.475940+0000","last_unstale":"2024-05-08T04:43:32.457552+0000","last_undegraded":"2024-05-08T04:43:32.457552+0000","last_fullsized":"2024-05-08T04:43:32.457552+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:03.366436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.458120+0000","last_change":"2024-05-08T04:43:30.472372+0000","last_active":"2024-05-08T04:43:31.458120+0000","last_peered":"2024-05-08T04:43:31.458120+0000","last_clean":"2024-05-08T04:43:31.458120+0000","last_became_active":"2024-05-08T04:43:30.472084+0000","last_became_peered":"2024-05-08T04:43:30.472084+0000","last_unstale":"2024-05-08T04:43:31.458120+0000","last_undegraded":"2024-05-08T04:43:31.458120+0000","last_fullsized":"2024-05-08T04:43:31.458120+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:53.044345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490416+0000","last_change":"2024-05-08T04:43:31.482283+0000","last_active":"2024-05-08T04:43:31.490416+0000","last_peered":"2024-05-08T04:43:31.490416+0000","last_clean":"2024-05-08T04:43:31.490416+0000","last_became_active":"2024-05-08T04:43:31.481954+0000","last_became_peered":"2024-05-08T04:43:31.481954+0000","last_unstale":"2024-05-08T04:43:31.490416+0000","last_undegraded":"2024-05-08T04:43:31.490416+0000","last_fullsized":"2024-05-08T04:43:31.490416+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:09.360952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.463577+0000","last_change":"2024-05-08T04:43:30.474689+0000","last_active":"2024-05-08T04:43:31.463577+0000","last_peered":"2024-05-08T04:43:31.463577+0000","last_clean":"2024-05-08T04:43:31.463577+0000","last_became_active":"2024-05-08T04:43:30.474393+0000","last_became_peered":"2024-05-08T04:43:30.474393+0000","last_unstale":"2024-05-08T04:43:31.463577+0000","last_undegraded":"2024-05-08T04:43:31.463577+0000","last_fullsized":"2024-05-08T04:43:31.463577+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:36.975260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458774+0000","last_change":"2024-05-08T04:43:31.478897+0000","last_active":"2024-05-08T04:43:32.458774+0000","last_peered":"2024-05-08T04:43:32.458774+0000","last_clean":"2024-05-08T04:43:32.458774+0000","last_became_active":"2024-05-08T04:43:31.477660+0000","last_became_peered":"2024-05-08T04:43:31.477660+0000","last_unstale":"2024-05-08T04:43:32.458774+0000","last_undegraded":"2024-05-08T04:43:32.458774+0000","last_fullsized":"2024-05-08T04:43:32.458774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:40.223740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.491868+0000","last_change":"2024-05-08T04:43:30.477574+0000","last_active":"2024-05-08T04:43:31.491868+0000","last_peered":"2024-05-08T04:43:31.491868+0000","last_clean":"2024-05-08T04:43:31.491868+0000","last_became_active":"2024-05-08T04:43:30.477208+0000","last_became_peered":"2024-05-08T04:43:30.477208+0000","last_unstale":"2024-05-08T04:43:31.491868+0000","last_undegraded":"2024-05-08T04:43:31.491868+0000","last_fullsized":"2024-05-08T04:43:31.491868+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:57.836878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459224+0000","last_change":"2024-05-08T04:43:31.479073+0000","last_active":"2024-05-08T04:43:32.459224+0000","last_peered":"2024-05-08T04:43:32.459224+0000","last_clean":"2024-05-08T04:43:32.459224+0000","last_became_active":"2024-05-08T04:43:31.478708+0000","last_became_peered":"2024-05-08T04:43:31.478708+0000","last_unstale":"2024-05-08T04:43:32.459224+0000","last_undegraded":"2024-05-08T04:43:32.459224+0000","last_fullsized":"2024-05-08T04:43:32.459224+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:16:03.757640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458898+0000","last_change":"2024-05-08T04:43:30.475273+0000","last_active":"2024-05-08T04:43:32.458898+0000","last_peered":"2024-05-08T04:43:32.458898+0000","last_clean":"2024-05-08T04:43:32.458898+0000","last_became_active":"2024-05-08T04:43:30.474818+0000","last_became_peered":"2024-05-08T04:43:30.474818+0000","last_unstale":"2024-05-08T04:43:32.458898+0000","last_undegraded":"2024-05-08T04:43:32.458898+0000","last_fullsized":"2024-05-08T04:43:32.458898+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:13.389423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457548+0000","last_change":"2024-05-08T04:43:31.470983+0000","last_active":"2024-05-08T04:43:32.457548+0000","last_peered":"2024-05-08T04:43:32.457548+0000","last_clean":"2024-05-08T04:43:32.457548+0000","last_became_active":"2024-05-08T04:43:31.470687+0000","last_became_peered":"2024-05-08T04:43:31.470687+0000","last_unstale":"2024-05-08T04:43:32.457548+0000","last_undegraded":"2024-05-08T04:43:32.457548+0000","last_fullsized":"2024-05-08T04:43:32.457548+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:01.300356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457392+0000","last_change":"2024-05-08T04:43:30.469508+0000","last_active":"2024-05-08T04:43:32.457392+0000","last_peered":"2024-05-08T04:43:32.457392+0000","last_clean":"2024-05-08T04:43:32.457392+0000","last_became_active":"2024-05-08T04:43:30.469254+0000","last_became_peered":"2024-05-08T04:43:30.469254+0000","last_unstale":"2024-05-08T04:43:32.457392+0000","last_undegraded":"2024-05-08T04:43:32.457392+0000","last_fullsized":"2024-05-08T04:43:32.457392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:56.733449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.484895+0000","last_change":"2024-05-08T04:43:31.478638+0000","last_active":"2024-05-08T04:43:31.484895+0000","last_peered":"2024-05-08T04:43:31.484895+0000","last_clean":"2024-05-08T04:43:31.484895+0000","last_became_active":"2024-05-08T04:43:31.478335+0000","last_became_peered":"2024-05-08T04:43:31.478335+0000","last_unstale":"2024-05-08T04:43:31.484895+0000","last_undegraded":"2024-05-08T04:43:31.484895+0000","last_fullsized":"2024-05-08T04:43:31.484895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.470461+0000","last_change":"2024-05-08T04:43:30.466280+0000","last_active":"2024-05-08T04:43:31.470461+0000","last_peered":"2024-05-08T04:43:31.470461+0000","last_clean":"2024-05-08T04:43:31.470461+0000","last_became_active":"2024-05-08T04:43:30.466019+0000","last_became_peered":"2024-05-08T04:43:30.466019+0000","last_unstale":"2024-05-08T04:43:31.470461+0000","last_undegraded":"2024-05-08T04:43:31.470461+0000","last_fullsized":"2024-05-08T04:43:31.470461+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:02.361679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459156+0000","last_change":"2024-05-08T04:43:31.475677+0000","last_active":"2024-05-08T04:43:32.459156+0000","last_peered":"2024-05-08T04:43:32.459156+0000","last_clean":"2024-05-08T04:43:32.459156+0000","last_became_active":"2024-05-08T04:43:31.475274+0000","last_became_peered":"2024-05-08T04:43:31.475274+0000","last_unstale":"2024-05-08T04:43:32.459156+0000","last_undegraded":"2024-05-08T04:43:32.459156+0000","last_fullsized":"2024-05-08T04:43:32.459156+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:46.331910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459618+0000","last_change":"2024-05-08T04:43:30.478500+0000","last_active":"2024-05-08T04:43:32.459618+0000","last_peered":"2024-05-08T04:43:32.459618+0000","last_clean":"2024-05-08T04:43:32.459618+0000","last_became_active":"2024-05-08T04:43:30.478153+0000","last_became_peered":"2024-05-08T04:43:30.478153+0000","last_unstale":"2024-05-08T04:43:32.459618+0000","last_undegraded":"2024-05-08T04:43:32.459618+0000","last_fullsized":"2024-05-08T04:43:32.459618+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:52.913803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.460188+0000","last_change":"2024-05-08T04:43:31.488981+0000","last_active":"2024-05-08T04:43:32.460188+0000","last_peered":"2024-05-08T04:43:32.460188+0000","last_clean":"2024-05-08T04:43:32.460188+0000","last_became_active":"2024-05-08T04:43:31.486691+0000","last_became_peered":"2024-05-08T04:43:31.486691+0000","last_unstale":"2024-05-08T04:43:32.460188+0000","last_undegraded":"2024-05-08T04:43:32.460188+0000","last_fullsized":"2024-05-08T04:43:32.460188+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:08.574905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.485533+0000","last_change":"2024-05-08T04:43:30.466252+0000","last_active":"2024-05-08T04:43:31.485533+0000","last_peered":"2024-05-08T04:43:31.485533+0000","last_clean":"2024-05-08T04:43:31.485533+0000","last_became_active":"2024-05-08T04:43:30.465955+0000","last_became_peered":"2024-05-08T04:43:30.465955+0000","last_unstale":"2024-05-08T04:43:31.485533+0000","last_undegraded":"2024-05-08T04:43:31.485533+0000","last_fullsized":"2024-05-08T04:43:31.485533+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.475023+0000","last_change":"2024-05-08T04:43:31.472728+0000","last_active":"2024-05-08T04:43:31.475023+0000","last_peered":"2024-05-08T04:43:31.475023+0000","last_clean":"2024-05-08T04:43:31.475023+0000","last_became_active":"2024-05-08T04:43:31.472418+0000","last_became_peered":"2024-05-08T04:43:31.472418+0000","last_unstale":"2024-05-08T04:43:31.475023+0000","last_undegraded":"2024-05-08T04:43:31.475023+0000","last_fullsized":"2024-05-08T04:43:31.475023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.086669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492551+0000","last_change":"2024-05-08T04:43:30.478007+0000","last_active":"2024-05-08T04:43:31.492551+0000","last_peered":"2024-05-08T04:43:31.492551+0000","last_clean":"2024-05-08T04:43:31.492551+0000","last_became_active":"2024-05-08T04:43:30.477742+0000","last_became_peered":"2024-05-08T04:43:30.477742+0000","last_unstale":"2024-05-08T04:43:31.492551+0000","last_undegraded":"2024-05-08T04:43:31.492551+0000","last_fullsized":"2024-05-08T04:43:31.492551+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:09.528428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492727+0000","last_change":"2024-05-08T04:43:31.490092+0000","last_active":"2024-05-08T04:43:31.492727+0000","last_peered":"2024-05-08T04:43:31.492727+0000","last_clean":"2024-05-08T04:43:31.492727+0000","last_became_active":"2024-05-08T04:43:31.489489+0000","last_became_peered":"2024-05-08T04:43:31.489489+0000","last_unstale":"2024-05-08T04:43:31.492727+0000","last_undegraded":"2024-05-08T04:43:31.492727+0000","last_fullsized":"2024-05-08T04:43:31.492727+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:15.841643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492568+0000","last_change":"2024-05-08T04:43:30.480111+0000","last_active":"2024-05-08T04:43:31.492568+0000","last_peered":"2024-05-08T04:43:31.492568+0000","last_clean":"2024-05-08T04:43:31.492568+0000","last_became_active":"2024-05-08T04:43:30.479714+0000","last_became_peered":"2024-05-08T04:43:30.479714+0000","last_unstale":"2024-05-08T04:43:31.492568+0000","last_undegraded":"2024-05-08T04:43:31.492568+0000","last_fullsized":"2024-05-08T04:43:31.492568+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:07.570149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.493473+0000","last_change":"2024-05-08T04:43:31.489681+0000","last_active":"2024-05-08T04:43:31.493473+0000","last_peered":"2024-05-08T04:43:31.493473+0000","last_clean":"2024-05-08T04:43:31.493473+0000","last_became_active":"2024-05-08T04:43:31.489349+0000","last_became_peered":"2024-05-08T04:43:31.489349+0000","last_unstale":"2024-05-08T04:43:31.493473+0000","last_undegraded":"2024-05-08T04:43:31.493473+0000","last_fullsized":"2024-05-08T04:43:31.493473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:32.704399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.458454+0000","last_change":"2024-05-08T04:43:30.474577+0000","last_active":"2024-05-08T04:43:31.458454+0000","last_peered":"2024-05-08T04:43:31.458454+0000","last_clean":"2024-05-08T04:43:31.458454+0000","last_became_active":"2024-05-08T04:43:30.473767+0000","last_became_peered":"2024-05-08T04:43:30.473767+0000","last_unstale":"2024-05-08T04:43:31.458454+0000","last_undegraded":"2024-05-08T04:43:31.458454+0000","last_fullsized":"2024-05-08T04:43:31.458454+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:02.361679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457238+0000","last_change":"2024-05-08T04:43:31.474426+0000","last_active":"2024-05-08T04:43:32.457238+0000","last_peered":"2024-05-08T04:43:32.457238+0000","last_clean":"2024-05-08T04:43:32.457238+0000","last_became_active":"2024-05-08T04:43:31.473622+0000","last_became_peered":"2024-05-08T04:43:31.473622+0000","last_unstale":"2024-05-08T04:43:32.457238+0000","last_undegraded":"2024-05-08T04:43:32.457238+0000","last_fullsized":"2024-05-08T04:43:32.457238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:35:02.611988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.490221+0000","last_change":"2024-05-08T04:43:30.470959+0000","last_active":"2024-05-08T04:43:31.490221+0000","last_peered":"2024-05-08T04:43:31.490221+0000","last_clean":"2024-05-08T04:43:31.490221+0000","last_became_active":"2024-05-08T04:43:30.470651+0000","last_became_peered":"2024-05-08T04:43:30.470651+0000","last_unstale":"2024-05-08T04:43:31.490221+0000","last_undegraded":"2024-05-08T04:43:31.490221+0000","last_fullsized":"2024-05-08T04:43:31.490221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:16.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.485726+0000","last_change":"2024-05-08T04:43:31.484048+0000","last_active":"2024-05-08T04:43:31.485726+0000","last_peered":"2024-05-08T04:43:31.485726+0000","last_clean":"2024-05-08T04:43:31.485726+0000","last_became_active":"2024-05-08T04:43:31.483130+0000","last_became_peered":"2024-05-08T04:43:31.483130+0000","last_unstale":"2024-05-08T04:43:31.485726+0000","last_undegraded":"2024-05-08T04:43:31.485726+0000","last_fullsized":"2024-05-08T04:43:31.485726+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:35.718248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462707+0000","last_change":"2024-05-08T04:43:30.478103+0000","last_active":"2024-05-08T04:43:31.462707+0000","last_peered":"2024-05-08T04:43:31.462707+0000","last_clean":"2024-05-08T04:43:31.462707+0000","last_became_active":"2024-05-08T04:43:30.477814+0000","last_became_peered":"2024-05-08T04:43:30.477814+0000","last_unstale":"2024-05-08T04:43:31.462707+0000","last_undegraded":"2024-05-08T04:43:31.462707+0000","last_fullsized":"2024-05-08T04:43:31.462707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:34.713492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.489192+0000","last_change":"2024-05-08T04:43:31.481488+0000","last_active":"2024-05-08T04:43:31.489192+0000","last_peered":"2024-05-08T04:43:31.489192+0000","last_clean":"2024-05-08T04:43:31.489192+0000","last_became_active":"2024-05-08T04:43:31.481230+0000","last_became_peered":"2024-05-08T04:43:31.481230+0000","last_unstale":"2024-05-08T04:43:31.489192+0000","last_undegraded":"2024-05-08T04:43:31.489192+0000","last_fullsized":"2024-05-08T04:43:31.489192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:37.585579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462820+0000","last_change":"2024-05-08T04:43:30.477308+0000","last_active":"2024-05-08T04:43:31.462820+0000","last_peered":"2024-05-08T04:43:31.462820+0000","last_clean":"2024-05-08T04:43:31.462820+0000","last_became_active":"2024-05-08T04:43:30.477067+0000","last_became_peered":"2024-05-08T04:43:30.477067+0000","last_unstale":"2024-05-08T04:43:31.462820+0000","last_undegraded":"2024-05-08T04:43:31.462820+0000","last_fullsized":"2024-05-08T04:43:31.462820+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:24.873246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458767+0000","last_change":"2024-05-08T04:43:31.477148+0000","last_active":"2024-05-08T04:43:32.458767+0000","last_peered":"2024-05-08T04:43:32.458767+0000","last_clean":"2024-05-08T04:43:32.458767+0000","last_became_active":"2024-05-08T04:43:31.476880+0000","last_became_peered":"2024-05-08T04:43:31.476880+0000","last_unstale":"2024-05-08T04:43:32.458767+0000","last_undegraded":"2024-05-08T04:43:32.458767+0000","last_fullsized":"2024-05-08T04:43:32.458767+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.900967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.455593+0000","last_change":"2024-05-08T04:43:30.474183+0000","last_active":"2024-05-08T04:43:31.455593+0000","last_peered":"2024-05-08T04:43:31.455593+0000","last_clean":"2024-05-08T04:43:31.455593+0000","last_became_active":"2024-05-08T04:43:30.473930+0000","last_became_peered":"2024-05-08T04:43:30.473930+0000","last_unstale":"2024-05-08T04:43:31.455593+0000","last_undegraded":"2024-05-08T04:43:31.455593+0000","last_fullsized":"2024-05-08T04:43:31.455593+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:16.854989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.472448+0000","last_change":"2024-05-08T04:43:31.467882+0000","last_active":"2024-05-08T04:43:31.472448+0000","last_peered":"2024-05-08T04:43:31.472448+0000","last_clean":"2024-05-08T04:43:31.472448+0000","last_became_active":"2024-05-08T04:43:31.467573+0000","last_became_peered":"2024-05-08T04:43:31.467573+0000","last_unstale":"2024-05-08T04:43:31.472448+0000","last_undegraded":"2024-05-08T04:43:31.472448+0000","last_fullsized":"2024-05-08T04:43:31.472448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:02.801555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457855+0000","last_change":"2024-05-08T04:43:30.468636+0000","last_active":"2024-05-08T04:43:32.457855+0000","last_peered":"2024-05-08T04:43:32.457855+0000","last_clean":"2024-05-08T04:43:32.457855+0000","last_became_active":"2024-05-08T04:43:30.468358+0000","last_became_peered":"2024-05-08T04:43:30.468358+0000","last_unstale":"2024-05-08T04:43:32.457855+0000","last_undegraded":"2024-05-08T04:43:32.457855+0000","last_fullsized":"2024-05-08T04:43:32.457855+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:45:50.652015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.457637+0000","last_change":"2024-05-08T04:43:31.479917+0000","last_active":"2024-05-08T04:43:32.457637+0000","last_peered":"2024-05-08T04:43:32.457637+0000","last_clean":"2024-05-08T04:43:32.457637+0000","last_became_active":"2024-05-08T04:43:31.479619+0000","last_became_peered":"2024-05-08T04:43:31.479619+0000","last_unstale":"2024-05-08T04:43:32.457637+0000","last_undegraded":"2024-05-08T04:43:32.457637+0000","last_fullsized":"2024-05-08T04:43:32.457637+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:58.841635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459392+0000","last_change":"2024-05-08T04:43:30.470703+0000","last_active":"2024-05-08T04:43:32.459392+0000","last_peered":"2024-05-08T04:43:32.459392+0000","last_clean":"2024-05-08T04:43:32.459392+0000","last_became_active":"2024-05-08T04:43:30.470326+0000","last_became_peered":"2024-05-08T04:43:30.470326+0000","last_unstale":"2024-05-08T04:43:32.459392+0000","last_undegraded":"2024-05-08T04:43:32.459392+0000","last_fullsized":"2024-05-08T04:43:32.459392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:27.457570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.491692+0000","last_change":"2024-05-08T04:43:31.484669+0000","last_active":"2024-05-08T04:43:31.491692+0000","last_peered":"2024-05-08T04:43:31.491692+0000","last_clean":"2024-05-08T04:43:31.491692+0000","last_became_active":"2024-05-08T04:43:31.484135+0000","last_became_peered":"2024-05-08T04:43:31.484135+0000","last_unstale":"2024-05-08T04:43:31.491692+0000","last_undegraded":"2024-05-08T04:43:31.491692+0000","last_fullsized":"2024-05-08T04:43:31.491692+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:17:25.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.469721+0000","last_change":"2024-05-08T04:43:30.474283+0000","last_active":"2024-05-08T04:43:31.469721+0000","last_peered":"2024-05-08T04:43:31.469721+0000","last_clean":"2024-05-08T04:43:31.469721+0000","last_became_active":"2024-05-08T04:43:30.473788+0000","last_became_peered":"2024-05-08T04:43:30.473788+0000","last_unstale":"2024-05-08T04:43:31.469721+0000","last_undegraded":"2024-05-08T04:43:31.469721+0000","last_fullsized":"2024-05-08T04:43:31.469721+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:00.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.480284+0000","last_change":"2024-05-08T04:43:31.475545+0000","last_active":"2024-05-08T04:43:31.480284+0000","last_peered":"2024-05-08T04:43:31.480284+0000","last_clean":"2024-05-08T04:43:31.480284+0000","last_became_active":"2024-05-08T04:43:31.474573+0000","last_became_peered":"2024-05-08T04:43:31.474573+0000","last_unstale":"2024-05-08T04:43:31.480284+0000","last_undegraded":"2024-05-08T04:43:31.480284+0000","last_fullsized":"2024-05-08T04:43:31.480284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:30.060696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.456171+0000","last_change":"2024-05-08T04:43:30.474802+0000","last_active":"2024-05-08T04:43:31.456171+0000","last_peered":"2024-05-08T04:43:31.456171+0000","last_clean":"2024-05-08T04:43:31.456171+0000","last_became_active":"2024-05-08T04:43:30.473013+0000","last_became_peered":"2024-05-08T04:43:30.473013+0000","last_unstale":"2024-05-08T04:43:31.456171+0000","last_undegraded":"2024-05-08T04:43:31.456171+0000","last_fullsized":"2024-05-08T04:43:31.456171+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:13.578238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458623+0000","last_change":"2024-05-08T04:43:31.473086+0000","last_active":"2024-05-08T04:43:32.458623+0000","last_peered":"2024-05-08T04:43:32.458623+0000","last_clean":"2024-05-08T04:43:32.458623+0000","last_became_active":"2024-05-08T04:43:31.472787+0000","last_became_peered":"2024-05-08T04:43:31.472787+0000","last_unstale":"2024-05-08T04:43:32.458623+0000","last_undegraded":"2024-05-08T04:43:32.458623+0000","last_fullsized":"2024-05-08T04:43:32.458623+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:55:45.500801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.467264+0000","last_change":"2024-05-08T04:43:30.465868+0000","last_active":"2024-05-08T04:43:31.467264+0000","last_peered":"2024-05-08T04:43:31.467264+0000","last_clean":"2024-05-08T04:43:31.467264+0000","last_became_active":"2024-05-08T04:43:30.465518+0000","last_became_peered":"2024-05-08T04:43:30.465518+0000","last_unstale":"2024-05-08T04:43:31.467264+0000","last_undegraded":"2024-05-08T04:43:31.467264+0000","last_fullsized":"2024-05-08T04:43:31.467264+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:53.044345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479271+0000","last_change":"2024-05-08T04:43:31.475699+0000","last_active":"2024-05-08T04:43:31.479271+0000","last_peered":"2024-05-08T04:43:31.479271+0000","last_clean":"2024-05-08T04:43:31.479271+0000","last_became_active":"2024-05-08T04:43:31.475371+0000","last_became_peered":"2024-05-08T04:43:31.475371+0000","last_unstale":"2024-05-08T04:43:31.479271+0000","last_undegraded":"2024-05-08T04:43:31.479271+0000","last_fullsized":"2024-05-08T04:43:31.479271+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:37.585579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459388+0000","last_change":"2024-05-08T04:43:30.478759+0000","last_active":"2024-05-08T04:43:32.459388+0000","last_peered":"2024-05-08T04:43:32.459388+0000","last_clean":"2024-05-08T04:43:32.459388+0000","last_became_active":"2024-05-08T04:43:30.478461+0000","last_became_peered":"2024-05-08T04:43:30.478461+0000","last_unstale":"2024-05-08T04:43:32.459388+0000","last_undegraded":"2024-05-08T04:43:32.459388+0000","last_fullsized":"2024-05-08T04:43:32.459388+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:05.757015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.479688+0000","last_change":"2024-05-08T04:43:31.475609+0000","last_active":"2024-05-08T04:43:31.479688+0000","last_peered":"2024-05-08T04:43:31.479688+0000","last_clean":"2024-05-08T04:43:31.479688+0000","last_became_active":"2024-05-08T04:43:31.475312+0000","last_became_peered":"2024-05-08T04:43:31.475312+0000","last_unstale":"2024-05-08T04:43:31.479688+0000","last_undegraded":"2024-05-08T04:43:31.479688+0000","last_fullsized":"2024-05-08T04:43:31.479688+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:13:01.455965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.462981+0000","last_change":"2024-05-08T04:43:30.476809+0000","last_active":"2024-05-08T04:43:31.462981+0000","last_peered":"2024-05-08T04:43:31.462981+0000","last_clean":"2024-05-08T04:43:31.462981+0000","last_became_active":"2024-05-08T04:43:30.476543+0000","last_became_peered":"2024-05-08T04:43:30.476543+0000","last_unstale":"2024-05-08T04:43:31.462981+0000","last_undegraded":"2024-05-08T04:43:31.462981+0000","last_fullsized":"2024-05-08T04:43:31.462981+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:14.524602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.492750+0000","last_change":"2024-05-08T04:43:31.484279+0000","last_active":"2024-05-08T04:43:31.492750+0000","last_peered":"2024-05-08T04:43:31.492750+0000","last_clean":"2024-05-08T04:43:31.492750+0000","last_became_active":"2024-05-08T04:43:31.483667+0000","last_became_peered":"2024-05-08T04:43:31.483667+0000","last_unstale":"2024-05-08T04:43:31.492750+0000","last_undegraded":"2024-05-08T04:43:31.492750+0000","last_fullsized":"2024-05-08T04:43:31.492750+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:09.360952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-08T04:43:31.455799+0000","last_change":"2024-05-08T04:43:30.471608+0000","last_active":"2024-05-08T04:43:31.455799+0000","last_peered":"2024-05-08T04:43:31.455799+0000","last_clean":"2024-05-08T04:43:31.455799+0000","last_became_active":"2024-05-08T04:43:30.471356+0000","last_became_peered":"2024-05-08T04:43:30.471356+0000","last_unstale":"2024-05-08T04:43:31.455799+0000","last_undegraded":"2024-05-08T04:43:31.455799+0000","last_fullsized":"2024-05-08T04:43:31.455799+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:03.134798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458442+0000","last_change":"2024-05-08T04:43:31.472449+0000","last_active":"2024-05-08T04:43:32.458442+0000","last_peered":"2024-05-08T04:43:32.458442+0000","last_clean":"2024-05-08T04:43:32.458442+0000","last_became_active":"2024-05-08T04:43:31.472164+0000","last_became_peered":"2024-05-08T04:43:31.472164+0000","last_unstale":"2024-05-08T04:43:32.458442+0000","last_undegraded":"2024-05-08T04:43:32.458442+0000","last_fullsized":"2024-05-08T04:43:32.458442+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:30.442611+0000","last_clean_scrub_stamp":"2024-05-08T04:43:30.442611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:48.832617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.459227+0000","last_change":"2024-05-08T04:43:30.472241+0000","last_active":"2024-05-08T04:43:32.459227+0000","last_peered":"2024-05-08T04:43:32.459227+0000","last_clean":"2024-05-08T04:43:32.459227+0000","last_became_active":"2024-05-08T04:43:30.470213+0000","last_became_peered":"2024-05-08T04:43:30.470213+0000","last_unstale":"2024-05-08T04:43:32.459227+0000","last_undegraded":"2024-05-08T04:43:32.459227+0000","last_fullsized":"2024-05-08T04:43:32.459227+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:43:29.437854+0000","last_clean_scrub_stamp":"2024-05-08T04:43:29.437854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:14.836887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-08T04:43:32.458267+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:43:32.458267+0000","last_peered":"2024-05-08T04:43:32.458267+0000","last_clean":"2024-05-08T04:43:32.458267+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:43:32.458267+0000","last_undegraded":"2024-05-08T04:43:32.458267+0000","last_fullsized":"2024-05-08T04:43:32.458267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":1276522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":685516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":683926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":683974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1568768,"data_stored":684064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":684520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":683904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2154496,"data_stored":1274272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:43:35.915 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T04:43:35.916 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T04:43:36.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 Processor -- start 2024-05-08T04:43:36.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 -- start start 2024-05-08T04:43:36.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ce10 0x7fdf8006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf8006e390 0x7fdf800ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800c6600 con 0x7fdf8015ba20 2024-05-08T04:43:36.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8015fbe0 con 0x7fdf8006e390 2024-05-08T04:43:36.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf86200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c6040 con 0x7fdf8015ce10 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf7fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf8006e390 0x7fdf800ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf7ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ce10 0x7fdf8006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf7f400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdf8015ba20 0x7fdf80160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:57500/0 (socket says 172.21.15.164:57500) 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf7fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf8006e390 0x7fdf800ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:42784/0 (socket says 172.21.15.164:42784) 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.095+0000 7fdf7f400640 1 -- 172.21.15.164:0/225990299 learned_addr learned my addr 172.21.15.164:0/225990299 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:36.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7fe00640 1 -- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ce10 msgr2=0x7fdf8006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:36.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7fe00640 1 --2- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ce10 0x7fdf8006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7fe00640 1 -- 172.21.15.164:0/225990299 >> v1:172.21.15.172:6789/0 conn(0x7fdf8015ba20 legacy=0x7fdf80160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7fe00640 1 -- 172.21.15.164:0/225990299 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800c6320 con 0x7fdf8006e390 2024-05-08T04:43:36.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7e000640 1 -- 172.21.15.164:0/225990299 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119087024 0 0) 0x7fdf800c6600 con 0x7fdf8015ba20 2024-05-08T04:43:36.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7fe00640 1 --2- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf800ae6e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ee148a53583f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7e000640 1 -- 172.21.15.164:0/225990299 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b0070 con 0x7fdf8006e390 2024-05-08T04:43:36.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7e000640 1 -- 172.21.15.164:0/225990299 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780ab600 con 0x7fdf8006e390 2024-05-08T04:43:36.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.099+0000 7fdf7e000640 1 -- 172.21.15.164:0/225990299 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780ab9d0 con 0x7fdf8006e390 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 msgr2=0x7fdf800ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf800ae6e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c410 comp rx=0 tx=0).stop 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- 172.21.15.164:0/225990299 shutdown_connections 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ce10 0x7fdf8006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- 172.21.15.164:0/225990299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf800ae6e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- 172.21.15.164:0/225990299 >> 172.21.15.164:0/225990299 conn(0x7fdf800cf770 msgr2=0x7fdf800cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- 172.21.15.164:0/225990299 shutdown_connections 2024-05-08T04:43:36.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- 172.21.15.164:0/225990299 wait complete. 2024-05-08T04:43:36.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 Processor -- start 2024-05-08T04:43:36.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- start start 2024-05-08T04:43:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf801f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ba20 0x7fdf800afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80168b70 con 0x7fdf8015ce10 2024-05-08T04:43:36.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800ce090 con 0x7fdf8006e390 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf86200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80168cf0 con 0x7fdf8015ba20 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59970/0 (socket says 172.21.15.164:59970) 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 -- 172.21.15.164:0/3471072150 learned_addr learned my addr 172.21.15.164:0/3471072150 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7f400640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf801f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7ea00640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ba20 0x7fdf800afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 -- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ba20 msgr2=0x7fdf800afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ba20 0x7fdf800afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 -- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 msgr2=0x7fdf801f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf801f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 -- 172.21.15.164:0/3471072150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80150cf0 con 0x7fdf8015ce10 2024-05-08T04:43:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.103+0000 7fdf7fe00640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdf780ae040 tx=0x7fdf780191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b8020 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780b4410 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b0040 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800b2880 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80150350 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80072d30 con 0x7fdf8015ce10 2024-05-08T04:43:36.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fdf780bb020 con 0x7fdf8015ce10 2024-05-08T04:43:36.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 0x7fdf60069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fdf7811d420 con 0x7fdf8015ce10 2024-05-08T04:43:36.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf7f400640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 0x7fdf60069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.107+0000 7fdf7f400640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 0x7fdf60069f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004fd0 tx=0x7fdf70001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.111+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf780c2030 con 0x7fdf8015ce10 2024-05-08T04:43:36.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.319+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdf801540f0 con 0x7fdf8015ce10 2024-05-08T04:43:36.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf77400640 1 -- 172.21.15.164:0/3471072150 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdf80150cf0 con 0x7fdf8015ce10 2024-05-08T04:43:36.324 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-05-08T04:43:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKATtmoOKFGxAAEwF3ja44T73OQ2M1rlqVdQ== 2024-05-08T04:43:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-05-08T04:43:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-05-08T04:43:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T04:43:36.325 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 msgr2=0x7fdf60069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 0x7fdf60069f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004fd0 tx=0x7fdf70001770 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 msgr2=0x7fdf80145b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdf780ae040 tx=0x7fdf780191b0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 shutdown_connections 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fdf60067aa0 0x7fdf60069f60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf8015ba20 0x7fdf800afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf8015ce10 0x7fdf80145b40 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 --2- 172.21.15.164:0/3471072150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf8006e390 0x7fdf801f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 >> 172.21.15.164:0/3471072150 conn(0x7fdf800cf770 msgr2=0x7fdf8006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:36.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 shutdown_connections 2024-05-08T04:43:36.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.323+0000 7fdf86200640 1 -- 172.21.15.164:0/3471072150 wait complete. 2024-05-08T04:43:36.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:36.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 Processor -- start 2024-05-08T04:43:36.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 -- start start 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f035806b3a0 0x7f03580afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 0x7f03580714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035815e100 con 0x7f035815dd20 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580b5a30 con 0x7f035806b3a0 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580b0220 con 0x7f03580ae2a0 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f035806b3a0 0x7f03580afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f035806b3a0 0x7f03580afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59976/0 (socket says 172.21.15.164:59976) 2024-05-08T04:43:36.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 -- 172.21.15.164:0/2914988597 learned_addr learned my addr 172.21.15.164:0/2914988597 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 -- 172.21.15.164:0/2914988597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 msgr2=0x7f03580714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ca00640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 0x7f03580714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 0x7f03580714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 -- 172.21.15.164:0/2914988597 >> v1:172.21.15.172:6789/0 conn(0x7f035815dd20 legacy=0x7f035806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f0357400640 1 -- 172.21.15.164:0/2914988597 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358161900 con 0x7f035806b3a0 2024-05-08T04:43:36.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.483+0000 7f035ca00640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 0x7f03580714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:36.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f0357400640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035806b3a0 0x7f03580afb80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0344009870 tx=0x7f034401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1edbadf558597a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f0356a00640 1 -- 172.21.15.164:0/2914988597 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440b0070 con 0x7f035806b3a0 2024-05-08T04:43:36.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f0356a00640 1 -- 172.21.15.164:0/2914988597 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440ab600 con 0x7f035806b3a0 2024-05-08T04:43:36.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f0356a00640 1 -- 172.21.15.164:0/2914988597 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440ab9d0 con 0x7f035806b3a0 2024-05-08T04:43:36.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 -- 172.21.15.164:0/2914988597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035806b3a0 msgr2=0x7f03580afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035806b3a0 0x7f03580afb80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0344009870 tx=0x7f034401c340 comp rx=0 tx=0).stop 2024-05-08T04:43:36.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 -- 172.21.15.164:0/2914988597 shutdown_connections 2024-05-08T04:43:36.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03580ae2a0 0x7f03580714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 --2- 172.21.15.164:0/2914988597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035806b3a0 0x7f03580afb80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 -- 172.21.15.164:0/2914988597 >> 172.21.15.164:0/2914988597 conn(0x7f03580b24d0 msgr2=0x7f03580b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:36.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 -- 172.21.15.164:0/2914988597 shutdown_connections 2024-05-08T04:43:36.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.487+0000 7f035ea00640 1 -- 172.21.15.164:0/2914988597 wait complete. 2024-05-08T04:43:36.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 Processor -- start 2024-05-08T04:43:36.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- start start 2024-05-08T04:43:36.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f035806b3a0 0x7f0358140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03580ae2a0 0x7f0358141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03581626c0 con 0x7f035815dd20 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580ae680 con 0x7f03580ae2a0 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035816ac00 con 0x7f035806b3a0 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59986/0 (socket says 172.21.15.164:59986) 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 -- 172.21.15.164:0/3692328249 learned_addr learned my addr 172.21.15.164:0/3692328249 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f0357e00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f035806b3a0 0x7f0358140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f0357400640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03580ae2a0 0x7f0358141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 -- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f035806b3a0 msgr2=0x7f0358140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f035806b3a0 0x7f0358140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 -- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03580ae2a0 msgr2=0x7f0358141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03580ae2a0 0x7f0358141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 -- 172.21.15.164:0/3692328249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358141b70 con 0x7f035815dd20 2024-05-08T04:43:36.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ca00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f035000cef0 tx=0x7f035000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035001a070 con 0x7f035815dd20 2024-05-08T04:43:36.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035000d860 con 0x7f035815dd20 2024-05-08T04:43:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035000dc50 con 0x7f035815dd20 2024-05-08T04:43:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0358141e50 con 0x7f035815dd20 2024-05-08T04:43:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.491+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0358142410 con 0x7f035815dd20 2024-05-08T04:43:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358069d60 con 0x7f035815dd20 2024-05-08T04:43:36.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f035001d070 con 0x7f035815dd20 2024-05-08T04:43:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f034fe00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 0x7f0338069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0350084270 con 0x7f035815dd20 2024-05-08T04:43:36.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f0357e00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 0x7f0338069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:36.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.495+0000 7f0357e00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 0x7f0338069f60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f03480098e0 tx=0x7f0348009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:36.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.499+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03500566b0 con 0x7f035815dd20 2024-05-08T04:43:36.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f035806bb40 con 0x7f035815dd20 2024-05-08T04:43:36.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f034fe00640 1 -- 172.21.15.164:0/3692328249 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f035005a750 con 0x7f035815dd20 2024-05-08T04:43:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:36.688 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:36.688 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 82 2024-05-08T04:43:36.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 msgr2=0x7f0338069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 0x7f0338069f60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f03480098e0 tx=0x7f0348009060 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 msgr2=0x7f035814f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f035000cef0 tx=0x7f035000a710 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 shutdown_connections 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0338067aa0 0x7f0338069f60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f035815dd20 0x7f035814f790 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03580ae2a0 0x7f0358141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 --2- 172.21.15.164:0/3692328249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f035806b3a0 0x7f0358140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 >> 172.21.15.164:0/3692328249 conn(0x7f03580b24d0 msgr2=0x7f03580b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 shutdown_connections 2024-05-08T04:43:36.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:36.687+0000 7f035ea00640 1 -- 172.21.15.164:0/3692328249 wait complete. 2024-05-08T04:43:37.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T04:43:30.835958+0000', 'modified': '2024-05-08T04:43:33.847549+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8400': {'gid': 8400, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6837/333884670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6835', 'nonce': 333884670}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 333884670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T04:43:37.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T04:43:37.112 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T04:43:37.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.247+0000 7f5171c00640 1 Processor -- start 2024-05-08T04:43:37.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.247+0000 7f5171c00640 1 -- start start 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f516c15ba70 0x7f516c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f516c15be50 0x7f516c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c0aeda0 0x7f516c0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c15fe70 con 0x7f516c15ba70 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c0cd9f0 con 0x7f516c15be50 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f5171c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0cdcd0 con 0x7f516c0aeda0 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f516c15ba70 0x7f516c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f516c15ba70 0x7f516c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59990/0 (socket says 172.21.15.164:59990) 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 -- 172.21.15.164:0/3150404891 learned_addr learned my addr 172.21.15.164:0/3150404891 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516b400640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c0aeda0 0x7f516c0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516a000640 1 --2- 172.21.15.164:0/3150404891 >> v2:172.21.15.172:3300/0 conn(0x7f516c15be50 0x7f516c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 -- 172.21.15.164:0/3150404891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c0aeda0 msgr2=0x7f516c0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c0aeda0 0x7f516c0cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 -- 172.21.15.164:0/3150404891 >> v2:172.21.15.172:3300/0 conn(0x7f516c15be50 msgr2=0x7f516c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 --2- 172.21.15.164:0/3150404891 >> v2:172.21.15.172:3300/0 conn(0x7f516c15be50 0x7f516c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.251+0000 7f516aa00640 1 -- 172.21.15.164:0/3150404891 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c15f820 con 0x7f516c15ba70 2024-05-08T04:43:37.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516aa00640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c15ba70 0x7f516c160300 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f516c15f780 tx=0x7f515c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8138f96b821c6bab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5169600640 1 -- 172.21.15.164:0/3150404891 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0a5750 con 0x7f516c15ba70 2024-05-08T04:43:37.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5169600640 1 -- 172.21.15.164:0/3150404891 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0a58f0 con 0x7f516c15ba70 2024-05-08T04:43:37.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5169600640 1 -- 172.21.15.164:0/3150404891 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0a5cc0 con 0x7f516c15ba70 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- 172.21.15.164:0/3150404891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c15ba70 msgr2=0x7f516c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c15ba70 0x7f516c160300 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f516c15f780 tx=0x7f515c01c9c0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- 172.21.15.164:0/3150404891 shutdown_connections 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c0aeda0 0x7f516c0cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- 172.21.15.164:0/3150404891 >> v2:172.21.15.172:3300/0 conn(0x7f516c15be50 0x7f516c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- 172.21.15.164:0/3150404891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c15ba70 0x7f516c160300 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- 172.21.15.164:0/3150404891 >> 172.21.15.164:0/3150404891 conn(0x7f516c06ac90 msgr2=0x7f516c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- 172.21.15.164:0/3150404891 shutdown_connections 2024-05-08T04:43:37.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- 172.21.15.164:0/3150404891 wait complete. 2024-05-08T04:43:37.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 Processor -- start 2024-05-08T04:43:37.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- start start 2024-05-08T04:43:37.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516c15ba70 0x7f516c071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 0x7f516c1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b3d40 con 0x7f516c0aeda0 2024-05-08T04:43:37.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b3bc0 con 0x7f516c15ba70 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f5171c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b3ec0 con 0x7f516c15be50 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59998/0 (socket says 172.21.15.164:59998) 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516aa00640 1 -- 172.21.15.164:0/3525308450 learned_addr learned my addr 172.21.15.164:0/3525308450 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516a000640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516c15ba70 0x7f516c071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.255+0000 7f516b400640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 0x7f516c1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 -- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 msgr2=0x7f516c1f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 0x7f516c1f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 -- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516c15ba70 msgr2=0x7f516c071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516c15ba70 0x7f516c071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 -- 172.21.15.164:0/3525308450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c070860 con 0x7f516c0aeda0 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516b400640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 0x7f516c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f516aa00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af040 tx=0x7f515c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5162a00640 1 -- 172.21.15.164:0/3525308450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0b8670 con 0x7f516c0aeda0 2024-05-08T04:43:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5162a00640 1 -- 172.21.15.164:0/3525308450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0be070 con 0x7f516c0aeda0 2024-05-08T04:43:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5162a00640 1 -- 172.21.15.164:0/3525308450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0b8a70 con 0x7f516c0aeda0 2024-05-08T04:43:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c070580 con 0x7f516c0aeda0 2024-05-08T04:43:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c15c2a0 con 0x7f516c0aeda0 2024-05-08T04:43:37.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.259+0000 7f5162a00640 1 -- 172.21.15.164:0/3525308450 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f515c0b8c10 con 0x7f516c0aeda0 2024-05-08T04:43:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5162a00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 0x7f514c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5162a00640 1 -- 172.21.15.164:0/3525308450 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f515c11eb80 con 0x7f516c0aeda0 2024-05-08T04:43:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516a000640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 0x7f514c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 Processor -- start 2024-05-08T04:43:37.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- start start 2024-05-08T04:43:37.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516a000640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 0x7f514c069f60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f516c158760 tx=0x7f5158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204c920 0x557c2204cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c220029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21eea5f0 con 0x557c22002f30 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e51da0 con 0x557c2204d240 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c21dda150 con 0x557c2204c920 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60012/0 (socket says 172.21.15.164:60012) 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 -- 172.21.15.164:0/3816452930 learned_addr learned my addr 172.21.15.164:0/3816452930 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516aa00640 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204c920 0x557c2204cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516a000640 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c220029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 -- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204c920 msgr2=0x557c2204cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204c920 0x557c2204cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 -- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 msgr2=0x557c220029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c220029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 -- 172.21.15.164:0/3816452930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c21df3ca0 con 0x557c22002f30 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f516b400640 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f516400ba30 tx=0x7f516400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc83c2488a84a4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5157400640 1 -- 172.21.15.164:0/3816452930 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164018030 con 0x557c22002f30 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5157400640 1 -- 172.21.15.164:0/3816452930 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164004020 con 0x557c22002f30 2024-05-08T04:43:37.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5157400640 1 -- 172.21.15.164:0/3816452930 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164004410 con 0x557c22002f30 2024-05-08T04:43:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- 172.21.15.164:0/3816452930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 msgr2=0x557c22003310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f516400ba30 tx=0x7f516400bef0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- 172.21.15.164:0/3816452930 shutdown_connections 2024-05-08T04:43:37.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c22002f30 0x557c22003310 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c220029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 --2- 172.21.15.164:0/3816452930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204c920 0x557c2204cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- 172.21.15.164:0/3816452930 >> 172.21.15.164:0/3816452930 conn(0x557c220092a0 msgr2=0x557c220012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- 172.21.15.164:0/3816452930 shutdown_connections 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.263+0000 7f5175003000 1 -- 172.21.15.164:0/3816452930 wait complete. 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- start start 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 Processor -- start 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c22053cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c22006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e0c990 con 0x557c2204c920 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e4de40 con 0x557c2204d240 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c21ddd900 con 0x557c22002f30 2024-05-08T04:43:37.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 10 client.0 _ll_register_callbacks cb 0x557c21f051c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60020/0 (socket says 172.21.15.164:60020) 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 -- 172.21.15.164:0/1701681726 learned_addr learned my addr 172.21.15.164:0/1701681726 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516b400640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c22006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516aa00640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c22053cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 -- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 msgr2=0x557c22053cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c22053cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 -- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 msgr2=0x557c22006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c22006aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 -- 172.21.15.164:0/1701681726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c21e0a630 con 0x557c2204c920 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516a000640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5158016700 tx=0x7f5158004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd3d33e7a56da72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:43:37.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158026020 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f5158014880 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5164004020 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158030050 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 10 client.8538 fetch_fsmap learned FSMap version 82 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x557c21dda3a0 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164004020 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5158033020 con 0x557c2204c920 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 10 client.8538 fetch_fsmap finished waiting for FSMap version 82 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 10 client.8538 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 20 client.8538 populate_metadata read hostname 'smithi164' 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220074b0 0x557c22007890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 4 client.8538 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 --> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c21deda50 con 0x557c220074b0 2024-05-08T04:43:37.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.267+0000 7f516aa00640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220074b0 0x557c22007890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f516aa00640 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220074b0 0x557c22007890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae06a137b26cc03d server_cookie=24446d4c7fb6661d in_seq=0 out_seq=0 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f514be00640 10 client.8538 ms_handle_connect on v2:172.21.15.164:6835/333884670 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f514be00640 1 -- 172.21.15.164:0/1701681726 <== mds.0 v2:172.21.15.164:6835/333884670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c21deda50 con 0x557c220074b0 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f514be00640 10 client.8538 handle_command_reply: tid=0 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 client.8538 shutdown 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 msgr2=0x557c22054230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5158016700 tx=0x7f5158004740 comp rx=0 tx=0).stop 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 shutdown_connections 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220074b0 0x557c22007890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204d240 0x557c22006aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c22054230 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f514be00640 0 client.8538 ms_handle_reset on v2:172.21.15.164:6835/333884670 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 --2- 172.21.15.164:0/1701681726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c22053cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 >> 172.21.15.164:0/1701681726 conn(0x557c220092a0 msgr2=0x557c220019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 shutdown_connections 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 1 -- 172.21.15.164:0/1701681726 wait complete. 2024-05-08T04:43:37.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.271+0000 7f5175003000 20 client.8538 trim_cache size 0 max 16384 2024-05-08T04:43:37.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 Processor -- start 2024-05-08T04:43:37.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- start start 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c220a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204c920 0x557c220a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1fb280 con 0x557c2204d240 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0afb20 con 0x557c2204c920 2024-05-08T04:43:37.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1fac80 con 0x557c22002f30 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60022/0 (socket says 172.21.15.164:60022) 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 -- 172.21.15.164:0/2255410637 learned_addr learned my addr 172.21.15.164:0/2255410637 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516aa00640 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c220a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516a000640 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204c920 0x557c220a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 -- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 msgr2=0x557c220a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c220a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 -- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204c920 msgr2=0x557c220a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204c920 0x557c220a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 -- 172.21.15.164:0/2255410637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2204d070 con 0x557c2204d240 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f516b400640 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5164008010 tx=0x7f5164014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4030ecd897588886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5156a00640 1 -- 172.21.15.164:0/2255410637 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400fa80 con 0x557c2204d240 2024-05-08T04:43:37.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5156a00640 1 -- 172.21.15.164:0/2255410637 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400fc20 con 0x557c2204d240 2024-05-08T04:43:37.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5156a00640 1 -- 172.21.15.164:0/2255410637 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164004020 con 0x557c2204d240 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- 172.21.15.164:0/2255410637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 msgr2=0x557c220a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5164008010 tx=0x7f5164014000 comp rx=0 tx=0).stop 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- 172.21.15.164:0/2255410637 shutdown_connections 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204d240 0x557c220a8ac0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c2204c920 0x557c220a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 --2- 172.21.15.164:0/2255410637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c22002f30 0x557c220a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- 172.21.15.164:0/2255410637 >> 172.21.15.164:0/2255410637 conn(0x557c220092a0 msgr2=0x557c22006220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- 172.21.15.164:0/2255410637 shutdown_connections 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.287+0000 7f5175003000 1 -- 172.21.15.164:0/2255410637 wait complete. 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- start start 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 Processor -- start 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c22002f30 0x557c22059620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 0x557c220a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e563d0 con 0x557c2204c920 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f0dd50 con 0x557c22002f30 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c21dec980 con 0x557c2204d240 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 10 client.0 _ll_register_callbacks cb 0x7f516400b450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T04:43:37.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60032/0 (socket says 172.21.15.164:60032) 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 -- 172.21.15.164:0/3786556728 learned_addr learned my addr 172.21.15.164:0/3786556728 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516aa00640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c22002f30 0x557c22059620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516b400640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 0x557c220a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 -- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 msgr2=0x557c220a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 0x557c220a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 -- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c22002f30 msgr2=0x557c22059620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c22002f30 0x557c22059620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 -- 172.21.15.164:0/3786556728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c21ed3600 con 0x557c2204c920 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516b400640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 0x557c220a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516a000640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5158010cf0 tx=0x7f5158015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4b1fd5eb0308aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158026070 con 0x557c2204c920 2024-05-08T04:43:37.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f5158014880 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f516400fc20 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158041050 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 10 client.8544 fetch_fsmap learned FSMap version 82 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x557c21de69b0 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400fc20 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5158044020 con 0x557c2204c920 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 10 client.8544 fetch_fsmap finished waiting for FSMap version 82 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 10 client.8544 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 20 client.8544 populate_metadata read hostname 'smithi164' 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220a9ee0 0x557c220aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 4 client.8544 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T04:43:37.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 --> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557c21e5f2e0 con 0x557c220a9ee0 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516aa00640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220a9ee0 0x557c220aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f516aa00640 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220a9ee0 0x557c220aa2c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89fa667f84d7386e server_cookie=853e95750a94670a in_seq=0 out_seq=0 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 10 client.8544 ms_handle_connect on v2:172.21.15.164:6835/333884670 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 1 -- 172.21.15.164:0/3786556728 <== mds.0 v2:172.21.15.164:6835/333884670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557c21e5f2e0 con 0x557c220a9ee0 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f514be00640 10 client.8544 handle_command_reply: tid=0 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.291+0000 7f5175003000 1 client.8544 shutdown 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 msgr2=0x557c220a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5158010cf0 tx=0x7f5158015000 comp rx=0 tx=0).stop 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 shutdown_connections 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:6835/333884670,v1:172.21.15.164:6837/333884670] conn(0x557c220a9ee0 0x557c220aa2c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c2204d240 0x557c220a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c2204c920 0x557c220a9460 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 --2- 172.21.15.164:0/3786556728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c22002f30 0x557c22059620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f514be00640 0 client.8544 ms_handle_reset on v2:172.21.15.164:6835/333884670 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 >> 172.21.15.164:0/3786556728 conn(0x557c220092a0 msgr2=0x557c220548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 shutdown_connections 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 1 -- 172.21.15.164:0/3786556728 wait complete. 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5175003000 20 client.8544 trim_cache size 0 max 16384 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "6150cf8f-12a4-433b-b39d-bc24e302fd2a", 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 8400, 2024-05-08T04:43:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 5.8422899260000003, 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 82, 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 126, 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 126, 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 10.767914972 2024-05-08T04:43:37.301 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 msgr2=0x7f514c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 0x7f514c069f60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f516c158760 tx=0x7f5158008040 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 msgr2=0x7f516c152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af040 tx=0x7f515c019760 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 shutdown_connections 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f514c067aa0 0x7f514c069f60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516c15be50 0x7f516c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516c15ba70 0x7f516c071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 --2- 172.21.15.164:0/3525308450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0aeda0 0x7f516c152c30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 >> 172.21.15.164:0/3525308450 conn(0x7f516c06ac90 msgr2=0x7f516c06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 shutdown_connections 2024-05-08T04:43:37.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.295+0000 7f5171c00640 1 -- 172.21.15.164:0/3525308450 wait complete. 2024-05-08T04:43:37.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:43:37.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.447+0000 7f091c400640 1 Processor -- start 2024-05-08T04:43:37.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.447+0000 7f091c400640 1 -- start start 2024-05-08T04:43:37.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f091c400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f09140b26c0 0x7f09140cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f091c400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091406c770 con 0x7f091415ab60 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f091c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091406ca50 con 0x7f09140cfbd0 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f091c400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140d0060 con 0x7f09140b26c0 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f091415ab60 0x7f0914072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:39724/0 (socket says 172.21.15.164:39724) 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919a00640 1 -- 172.21.15.164:0/3944788314 learned_addr learned my addr 172.21.15.164:0/3944788314 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:37.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919000640 1 --2- 172.21.15.164:0/3944788314 >> v2:172.21.15.164:3300/0 conn(0x7f09140b26c0 0x7f09140cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919000640 1 -- 172.21.15.164:0/3944788314 >> v1:172.21.15.172:6789/0 conn(0x7f09140cfbd0 legacy=0x7f09140cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:37.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919000640 1 -- 172.21.15.164:0/3944788314 >> v1:172.21.15.164:6789/0 conn(0x7f091415ab60 legacy=0x7f0914072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0913e00640 1 -- 172.21.15.164:0/3944788314 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80516014 0 0) 0x7f091406c770 con 0x7f091415ab60 2024-05-08T04:43:37.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.451+0000 7f0919000640 1 -- 172.21.15.164:0/3944788314 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091406cd30 con 0x7f09140b26c0 2024-05-08T04:43:37.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f0919000640 1 --2- 172.21.15.164:0/3944788314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f09140cf690 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f09000097c0 tx=0x7f090001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a12278416d1bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f0913e00640 1 -- 172.21.15.164:0/3944788314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000b0070 con 0x7f09140b26c0 2024-05-08T04:43:37.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f0913e00640 1 -- 172.21.15.164:0/3944788314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09000ab600 con 0x7f09140b26c0 2024-05-08T04:43:37.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f0913e00640 1 -- 172.21.15.164:0/3944788314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000ab9d0 con 0x7f09140b26c0 2024-05-08T04:43:37.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- 172.21.15.164:0/3944788314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 msgr2=0x7f09140cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 --2- 172.21.15.164:0/3944788314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f09140cf690 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f09000097c0 tx=0x7f090001c460 comp rx=0 tx=0).stop 2024-05-08T04:43:37.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- 172.21.15.164:0/3944788314 shutdown_connections 2024-05-08T04:43:37.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 --2- 172.21.15.164:0/3944788314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f09140cf690 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- 172.21.15.164:0/3944788314 >> 172.21.15.164:0/3944788314 conn(0x7f091406ac90 msgr2=0x7f091406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- 172.21.15.164:0/3944788314 shutdown_connections 2024-05-08T04:43:37.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- 172.21.15.164:0/3944788314 wait complete. 2024-05-08T04:43:37.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 Processor -- start 2024-05-08T04:43:37.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.455+0000 7f091c400640 1 -- start start 2024-05-08T04:43:37.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09140cfbd0 0x7f0914158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f091415ab60 0x7f0914139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09140aeeb0 con 0x7f09140b26c0 2024-05-08T04:43:37.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09140aed30 con 0x7f091415ab60 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0914168980 con 0x7f09140cfbd0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60042/0 (socket says 172.21.15.164:60042) 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 -- 172.21.15.164:0/3654876841 learned_addr learned my addr 172.21.15.164:0/3654876841 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919000640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09140cfbd0 0x7f0914158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091a400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f091415ab60 0x7f0914139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 -- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09140cfbd0 msgr2=0x7f0914158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09140cfbd0 0x7f0914158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 -- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f091415ab60 msgr2=0x7f0914139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f091415ab60 0x7f0914139df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 -- 172.21.15.164:0/3654876841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914154e30 con 0x7f09140b26c0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0919a00640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f090800c480 tx=0x7f090800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908018070 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0908007500 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908004a30 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914155110 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09141556d0 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0914155eb0 con 0x7f09140b26c0 2024-05-08T04:43:37.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.459+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f0908004bd0 con 0x7f09140b26c0 2024-05-08T04:43:37.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.463+0000 7f0911600640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 0x7f08f4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:37.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.463+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0908082e20 con 0x7f09140b26c0 2024-05-08T04:43:37.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.463+0000 7f0919000640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 0x7f08f4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:37.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.463+0000 7f0919000640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 0x7f08f4069f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f09000ad000 tx=0x7f09000a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:37.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.467+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0914155eb0 con 0x7f09140b26c0 2024-05-08T04:43:37.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0914069d60 con 0x7f09140b26c0 2024-05-08T04:43:37.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f0911600640 1 -- 172.21.15.164:0/3654876841 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0908055020 con 0x7f09140b26c0 2024-05-08T04:43:37.660 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:43:37.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 msgr2=0x7f08f4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 0x7f08f4069f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f09000ad000 tx=0x7f09000a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:37.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 msgr2=0x7f091415e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:37.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f090800c480 tx=0x7f090800c940 comp rx=0 tx=0).stop 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 shutdown_connections 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f08f4067aa0 0x7f08f4069f60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09140cfbd0 0x7f0914158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f091415ab60 0x7f0914139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 --2- 172.21.15.164:0/3654876841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09140b26c0 0x7f091415e610 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 >> 172.21.15.164:0/3654876841 conn(0x7f091406ac90 msgr2=0x7f091406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 shutdown_connections 2024-05-08T04:43:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:37.659+0000 7f091c400640 1 -- 172.21.15.164:0/3654876841 wait complete. 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:43:37.681 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:37.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.682 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-05-08T04:43:37.691 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42049sec preferred_lft 42049sec 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:a4:0e:8b:5e brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:43:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:37.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.694 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:43:37.752 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-05-08T04:43:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.754 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:43:37.810 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:43:37.811 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:37.811 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:37.811 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:37.811 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:43:37.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.811 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-05-08T04:43:37.811 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-05-08T04:43:37.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:43:37.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:43:37.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:43:37.812 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-05-08T04:43:37.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.898 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:43:37.951 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:43:37.951 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:43:37.951 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:37.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:37.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-05-08T04:43:38.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:38.014 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:38.062 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T04:43:38.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:38.063 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:38.113 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:43:38.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.114 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:38.202 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:38.202 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:38.202 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:38.217 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:38.217 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:38.217 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:38.217 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:38.217 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:43:38.228 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:43:38.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:43:38.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:38.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.252 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:43:38.311 INFO:teuthology.orchestra.run.smithi164.stdout:client7109 2024-05-08T04:43:38.311 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:38.311 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:38.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T04:43:38.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.543+0000 7f7a35000640 1 Processor -- start 2024-05-08T04:43:38.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.543+0000 7f7a35000640 1 -- start start 2024-05-08T04:43:38.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a35000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a3015ab60 0x7f7a30072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a35000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300cfda0 0x7f7a300cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a35000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a30072640 con 0x7f7a300b26c0 2024-05-08T04:43:38.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a35000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3006c770 con 0x7f7a3015ab60 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a35000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3006ca50 con 0x7f7a300cfda0 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a3015ab60 0x7f7a30072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a3015ab60 0x7f7a30072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60056/0 (socket says 172.21.15.164:60056) 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 -- 172.21.15.164:0/407492475 learned_addr learned my addr 172.21.15.164:0/407492475 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2e800640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300cfda0 0x7f7a300cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 -- 172.21.15.164:0/407492475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300cfda0 msgr2=0x7f7a300cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:38.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300cfda0 0x7f7a300cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 -- 172.21.15.164:0/407492475 >> v1:172.21.15.172:6789/0 conn(0x7f7a300b26c0 legacy=0x7f7a300cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.547+0000 7f7a2de00640 1 -- 172.21.15.164:0/407492475 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3006cd30 con 0x7f7a3015ab60 2024-05-08T04:43:38.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a2de00640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a30072100 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009870 tx=0x7f7a2001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27aff40282ee07a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:38.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a2d400640 1 -- 172.21.15.164:0/407492475 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200b2070 con 0x7f7a3015ab60 2024-05-08T04:43:38.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a2d400640 1 -- 172.21.15.164:0/407492475 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a200a9070 con 0x7f7a3015ab60 2024-05-08T04:43:38.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a2d400640 1 -- 172.21.15.164:0/407492475 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200ad880 con 0x7f7a3015ab60 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- 172.21.15.164:0/407492475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 msgr2=0x7f7a30072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a30072100 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009870 tx=0x7f7a2001c560 comp rx=0 tx=0).stop 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- 172.21.15.164:0/407492475 shutdown_connections 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300cfda0 0x7f7a300cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- 172.21.15.164:0/407492475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a30072100 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- 172.21.15.164:0/407492475 >> 172.21.15.164:0/407492475 conn(0x7f7a3006ac90 msgr2=0x7f7a3006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:38.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- 172.21.15.164:0/407492475 shutdown_connections 2024-05-08T04:43:38.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- 172.21.15.164:0/407492475 wait complete. 2024-05-08T04:43:38.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 Processor -- start 2024-05-08T04:43:38.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 -- start start 2024-05-08T04:43:38.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300b26c0 0x7f7a30152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a300cfda0 0x7f7a3014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.551+0000 7f7a35000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300aeeb0 con 0x7f7a3015ab60 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300aed30 con 0x7f7a300cfda0 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30167970 con 0x7f7a300b26c0 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60060/0 (socket says 172.21.15.164:60060) 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 -- 172.21.15.164:0/2809710271 learned_addr learned my addr 172.21.15.164:0/2809710271 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2de00640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300b26c0 0x7f7a30152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a27e00640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a300cfda0 0x7f7a3014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 -- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300b26c0 msgr2=0x7f7a30152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300b26c0 0x7f7a30152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 -- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a300cfda0 msgr2=0x7f7a3014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a300cfda0 0x7f7a3014e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 -- 172.21.15.164:0/2809710271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a301533b0 con 0x7f7a3015ab60 2024-05-08T04:43:38.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a2e800640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800efc0 tx=0x7f7a2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:38.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28018020 con 0x7f7a3015ab60 2024-05-08T04:43:38.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28014480 con 0x7f7a3015ab60 2024-05-08T04:43:38.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28014870 con 0x7f7a3015ab60 2024-05-08T04:43:38.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3014eea0 con 0x7f7a3015ab60 2024-05-08T04:43:38.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a301fad20 con 0x7f7a3015ab60 2024-05-08T04:43:38.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30069d60 con 0x7f7a3015ab60 2024-05-08T04:43:38.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.555+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f7a2801b020 con 0x7f7a3015ab60 2024-05-08T04:43:38.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.559+0000 7f7a26000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 0x7f7a0c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:38.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.559+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7a28084680 con 0x7f7a3015ab60 2024-05-08T04:43:38.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.559+0000 7f7a2de00640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 0x7f7a0c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:38.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.559+0000 7f7a2de00640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 0x7f7a0c069f60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7a200af000 tx=0x7f7a20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:38.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.563+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a2808b070 con 0x7f7a3015ab60 2024-05-08T04:43:38.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7a3006d9a0 con 0x7f7a3015ab60 2024-05-08T04:43:38.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a26000640 1 -- 172.21.15.164:0/2809710271 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7a28022030 con 0x7f7a3015ab60 2024-05-08T04:43:38.756 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 msgr2=0x7f7a0c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 0x7f7a0c069f60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7a200af000 tx=0x7f7a20008040 comp rx=0 tx=0).stop 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 msgr2=0x7f7a3014c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800efc0 tx=0x7f7a2800c490 comp rx=0 tx=0).stop 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 shutdown_connections 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7a0c067aa0 0x7f7a0c069f60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3015ab60 0x7f7a3014c630 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a300cfda0 0x7f7a3014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 --2- 172.21.15.164:0/2809710271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a300b26c0 0x7f7a30152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 >> 172.21.15.164:0/2809710271 conn(0x7f7a3006ac90 msgr2=0x7f7a30153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 shutdown_connections 2024-05-08T04:43:38.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:38.755+0000 7f7a35000640 1 -- 172.21.15.164:0/2809710271 wait complete. 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T04:43:38.777 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:38.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.778 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-05-08T04:43:38.788 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42052sec preferred_lft 42052sec 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 02:42:79:c8:8f:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.789 INFO:teuthology.orchestra.run.smithi172.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-08T04:43:38.790 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-05-08T04:43:38.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.791 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T04:43:38.853 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-05-08T04:43:38.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.856 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-05-08T04:43:38.913 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-05-08T04:43:38.913 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T04:43:38.913 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T04:43:38.913 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T04:43:38.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:38.914 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-05-08T04:43:38.915 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-05-08T04:43:38.915 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T04:43:38.915 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T04:43:38.915 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T04:43:38.915 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-05-08T04:43:39.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:39.011 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:43:39.020 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:43:39.020 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:43:39.021 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:39.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:39.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-05-08T04:43:39.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:39.086 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:39.138 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T04:43:39.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T04:43:39.138 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:39.193 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T04:43:39.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:39.193 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:39.288 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:39.288 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:39.288 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:39.303 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:39.304 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:39.304 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:39.304 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:39.304 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T04:43:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T04:43:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-05-08T04:43:39.314 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-05-08T04:43:39.314 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T04:43:39.314 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-05-08T04:43:39.314 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-05-08T04:43:39.314 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=6150cf8f-12a4-433b-b39d-bc24e302fd2a" will pass to kernel 2024-05-08T04:43:39.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T04:43:39.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:39.317 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:39.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:39.338 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T04:43:39.391 INFO:teuthology.orchestra.run.smithi172.stdout:client7118 2024-05-08T04:43:39.392 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:39.392 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:39.441 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-08T04:43:39.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 Processor -- start 2024-05-08T04:43:39.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 -- start start 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f579406cce0 0x7f57940ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f57940cecc0 0x7f57940ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579415e870 con 0x7f579415ab10 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57940cf130 con 0x7f57940cecc0 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f579b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579406bb80 con 0x7f579406cce0 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5798a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f579415ab10 0x7f57940ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42154/0 (socket says 172.21.15.164:42154) 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5798a00640 1 -- 172.21.15.164:0/4094748131 learned_addr learned my addr 172.21.15.164:0/4094748131 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5799400640 1 --2- 172.21.15.164:0/4094748131 >> v2:172.21.15.172:3300/0 conn(0x7f57940cecc0 0x7f57940ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f578be00640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f579406cce0 0x7f57940ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5793e00640 1 -- 172.21.15.164:0/4094748131 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545506390 0 0) 0x7f579415e870 con 0x7f579415ab10 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5793e00640 1 -- 172.21.15.164:0/4094748131 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5778003530 con 0x7f579415ab10 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5799400640 1 -- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f579406cce0 msgr2=0x7f57940ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5799400640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f579406cce0 0x7f57940ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5799400640 1 -- 172.21.15.164:0/4094748131 >> v1:172.21.15.164:6789/0 conn(0x7f579415ab10 legacy=0x7f57940ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:39.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.591+0000 7f5799400640 1 -- 172.21.15.164:0/4094748131 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579415eb50 con 0x7f57940cecc0 2024-05-08T04:43:39.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f5799400640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57940cecc0 0x7f57940ca6e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd72a8190f6531 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:39.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f5793e00640 1 -- 172.21.15.164:0/4094748131 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0b2070 con 0x7f57940cecc0 2024-05-08T04:43:39.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f5793e00640 1 -- 172.21.15.164:0/4094748131 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0a5880 con 0x7f57940cecc0 2024-05-08T04:43:39.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f5793e00640 1 -- 172.21.15.164:0/4094748131 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0a5c50 con 0x7f57940cecc0 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 -- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57940cecc0 msgr2=0x7f57940ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57940cecc0 0x7f57940ca6e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c01c8b0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 -- 172.21.15.164:0/4094748131 shutdown_connections 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57940cecc0 0x7f57940ca6e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 --2- 172.21.15.164:0/4094748131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f579406cce0 0x7f57940ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 -- 172.21.15.164:0/4094748131 >> 172.21.15.164:0/4094748131 conn(0x7f57940cf770 msgr2=0x7f57940cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 -- 172.21.15.164:0/4094748131 shutdown_connections 2024-05-08T04:43:39.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.595+0000 7f579b400640 1 -- 172.21.15.164:0/4094748131 wait complete. 2024-05-08T04:43:39.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 Processor -- start 2024-05-08T04:43:39.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- start start 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f579406cce0 0x7f579414b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57940cecc0 0x7f5794152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940aedf0 con 0x7f579415ab10 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579415aef0 con 0x7f579406cce0 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940aef70 con 0x7f57940cecc0 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54998/0 (socket says 172.21.15.164:54998) 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 -- 172.21.15.164:0/1698817589 learned_addr learned my addr 172.21.15.164:0/1698817589 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f578be00640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57940cecc0 0x7f5794152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5798a00640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f579406cce0 0x7f579414b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 -- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57940cecc0 msgr2=0x7f5794152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:39.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57940cecc0 0x7f5794152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 -- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f579406cce0 msgr2=0x7f579414b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:39.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f579406cce0 0x7f579414b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:39.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 -- 172.21.15.164:0/1698817589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579414ff20 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5799400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f578c01af60 tx=0x7f578c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0b2050 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0be070 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5794150200 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0ad850 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.599+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941507c0 con 0x7f579415ab10 2024-05-08T04:43:39.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5794072d30 con 0x7f579415ab10 2024-05-08T04:43:39.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f578c0ad9f0 con 0x7f579415ab10 2024-05-08T04:43:39.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f5791600640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 0x7f5770069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:39.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f578c11eb30 con 0x7f579415ab10 2024-05-08T04:43:39.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f5798a00640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 0x7f5770069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:39.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.603+0000 7f5798a00640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 0x7f5770069f60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5784005890 tx=0x7f5784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:39.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.607+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f578c016560 con 0x7f579415ab10 2024-05-08T04:43:39.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:39.795+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57940c9b30 con 0x7f579415ab10 2024-05-08T04:43:40.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f5791600640 1 -- 172.21.15.164:0/1698817589 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 136+0+0 (secure 0 0 0) 0x7f578c0f16b0 con 0x7f579415ab10 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 msgr2=0x7f5770069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 0x7f5770069f60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5784005890 tx=0x7f5784009040 comp rx=0 tx=0).stop 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 msgr2=0x7f57940ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f578c01af60 tx=0x7f578c019650 comp rx=0 tx=0).stop 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 shutdown_connections 2024-05-08T04:43:40.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5770067aa0 0x7f5770069f60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57940cecc0 0x7f5794152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f579415ab10 0x7f57940ad890 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 --2- 172.21.15.164:0/1698817589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f579406cce0 0x7f579414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 >> 172.21.15.164:0/1698817589 conn(0x7f57940cf770 msgr2=0x7f57940ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 shutdown_connections 2024-05-08T04:43:40.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.559+0000 7f579b400640 1 -- 172.21.15.164:0/1698817589 wait complete. 2024-05-08T04:43:40.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-08T04:43:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 Processor -- start 2024-05-08T04:43:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 -- start start 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f70f00714f0 0x7f70f006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00b2dc0 con 0x7f70f006d0d0 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f006d880 con 0x7f70f0071fe0 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ef200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f00b4870 con 0x7f70f00714f0 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70e5400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f70f006d0d0 0x7f70f00718d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42156/0 (socket says 172.21.15.164:42156) 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70e5400640 1 -- 172.21.15.164:0/842732234 learned_addr learned my addr 172.21.15.164:0/842732234 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ede00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f70f00714f0 0x7f70f006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ede00640 1 -- 172.21.15.164:0/842732234 >> v1:172.21.15.172:6789/0 conn(0x7f70f0071fe0 legacy=0x7f70f0163a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ede00640 1 -- 172.21.15.164:0/842732234 >> v1:172.21.15.164:6789/0 conn(0x7f70f006d0d0 legacy=0x7f70f00718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ed400640 1 -- 172.21.15.164:0/842732234 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894295915 0 0) 0x7f70f00b2dc0 con 0x7f70f006d0d0 2024-05-08T04:43:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.719+0000 7f70ede00640 1 -- 172.21.15.164:0/842732234 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f006db60 con 0x7f70f00714f0 2024-05-08T04:43:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ede00640 1 --2- 172.21.15.164:0/842732234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f00714f0 0x7f70f006cb90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f70e00097f0 tx=0x7f70e001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288884df93807ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ed400640 1 -- 172.21.15.164:0/842732234 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e00b0070 con 0x7f70f00714f0 2024-05-08T04:43:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ed400640 1 -- 172.21.15.164:0/842732234 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00ab600 con 0x7f70f00714f0 2024-05-08T04:43:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ed400640 1 -- 172.21.15.164:0/842732234 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e00ab9d0 con 0x7f70f00714f0 2024-05-08T04:43:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 -- 172.21.15.164:0/842732234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f00714f0 msgr2=0x7f70f006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 --2- 172.21.15.164:0/842732234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f00714f0 0x7f70f006cb90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f70e00097f0 tx=0x7f70e001c3a0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 -- 172.21.15.164:0/842732234 shutdown_connections 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 --2- 172.21.15.164:0/842732234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f00714f0 0x7f70f006cb90 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 -- 172.21.15.164:0/842732234 >> 172.21.15.164:0/842732234 conn(0x7f70f00d03e0 msgr2=0x7f70f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 -- 172.21.15.164:0/842732234 shutdown_connections 2024-05-08T04:43:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.723+0000 7f70ef200640 1 -- 172.21.15.164:0/842732234 wait complete. 2024-05-08T04:43:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 Processor -- start 2024-05-08T04:43:40.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- start start 2024-05-08T04:43:40.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 0x7f70f00ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:40.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:40.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70f0149eb0 0x7f70f0155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:40.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00cc810 con 0x7f70f0071fe0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00cc690 con 0x7f70f00714f0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0168270 con 0x7f70f0149eb0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55014/0 (socket says 172.21.15.164:55014) 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 -- 172.21.15.164:0/3617435122 learned_addr learned my addr 172.21.15.164:0/3617435122 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ede00640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 0x7f70f00ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ee800640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70f0149eb0 0x7f70f0155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 -- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70f0149eb0 msgr2=0x7f70f0155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70f0149eb0 0x7f70f0155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 -- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 msgr2=0x7f70f00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 0x7f70f00ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 -- 172.21.15.164:0/3617435122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00cd600 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ede00640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 0x7f70f00ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e5400640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c550 tx=0x7f70d800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8018070 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8007580 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8004a30 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f00cd8e0 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f014a340 con 0x7f70f0071fe0 2024-05-08T04:43:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.727+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f004ef30 con 0x7f70f0071fe0 2024-05-08T04:43:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.731+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f70d8004bd0 con 0x7f70f0071fe0 2024-05-08T04:43:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.731+0000 7f70e6a00640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 0x7f70cc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.731+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f70d8082f10 con 0x7f70f0071fe0 2024-05-08T04:43:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.731+0000 7f70ede00640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 0x7f70cc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.731+0000 7f70ede00640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 0x7f70cc069f60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ad000 tx=0x7f70e00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.735+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70d8055720 con 0x7f70f0071fe0 2024-05-08T04:43:40.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f70f0164d60 con 0x7f70cc067aa0 2024-05-08T04:43:40.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70e6a00640 1 -- 172.21.15.164:0/3617435122 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f70f0164d60 con 0x7f70cc067aa0 2024-05-08T04:43:40.912 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-05-08T04:43:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-05-08T04:43:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephfs" 2024-05-08T04:43:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-08T04:43:40.913 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 msgr2=0x7f70cc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 0x7f70cc069f60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ad000 tx=0x7f70e00a6000 comp rx=0 tx=0).stop 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 msgr2=0x7f70f0156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c550 tx=0x7f70d800ca10 comp rx=0 tx=0).stop 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 shutdown_connections 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f70cc067aa0 0x7f70cc069f60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70f0149eb0 0x7f70f0155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70f0071fe0 0x7f70f0156030 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 --2- 172.21.15.164:0/3617435122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70f00714f0 0x7f70f00ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:40.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.911+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 >> 172.21.15.164:0/3617435122 conn(0x7f70f00d03e0 msgr2=0x7f70f015c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:40.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.915+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 shutdown_connections 2024-05-08T04:43:40.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:40.915+0000 7f70ef200640 1 -- 172.21.15.164:0/3617435122 wait complete. 2024-05-08T04:43:40.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T04:43:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 Processor -- start 2024-05-08T04:43:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 -- start start 2024-05-08T04:43:41.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05ec06cce0 0x7f05ec0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec15e870 con 0x7f05ec15ab10 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0cf130 con 0x7f05ec06cce0 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.071+0000 7f05f2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec06bb80 con 0x7f05ec0cecc0 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05eb400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f05ec15ab10 0x7f05ec0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42164/0 (socket says 172.21.15.164:42164) 2024-05-08T04:43:41.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05eb400640 1 -- 172.21.15.164:0/1787620535 learned_addr learned my addr 172.21.15.164:0/1787620535 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 --2- 172.21.15.164:0/1787620535 >> v2:172.21.15.172:3300/0 conn(0x7f05ec06cce0 0x7f05ec0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05ebe00640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465129881 0 0) 0x7f05ec15e870 con 0x7f05ec15ab10 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d0003530 con 0x7f05ec15ab10 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 -- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 msgr2=0x7f05ec0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec0ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100014203 0 0) 0x7f05d0003530 con 0x7f05ec15ab10 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 -- 172.21.15.164:0/1787620535 >> v1:172.21.15.164:6789/0 conn(0x7f05ec15ab10 legacy=0x7f05ec0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 -- 172.21.15.164:0/1787620535 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec15eb50 con 0x7f05ec06cce0 2024-05-08T04:43:41.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05ebe00640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:41.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.075+0000 7f05e2a00640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec06cce0 0x7f05ec0ce780 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ba60 tx=0x7f05d401e9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f897ea7418e6f2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40b4070 con 0x7f05ec06cce0 2024-05-08T04:43:41.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40a77e0 con 0x7f05ec06cce0 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05eaa00640 1 -- 172.21.15.164:0/1787620535 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc001680 con 0x7f05ec06cce0 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec06cce0 msgr2=0x7f05ec0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec06cce0 0x7f05ec0ce780 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ba60 tx=0x7f05d401e9d0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- 172.21.15.164:0/1787620535 shutdown_connections 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- 172.21.15.164:0/1787620535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec06cce0 0x7f05ec0ce780 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- 172.21.15.164:0/1787620535 >> 172.21.15.164:0/1787620535 conn(0x7f05ec0cf770 msgr2=0x7f05ec0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- 172.21.15.164:0/1787620535 shutdown_connections 2024-05-08T04:43:41.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- 172.21.15.164:0/1787620535 wait complete. 2024-05-08T04:43:41.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 Processor -- start 2024-05-08T04:43:41.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- start start 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec15ab10 0x7f05ec14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0aedf0 con 0x7f05ec06cce0 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec15aef0 con 0x7f05ec15ab10 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05f2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0aef70 con 0x7f05ec0cecc0 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.079+0000 7f05eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55022/0 (socket says 172.21.15.164:55022) 2024-05-08T04:43:41.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 -- 172.21.15.164:0/95169401 learned_addr learned my addr 172.21.15.164:0/95169401 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05ebe00640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec15ab10 0x7f05ec14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e2a00640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 -- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 msgr2=0x7f05ec14a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec14a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 -- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec15ab10 msgr2=0x7f05ec14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec15ab10 0x7f05ec14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 -- 172.21.15.164:0/95169401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0c6bd0 con 0x7f05ec06cce0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05eb400640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c700 tx=0x7f05dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc017070 con 0x7f05ec06cce0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc005ae0 con 0x7f05ec06cce0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec14ff30 con 0x7f05ec06cce0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007070 con 0x7f05ec06cce0 2024-05-08T04:43:41.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec1504f0 con 0x7f05ec06cce0 2024-05-08T04:43:41.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec14ab00 con 0x7f05ec06cce0 2024-05-08T04:43:41.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f05dc01a070 con 0x7f05ec06cce0 2024-05-08T04:43:41.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.083+0000 7f05e3e00640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 0x7f05c8069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.087+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f05dc0831f0 con 0x7f05ec06cce0 2024-05-08T04:43:41.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.087+0000 7f05e2a00640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 0x7f05c8069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.087+0000 7f05e2a00640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 0x7f05c8069f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f05d40b1000 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.091+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05dc0558f0 con 0x7f05ec06cce0 2024-05-08T04:43:41.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05ec072d30 con 0x7f05ec06cce0 2024-05-08T04:43:41.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05e3e00640 1 -- 172.21.15.164:0/95169401 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05dc0551f0 con 0x7f05ec06cce0 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 msgr2=0x7f05c8069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 0x7f05c8069f90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f05d40b1000 tx=0x7f05d4008040 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 msgr2=0x7f05ec153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c700 tx=0x7f05dc00cbc0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 shutdown_connections 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f05c8067ad0 0x7f05c8069f90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05ec0cecc0 0x7f05ec14a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05ec15ab10 0x7f05ec14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 --2- 172.21.15.164:0/95169401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05ec06cce0 0x7f05ec153470 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 >> 172.21.15.164:0/95169401 conn(0x7f05ec0cf770 msgr2=0x7f05ec0ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 shutdown_connections 2024-05-08T04:43:41.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.247+0000 7f05f2000640 1 -- 172.21.15.164:0/95169401 wait complete. 2024-05-08T04:43:41.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol45980 2024-05-08T04:43:41.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.403+0000 7fc875800640 1 Processor -- start 2024-05-08T04:43:41.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.403+0000 7fc875800640 1 -- start start 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc875800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 0x7fc87006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc875800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc8700cee90 0x7fc8700aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc875800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87015e870 con 0x7fc8700ae6e0 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc875800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87006bb80 con 0x7fc8700cee90 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc875800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87015eb50 con 0x7fc87015ab10 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc8700cee90 0x7fc8700aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc8700cee90 0x7fc8700aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55036/0 (socket says 172.21.15.164:55036) 2024-05-08T04:43:41.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 -- 172.21.15.164:0/3100820689 learned_addr learned my addr 172.21.15.164:0/3100820689 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86ea00640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 0x7fc87006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 -- 172.21.15.164:0/3100820689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 msgr2=0x7fc87006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 0x7fc87006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 -- 172.21.15.164:0/3100820689 >> v1:172.21.15.172:6789/0 conn(0x7fc8700ae6e0 legacy=0x7fc8700ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86f400640 1 -- 172.21.15.164:0/3100820689 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87015ee30 con 0x7fc8700cee90 2024-05-08T04:43:41.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.407+0000 7fc86ea00640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 0x7fc87006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:41.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc86f400640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc8700aeb40 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fc86800bb10 tx=0x7fc86801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10ec53d9d373544 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc86e000640 1 -- 172.21.15.164:0/3100820689 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680b4070 con 0x7fc8700cee90 2024-05-08T04:43:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc86e000640 1 -- 172.21.15.164:0/3100820689 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8680a7c00 con 0x7fc8700cee90 2024-05-08T04:43:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc86e000640 1 -- 172.21.15.164:0/3100820689 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680af440 con 0x7fc8700cee90 2024-05-08T04:43:41.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- 172.21.15.164:0/3100820689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 msgr2=0x7fc8700aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc8700aeb40 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fc86800bb10 tx=0x7fc86801eb70 comp rx=0 tx=0).stop 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- 172.21.15.164:0/3100820689 shutdown_connections 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc8700aeb40 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 --2- 172.21.15.164:0/3100820689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc87015ab10 0x7fc87006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- 172.21.15.164:0/3100820689 >> 172.21.15.164:0/3100820689 conn(0x7fc8700cf770 msgr2=0x7fc8700cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- 172.21.15.164:0/3100820689 shutdown_connections 2024-05-08T04:43:41.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- 172.21.15.164:0/3100820689 wait complete. 2024-05-08T04:43:41.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 Processor -- start 2024-05-08T04:43:41.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.411+0000 7fc875800640 1 -- start start 2024-05-08T04:43:41.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8700ae6e0 0x7fc87014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc87015ab10 0x7fc8700c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700cb070 con 0x7fc8700cee90 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700caef0 con 0x7fc87015ab10 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700cb1f0 con 0x7fc8700ae6e0 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55046/0 (socket says 172.21.15.164:55046) 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 -- 172.21.15.164:0/3203256775 learned_addr learned my addr 172.21.15.164:0/3203256775 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc86ea00640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8700ae6e0 0x7fc87014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc86f400640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc87015ab10 0x7fc8700c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 -- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8700ae6e0 msgr2=0x7fc87014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8700ae6e0 0x7fc87014c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 -- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc87015ab10 msgr2=0x7fc8700c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc87015ab10 0x7fc8700c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 -- 172.21.15.164:0/3203256775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87015c4c0 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc866000640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c450 tx=0x7fc85800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858016020 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8580052d0 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87015c7a0 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858007260 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87015cd60 con 0x7fc8700cee90 2024-05-08T04:43:41.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.415+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc870072d30 con 0x7fc8700cee90 2024-05-08T04:43:41.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.419+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7fc858007400 con 0x7fc8700cee90 2024-05-08T04:43:41.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.419+0000 7fc867400640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 0x7fc84c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.419+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc858083470 con 0x7fc8700cee90 2024-05-08T04:43:41.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.419+0000 7fc86ea00640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 0x7fc84c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.419+0000 7fc86ea00640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 0x7fc84c069f60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc860009bd0 tx=0x7fc86000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.423+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc858020020 con 0x7fc8700cee90 2024-05-08T04:43:41.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.607+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol45980", "target": ["mon-mgr", ""]}) v1 -- 0x7fc87004f010 con 0x7fc84c067aa0 2024-05-08T04:43:41.613 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:41.611+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:41.613 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:41.611+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:41.614 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:41.611+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:41.614 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:41.611+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:41.614 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:41.611+0000 7ffa56200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:41.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.643+0000 7fc867400640 1 -- 172.21.15.164:0/3203256775 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc87004f010 con 0x7fc84c067aa0 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 msgr2=0x7fc84c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 0x7fc84c069f60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc860009bd0 tx=0x7fc86000b040 comp rx=0 tx=0).stop 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 msgr2=0x7fc87015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c450 tx=0x7fc85800c910 comp rx=0 tx=0).stop 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 shutdown_connections 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc84c067aa0 0x7fc84c069f60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc87015ab10 0x7fc8700c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8700cee90 0x7fc87015d560 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 --2- 172.21.15.164:0/3203256775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8700ae6e0 0x7fc87014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 >> 172.21.15.164:0/3203256775 conn(0x7fc8700cf770 msgr2=0x7fc8700cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:41.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 shutdown_connections 2024-05-08T04:43:41.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.647+0000 7fc875800640 1 -- 172.21.15.164:0/3203256775 wait complete. 2024-05-08T04:43:41.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol45980 subvol_snap74790 2024-05-08T04:43:41.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.803+0000 7f0a54800640 1 Processor -- start 2024-05-08T04:43:41.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.803+0000 7f0a54800640 1 -- start start 2024-05-08T04:43:41.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.803+0000 7f0a54800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a4c15ba20 0x7f0a4c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a54800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a54800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0c6600 con 0x7f0a4c06de50 2024-05-08T04:43:41.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a54800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c15fbe0 con 0x7f0a4c15ba20 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a54800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c6040 con 0x7f0a4c0aedf0 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a4c15ba20 0x7f0a4c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0a4c06de50 0x7f0a4c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42168/0 (socket says 172.21.15.164:42168) 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51400640 1 -- 172.21.15.164:0/488579746 learned_addr learned my addr 172.21.15.164:0/488579746 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a52800640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 -- 172.21.15.164:0/488579746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 msgr2=0x7f0a4c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 -- 172.21.15.164:0/488579746 >> v1:172.21.15.164:6789/0 conn(0x7f0a4c06de50 legacy=0x7f0a4c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 -- 172.21.15.164:0/488579746 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0c6320 con 0x7f0a4c15ba20 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a50a00640 1 -- 172.21.15.164:0/488579746 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613617123 0 0) 0x7f0a4c0c6600 con 0x7f0a4c06de50 2024-05-08T04:43:41.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a52800640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:41.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a51e00640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c160070 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097c0 tx=0x7f0a4001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621993c972063ff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a50a00640 1 -- 172.21.15.164:0/488579746 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400ab610 con 0x7f0a4c15ba20 2024-05-08T04:43:41.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a50a00640 1 -- 172.21.15.164:0/488579746 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a400b0070 con 0x7f0a4c15ba20 2024-05-08T04:43:41.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.807+0000 7f0a50a00640 1 -- 172.21.15.164:0/488579746 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400ab9d0 con 0x7f0a4c15ba20 2024-05-08T04:43:41.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- 172.21.15.164:0/488579746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c15ba20 msgr2=0x7f0a4c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c160070 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097c0 tx=0x7f0a4001c470 comp rx=0 tx=0).stop 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- 172.21.15.164:0/488579746 shutdown_connections 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- 172.21.15.164:0/488579746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c160070 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- 172.21.15.164:0/488579746 >> 172.21.15.164:0/488579746 conn(0x7f0a4c0cf770 msgr2=0x7f0a4c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- 172.21.15.164:0/488579746 shutdown_connections 2024-05-08T04:43:41.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- 172.21.15.164:0/488579746 wait complete. 2024-05-08T04:43:41.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 Processor -- start 2024-05-08T04:43:41.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- start start 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0ce000 con 0x7f0a4c06de50 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0cde80 con 0x7f0a4c15ba20 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a54800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c168b70 con 0x7f0a4c0aedf0 2024-05-08T04:43:41.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55076/0 (socket says 172.21.15.164:55076) 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 -- 172.21.15.164:0/3422765952 learned_addr learned my addr 172.21.15.164:0/3422765952 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51400640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a52800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 -- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 msgr2=0x7f0a4c14cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c14cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 -- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 msgr2=0x7f0a4c0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c0b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a51e00640 1 -- 172.21.15.164:0/3422765952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c154b70 con 0x7f0a4c06de50 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.811+0000 7f0a52800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a51e00640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0a400ad040 tx=0x7f0a4001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400bf610 con 0x7f0a4c06de50 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a400bf7d0 con 0x7f0a4c06de50 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400bfbc0 con 0x7f0a4c06de50 2024-05-08T04:43:41.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c154e50 con 0x7f0a4c06de50 2024-05-08T04:43:41.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c155410 con 0x7f0a4c06de50 2024-05-08T04:43:41.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c072d30 con 0x7f0a4c06de50 2024-05-08T04:43:41.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f0a400bd020 con 0x7f0a4c06de50 2024-05-08T04:43:41.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 0x7f0a30069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:41.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0a4011e1a0 con 0x7f0a4c06de50 2024-05-08T04:43:41.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.815+0000 7f0a51400640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 0x7f0a30069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:41.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.819+0000 7f0a51400640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 0x7f0a30069f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c15b1e0 tx=0x7f0a3c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:41.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:41.819+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a400a8050 con 0x7f0a4c06de50 2024-05-08T04:43:42.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.003+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol45980", "snap_name": "subvol_snap74790", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a4c04f010 con 0x7f0a30067aa0 2024-05-08T04:43:42.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a46000640 1 -- 172.21.15.164:0/3422765952 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0a4c04f010 con 0x7f0a30067aa0 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 msgr2=0x7f0a30069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 0x7f0a30069f60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c15b1e0 tx=0x7f0a3c007b60 comp rx=0 tx=0).stop 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 msgr2=0x7f0a4c0cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0a400ad040 tx=0x7f0a4001acb0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 shutdown_connections 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0a30067aa0 0x7f0a30069f60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a4c0aedf0 0x7f0a4c14cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a4c15ba20 0x7f0a4c0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 --2- 172.21.15.164:0/3422765952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a4c06de50 0x7f0a4c0cb890 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 >> 172.21.15.164:0/3422765952 conn(0x7f0a4c0cf770 msgr2=0x7f0a4c153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 shutdown_connections 2024-05-08T04:43:42.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.019+0000 7f0a54800640 1 -- 172.21.15.164:0/3422765952 wait complete. 2024-05-08T04:43:42.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol45980 --retain-snapshots 2024-05-08T04:43:42.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 Processor -- start 2024-05-08T04:43:42.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 -- start start 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c7006efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c700d0c20 0x7f4c700ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c700cfe60 con 0x7f4c7006f520 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c700d0790 con 0x7f4c700d0c20 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c74a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700d0140 con 0x7f4c700cdb10 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c700d0c20 0x7f4c700ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6e800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4c7006f520 0x7f4c70161b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46138/0 (socket says 172.21.15.164:46138) 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6e800640 1 -- 172.21.15.164:0/3646986926 learned_addr learned my addr 172.21.15.164:0/3646986926 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6de00640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c7006efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6d400640 1 -- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 msgr2=0x7f4c7006efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6d400640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c7006efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6d400640 1 -- 172.21.15.164:0/3646986926 >> v1:172.21.15.172:6789/0 conn(0x7f4c7006f520 legacy=0x7f4c70161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6d400640 1 -- 172.21.15.164:0/3646986926 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700d0420 con 0x7f4c700d0c20 2024-05-08T04:43:42.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.179+0000 7f4c6ca00640 1 -- 172.21.15.164:0/3646986926 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544605259 0 0) 0x7f4c700cfe60 con 0x7f4c7006f520 2024-05-08T04:43:42.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c6d400640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c700d0c20 0x7f4c700ce220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097f0 tx=0x7f4c5c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671a0e6422e706e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c6ca00640 1 -- 172.21.15.164:0/3646986926 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0b0070 con 0x7f4c700d0c20 2024-05-08T04:43:42.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c6ca00640 1 -- 172.21.15.164:0/3646986926 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0ab600 con 0x7f4c700d0c20 2024-05-08T04:43:42.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c6ca00640 1 -- 172.21.15.164:0/3646986926 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ab9d0 con 0x7f4c700d0c20 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 -- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c700d0c20 msgr2=0x7f4c700ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c700d0c20 0x7f4c700ce220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097f0 tx=0x7f4c5c01c410 comp rx=0 tx=0).stop 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 -- 172.21.15.164:0/3646986926 shutdown_connections 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c700d0c20 0x7f4c700ce220 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 --2- 172.21.15.164:0/3646986926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c7006efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 -- 172.21.15.164:0/3646986926 >> 172.21.15.164:0/3646986926 conn(0x7f4c700b34f0 msgr2=0x7f4c700b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 -- 172.21.15.164:0/3646986926 shutdown_connections 2024-05-08T04:43:42.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.183+0000 7f4c74a00640 1 -- 172.21.15.164:0/3646986926 wait complete. 2024-05-08T04:43:42.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 Processor -- start 2024-05-08T04:43:42.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- start start 2024-05-08T04:43:42.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c7006f520 0x7f4c70151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c700c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7016b260 con 0x7f4c700d0c20 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7016b0e0 con 0x7f4c7006f520 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7016b3e0 con 0x7f4c700cdb10 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55086/0 (socket says 172.21.15.164:55086) 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 -- 172.21.15.164:0/1514514089 learned_addr learned my addr 172.21.15.164:0/1514514089 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6de00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c7006f520 0x7f4c70151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6d400640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c700c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 -- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 msgr2=0x7f4c700c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c700c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 -- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c7006f520 msgr2=0x7f4c70151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c7006f520 0x7f4c70151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 -- 172.21.15.164:0/1514514089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c701523e0 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6d400640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c700c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c6e800640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800c450 tx=0x7f4c6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680074e0 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680047f0 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700b5c10 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68007680 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.187+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c700b61d0 con 0x7f4c700d0c20 2024-05-08T04:43:42.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7009fae0 con 0x7f4c700d0c20 2024-05-08T04:43:42.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f4c68011070 con 0x7f4c700d0c20 2024-05-08T04:43:42.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c66000640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 0x7f4c50069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4c68083760 con 0x7f4c700d0c20 2024-05-08T04:43:42.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c6de00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 0x7f4c50069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.191+0000 7f4c6de00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 0x7f4c50069f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4c600113e0 tx=0x7f4c6000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.195+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c68055f70 con 0x7f4c700d0c20 2024-05-08T04:43:42.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.375+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol45980", "retain_snapshots": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4c7004f010 con 0x7f4c50067aa0 2024-05-08T04:43:42.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.395+0000 7f4c66000640 1 -- 172.21.15.164:0/1514514089 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4c7004f010 con 0x7f4c50067aa0 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 msgr2=0x7f4c50069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 0x7f4c50069f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4c600113e0 tx=0x7f4c6000b040 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 msgr2=0x7f4c70153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800c450 tx=0x7f4c6800c910 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 shutdown_connections 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4c50067aa0 0x7f4c50069f60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c700cdb10 0x7f4c700c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c700d0c20 0x7f4c70153640 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 --2- 172.21.15.164:0/1514514089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c7006f520 0x7f4c70151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 >> 172.21.15.164:0/1514514089 conn(0x7f4c700b34f0 msgr2=0x7f4c700b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:42.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 shutdown_connections 2024-05-08T04:43:42.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.399+0000 7f4c74a00640 1 -- 172.21.15.164:0/1514514089 wait complete. 2024-05-08T04:43:42.406 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.403+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.406 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.403+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.406 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.403+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.406 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.403+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.406 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.403+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.423 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.423+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.423 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.423+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.423 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.423+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.423 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.423+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.423 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.423+0000 7ffa58000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol45980 2024-05-08T04:43:42.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.431+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.431+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.431+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.431+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:42.431+0000 7ffa59400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T04:43:42.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.567+0000 7f475c400640 1 Processor -- start 2024-05-08T04:43:42.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.567+0000 7f475c400640 1 -- start start 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 0x7f4754168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475c400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540cfe60 con 0x7f47540cdb10 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540d0140 con 0x7f47540d0c20 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540d0790 con 0x7f47540ce930 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f4759a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f47540cdb10 0x7f475406efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42176/0 (socket says 172.21.15.164:42176) 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f4759a00640 1 -- 172.21.15.164:0/3832668221 learned_addr learned my addr 172.21.15.164:0/3832668221 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475a400640 1 --2- 172.21.15.164:0/3832668221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 0x7f4754168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f4753e00640 1 -- 172.21.15.164:0/3832668221 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980612917 0 0) 0x7f47540cfe60 con 0x7f47540cdb10 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f4753e00640 1 -- 172.21.15.164:0/3832668221 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c003530 con 0x7f47540cdb10 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475a400640 1 -- 172.21.15.164:0/3832668221 >> v1:172.21.15.172:6789/0 conn(0x7f47540d0c20 legacy=0x7f47540ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475a400640 1 -- 172.21.15.164:0/3832668221 >> v1:172.21.15.164:6789/0 conn(0x7f47540cdb10 legacy=0x7f475406efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.571+0000 7f475a400640 1 -- 172.21.15.164:0/3832668221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540d0420 con 0x7f47540ce930 2024-05-08T04:43:42.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475a400640 1 --2- 172.21.15.164:0/3832668221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 0x7f4754168ec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f474c009dc0 tx=0x7f474c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2d0f3231d0c1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f4753e00640 1 -- 172.21.15.164:0/3832668221 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c0b2070 con 0x7f47540ce930 2024-05-08T04:43:42.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f4753e00640 1 -- 172.21.15.164:0/3832668221 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c0a58f0 con 0x7f47540ce930 2024-05-08T04:43:42.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f4753e00640 1 -- 172.21.15.164:0/3832668221 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c0a5cc0 con 0x7f47540ce930 2024-05-08T04:43:42.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- 172.21.15.164:0/3832668221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 msgr2=0x7f4754168ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 --2- 172.21.15.164:0/3832668221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 0x7f4754168ec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f474c009dc0 tx=0x7f474c01c900 comp rx=0 tx=0).stop 2024-05-08T04:43:42.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- 172.21.15.164:0/3832668221 shutdown_connections 2024-05-08T04:43:42.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 --2- 172.21.15.164:0/3832668221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540ce930 0x7f4754168ec0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- 172.21.15.164:0/3832668221 >> 172.21.15.164:0/3832668221 conn(0x7f47540b34f0 msgr2=0x7f47540b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:42.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- 172.21.15.164:0/3832668221 shutdown_connections 2024-05-08T04:43:42.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- 172.21.15.164:0/3832668221 wait complete. 2024-05-08T04:43:42.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 Processor -- start 2024-05-08T04:43:42.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.575+0000 7f475c400640 1 -- start start 2024-05-08T04:43:42.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47540cdb10 0x7f47540c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 0x7f475406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475416bda0 con 0x7f47540ce930 2024-05-08T04:43:42.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475416bc20 con 0x7f47540cdb10 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475416bf20 con 0x7f47540d0c20 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55144/0 (socket says 172.21.15.164:55144) 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 -- 172.21.15.164:0/1516408125 learned_addr learned my addr 172.21.15.164:0/1516408125 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759a00640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47540cdb10 0x7f47540c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475a400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 0x7f475406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 -- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 msgr2=0x7f475406c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 0x7f475406c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 -- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47540cdb10 msgr2=0x7f47540c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:42.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47540cdb10 0x7f47540c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 -- 172.21.15.164:0/1516408125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540c5bd0 con 0x7f47540ce930 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475a400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 0x7f475406c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4759000640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f474000c340 tx=0x7f474000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400052d0 con 0x7f47540ce930 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740005470 con 0x7f47540ce930 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740007260 con 0x7f47540ce930 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47540c5eb0 con 0x7f47540ce930 2024-05-08T04:43:42.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47540c6470 con 0x7f47540ce930 2024-05-08T04:43:42.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475409fae0 con 0x7f47540ce930 2024-05-08T04:43:42.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.579+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 87747+0+0 (secure 0 0 0) 0x7f4740007400 con 0x7f47540ce930 2024-05-08T04:43:42.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.583+0000 7f4751600640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 0x7f4734069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:42.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.583+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4740083020 con 0x7f47540ce930 2024-05-08T04:43:42.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.583+0000 7f4759a00640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 0x7f4734069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:42.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.583+0000 7f4759a00640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 0x7f4734069f60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47480011f0 tx=0x7f4748007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:42.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.587+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47400551e0 con 0x7f47540ce930 2024-05-08T04:43:42.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:42.763+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol45980", "target": ["mon-mgr", ""]}) v1 -- 0x7f475404f010 con 0x7f4734067aa0 2024-05-08T04:43:43.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:43.583+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f4740051850 con 0x7f47540ce930 2024-05-08T04:43:46.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.551+0000 7f4751600640 1 -- 172.21.15.164:0/1516408125 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+306 (secure 0 0 0) 0x7f475404f010 con 0x7f4734067aa0 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "features": [ 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-clone", 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-autoprotect", 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-retention" 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flavor": 2, 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "mon_addrs": [ 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.164:6789", 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6789", 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6790" 2024-05-08T04:43:46.553 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:46.554 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "snapshot-retained", 2024-05-08T04:43:46.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "subvolume" 2024-05-08T04:43:46.554 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.551+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 msgr2=0x7f4734069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.551+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 0x7f4734069f60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47480011f0 tx=0x7f4748007d40 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.551+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 msgr2=0x7f475406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.551+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f474000c340 tx=0x7f474000c800 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 shutdown_connections 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4734067aa0 0x7f4734069f60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47540d0c20 0x7f475406c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47540ce930 0x7f475406bda0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 --2- 172.21.15.164:0/1516408125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47540cdb10 0x7f47540c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 >> 172.21.15.164:0/1516408125 conn(0x7f47540b34f0 msgr2=0x7f47540b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:46.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 shutdown_connections 2024-05-08T04:43:46.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.555+0000 7f475c400640 1 -- 172.21.15.164:0/1516408125 wait complete. 2024-05-08T04:43:46.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol45980 2024-05-08T04:43:46.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 Processor -- start 2024-05-08T04:43:46.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 -- start start 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f32e415ab10 0x7f32e406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40cee90 0x7f32e40aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e415e870 con 0x7f32e40ae6e0 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e406bb80 con 0x7f32e415ab10 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ec000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415eb50 con 0x7f32e40cee90 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f32e415ab10 0x7f32e406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f32e415ab10 0x7f32e406cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55156/0 (socket says 172.21.15.164:55156) 2024-05-08T04:43:46.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 -- 172.21.15.164:0/3261463575 learned_addr learned my addr 172.21.15.164:0/3261463575 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:46.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32ea000640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40cee90 0x7f32e40aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 -- 172.21.15.164:0/3261463575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40cee90 msgr2=0x7f32e40aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:46.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40cee90 0x7f32e40aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 -- 172.21.15.164:0/3261463575 >> v1:172.21.15.172:6789/0 conn(0x7f32e40ae6e0 legacy=0x7f32e40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.767+0000 7f32e9600640 1 -- 172.21.15.164:0/3261463575 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e415ee30 con 0x7f32e415ab10 2024-05-08T04:43:46.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32e9600640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e415ab10 0x7f32e406cce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c353ba9e4eda8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:46.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32e3e00640 1 -- 172.21.15.164:0/3261463575 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d80ab660 con 0x7f32e415ab10 2024-05-08T04:43:46.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32e3e00640 1 -- 172.21.15.164:0/3261463575 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d80b0070 con 0x7f32e415ab10 2024-05-08T04:43:46.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32e3e00640 1 -- 172.21.15.164:0/3261463575 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d80aba20 con 0x7f32e415ab10 2024-05-08T04:43:46.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 -- 172.21.15.164:0/3261463575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e415ab10 msgr2=0x7f32e406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e415ab10 0x7f32e406cce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d801c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 -- 172.21.15.164:0/3261463575 shutdown_connections 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40cee90 0x7f32e40aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 --2- 172.21.15.164:0/3261463575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e415ab10 0x7f32e406cce0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 -- 172.21.15.164:0/3261463575 >> 172.21.15.164:0/3261463575 conn(0x7f32e40cf770 msgr2=0x7f32e40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 -- 172.21.15.164:0/3261463575 shutdown_connections 2024-05-08T04:43:46.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.771+0000 7f32ec000640 1 -- 172.21.15.164:0/3261463575 wait complete. 2024-05-08T04:43:46.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 Processor -- start 2024-05-08T04:43:46.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- start start 2024-05-08T04:43:46.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40ae6e0 0x7f32e415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32e415ab10 0x7f32e4159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40cb070 con 0x7f32e40cee90 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40caef0 con 0x7f32e415ab10 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40cb1f0 con 0x7f32e40ae6e0 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55166/0 (socket says 172.21.15.164:55166) 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 -- 172.21.15.164:0/2466221667 learned_addr learned my addr 172.21.15.164:0/2466221667 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e9600640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40ae6e0 0x7f32e415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ea000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32e415ab10 0x7f32e4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 -- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40ae6e0 msgr2=0x7f32e415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40ae6e0 0x7f32e415bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 -- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32e415ab10 msgr2=0x7f32e4159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32e415ab10 0x7f32e4159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 -- 172.21.15.164:0/2466221667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e415c650 con 0x7f32e40cee90 2024-05-08T04:43:46.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e8c00640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f32d000c500 tx=0x7f32d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0016020 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d0005220 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e415c930 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0007030 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.775+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e415cef0 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e4072d30 con 0x7f32e40cee90 2024-05-08T04:43:46.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f32d00071d0 con 0x7f32e40cee90 2024-05-08T04:43:46.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32e1600640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 0x7f32c406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:46.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f32d0083670 con 0x7f32e40cee90 2024-05-08T04:43:46.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32e9600640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 0x7f32c406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:46.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.779+0000 7f32e9600640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 0x7f32c406a1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f32d80af000 tx=0x7f32d80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:46.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.783+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32d0020020 con 0x7f32e40cee90 2024-05-08T04:43:46.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.959+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol45980", "target": ["mon-mgr", ""]}) v1 -- 0x7f32e406d420 con 0x7f32c4067ce0 2024-05-08T04:43:46.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.987+0000 7f32e1600640 1 -- 172.21.15.164:0/2466221667 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32e406d420 con 0x7f32c4067ce0 2024-05-08T04:43:46.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 msgr2=0x7f32c406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 0x7f32c406a1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f32d80af000 tx=0x7f32d80a7000 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 msgr2=0x7f32e4159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f32d000c500 tx=0x7f32d000c9c0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 shutdown_connections 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f32c4067ce0 0x7f32c406a1a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32e415ab10 0x7f32e4159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32e40cee90 0x7f32e4159000 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 --2- 172.21.15.164:0/2466221667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32e40ae6e0 0x7f32e415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 >> 172.21.15.164:0/2466221667 conn(0x7f32e40cf770 msgr2=0x7f32e406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 shutdown_connections 2024-05-08T04:43:46.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:46.991+0000 7f32ec000640 1 -- 172.21.15.164:0/2466221667 wait complete. 2024-05-08T04:43:47.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol45980 2024-05-08T04:43:47.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 Processor -- start 2024-05-08T04:43:47.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 -- start start 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e240ae6e0 0x7f8e240aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e240c6600 con 0x7f8e2415ba20 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e240c6040 con 0x7f8e2415ce10 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.147+0000 7f8e29400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2415fbe0 con 0x7f8e240ae6e0 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e240ae6e0 0x7f8e240aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e240ae6e0 0x7f8e240aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55176/0 (socket says 172.21.15.164:55176) 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e22800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8e2415ba20 0x7f8e24160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42192/0 (socket says 172.21.15.164:42192) 2024-05-08T04:43:47.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 -- 172.21.15.164:0/2511933365 learned_addr learned my addr 172.21.15.164:0/2511933365 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:47.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 -- 172.21.15.164:0/2511933365 >> v1:172.21.15.172:6789/0 conn(0x7f8e2415ce10 legacy=0x7f8e2406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 -- 172.21.15.164:0/2511933365 >> v1:172.21.15.164:6789/0 conn(0x7f8e2415ba20 legacy=0x7f8e24160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 -- 172.21.15.164:0/2511933365 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240c6320 con 0x7f8e240ae6e0 2024-05-08T04:43:47.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e21e00640 1 -- 172.21.15.164:0/2511933365 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691232579 0 0) 0x7f8e240c6600 con 0x7f8e2415ba20 2024-05-08T04:43:47.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e23200640 1 --2- 172.21.15.164:0/2511933365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e240aeb20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009870 tx=0x7f8e1c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa149b2ad55fae16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e21e00640 1 -- 172.21.15.164:0/2511933365 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0b2070 con 0x7f8e240ae6e0 2024-05-08T04:43:47.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e21e00640 1 -- 172.21.15.164:0/2511933365 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c0a9070 con 0x7f8e240ae6e0 2024-05-08T04:43:47.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.151+0000 7f8e21e00640 1 -- 172.21.15.164:0/2511933365 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0ad880 con 0x7f8e240ae6e0 2024-05-08T04:43:47.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- 172.21.15.164:0/2511933365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 msgr2=0x7f8e240aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 --2- 172.21.15.164:0/2511933365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e240aeb20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009870 tx=0x7f8e1c01c560 comp rx=0 tx=0).stop 2024-05-08T04:43:47.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- 172.21.15.164:0/2511933365 shutdown_connections 2024-05-08T04:43:47.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 --2- 172.21.15.164:0/2511933365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e240aeb20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- 172.21.15.164:0/2511933365 >> 172.21.15.164:0/2511933365 conn(0x7f8e240cf770 msgr2=0x7f8e240cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:47.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- 172.21.15.164:0/2511933365 shutdown_connections 2024-05-08T04:43:47.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- 172.21.15.164:0/2511933365 wait complete. 2024-05-08T04:43:47.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 Processor -- start 2024-05-08T04:43:47.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- start start 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e2415ba20 0x7f8e240b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e2415ce10 0x7f8e240b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240ce000 con 0x7f8e240ae6e0 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240cde80 con 0x7f8e2415ce10 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e29400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24169b80 con 0x7f8e2415ba20 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55182/0 (socket says 172.21.15.164:55182) 2024-05-08T04:43:47.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 -- 172.21.15.164:0/3373391254 learned_addr learned my addr 172.21.15.164:0/3373391254 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e19e00640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e2415ba20 0x7f8e240b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e23200640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e2415ce10 0x7f8e240b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 -- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e2415ba20 msgr2=0x7f8e240b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e2415ba20 0x7f8e240b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 -- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e2415ce10 msgr2=0x7f8e240b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e2415ce10 0x7f8e240b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 -- 172.21.15.164:0/3373391254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240b4ad0 con 0x7f8e240ae6e0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e22800640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c8c0 tx=0x7f8e1400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.155+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14018070 con 0x7f8e240ae6e0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1400a640 con 0x7f8e240ae6e0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e240b4db0 con 0x7f8e240ae6e0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400aa70 con 0x7f8e240ae6e0 2024-05-08T04:43:47.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e240b5370 con 0x7f8e240ae6e0 2024-05-08T04:43:47.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24150b20 con 0x7f8e240ae6e0 2024-05-08T04:43:47.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f8e1401b070 con 0x7f8e240ae6e0 2024-05-08T04:43:47.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e1b400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 0x7f8e0006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8e140837e0 con 0x7f8e240ae6e0 2024-05-08T04:43:47.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.159+0000 7f8e19e00640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 0x7f8e0006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.163+0000 7f8e19e00640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 0x7f8e0006a1a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e24155370 tx=0x7f8e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.163+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e14055f90 con 0x7f8e240ae6e0 2024-05-08T04:43:47.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.339+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol45980", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e24072d30 con 0x7f8e00067ce0 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e1b400640 1 -- 172.21.15.164:0/3373391254 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+717 (secure 0 0 0) 0x7f8e24072d30 con 0x7f8e00067ce0 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "atime": "2024-05-08 04:43:46", 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_pcent": "undefined", 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_quota": "infinite", 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "bytes_used": 0, 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "created_at": "2024-05-08 04:43:46", 2024-05-08T04:43:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ctime": "2024-05-08 04:43:46", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "data_pool": "cephfs_data", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "features": [ 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-clone", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-autoprotect", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot-retention" 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flavor": 2, 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "gid": 0, 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": 16877, 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mon_addrs": [ 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.164:6789", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6789", 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "172.21.15.172:6790" 2024-05-08T04:43:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "mtime": "2024-05-08 04:43:46", 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/volumes/_nogroup/subvol45980/e8ebed11-6ba7-4660-9e76-bc58498ab6c8", 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_namespace": "", 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "complete", 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "subvolume", 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "uid": 0 2024-05-08T04:43:47.358 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 msgr2=0x7f8e0006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 0x7f8e0006a1a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e24155370 tx=0x7f8e0c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 msgr2=0x7f8e2414efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c8c0 tx=0x7f8e1400cd80 comp rx=0 tx=0).stop 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 shutdown_connections 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8e00067ce0 0x7f8e0006a1a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e2415ba20 0x7f8e240b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e2415ce10 0x7f8e240b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 --2- 172.21.15.164:0/3373391254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e240ae6e0 0x7f8e2414efe0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 >> 172.21.15.164:0/3373391254 conn(0x7f8e240cf770 msgr2=0x7f8e240cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 shutdown_connections 2024-05-08T04:43:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.355+0000 7f8e29400640 1 -- 172.21.15.164:0/3373391254 wait complete. 2024-05-08T04:43:47.378 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol45980 subvol_snap74790 2024-05-08T04:43:47.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 Processor -- start 2024-05-08T04:43:47.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 -- start start 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76a0165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a015e870 con 0x7f76a006cce0 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a015eb50 con 0x7f76a015ab10 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f76a5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006bb80 con 0x7f76a00cee90 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f7695400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f76a006cce0 0x7f76a00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42206/0 (socket says 172.21.15.164:42206) 2024-05-08T04:43:47.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f7695400640 1 -- 172.21.15.164:0/2960950501 learned_addr learned my addr 172.21.15.164:0/2960950501 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682868088 0 0) 0x7f76a015e870 con 0x7f76a006cce0 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7684003530 con 0x7f76a006cce0 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.523+0000 7f769e800640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76a0165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2482030070 0 0) 0x7f7684003530 con 0x7f76a006cce0 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 msgr2=0x7f76a0165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:47.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76a0165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 >> v1:172.21.15.172:6789/0 conn(0x7f76a015ab10 legacy=0x7f76a00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a015ee30 con 0x7f76a006cce0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f7688001680 con 0x7f76a006cce0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 >> v1:172.21.15.164:6789/0 conn(0x7f76a006cce0 legacy=0x7f76a00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7684003fc0 0x7f7684004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7684004d50 0x7f76840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006bb80 con 0x7f7684004d50 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768400bbc0 con 0x7f7684003fc0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768400bdc0 con 0x7f76a00cee90 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7684004d50 0x7f76840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f7695400640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7684003fc0 0x7f7684004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769de00640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 -- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 msgr2=0x7f76840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:47.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76a00cee90 0x7f76840039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 -- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7684003fc0 msgr2=0x7f7684004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:47.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7684003fc0 0x7f7684004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 -- 172.21.15.164:0/2960950501 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684003530 con 0x7f7684004d50 2024-05-08T04:43:47.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.527+0000 7f769e800640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7684004d50 0x7f76840051d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ca210 tx=0x7f769801ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836219162d33d4ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980b4070 con 0x7f7684004d50 2024-05-08T04:43:47.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980a7d30 con 0x7f7684004d50 2024-05-08T04:43:47.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769d400640 1 -- 172.21.15.164:0/2960950501 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698009c10 con 0x7f7684004d50 2024-05-08T04:43:47.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- 172.21.15.164:0/2960950501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7684004d50 msgr2=0x7f76840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 --2- 172.21.15.164:0/2960950501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7684004d50 0x7f76840051d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ca210 tx=0x7f769801ee30 comp rx=0 tx=0).stop 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769e800640 1 -- 172.21.15.164:0/2960950501 reap_dead start 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- 172.21.15.164:0/2960950501 shutdown_connections 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- 172.21.15.164:0/2960950501 >> 172.21.15.164:0/2960950501 conn(0x7f76a00cf770 msgr2=0x7f76a00cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- 172.21.15.164:0/2960950501 shutdown_connections 2024-05-08T04:43:47.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- 172.21.15.164:0/2960950501 wait complete. 2024-05-08T04:43:47.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 Processor -- start 2024-05-08T04:43:47.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- start start 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7684004d50 0x7f76a015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a0151c30 0x7f76a01f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76a01520b0 0x7f76a01f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a015aef0 con 0x7f76a01520b0 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a015f3f0 con 0x7f76a0151c30 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f76a5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00aedf0 con 0x7f7684004d50 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76a01520b0 0x7f76a01f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.531+0000 7f769e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76a01520b0 0x7f76a01f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55202/0 (socket says 172.21.15.164:55202) 2024-05-08T04:43:47.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 -- 172.21.15.164:0/96280483 learned_addr learned my addr 172.21.15.164:0/96280483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7695400640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a0151c30 0x7f76a01f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769de00640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7684004d50 0x7f76a015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 -- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7684004d50 msgr2=0x7f76a015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7684004d50 0x7f76a015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 -- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a0151c30 msgr2=0x7f76a01f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a0151c30 0x7f76a01f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 -- 172.21.15.164:0/96280483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00c6a90 con 0x7f76a01520b0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7695400640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76a0151c30 0x7f76a01f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f769e800640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76a01520b0 0x7f76a01f7ca0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f76980b2040 tx=0x7f7698009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980be070 con 0x7f76a01520b0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980a7b30 con 0x7f76a01520b0 2024-05-08T04:43:47.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980b4070 con 0x7f76a01520b0 2024-05-08T04:43:47.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f76a5000640 1 -- 172.21.15.164:0/96280483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a00c6d70 con 0x7f76a01520b0 2024-05-08T04:43:47.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f76a5000640 1 -- 172.21.15.164:0/96280483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00b2e00 con 0x7f76a01520b0 2024-05-08T04:43:47.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f76a5000640 1 -- 172.21.15.164:0/96280483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a00b2910 con 0x7f76a01520b0 2024-05-08T04:43:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f76980c1070 con 0x7f76a01520b0 2024-05-08T04:43:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f767c067d10 0x7f767c06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.535+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f769811de40 con 0x7f76a01520b0 2024-05-08T04:43:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.539+0000 7f769de00640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f767c067d10 0x7f767c06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:47.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.539+0000 7f769de00640 1 --2- 172.21.15.164:0/96280483 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f767c067d10 0x7f767c06a1d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7690007fd0 tx=0x7f7690009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:47.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.539+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7698010040 con 0x7f76a01520b0 2024-05-08T04:43:47.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.727+0000 7f76a5000640 1 -- 172.21.15.164:0/96280483 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol45980", "snap_name": "subvol_snap74790", "target": ["mon-mgr", ""]}) v1 -- 0x7f76a00c6840 con 0x7f767c067d10 2024-05-08T04:43:47.733 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-05-08T04:43:47.731+0000 7ffa56200640 -1 mgr.server reply reply (22) Invalid argument failed to open clone index 2024-05-08T04:43:47.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:47.731+0000 7f7696a00640 1 -- 172.21.15.164:0/96280483 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: -22 failed to open clone index) v1 ==== 34+0+0 (secure 0 0 0) 0x7f76a00c6840 con 0x7f767c067d10 2024-05-08T04:43:47.734 INFO:teuthology.orchestra.run.smithi164.stderr:Error EINVAL: failed to open clone index 2024-05-08T04:43:47.743 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T04:43:47.743 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T04:43:47.744 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:47.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T04:43:47.873 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T04:43:47.873 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:47.873 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:47.882 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:47.882 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:47.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:48.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.111+0000 7f793ca00640 1 Processor -- start 2024-05-08T04:43:48.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.111+0000 7f793ca00640 1 -- start start 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f793ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79380cfda0 0x7f7938160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f793ca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f793815be50 0x7f79380b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f793ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79380cd9f0 con 0x7f793815ba70 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f793ca00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793815fe70 con 0x7f793815be50 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f793ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79380cdcd0 con 0x7f79380cfda0 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f793815be50 0x7f79380b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f793815be50 0x7f79380b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55218/0 (socket says 172.21.15.164:55218) 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 -- 172.21.15.164:0/3235335984 learned_addr learned my addr 172.21.15.164:0/3235335984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:48.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f792fe00640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79380cfda0 0x7f7938160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 -- 172.21.15.164:0/3235335984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79380cfda0 msgr2=0x7f7938160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:48.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79380cfda0 0x7f7938160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 -- 172.21.15.164:0/3235335984 >> v1:172.21.15.172:6789/0 conn(0x7f793815ba70 legacy=0x7f79380cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:48.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.115+0000 7f7936800640 1 -- 172.21.15.164:0/3235335984 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793815f820 con 0x7f793815be50 2024-05-08T04:43:48.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f7936800640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f793815be50 0x7f79380b31a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f793001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816f9bea070e1ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f7935400640 1 -- 172.21.15.164:0/3235335984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300a5750 con 0x7f793815be50 2024-05-08T04:43:48.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f7935400640 1 -- 172.21.15.164:0/3235335984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79300a58f0 con 0x7f793815be50 2024-05-08T04:43:48.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f7935400640 1 -- 172.21.15.164:0/3235335984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300a5cc0 con 0x7f793815be50 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- 172.21.15.164:0/3235335984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f793815be50 msgr2=0x7f79380b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f793815be50 0x7f79380b31a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f793001c9c0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- 172.21.15.164:0/3235335984 shutdown_connections 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f793815be50 0x7f79380b31a0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 --2- 172.21.15.164:0/3235335984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79380cfda0 0x7f7938160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- 172.21.15.164:0/3235335984 >> 172.21.15.164:0/3235335984 conn(0x7f793806ac90 msgr2=0x7f793806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- 172.21.15.164:0/3235335984 shutdown_connections 2024-05-08T04:43:48.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- 172.21.15.164:0/3235335984 wait complete. 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 Processor -- start 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.119+0000 7f793ca00640 1 -- start start 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793815ba70 0x7f793815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 0x7f793815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938168d40 con 0x7f79380cfda0 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938168bc0 con 0x7f793815be50 2024-05-08T04:43:48.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7938168ec0 con 0x7f793815ba70 2024-05-08T04:43:48.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55234/0 (socket says 172.21.15.164:55234) 2024-05-08T04:43:48.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 -- 172.21.15.164:0/3386761199 learned_addr learned my addr 172.21.15.164:0/3386761199 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:48.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f792fe00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793815ba70 0x7f793815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7936800640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 0x7f793815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 -- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793815ba70 msgr2=0x7f793815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793815ba70 0x7f793815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 -- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 msgr2=0x7f793815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 0x7f793815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 -- 172.21.15.164:0/3386761199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793815d150 con 0x7f79380cfda0 2024-05-08T04:43:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7936800640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 0x7f793815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f7935e00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f792800c450 tx=0x7f792800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928012830 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928017070 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928012c30 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793815c390 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793815c950 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7938069d60 con 0x7f79380cfda0 2024-05-08T04:43:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.123+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f792801c460 con 0x7f79380cfda0 2024-05-08T04:43:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.127+0000 7f792e000640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 0x7f791406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.127+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7928083b90 con 0x7f79380cfda0 2024-05-08T04:43:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.127+0000 7f792fe00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 0x7f791406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.127+0000 7f792fe00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 0x7f791406a1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79380d01c0 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.131+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7928056340 con 0x7f79380cfda0 2024-05-08T04:43:48.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.287+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79380b2440 con 0x7f79380cfda0 2024-05-08T04:43:48.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.287+0000 7f792e000640 1 -- 172.21.15.164:0/3386761199 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f7928055c40 con 0x7f79380cfda0 2024-05-08T04:43:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:48.289 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 msgr2=0x7f791406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 0x7f791406a1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79380d01c0 tx=0x7f7920008040 comp rx=0 tx=0).stop 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 msgr2=0x7f79381528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f792800c450 tx=0x7f792800c910 comp rx=0 tx=0).stop 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 shutdown_connections 2024-05-08T04:43:48.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7914067ce0 0x7f791406a1a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793815ba70 0x7f793815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793815be50 0x7f793815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 --2- 172.21.15.164:0/3386761199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79380cfda0 0x7f79381528c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 >> 172.21.15.164:0/3386761199 conn(0x7f793806ac90 msgr2=0x7f793806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 shutdown_connections 2024-05-08T04:43:48.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.291+0000 7f793ca00640 1 -- 172.21.15.164:0/3386761199 wait complete. 2024-05-08T04:43:48.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:43:29.191800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:43:29.875561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:48.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:48.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 Processor -- start 2024-05-08T04:43:48.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 -- start start 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 0x7f078c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 0x7f078c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c15e870 con 0x7f078c15ab10 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c0cf130 con 0x7f078c06cce0 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0792800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c06bb80 con 0x7f078c0cecc0 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078b400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f078c15ab10 0x7f078c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:46208/0 (socket says 172.21.15.164:46208) 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078b400640 1 -- 172.21.15.164:0/3298513173 learned_addr learned my addr 172.21.15.164:0/3298513173 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078be00640 1 --2- 172.21.15.164:0/3298513173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 0x7f078c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0782a00640 1 --2- 172.21.15.164:0/3298513173 >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 0x7f078c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078aa00640 1 -- 172.21.15.164:0/3298513173 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34977669 0 0) 0x7f078c15e870 con 0x7f078c15ab10 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078aa00640 1 -- 172.21.15.164:0/3298513173 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0770003530 con 0x7f078c15ab10 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078be00640 1 -- 172.21.15.164:0/3298513173 >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 msgr2=0x7f078c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078be00640 1 --2- 172.21.15.164:0/3298513173 >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 0x7f078c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078be00640 1 -- 172.21.15.164:0/3298513173 >> v1:172.21.15.172:6789/0 conn(0x7f078c15ab10 legacy=0x7f078c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f078be00640 1 -- 172.21.15.164:0/3298513173 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c15eb50 con 0x7f078c0cecc0 2024-05-08T04:43:48.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.451+0000 7f0782a00640 1 --2- 172.21.15.164:0/3298513173 >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 0x7f078c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:48.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f078be00640 1 --2- 172.21.15.164:0/3298513173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 0x7f078c0ca6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0784009d90 tx=0x7f078401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d13368b3c8b03a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f078aa00640 1 -- 172.21.15.164:0/3298513173 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840b2070 con 0x7f078c0cecc0 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f078aa00640 1 -- 172.21.15.164:0/3298513173 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07840a5880 con 0x7f078c0cecc0 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f078aa00640 1 -- 172.21.15.164:0/3298513173 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840a5c50 con 0x7f078c0cecc0 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 -- 172.21.15.164:0/3298513173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 msgr2=0x7f078c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 --2- 172.21.15.164:0/3298513173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 0x7f078c0ca6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0784009d90 tx=0x7f078401c8b0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 -- 172.21.15.164:0/3298513173 shutdown_connections 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 --2- 172.21.15.164:0/3298513173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c0cecc0 0x7f078c0ca6e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 --2- 172.21.15.164:0/3298513173 >> v2:172.21.15.172:3300/0 conn(0x7f078c06cce0 0x7f078c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 -- 172.21.15.164:0/3298513173 >> 172.21.15.164:0/3298513173 conn(0x7f078c0cf770 msgr2=0x7f078c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 -- 172.21.15.164:0/3298513173 shutdown_connections 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.455+0000 7f0792800640 1 -- 172.21.15.164:0/3298513173 wait complete. 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 Processor -- start 2024-05-08T04:43:48.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- start start 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f078c06cce0 0x7f078c14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c15ab10 0x7f078c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0aedf0 con 0x7f078c0cecc0 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c15aef0 con 0x7f078c06cce0 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0aef70 con 0x7f078c15ab10 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55242/0 (socket says 172.21.15.164:55242) 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 -- 172.21.15.164:0/3350051132 learned_addr learned my addr 172.21.15.164:0/3350051132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f078b400640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f078c06cce0 0x7f078c14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f078be00640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c15ab10 0x7f078c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 -- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c15ab10 msgr2=0x7f078c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c15ab10 0x7f078c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 -- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f078c06cce0 msgr2=0x7f078c14dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f078c06cce0 0x7f078c14dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 -- 172.21.15.164:0/3350051132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c152480 con 0x7f078c0cecc0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0782a00640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f077400e9b0 tx=0x7f077400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774017070 con 0x7f078c0cecc0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400ca00 con 0x7f078c0cecc0 2024-05-08T04:43:48.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400cdd0 con 0x7f078c0cecc0 2024-05-08T04:43:48.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c158e90 con 0x7f078c0cecc0 2024-05-08T04:43:48.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.459+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c152720 con 0x7f078c0cecc0 2024-05-08T04:43:48.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c072d30 con 0x7f078c0cecc0 2024-05-08T04:43:48.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f077401a070 con 0x7f078c0cecc0 2024-05-08T04:43:48.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f0783e00640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 0x7f076806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0774083c90 con 0x7f078c0cecc0 2024-05-08T04:43:48.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f078b400640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 0x7f076806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.463+0000 7f078b400640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 0x7f076806a1a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f077c0011f0 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.467+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0774056440 con 0x7f078c0cecc0 2024-05-08T04:43:48.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f078c04f010 con 0x7f078c0cecc0 2024-05-08T04:43:48.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0783e00640 1 -- 172.21.15.164:0/3350051132 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0774055d40 con 0x7f078c0cecc0 2024-05-08T04:43:48.627 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 msgr2=0x7f076806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 0x7f076806a1a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f077c0011f0 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 msgr2=0x7f078c14bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f077400e9b0 tx=0x7f077400ee70 comp rx=0 tx=0).stop 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 shutdown_connections 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0768067ce0 0x7f076806a1a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f078c15ab10 0x7f078c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f078c0cecc0 0x7f078c14bfe0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 --2- 172.21.15.164:0/3350051132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f078c06cce0 0x7f078c14dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 >> 172.21.15.164:0/3350051132 conn(0x7f078c0cf770 msgr2=0x7f078c0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 shutdown_connections 2024-05-08T04:43:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.627+0000 7f0792800640 1 -- 172.21.15.164:0/3350051132 wait complete. 2024-05-08T04:43:48.648 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:48.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 Processor -- start 2024-05-08T04:43:48.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 -- start start 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47240cee90 0x7f4724165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472415e870 con 0x7f472415ab10 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472415eb50 con 0x7f472406cce0 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4729c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472406bb80 con 0x7f47240cee90 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47240cee90 0x7f4724165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47240cee90 0x7f4724165ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55250/0 (socket says 172.21.15.164:55250) 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4722a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f472415ab10 0x7f47240ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42220/0 (socket says 172.21.15.164:42220) 2024-05-08T04:43:48.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 -- 172.21.15.164:0/2943194304 learned_addr learned my addr 172.21.15.164:0/2943194304 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:48.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 -- 172.21.15.164:0/2943194304 >> v1:172.21.15.172:6789/0 conn(0x7f472406cce0 legacy=0x7f47240ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:48.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 -- 172.21.15.164:0/2943194304 >> v1:172.21.15.164:6789/0 conn(0x7f472415ab10 legacy=0x7f47240ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.787+0000 7f4723400640 1 -- 172.21.15.164:0/2943194304 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472415ee30 con 0x7f47240cee90 2024-05-08T04:43:48.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4723400640 1 --2- 172.21.15.164:0/2943194304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47240cee90 0x7f4724165ad0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f471c009870 tx=0x7f471c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e784aa9bcfba895 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4721600640 1 -- 172.21.15.164:0/2943194304 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0b1070 con 0x7f47240cee90 2024-05-08T04:43:48.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4721600640 1 -- 172.21.15.164:0/2943194304 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0ac410 con 0x7f47240cee90 2024-05-08T04:43:48.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4721600640 1 -- 172.21.15.164:0/2943194304 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0ac7e0 con 0x7f47240cee90 2024-05-08T04:43:48.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 -- 172.21.15.164:0/2943194304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47240cee90 msgr2=0x7f4724165ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 --2- 172.21.15.164:0/2943194304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47240cee90 0x7f4724165ad0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f471c009870 tx=0x7f471c01c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 -- 172.21.15.164:0/2943194304 shutdown_connections 2024-05-08T04:43:48.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 --2- 172.21.15.164:0/2943194304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47240cee90 0x7f4724165ad0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 -- 172.21.15.164:0/2943194304 >> 172.21.15.164:0/2943194304 conn(0x7f47240cf770 msgr2=0x7f47240cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 -- 172.21.15.164:0/2943194304 shutdown_connections 2024-05-08T04:43:48.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 -- 172.21.15.164:0/2943194304 wait complete. 2024-05-08T04:43:48.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.791+0000 7f4729c00640 1 Processor -- start 2024-05-08T04:43:48.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- start start 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472406cce0 0x7f4724150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 0x7f47241563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240aef70 con 0x7f472415ab10 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240aedf0 con 0x7f47240cee90 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4724168830 con 0x7f472406cce0 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55262/0 (socket says 172.21.15.164:55262) 2024-05-08T04:43:48.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 -- 172.21.15.164:0/48161067 learned_addr learned my addr 172.21.15.164:0/48161067 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4722a00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472406cce0 0x7f4724150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4722000640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 0x7f47241563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 -- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472406cce0 msgr2=0x7f4724150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472406cce0 0x7f4724150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 -- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 msgr2=0x7f47241563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 0x7f47241563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 -- 172.21.15.164:0/48161067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724158af0 con 0x7f472415ab10 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4722000640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 0x7f47241563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4723400640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f471c0190b0 tx=0x7f471c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0bc070 con 0x7f472415ab10 2024-05-08T04:43:48.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0b7440 con 0x7f472415ab10 2024-05-08T04:43:48.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0b1070 con 0x7f472415ab10 2024-05-08T04:43:48.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4724156a30 con 0x7f472415ab10 2024-05-08T04:43:48.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724159050 con 0x7f472415ab10 2024-05-08T04:43:48.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.795+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4724072d30 con 0x7f472415ab10 2024-05-08T04:43:48.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.799+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f471c0b63e0 con 0x7f472415ab10 2024-05-08T04:43:48.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.799+0000 7f471aa00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 0x7f470406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:48.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.799+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f471c11d4f0 con 0x7f472415ab10 2024-05-08T04:43:48.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.799+0000 7f4722a00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 0x7f470406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:48.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.799+0000 7f4722a00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 0x7f470406a1a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47140012f0 tx=0x7f4714001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:48.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.803+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4724158af0 con 0x7f472415ab10 2024-05-08T04:43:48.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.955+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f472404f010 con 0x7f472415ab10 2024-05-08T04:43:48.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.955+0000 7f471aa00640 1 -- 172.21.15.164:0/48161067 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f471c016150 con 0x7f472415ab10 2024-05-08T04:43:48.958 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 msgr2=0x7f470406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 0x7f470406a1a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47140012f0 tx=0x7f4714001700 comp rx=0 tx=0).stop 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 msgr2=0x7f472414b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f471c0190b0 tx=0x7f471c019240 comp rx=0 tx=0).stop 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 shutdown_connections 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4704067ce0 0x7f470406a1a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415ab10 0x7f472414b1d0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47240cee90 0x7f47241563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 --2- 172.21.15.164:0/48161067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472406cce0 0x7f4724150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:48.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 >> 172.21.15.164:0/48161067 conn(0x7f47240cf770 msgr2=0x7f47240ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:48.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 shutdown_connections 2024-05-08T04:43:48.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:48.959+0000 7f4729c00640 1 -- 172.21.15.164:0/48161067 wait complete. 2024-05-08T04:43:48.979 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:49.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 Processor -- start 2024-05-08T04:43:49.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 -- start start 2024-05-08T04:43:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015ab10 0x7fe94006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9400ae6e0 0x7fe9400aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9400ce780 0x7fe9400cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9400cf130 con 0x7fe9400ce780 2024-05-08T04:43:49.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94006bb80 con 0x7fe9400ae6e0 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.115+0000 7fe948000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015e870 con 0x7fe94015ab10 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9400ce780 0x7fe9400cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9400ce780 0x7fe9400cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55264/0 (socket says 172.21.15.164:55264) 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 -- 172.21.15.164:0/3577729034 learned_addr learned my addr 172.21.15.164:0/3577729034 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe945600640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015ab10 0x7fe94006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe944c00640 1 --2- 172.21.15.164:0/3577729034 >> v2:172.21.15.172:3300/0 conn(0x7fe9400ae6e0 0x7fe9400aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 -- 172.21.15.164:0/3577729034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015ab10 msgr2=0x7fe94006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015ab10 0x7fe94006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 -- 172.21.15.164:0/3577729034 >> v2:172.21.15.172:3300/0 conn(0x7fe9400ae6e0 msgr2=0x7fe9400aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 --2- 172.21.15.164:0/3577729034 >> v2:172.21.15.172:3300/0 conn(0x7fe9400ae6e0 0x7fe9400aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 -- 172.21.15.164:0/3577729034 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94015eb50 con 0x7fe9400ce780 2024-05-08T04:43:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe946000640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400ce780 0x7fe9400cebf0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe9400cd2f0 tx=0x7fe93801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394401bf5a8e3de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe93fe00640 1 -- 172.21.15.164:0/3577729034 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380a57b0 con 0x7fe9400ce780 2024-05-08T04:43:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe93fe00640 1 -- 172.21.15.164:0/3577729034 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9380a5950 con 0x7fe9400ce780 2024-05-08T04:43:49.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.119+0000 7fe93fe00640 1 -- 172.21.15.164:0/3577729034 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380a5d20 con 0x7fe9400ce780 2024-05-08T04:43:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- 172.21.15.164:0/3577729034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400ce780 msgr2=0x7fe9400cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400ce780 0x7fe9400cebf0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe9400cd2f0 tx=0x7fe93801c930 comp rx=0 tx=0).stop 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- 172.21.15.164:0/3577729034 shutdown_connections 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9400ce780 0x7fe9400cebf0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- 172.21.15.164:0/3577729034 >> v2:172.21.15.172:3300/0 conn(0x7fe9400ae6e0 0x7fe9400aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- 172.21.15.164:0/3577729034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe94015ab10 0x7fe94006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- 172.21.15.164:0/3577729034 >> 172.21.15.164:0/3577729034 conn(0x7fe9400cf770 msgr2=0x7fe9400cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- 172.21.15.164:0/3577729034 shutdown_connections 2024-05-08T04:43:49.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- 172.21.15.164:0/3577729034 wait complete. 2024-05-08T04:43:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 Processor -- start 2024-05-08T04:43:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- start start 2024-05-08T04:43:49.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9400ae6e0 0x7fe9400adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9400ce780 0x7fe940154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c5b70 con 0x7fe94015ab10 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94015aef0 con 0x7fe9400ae6e0 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe948000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c5cf0 con 0x7fe9400ce780 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe945600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9400ae6e0 0x7fe9400adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55274/0 (socket says 172.21.15.164:55274) 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 -- 172.21.15.164:0/847893382 learned_addr learned my addr 172.21.15.164:0/847893382 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe944c00640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9400ce780 0x7fe940154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 -- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9400ce780 msgr2=0x7fe940154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9400ce780 0x7fe940154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 -- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9400ae6e0 msgr2=0x7fe9400adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9400ae6e0 0x7fe9400adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 -- 172.21.15.164:0/847893382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400ae160 con 0x7fe94015ab10 2024-05-08T04:43:49.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.123+0000 7fe946000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe9380af040 tx=0x7fe9380196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380bc020 con 0x7fe94015ab10 2024-05-08T04:43:49.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9380b8670 con 0x7fe94015ab10 2024-05-08T04:43:49.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380b8a60 con 0x7fe94015ab10 2024-05-08T04:43:49.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9400704c0 con 0x7fe94015ab10 2024-05-08T04:43:49.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe940070a80 con 0x7fe94015ab10 2024-05-08T04:43:49.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe940072d30 con 0x7fe94015ab10 2024-05-08T04:43:49.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fe9380bf020 con 0x7fe94015ab10 2024-05-08T04:43:49.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 0x7fe92006a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe93811edd0 con 0x7fe94015ab10 2024-05-08T04:43:49.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe945600640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 0x7fe92006a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.127+0000 7fe945600640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 0x7fe92006a1d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caef0 tx=0x7fe934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.131+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9380c6030 con 0x7fe94015ab10 2024-05-08T04:43:49.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe940070c50 con 0x7fe94015ab10 2024-05-08T04:43:49.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe93d600640 1 -- 172.21.15.164:0/847893382 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9380165e0 con 0x7fe94015ab10 2024-05-08T04:43:49.288 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 msgr2=0x7fe92006a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 0x7fe92006a1d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe9400caef0 tx=0x7fe934008040 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 msgr2=0x7fe940154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe9380af040 tx=0x7fe9380196d0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 shutdown_connections 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe920067d10 0x7fe92006a1d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9400ce780 0x7fe940154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe94015ab10 0x7fe940154550 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 --2- 172.21.15.164:0/847893382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9400ae6e0 0x7fe9400adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 >> 172.21.15.164:0/847893382 conn(0x7fe9400cf770 msgr2=0x7fe9400cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:49.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 shutdown_connections 2024-05-08T04:43:49.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.287+0000 7fe948000640 1 -- 172.21.15.164:0/847893382 wait complete. 2024-05-08T04:43:49.310 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:43:49.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 Processor -- start 2024-05-08T04:43:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 -- start start 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc15bc50 0x7f7bcc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7bcc0ccfe0 0x7f7bcc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b42a0 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cd4e0 con 0x7f7bcc0ccfe0 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bd0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cd7c0 con 0x7f7bcc15bc50 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42050/0 (socket says 172.21.15.164:42050) 2024-05-08T04:43:49.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 -- 172.21.15.164:0/4047892588 learned_addr learned my addr 172.21.15.164:0/4047892588 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bc9e00640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc15bc50 0x7f7bcc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bc9400640 1 --2- 172.21.15.164:0/4047892588 >> v2:172.21.15.172:3300/0 conn(0x7f7bcc0ccfe0 0x7f7bcc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 -- 172.21.15.164:0/4047892588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc15bc50 msgr2=0x7f7bcc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc15bc50 0x7f7bcc0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 -- 172.21.15.164:0/4047892588 >> v2:172.21.15.172:3300/0 conn(0x7f7bcc0ccfe0 msgr2=0x7f7bcc0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 --2- 172.21.15.164:0/4047892588 >> v2:172.21.15.172:3300/0 conn(0x7f7bcc0ccfe0 0x7f7bcc0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.447+0000 7f7bca800640 1 -- 172.21.15.164:0/4047892588 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0cf590 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bca800640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc071020 tx=0x7f7bc401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b4a406805ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bc8a00640 1 -- 172.21.15.164:0/4047892588 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40ab620 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bc8a00640 1 -- 172.21.15.164:0/4047892588 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc40b0070 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bc8a00640 1 -- 172.21.15.164:0/4047892588 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40ab9e0 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 -- 172.21.15.164:0/4047892588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 msgr2=0x7f7bcc0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc071020 tx=0x7f7bc401c460 comp rx=0 tx=0).stop 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 -- 172.21.15.164:0/4047892588 shutdown_connections 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc0d05b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/4047892588 >> v2:172.21.15.172:3300/0 conn(0x7f7bcc0ccfe0 0x7f7bcc0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/4047892588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc15bc50 0x7f7bcc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 -- 172.21.15.164:0/4047892588 >> 172.21.15.164:0/4047892588 conn(0x7f7bcc0b34f0 msgr2=0x7f7bcc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:49.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 -- 172.21.15.164:0/4047892588 shutdown_connections 2024-05-08T04:43:49.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 -- 172.21.15.164:0/4047892588 wait complete. 2024-05-08T04:43:49.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.451+0000 7f7bd0a00640 1 Processor -- start 2024-05-08T04:43:49.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- start start 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc0ccfe0 0x7f7bcc15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bcc15bc50 0x7f7bcc15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cea90 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0ce910 con 0x7f7bcc15bc50 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cec10 con 0x7f7bcc0ccfe0 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42056/0 (socket says 172.21.15.164:42056) 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 -- 172.21.15.164:0/1932329247 learned_addr learned my addr 172.21.15.164:0/1932329247 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bca800640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bcc15bc50 0x7f7bcc15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9400640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc0ccfe0 0x7f7bcc15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 -- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc0ccfe0 msgr2=0x7f7bcc15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc0ccfe0 0x7f7bcc15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 -- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bcc15bc50 msgr2=0x7f7bcc15c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bcc15bc50 0x7f7bcc15c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 -- 172.21.15.164:0/1932329247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0caab0 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc9e00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc007e70 tx=0x7f7bbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc014620 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc019070 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc014a00 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc0cad90 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc0cb350 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.455+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc069d60 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.459+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f7bbc014ba0 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.459+0000 7f7bc2000640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 0x7f7bac06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.459+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7bbc084590 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.459+0000 7f7bc9400640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 0x7f7bac06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.459+0000 7f7bc9400640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 0x7f7bac06a1a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc13e9f0 tx=0x7f7bb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.463+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7bbc011260 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7bcc0b7940 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bc2000640 1 -- 172.21.15.164:0/1932329247 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7bbc056d40 con 0x7f7bcc0b4d50 2024-05-08T04:43:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:43:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 msgr2=0x7f7bac06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 0x7f7bac06a1a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc13e9f0 tx=0x7f7bb8008040 comp rx=0 tx=0).stop 2024-05-08T04:43:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 msgr2=0x7f7bcc1519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc007e70 tx=0x7f7bbc00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 shutdown_connections 2024-05-08T04:43:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7bac067ce0 0x7f7bac06a1a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bcc0ccfe0 0x7f7bcc15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bcc15bc50 0x7f7bcc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 --2- 172.21.15.164:0/1932329247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc0b4d50 0x7f7bcc1519c0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 >> 172.21.15.164:0/1932329247 conn(0x7f7bcc0b34f0 msgr2=0x7f7bcc159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 shutdown_connections 2024-05-08T04:43:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.635+0000 7f7bd0a00640 1 -- 172.21.15.164:0/1932329247 wait complete. 2024-05-08T04:43:49.657 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T04:43:49.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:49.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T04:43:49.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:43:49.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:49.684 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T04:43:49.739 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:49.739 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:49.749 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:49.749 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:49.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:49.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 Processor -- start 2024-05-08T04:43:49.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 -- start start 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7930161950 0x7f79300cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 0x7f793015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7930161fe0 con 0x7f7930144200 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7930162880 con 0x7f7930161950 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7938600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79301622c0 con 0x7f793015ea20 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7930161950 0x7f79300cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7930161950 0x7f79300cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42062/0 (socket says 172.21.15.164:42062) 2024-05-08T04:43:49.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 -- 172.21.15.164:0/1493106654 learned_addr learned my addr 172.21.15.164:0/1493106654 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7936600640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 0x7f793015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 -- 172.21.15.164:0/1493106654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 msgr2=0x7f793015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 0x7f793015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 -- 172.21.15.164:0/1493106654 >> v1:172.21.15.164:6789/0 conn(0x7f7930144200 legacy=0x7f79300cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7935200640 1 -- 172.21.15.164:0/1493106654 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79301625a0 con 0x7f7930161950 2024-05-08T04:43:49.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.939+0000 7f7936600640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 0x7f793015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:49.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7935200640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930161950 0x7f79300cfd60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f791c009870 tx=0x7f791c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698e9fc67efe7766 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f792fe00640 1 -- 172.21.15.164:0/1493106654 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0b0070 con 0x7f7930161950 2024-05-08T04:43:49.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f792fe00640 1 -- 172.21.15.164:0/1493106654 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c0ab600 con 0x7f7930161950 2024-05-08T04:43:49.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f792fe00640 1 -- 172.21.15.164:0/1493106654 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0ab9d0 con 0x7f7930161950 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 -- 172.21.15.164:0/1493106654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930161950 msgr2=0x7f79300cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930161950 0x7f79300cfd60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f791c009870 tx=0x7f791c01c4d0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 -- 172.21.15.164:0/1493106654 shutdown_connections 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f793015ea20 0x7f793015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 --2- 172.21.15.164:0/1493106654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930161950 0x7f79300cfd60 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 -- 172.21.15.164:0/1493106654 >> 172.21.15.164:0/1493106654 conn(0x7f793006ac90 msgr2=0x7f793006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 -- 172.21.15.164:0/1493106654 shutdown_connections 2024-05-08T04:43:49.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.943+0000 7f7938600640 1 -- 172.21.15.164:0/1493106654 wait complete. 2024-05-08T04:43:49.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 Processor -- start 2024-05-08T04:43:49.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- start start 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793015ea20 0x7f793013d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7930161950 0x7f7930142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300cf370 con 0x7f7930144200 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300cf1f0 con 0x7f793015ea20 2024-05-08T04:43:49.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793016acf0 con 0x7f7930161950 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42064/0 (socket says 172.21.15.164:42064) 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 -- 172.21.15.164:0/1838261807 learned_addr learned my addr 172.21.15.164:0/1838261807 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7936600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7930161950 0x7f7930142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935200640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793015ea20 0x7f793013d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 -- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7930161950 msgr2=0x7f7930142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7930161950 0x7f7930142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 -- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793015ea20 msgr2=0x7f793013d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793015ea20 0x7f793013d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 -- 172.21.15.164:0/1838261807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793013bfd0 con 0x7f7930144200 2024-05-08T04:43:49.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7935c00640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7924007fd0 tx=0x7f792400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924017070 con 0x7f7930144200 2024-05-08T04:43:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240052d0 con 0x7f7930144200 2024-05-08T04:43:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793013dae0 con 0x7f7930144200 2024-05-08T04:43:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924003840 con 0x7f7930144200 2024-05-08T04:43:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.947+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793013c270 con 0x7f7930144200 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7930069d60 con 0x7f7930144200 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f792401a070 con 0x7f7930144200 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f792d600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 0x7f791006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f79240830f0 con 0x7f7930144200 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f7935200640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 0x7f791006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.951+0000 7f7935200640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 0x7f791006a1a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f791c0ad000 tx=0x7f791c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:49.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:49.955+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7924005480 con 0x7f7930144200 2024-05-08T04:43:50.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.111+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79300af110 con 0x7f7930144200 2024-05-08T04:43:50.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.111+0000 7f792d600640 1 -- 172.21.15.164:0/1838261807 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f79240558a0 con 0x7f7930144200 2024-05-08T04:43:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:50.114 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 msgr2=0x7f791006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 0x7f791006a1a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f791c0ad000 tx=0x7f791c008040 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 msgr2=0x7f793013ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7924007fd0 tx=0x7f792400c890 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 shutdown_connections 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7910067ce0 0x7f791006a1a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7930161950 0x7f7930142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f793015ea20 0x7f793013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 --2- 172.21.15.164:0/1838261807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7930144200 0x7f793013ca50 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 >> 172.21.15.164:0/1838261807 conn(0x7f793006ac90 msgr2=0x7f79300adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 shutdown_connections 2024-05-08T04:43:50.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.115+0000 7f7938600640 1 -- 172.21.15.164:0/1838261807 wait complete. 2024-05-08T04:43:50.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:43:29.191800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:43:29.875561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:50.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:50.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.271+0000 7f86d3a00640 1 Processor -- start 2024-05-08T04:43:50.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.271+0000 7f86d3a00640 1 -- start start 2024-05-08T04:43:50.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d3a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f86cc15ba20 0x7f86cc160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86cc06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d3a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86cc0c6600 con 0x7f86cc06de50 2024-05-08T04:43:50.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d3a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15fbe0 con 0x7f86cc15ba20 2024-05-08T04:43:50.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc0c6040 con 0x7f86cc0aedf0 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cbe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f86cc06de50 0x7f86cc0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58788/0 (socket says 172.21.15.164:58788) 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cbe00640 1 -- 172.21.15.164:0/1486377984 learned_addr learned my addr 172.21.15.164:0/1486377984 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1000640 1 --2- 172.21.15.164:0/1486377984 >> v2:172.21.15.172:3300/0 conn(0x7f86cc15ba20 0x7f86cc160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86cc06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446875289 0 0) 0x7f86cc0c6600 con 0x7f86cc06de50 2024-05-08T04:43:50.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b0003530 con 0x7f86cc06de50 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848440035 0 0) 0x7f86b0003530 con 0x7f86cc06de50 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 msgr2=0x7f86cc06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86cc06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 >> v2:172.21.15.172:3300/0 conn(0x7f86cc15ba20 msgr2=0x7f86cc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 --2- 172.21.15.164:0/1486377984 >> v2:172.21.15.172:3300/0 conn(0x7f86cc15ba20 0x7f86cc160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0c6320 con 0x7f86cc06de50 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f86b8001680 con 0x7f86cc06de50 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 >> v1:172.21.15.164:6789/0 conn(0x7f86cc06de50 legacy=0x7f86cc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86b00039c0 0x7f86b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86b0004d50 0x7f86b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc15fbe0 con 0x7f86b0004d50 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc0c6040 con 0x7f86b00039c0 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000b9d0 con 0x7f86cc0aedf0 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86b0004d50 0x7f86b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86cbe00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1000640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86b00039c0 0x7f86b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 -- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 msgr2=0x7f86b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:50.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0aedf0 0x7f86b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 -- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86b00039c0 msgr2=0x7f86b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:50.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86b00039c0 0x7f86b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.275+0000 7f86d1a00640 1 -- 172.21.15.164:0/1486377984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0003530 con 0x7f86b0004d50 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d1a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86b0004d50 0x7f86b00051d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009ae0 tx=0x7f86c401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e001d75fb43205 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40b2070 con 0x7f86b0004d50 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c40a5660 con 0x7f86b0004d50 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86cb400640 1 -- 172.21.15.164:0/1486377984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40096b0 con 0x7f86b0004d50 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- 172.21.15.164:0/1486377984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86b0004d50 msgr2=0x7f86b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 --2- 172.21.15.164:0/1486377984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86b0004d50 0x7f86b00051d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009ae0 tx=0x7f86c401ca70 comp rx=0 tx=0).stop 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d1a00640 1 -- 172.21.15.164:0/1486377984 reap_dead start 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- 172.21.15.164:0/1486377984 shutdown_connections 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- 172.21.15.164:0/1486377984 >> 172.21.15.164:0/1486377984 conn(0x7f86cc0cf770 msgr2=0x7f86cc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- 172.21.15.164:0/1486377984 shutdown_connections 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- 172.21.15.164:0/1486377984 wait complete. 2024-05-08T04:43:50.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 Processor -- start 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.279+0000 7f86d3a00640 1 -- start start 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86cc14c670 0x7f86cc156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 0x7f86cc154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1605b0 con 0x7f86cc1571d0 2024-05-08T04:43:50.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc0cf410 con 0x7f86cc14c670 2024-05-08T04:43:50.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc15ce10 con 0x7f86cc0ca970 2024-05-08T04:43:50.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42078/0 (socket says 172.21.15.164:42078) 2024-05-08T04:43:50.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 -- 172.21.15.164:0/509185503 learned_addr learned my addr 172.21.15.164:0/509185503 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:50.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d1000640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86cc14c670 0x7f86cc156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d1a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 0x7f86cc154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 -- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 msgr2=0x7f86cc154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 0x7f86cc154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 -- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86cc14c670 msgr2=0x7f86cc156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86cc14c670 0x7f86cc156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 -- 172.21.15.164:0/509185503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc157650 con 0x7f86cc1571d0 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d1a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 0x7f86cc154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86cbe00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f86b8007340 tx=0x7f86b8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b800f9c0 con 0x7f86cc1571d0 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b8018070 con 0x7f86cc1571d0 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc0cb1a0 con 0x7f86cc1571d0 2024-05-08T04:43:50.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b800fdc0 con 0x7f86cc1571d0 2024-05-08T04:43:50.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc0cb760 con 0x7f86cc1571d0 2024-05-08T04:43:50.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc072d30 con 0x7f86cc1571d0 2024-05-08T04:43:50.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.283+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f86b801c410 con 0x7f86cc1571d0 2024-05-08T04:43:50.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.287+0000 7f86c8c00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 0x7f86ac06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.287+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f86b8083a20 con 0x7f86cc1571d0 2024-05-08T04:43:50.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.287+0000 7f86d1000640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 0x7f86ac06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.287+0000 7f86d1000640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 0x7f86ac06a1d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f86bc014450 tx=0x7f86bc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.291+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86b8056230 con 0x7f86cc1571d0 2024-05-08T04:43:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.443+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86cc0cea50 con 0x7f86cc1571d0 2024-05-08T04:43:50.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.443+0000 7f86c8c00640 1 -- 172.21.15.164:0/509185503 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86b8055b30 con 0x7f86cc1571d0 2024-05-08T04:43:50.446 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:50.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 msgr2=0x7f86ac06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 0x7f86ac06a1d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f86bc014450 tx=0x7f86bc00b040 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 msgr2=0x7f86cc0ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f86b8007340 tx=0x7f86b8004f90 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 shutdown_connections 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86ac067d10 0x7f86ac06a1d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86cc0ca970 0x7f86cc154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86cc1571d0 0x7f86cc0ca430 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 --2- 172.21.15.164:0/509185503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86cc14c670 0x7f86cc156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 >> 172.21.15.164:0/509185503 conn(0x7f86cc0cf770 msgr2=0x7f86cc0cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 shutdown_connections 2024-05-08T04:43:50.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.447+0000 7f86d3a00640 1 -- 172.21.15.164:0/509185503 wait complete. 2024-05-08T04:43:50.468 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:50.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 Processor -- start 2024-05-08T04:43:50.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 -- start start 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f09a415d9b0 0x7f09a40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a4070c40 0x7f09a40b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a4071e40 con 0x7f09a40729e0 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40af2b0 con 0x7f09a415d9b0 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40adaa0 con 0x7f09a4070c40 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f09a415d9b0 0x7f09a40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a2a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f09a40729e0 0x7f09a4161630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58796/0 (socket says 172.21.15.164:58796) 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a2a00640 1 -- 172.21.15.164:0/3472699414 learned_addr learned my addr 172.21.15.164:0/3472699414 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:50.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3e00640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a4070c40 0x7f09a40b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3400640 1 -- 172.21.15.164:0/3472699414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a4070c40 msgr2=0x7f09a40b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:50.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3400640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a4070c40 0x7f09a40b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3400640 1 -- 172.21.15.164:0/3472699414 >> v1:172.21.15.164:6789/0 conn(0x7f09a40729e0 legacy=0x7f09a4161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a3400640 1 -- 172.21.15.164:0/3472699414 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40add80 con 0x7f09a415d9b0 2024-05-08T04:43:50.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.607+0000 7f09a2000640 1 -- 172.21.15.164:0/3472699414 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313848833 0 0) 0x7f09a4071e40 con 0x7f09a40729e0 2024-05-08T04:43:50.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a3400640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40b2ea0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f09940097c0 tx=0x7f099401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccd1c8ca8e988ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a2000640 1 -- 172.21.15.164:0/3472699414 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940b0070 con 0x7f09a415d9b0 2024-05-08T04:43:50.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a2000640 1 -- 172.21.15.164:0/3472699414 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09940ab600 con 0x7f09a415d9b0 2024-05-08T04:43:50.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a2000640 1 -- 172.21.15.164:0/3472699414 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940ab9d0 con 0x7f09a415d9b0 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- 172.21.15.164:0/3472699414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 msgr2=0x7f09a40b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40b2ea0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f09940097c0 tx=0x7f099401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- 172.21.15.164:0/3472699414 shutdown_connections 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a4070c40 0x7f09a40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 --2- 172.21.15.164:0/3472699414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40b2ea0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- 172.21.15.164:0/3472699414 >> 172.21.15.164:0/3472699414 conn(0x7f09a406ac90 msgr2=0x7f09a406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- 172.21.15.164:0/3472699414 shutdown_connections 2024-05-08T04:43:50.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- 172.21.15.164:0/3472699414 wait complete. 2024-05-08T04:43:50.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 Processor -- start 2024-05-08T04:43:50.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.611+0000 7f09a9000640 1 -- start start 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a4070c40 0x7f09a4160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 0x7f09a413c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a416ab50 con 0x7f09a415d9b0 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a416a9d0 con 0x7f09a4070c40 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a416acd0 con 0x7f09a40729e0 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42102/0 (socket says 172.21.15.164:42102) 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 -- 172.21.15.164:0/3122938644 learned_addr learned my addr 172.21.15.164:0/3122938644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3400640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a4070c40 0x7f09a4160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a2a00640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 0x7f09a413c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 -- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 msgr2=0x7f09a413c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 0x7f09a413c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 -- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a4070c40 msgr2=0x7f09a4160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a4070c40 0x7f09a4160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 -- 172.21.15.164:0/3122938644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a414d9e0 con 0x7f09a415d9b0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a2a00640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 0x7f09a413c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a3e00640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f099c00efc0 tx=0x7f099c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c018020 con 0x7f09a415d9b0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c014480 con 0x7f09a415d9b0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c009e60 con 0x7f09a415d9b0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a414dcc0 con 0x7f09a415d9b0 2024-05-08T04:43:50.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a414e280 con 0x7f09a415d9b0 2024-05-08T04:43:50.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.615+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a404ef30 con 0x7f09a415d9b0 2024-05-08T04:43:50.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.619+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f099c01b020 con 0x7f09a415d9b0 2024-05-08T04:43:50.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.619+0000 7f099b400640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 0x7f098406a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.619+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f099c084a80 con 0x7f09a415d9b0 2024-05-08T04:43:50.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.619+0000 7f09a3400640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 0x7f098406a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.619+0000 7f09a3400640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 0x7f098406a1d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f09940ad000 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.623+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f099c022030 con 0x7f09a415d9b0 2024-05-08T04:43:50.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09a41510d0 con 0x7f09a415d9b0 2024-05-08T04:43:50.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f099b400640 1 -- 172.21.15.164:0/3122938644 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f099c010040 con 0x7f09a415d9b0 2024-05-08T04:43:50.776 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:50.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 msgr2=0x7f098406a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 0x7f098406a1d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f09940ad000 tx=0x7f0994008040 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 msgr2=0x7f09a40afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f099c00efc0 tx=0x7f099c00c490 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 shutdown_connections 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0984067d10 0x7f098406a1d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a40729e0 0x7f09a413c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a415d9b0 0x7f09a40afa10 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 --2- 172.21.15.164:0/3122938644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a4070c40 0x7f09a4160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 >> 172.21.15.164:0/3122938644 conn(0x7f09a406ac90 msgr2=0x7f09a4153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 shutdown_connections 2024-05-08T04:43:50.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.775+0000 7f09a9000640 1 -- 172.21.15.164:0/3122938644 wait complete. 2024-05-08T04:43:50.797 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:50.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 Processor -- start 2024-05-08T04:43:50.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 -- start start 2024-05-08T04:43:50.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1c0c6600 con 0x7f3e1c15ce10 2024-05-08T04:43:50.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1c0c6040 con 0x7f3e1c15ba20 2024-05-08T04:43:50.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e22e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1c15fbe0 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42112/0 (socket says 172.21.15.164:42112) 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 -- 172.21.15.164:0/3265883065 learned_addr learned my addr 172.21.15.164:0/3265883065 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e13400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3e1c15ce10 0x7f3e1c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58812/0 (socket says 172.21.15.164:58812) 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 -- 172.21.15.164:0/3265883065 >> v1:172.21.15.172:6789/0 conn(0x7f3e1c15ba20 legacy=0x7f3e1c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 -- 172.21.15.164:0/3265883065 >> v1:172.21.15.164:6789/0 conn(0x7f3e1c15ce10 legacy=0x7f3e1c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.935+0000 7f3e20e00640 1 -- 172.21.15.164:0/3265883065 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c0c6320 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e20e00640 1 --2- 172.21.15.164:0/3265883065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009e50 tx=0x7f3e1401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe67d83b2daecacb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e1b400640 1 -- 172.21.15.164:0/3265883065 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140a5700 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e1b400640 1 -- 172.21.15.164:0/3265883065 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e140a58a0 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e1b400640 1 -- 172.21.15.164:0/3265883065 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140a5c70 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 -- 172.21.15.164:0/3265883065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c0ae6e0 msgr2=0x7f3e1c0aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 --2- 172.21.15.164:0/3265883065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009e50 tx=0x7f3e1401ca30 comp rx=0 tx=0).stop 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 -- 172.21.15.164:0/3265883065 shutdown_connections 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 --2- 172.21.15.164:0/3265883065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c0ae6e0 0x7f3e1c0aeb20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 -- 172.21.15.164:0/3265883065 >> 172.21.15.164:0/3265883065 conn(0x7f3e1c0cf770 msgr2=0x7f3e1c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 -- 172.21.15.164:0/3265883065 shutdown_connections 2024-05-08T04:43:50.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.939+0000 7f3e22e00640 1 -- 172.21.15.164:0/3265883065 wait complete. 2024-05-08T04:43:50.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 Processor -- start 2024-05-08T04:43:50.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- start start 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 0x7f3e1c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c15ba20 0x7f3e1c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c0ce000 con 0x7f3e1c15ce10 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c0cde80 con 0x7f3e1c15ba20 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c169b80 con 0x7f3e1c0ae6e0 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42124/0 (socket says 172.21.15.164:42124) 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 -- 172.21.15.164:0/2676941050 learned_addr learned my addr 172.21.15.164:0/2676941050 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e13400640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c15ba20 0x7f3e1c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e1be00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 0x7f3e1c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 -- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 msgr2=0x7f3e1c153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 0x7f3e1c153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 -- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c15ba20 msgr2=0x7f3e1c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c15ba20 0x7f3e1c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 -- 172.21.15.164:0/2676941050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c153f80 con 0x7f3e1c15ce10 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e1be00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 0x7f3e1c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e20e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3e140af040 tx=0x7f3e140197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140b8470 con 0x7f3e1c15ce10 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e140b8670 con 0x7f3e1c15ce10 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c154200 con 0x7f3e1c15ce10 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140b8a60 con 0x7f3e1c15ce10 2024-05-08T04:43:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.943+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c0b0660 con 0x7f3e1c15ce10 2024-05-08T04:43:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c072d30 con 0x7f3e1c15ce10 2024-05-08T04:43:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f3e140c0020 con 0x7f3e1c15ce10 2024-05-08T04:43:50.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e18c00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 0x7f3df806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:50.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3e1411e5a0 con 0x7f3e1c15ce10 2024-05-08T04:43:50.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e1be00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 0x7f3df806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:50.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.947+0000 7f3e1be00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 0x7f3df806a1a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c001710 tx=0x7f3e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:50.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:50.951+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e140c7020 con 0x7f3e1c15ce10 2024-05-08T04:43:51.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.103+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e1c04f010 con 0x7f3e1c15ce10 2024-05-08T04:43:51.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.103+0000 7f3e18c00640 1 -- 172.21.15.164:0/2676941050 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e1c153f80 con 0x7f3e1c15ce10 2024-05-08T04:43:51.106 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 msgr2=0x7f3df806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 0x7f3df806a1a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c001710 tx=0x7f3e0c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 msgr2=0x7f3e1c14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3e140af040 tx=0x7f3e140197d0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 shutdown_connections 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3df8067ce0 0x7f3df806a1a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e1c15ce10 0x7f3e1c14fea0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c15ba20 0x7f3e1c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 --2- 172.21.15.164:0/2676941050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e1c0ae6e0 0x7f3e1c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 >> 172.21.15.164:0/2676941050 conn(0x7f3e1c0cf770 msgr2=0x7f3e1c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:51.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 shutdown_connections 2024-05-08T04:43:51.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.107+0000 7f3e22e00640 1 -- 172.21.15.164:0/2676941050 wait complete. 2024-05-08T04:43:51.126 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T04:43:51.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.259+0000 7f4cedc00640 1 Processor -- start 2024-05-08T04:43:51.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 -- start start 2024-05-08T04:43:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ce80cfda0 0x7f4ce8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ce815be50 0x7f4ce80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce80cd9f0 con 0x7f4ce815ba70 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce815fe70 con 0x7f4ce815be50 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cedc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce80cdcd0 con 0x7f4ce80cfda0 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ce815be50 0x7f4ce80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ce815be50 0x7f4ce80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42126/0 (socket says 172.21.15.164:42126) 2024-05-08T04:43:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 -- 172.21.15.164:0/853005346 learned_addr learned my addr 172.21.15.164:0/853005346 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce6a00640 1 --1- 172.21.15.164:0/853005346 >> v1:172.21.15.164:6789/0 conn(0x7f4ce815ba70 0x7f4ce80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58828/0 (socket says 172.21.15.164:58828) 2024-05-08T04:43:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4cde000640 1 --2- 172.21.15.164:0/853005346 >> v2:172.21.15.172:3300/0 conn(0x7f4ce80cfda0 0x7f4ce8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 -- 172.21.15.164:0/853005346 >> v2:172.21.15.172:3300/0 conn(0x7f4ce80cfda0 msgr2=0x7f4ce8160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 --2- 172.21.15.164:0/853005346 >> v2:172.21.15.172:3300/0 conn(0x7f4ce80cfda0 0x7f4ce8160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 -- 172.21.15.164:0/853005346 >> v1:172.21.15.164:6789/0 conn(0x7f4ce815ba70 legacy=0x7f4ce80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce7400640 1 -- 172.21.15.164:0/853005346 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce815f820 con 0x7f4ce815be50 2024-05-08T04:43:51.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.263+0000 7f4ce6000640 1 -- 172.21.15.164:0/853005346 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044602103 0 0) 0x7f4ce80cd9f0 con 0x7f4ce815ba70 2024-05-08T04:43:51.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4ce7400640 1 --2- 172.21.15.164:0/853005346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce815be50 0x7f4ce80b31a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4ce815f780 tx=0x7f4ce001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a314b1dcbe812f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4ce6000640 1 -- 172.21.15.164:0/853005346 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00ab600 con 0x7f4ce815be50 2024-05-08T04:43:51.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4ce6000640 1 -- 172.21.15.164:0/853005346 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce00b0070 con 0x7f4ce815be50 2024-05-08T04:43:51.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4ce6000640 1 -- 172.21.15.164:0/853005346 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00ab9c0 con 0x7f4ce815be50 2024-05-08T04:43:51.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- 172.21.15.164:0/853005346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce815be50 msgr2=0x7f4ce80b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 --2- 172.21.15.164:0/853005346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce815be50 0x7f4ce80b31a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4ce815f780 tx=0x7f4ce001c4f0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- 172.21.15.164:0/853005346 shutdown_connections 2024-05-08T04:43:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 --2- 172.21.15.164:0/853005346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce815be50 0x7f4ce80b31a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 --2- 172.21.15.164:0/853005346 >> v2:172.21.15.172:3300/0 conn(0x7f4ce80cfda0 0x7f4ce8160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- 172.21.15.164:0/853005346 >> 172.21.15.164:0/853005346 conn(0x7f4ce806ac90 msgr2=0x7f4ce806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- 172.21.15.164:0/853005346 shutdown_connections 2024-05-08T04:43:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- 172.21.15.164:0/853005346 wait complete. 2024-05-08T04:43:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 Processor -- start 2024-05-08T04:43:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.267+0000 7f4cedc00640 1 -- start start 2024-05-08T04:43:51.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce815ba70 0x7f4ce814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce815be50 0x7f4ce8150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8168d40 con 0x7f4ce80cfda0 2024-05-08T04:43:51.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8168bc0 con 0x7f4ce815be50 2024-05-08T04:43:51.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8168ec0 con 0x7f4ce815ba70 2024-05-08T04:43:51.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42128/0 (socket says 172.21.15.164:42128) 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 -- 172.21.15.164:0/1117527269 learned_addr learned my addr 172.21.15.164:0/1117527269 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cde000640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce815ba70 0x7f4ce814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce7400640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce815be50 0x7f4ce8150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 -- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce815ba70 msgr2=0x7f4ce814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce815ba70 0x7f4ce814ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 -- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce815be50 msgr2=0x7f4ce8150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce815be50 0x7f4ce8150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 -- 172.21.15.164:0/1117527269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce814e800 con 0x7f4ce80cfda0 2024-05-08T04:43:51.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4ce6a00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8001570 tx=0x7f4cd800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8018070 con 0x7f4ce80cfda0 2024-05-08T04:43:51.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800a640 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce8151c20 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800aa30 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce81521e0 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.271+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce8069d60 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.275+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f4cd801b070 con 0x7f4ce80cfda0 2024-05-08T04:43:51.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.275+0000 7f4cdf400640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 0x7f4cc406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.275+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4cd8083300 con 0x7f4ce80cfda0 2024-05-08T04:43:51.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.275+0000 7f4cde000640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 0x7f4cc406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.275+0000 7f4cde000640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 0x7f4cc406a1a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4ce815ee80 tx=0x7f4cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.279+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cd80554c0 con 0x7f4ce80cfda0 2024-05-08T04:43:51.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.443+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ce80cf3e0 con 0x7f4ce80cfda0 2024-05-08T04:43:51.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.443+0000 7f4cdf400640 1 -- 172.21.15.164:0/1117527269 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4cd8059560 con 0x7f4ce80cfda0 2024-05-08T04:43:51.446 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 msgr2=0x7f4cc406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 0x7f4cc406a1a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4ce815ee80 tx=0x7f4cd0008040 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 msgr2=0x7f4ce814e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8001570 tx=0x7f4cd800cd20 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 shutdown_connections 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4cc4067ce0 0x7f4cc406a1a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ce815ba70 0x7f4ce814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce815be50 0x7f4ce8150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 --2- 172.21.15.164:0/1117527269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80cfda0 0x7f4ce814e2c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 >> 172.21.15.164:0/1117527269 conn(0x7f4ce806ac90 msgr2=0x7f4ce8158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 shutdown_connections 2024-05-08T04:43:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.447+0000 7f4cedc00640 1 -- 172.21.15.164:0/1117527269 wait complete. 2024-05-08T04:43:51.467 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T04:43:51.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:51.467 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-08T04:43:51.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:43:51.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:43:51.496 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T04:43:51.551 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:51.551 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:51.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:51.560 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:51.560 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:51.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:43:51.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:51.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.739+0000 7fa6cf200640 1 Processor -- start 2024-05-08T04:43:51.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.739+0000 7fa6cf200640 1 -- start start 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cf200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa6c80cfda0 0x7fa6c8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cf200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80cd9f0 con 0x7fa6c815ba70 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cf200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80cdcd0 con 0x7fa6c815be50 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cf200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c815fe70 con 0x7fa6c80cfda0 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa6c815ba70 0x7fa6c80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58830/0 (socket says 172.21.15.164:58830) 2024-05-08T04:43:51.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 -- 172.21.15.164:0/4181081050 learned_addr learned my addr 172.21.15.164:0/4181081050 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122674447 0 0) 0x7fa6c80cd9f0 con 0x7fa6c815ba70 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ac003530 con 0x7fa6c815ba70 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7400640 1 --2- 172.21.15.164:0/4181081050 >> v2:172.21.15.172:3300/0 conn(0x7fa6c80cfda0 0x7fa6c8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224579043 0 0) 0x7fa6ac003530 con 0x7fa6c815ba70 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 >> v2:172.21.15.172:3300/0 conn(0x7fa6c80cfda0 msgr2=0x7fa6c8160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 --2- 172.21.15.164:0/4181081050 >> v2:172.21.15.172:3300/0 conn(0x7fa6c80cfda0 0x7fa6c8160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 >> v1:172.21.15.172:6789/0 conn(0x7fa6c815be50 legacy=0x7fa6c80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c815f820 con 0x7fa6c815ba70 2024-05-08T04:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fa6b8001680 con 0x7fa6c815ba70 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 >> v1:172.21.15.164:6789/0 conn(0x7fa6c815ba70 legacy=0x7fa6c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 0x7fa6ac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c815fe70 con 0x7fa6ac003a10 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c170 con 0x7fa6ac004810 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c370 con 0x7fa6ac005220 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 0x7fa6ac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7400640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cd200640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 -- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6ac005220 msgr2=0x7fa6ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 -- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac004810 msgr2=0x7fa6ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7e00640 1 -- 172.21.15.164:0/4181081050 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac003530 con 0x7fa6ac003a10 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6c7400640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cd200640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:51.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.743+0000 7fa6cd200640 1 -- 172.21.15.164:0/4181081050 reap_dead start 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 0x7fa6ac003e30 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c110 tx=0x7fa6b801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6a03c92a95a602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8007c30 con 0x7fa6ac003a10 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b80a9020 con 0x7fa6ac003a10 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6c6a00640 1 -- 172.21.15.164:0/4181081050 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80a5b00 con 0x7fa6ac003a10 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 msgr2=0x7fa6ac003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 0x7fa6ac003e30 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c110 tx=0x7fa6b801cb70 comp rx=0 tx=0).stop 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- 172.21.15.164:0/4181081050 shutdown_connections 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 --2- 172.21.15.164:0/4181081050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac003a10 0x7fa6ac003e30 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- 172.21.15.164:0/4181081050 >> 172.21.15.164:0/4181081050 conn(0x7fa6c806ac90 msgr2=0x7fa6c806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- 172.21.15.164:0/4181081050 shutdown_connections 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- 172.21.15.164:0/4181081050 wait complete. 2024-05-08T04:43:51.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 Processor -- start 2024-05-08T04:43:51.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.747+0000 7fa6cf200640 1 -- start start 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 0x7fa6c814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 0x7fa6c8144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80ae860 con 0x7fa6c81f4c30 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80ae6e0 con 0x7fa6ac003a10 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80ae9e0 con 0x7fa6c8151160 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42146/0 (socket says 172.21.15.164:42146) 2024-05-08T04:43:51.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 -- 172.21.15.164:0/3526129213 learned_addr learned my addr 172.21.15.164:0/3526129213 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6c7400640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 0x7fa6c8144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 0x7fa6c814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 -- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 msgr2=0x7fa6c8144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 0x7fa6c8144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 -- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 msgr2=0x7fa6c814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 0x7fa6c814d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 -- 172.21.15.164:0/3526129213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c814e4c0 con 0x7fa6c81f4c30 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 0x7fa6c814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6c7400640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 0x7fa6c8144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cd200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa6c815c230 tx=0x7fa6c000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0003980 con 0x7fa6c81f4c30 2024-05-08T04:43:51.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0018070 con 0x7fa6c81f4c30 2024-05-08T04:43:51.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c815d9f0 con 0x7fa6c81f4c30 2024-05-08T04:43:51.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0003d60 con 0x7fa6c81f4c30 2024-05-08T04:43:51.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c81f50e0 con 0x7fa6c81f4c30 2024-05-08T04:43:51.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8069d60 con 0x7fa6c81f4c30 2024-05-08T04:43:51.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.751+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fa6c00050d0 con 0x7fa6c81f4c30 2024-05-08T04:43:51.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.755+0000 7fa6bfe00640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 0x7fa6a806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:51.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.755+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa6c0083820 con 0x7fa6c81f4c30 2024-05-08T04:43:51.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.755+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 0x7fa6a806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:51.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.755+0000 7fa6c7e00640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 0x7fa6a806a1d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80aa000 tx=0x7fa6c80cdea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:51.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.759+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6c0022020 con 0x7fa6c81f4c30 2024-05-08T04:43:51.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.943+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6c806e210 con 0x7fa6c81f4c30 2024-05-08T04:43:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.943+0000 7fa6bfe00640 1 -- 172.21.15.164:0/3526129213 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fa6c0055f10 con 0x7fa6c81f4c30 2024-05-08T04:43:51.947 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:51.947 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:51.947 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 83 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 msgr2=0x7fa6a806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 0x7fa6a806a1d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80aa000 tx=0x7fa6c80cdea0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 msgr2=0x7fa6c80b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa6c815c230 tx=0x7fa6c000eb90 comp rx=0 tx=0).stop 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 shutdown_connections 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa6a8067d10 0x7fa6a806a1d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6c8151160 0x7fa6c8144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6c81f4c30 0x7fa6c80b0260 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 --2- 172.21.15.164:0/3526129213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac003a10 0x7fa6c814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 >> 172.21.15.164:0/3526129213 conn(0x7fa6c806ac90 msgr2=0x7fa6c806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 shutdown_connections 2024-05-08T04:43:51.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:51.947+0000 7fa6cf200640 1 -- 172.21.15.164:0/3526129213 wait complete. 2024-05-08T04:43:51.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:52.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 Processor -- start 2024-05-08T04:43:52.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 -- start start 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ba415dd70 0x7f5ba41619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba406cc70 con 0x7f5ba41625f0 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba406cf50 con 0x7f5ba406c7b0 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba9c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba40b1570 con 0x7f5ba415dd70 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba3400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5ba41625f0 0x7f5ba40b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58840/0 (socket says 172.21.15.164:58840) 2024-05-08T04:43:52.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba3400640 1 -- 172.21.15.164:0/1722141446 learned_addr learned my addr 172.21.15.164:0/1722141446 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:52.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/1722141446 >> v2:172.21.15.172:3300/0 conn(0x7f5ba415dd70 0x7f5ba41619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747692753 0 0) 0x7f5ba406cc70 con 0x7f5ba41625f0 2024-05-08T04:43:52.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b88003530 con 0x7f5ba41625f0 2024-05-08T04:43:52.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038978455 0 0) 0x7f5ba406cf50 con 0x7f5ba406c7b0 2024-05-08T04:43:52.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba406cc70 con 0x7f5ba406c7b0 2024-05-08T04:43:52.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1974225357 0 0) 0x7f5b88003530 con 0x7f5ba41625f0 2024-05-08T04:43:52.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba2a00640 1 -- 172.21.15.164:0/1722141446 >> v1:172.21.15.172:6789/0 conn(0x7f5ba406c7b0 legacy=0x7f5ba4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba2a00640 1 -- 172.21.15.164:0/1722141446 >> v1:172.21.15.164:6789/0 conn(0x7f5ba41625f0 legacy=0x7f5ba40b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.103+0000 7f5ba2a00640 1 -- 172.21.15.164:0/1722141446 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40b1850 con 0x7f5ba415dd70 2024-05-08T04:43:52.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/1722141446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba41619a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b940097c0 tx=0x7f5b9401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3510ce457069ea52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940b0070 con 0x7f5ba415dd70 2024-05-08T04:43:52.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940ab400 con 0x7f5ba415dd70 2024-05-08T04:43:52.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba1600640 1 -- 172.21.15.164:0/1722141446 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c001a00 con 0x7f5ba415dd70 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 -- 172.21.15.164:0/1722141446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 msgr2=0x7f5ba41619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/1722141446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba41619a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b940097c0 tx=0x7f5b9401c3c0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 -- 172.21.15.164:0/1722141446 shutdown_connections 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/1722141446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba41619a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 -- 172.21.15.164:0/1722141446 >> 172.21.15.164:0/1722141446 conn(0x7f5ba40b34f0 msgr2=0x7f5ba40b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 -- 172.21.15.164:0/1722141446 shutdown_connections 2024-05-08T04:43:52.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.107+0000 7f5ba9c00640 1 -- 172.21.15.164:0/1722141446 wait complete. 2024-05-08T04:43:52.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 Processor -- start 2024-05-08T04:43:52.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- start start 2024-05-08T04:43:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba40b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba41625f0 0x7f5ba40b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40b3060 con 0x7f5ba406c7b0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40b2ee0 con 0x7f5ba415dd70 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba416bcf0 con 0x7f5ba41625f0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42150/0 (socket says 172.21.15.164:42150) 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 -- 172.21.15.164:0/3991295659 learned_addr learned my addr 172.21.15.164:0/3991295659 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 -- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba41625f0 msgr2=0x7f5ba40b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba41625f0 0x7f5ba40b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 -- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 msgr2=0x7f5ba40b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba40b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 -- 172.21.15.164:0/3991295659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40b4810 con 0x7f5ba406c7b0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba2a00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5b94018fd0 tx=0x7f5b94019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940b9020 con 0x7f5ba406c7b0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b94003820 con 0x7f5ba406c7b0 2024-05-08T04:43:52.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940b0040 con 0x7f5ba406c7b0 2024-05-08T04:43:52.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba40c5420 con 0x7f5ba406c7b0 2024-05-08T04:43:52.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba4160890 con 0x7f5ba406c7b0 2024-05-08T04:43:52.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.111+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba4069d60 con 0x7f5ba406c7b0 2024-05-08T04:43:52.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.115+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f5b940bc020 con 0x7f5ba406c7b0 2024-05-08T04:43:52.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.115+0000 7f5b9aa00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 0x7f5b8406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.115+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f5b9411dc10 con 0x7f5ba406c7b0 2024-05-08T04:43:52.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.115+0000 7f5ba2000640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 0x7f5b8406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.115+0000 7f5ba2000640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 0x7f5b8406a1a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5b90007fd0 tx=0x7f5b90007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.119+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b940c3030 con 0x7f5ba406c7b0 2024-05-08T04:43:52.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ba40b4010 con 0x7f5ba406c7b0 2024-05-08T04:43:52.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5b9aa00640 1 -- 172.21.15.164:0/3991295659 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f5b94015fb0 con 0x7f5ba406c7b0 2024-05-08T04:43:52.276 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:52.277 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:52.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 msgr2=0x7f5b8406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 0x7f5b8406a1a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5b90007fd0 tx=0x7f5b90007960 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 msgr2=0x7f5ba415fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5b94018fd0 tx=0x7f5b94019160 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 shutdown_connections 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5b84067ce0 0x7f5b8406a1a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ba41625f0 0x7f5ba40b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ba415dd70 0x7f5ba40b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 --2- 172.21.15.164:0/3991295659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba406c7b0 0x7f5ba415fd80 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 >> 172.21.15.164:0/3991295659 conn(0x7f5ba40b34f0 msgr2=0x7f5ba40ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 shutdown_connections 2024-05-08T04:43:52.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.275+0000 7f5ba9c00640 1 -- 172.21.15.164:0/3991295659 wait complete. 2024-05-08T04:43:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T04:38:17.494575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T04:43:29.191800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T04:43:29.875561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T04:43:52.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T04:43:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 Processor -- start 2024-05-08T04:43:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 -- start start 2024-05-08T04:43:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce415df80 0x7f2ce40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2ce406ac90 0x7f2ce406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce40caaa0 con 0x7f2ce40cf730 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce415e3e0 con 0x7f2ce406ac90 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce415eea0 con 0x7f2ce415df80 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce8a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2ce40cf730 0x7f2ce406d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58856/0 (socket says 172.21.15.164:58856) 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce8a00640 1 -- 172.21.15.164:0/1840579446 learned_addr learned my addr 172.21.15.164:0/1840579446 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9400640 1 --2- 172.21.15.164:0/1840579446 >> v2:172.21.15.172:3300/0 conn(0x7f2ce406ac90 0x7f2ce406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce3e00640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce415df80 0x7f2ce40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938653896 0 0) 0x7f2ce40caaa0 con 0x7f2ce40cf730 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ccc003530 con 0x7f2ce40cf730 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9400640 1 -- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce415df80 msgr2=0x7f2ce40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9400640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce415df80 0x7f2ce40ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9400640 1 -- 172.21.15.164:0/1840579446 >> v1:172.21.15.164:6789/0 conn(0x7f2ce40cf730 legacy=0x7f2ce406d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848483757 0 0) 0x7f2ccc003530 con 0x7f2ce40cf730 2024-05-08T04:43:52.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.439+0000 7f2ce9400640 1 -- 172.21.15.164:0/1840579446 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce415e600 con 0x7f2ce406ac90 2024-05-08T04:43:52.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9400640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce406ac90 0x7f2ce406b120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009d90 tx=0x7f2cdc01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfb9eb9d8f881e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0ac070 con 0x7f2ce406ac90 2024-05-08T04:43:52.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc0a6410 con 0x7f2ce406ac90 2024-05-08T04:43:52.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce3400640 1 -- 172.21.15.164:0/1840579446 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8001680 con 0x7f2ce406ac90 2024-05-08T04:43:52.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 -- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce406ac90 msgr2=0x7f2ce406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce406ac90 0x7f2ce406b120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009d90 tx=0x7f2cdc01c800 comp rx=0 tx=0).stop 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 -- 172.21.15.164:0/1840579446 shutdown_connections 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce406ac90 0x7f2ce406b120 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/1840579446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce415df80 0x7f2ce40ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 -- 172.21.15.164:0/1840579446 >> 172.21.15.164:0/1840579446 conn(0x7f2ce40b2500 msgr2=0x7f2ce40b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 -- 172.21.15.164:0/1840579446 shutdown_connections 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.443+0000 7f2ce9e00640 1 -- 172.21.15.164:0/1840579446 wait complete. 2024-05-08T04:43:52.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 Processor -- start 2024-05-08T04:43:52.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- start start 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 0x7f2ce4156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce40cf730 0x7f2ce4162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce416a910 con 0x7f2ce415df80 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4072340 con 0x7f2ce40cf730 2024-05-08T04:43:52.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce416aa90 con 0x7f2ce406ac90 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42164/0 (socket says 172.21.15.164:42164) 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 -- 172.21.15.164:0/763148100 learned_addr learned my addr 172.21.15.164:0/763148100 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce3e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce40cf730 0x7f2ce4162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce8a00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 0x7f2ce4156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 -- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 msgr2=0x7f2ce4156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 0x7f2ce4156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 -- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce40cf730 msgr2=0x7f2ce4162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce40cf730 0x7f2ce4162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 -- 172.21.15.164:0/763148100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40b3fa0 con 0x7f2ce415df80 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce8a00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 0x7f2ce4156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9400640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc01aeb0 tx=0x7f2cdc0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0ac070 con 0x7f2ce415df80 2024-05-08T04:43:52.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc0b1650 con 0x7f2ce415df80 2024-05-08T04:43:52.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc009450 con 0x7f2ce415df80 2024-05-08T04:43:52.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce41635a0 con 0x7f2ce415df80 2024-05-08T04:43:52.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.447+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce40b4520 con 0x7f2ce415df80 2024-05-08T04:43:52.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce404ef30 con 0x7f2ce415df80 2024-05-08T04:43:52.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f2cdc0af070 con 0x7f2ce415df80 2024-05-08T04:43:52.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce0c00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 0x7f2cc406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2cdc11e3b0 con 0x7f2ce415df80 2024-05-08T04:43:52.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce8a00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 0x7f2cc406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.451+0000 7f2ce8a00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 0x7f2cc406a1a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8001b80 tx=0x7f2cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.455+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cdc0c1060 con 0x7f2ce415df80 2024-05-08T04:43:52.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.615+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ce41686e0 con 0x7f2ce415df80 2024-05-08T04:43:52.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.615+0000 7f2ce0c00640 1 -- 172.21.15.164:0/763148100 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cdc016480 con 0x7f2ce415df80 2024-05-08T04:43:52.618 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 msgr2=0x7f2cc406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 0x7f2cc406a1a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8001b80 tx=0x7f2cd8009040 comp rx=0 tx=0).stop 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 msgr2=0x7f2ce415f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc01aeb0 tx=0x7f2cdc0195a0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 shutdown_connections 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2cc4067ce0 0x7f2cc406a1a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ce415df80 0x7f2ce415f0b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ce40cf730 0x7f2ce4162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 --2- 172.21.15.164:0/763148100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ce406ac90 0x7f2ce4156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 >> 172.21.15.164:0/763148100 conn(0x7f2ce40b2500 msgr2=0x7f2ce40b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 shutdown_connections 2024-05-08T04:43:52.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.619+0000 7f2ce9e00640 1 -- 172.21.15.164:0/763148100 wait complete. 2024-05-08T04:43:52.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T04:43:52.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.775+0000 7f62ef000640 1 Processor -- start 2024-05-08T04:43:52.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.775+0000 7f62ef000640 1 -- start start 2024-05-08T04:43:52.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ef000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 0x7f62e8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ef000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ef000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80c6600 con 0x7f62e806de50 2024-05-08T04:43:52.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ef000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e815fbe0 con 0x7f62e815ba20 2024-05-08T04:43:52.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ef000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c6040 con 0x7f62e80aedf0 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62e7e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 0x7f62e8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:45862/0 (socket says 172.21.15.164:45862) 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 -- 172.21.15.164:0/57513758 learned_addr learned my addr 172.21.15.164:0/57513758 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 -- 172.21.15.164:0/57513758 >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 msgr2=0x7f62e8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 --2- 172.21.15.164:0/57513758 >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 0x7f62e8160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 -- 172.21.15.164:0/57513758 >> v1:172.21.15.172:6789/0 conn(0x7f62e806de50 legacy=0x7f62e80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62ed000640 1 -- 172.21.15.164:0/57513758 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e80c6320 con 0x7f62e80aedf0 2024-05-08T04:43:52.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.779+0000 7f62e7e00640 1 --2- 172.21.15.164:0/57513758 >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 0x7f62e8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:52.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ed000640 1 --2- 172.21.15.164:0/57513758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009870 tx=0x7f62e001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd91342e19da83e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62e6a00640 1 -- 172.21.15.164:0/57513758 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00aa430 con 0x7f62e80aedf0 2024-05-08T04:43:52.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62e6a00640 1 -- 172.21.15.164:0/57513758 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e00af070 con 0x7f62e80aedf0 2024-05-08T04:43:52.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62e6a00640 1 -- 172.21.15.164:0/57513758 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00aa7f0 con 0x7f62e80aedf0 2024-05-08T04:43:52.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- 172.21.15.164:0/57513758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 msgr2=0x7f62e806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 --2- 172.21.15.164:0/57513758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009870 tx=0x7f62e001c530 comp rx=0 tx=0).stop 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- 172.21.15.164:0/57513758 shutdown_connections 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 --2- 172.21.15.164:0/57513758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e80aedf0 0x7f62e806e2c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 --2- 172.21.15.164:0/57513758 >> v2:172.21.15.172:3300/0 conn(0x7f62e815ba20 0x7f62e8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- 172.21.15.164:0/57513758 >> 172.21.15.164:0/57513758 conn(0x7f62e80cf770 msgr2=0x7f62e80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- 172.21.15.164:0/57513758 shutdown_connections 2024-05-08T04:43:52.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- 172.21.15.164:0/57513758 wait complete. 2024-05-08T04:43:52.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 Processor -- start 2024-05-08T04:43:52.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 -- start start 2024-05-08T04:43:52.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.783+0000 7f62ef000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e80aedf0 0x7f62e8150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e815ba20 0x7f62e81f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80ce000 con 0x7f62e806de50 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80cde80 con 0x7f62e80aedf0 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8168b70 con 0x7f62e815ba20 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42180/0 (socket says 172.21.15.164:42180) 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 -- 172.21.15.164:0/285672293 learned_addr learned my addr 172.21.15.164:0/285672293 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ed000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e815ba20 0x7f62e81f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7400640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e80aedf0 0x7f62e8150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 -- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e815ba20 msgr2=0x7f62e81f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e815ba20 0x7f62e81f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 -- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e80aedf0 msgr2=0x7f62e8150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e80aedf0 0x7f62e8150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 -- 172.21.15.164:0/285672293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8157cf0 con 0x7f62e806de50 2024-05-08T04:43:52.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62e7e00640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009ec0 tx=0x7f62d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8018020 con 0x7f62e806de50 2024-05-08T04:43:52.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8014680 con 0x7f62e806de50 2024-05-08T04:43:52.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e8157fd0 con 0x7f62e806de50 2024-05-08T04:43:52.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8014a50 con 0x7f62e806de50 2024-05-08T04:43:52.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e8158590 con 0x7f62e806de50 2024-05-08T04:43:52.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e8072d30 con 0x7f62e806de50 2024-05-08T04:43:52.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.787+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f62d801b020 con 0x7f62e806de50 2024-05-08T04:43:52.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.791+0000 7f62dfe00640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 0x7f62c806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:52.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.791+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f62d8084be0 con 0x7f62e806de50 2024-05-08T04:43:52.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.791+0000 7f62e7400640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 0x7f62c806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:52.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.791+0000 7f62e7400640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 0x7f62c806a1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cce60 tx=0x7f62d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:52.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.795+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62d8022030 con 0x7f62e806de50 2024-05-08T04:43:52.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.947+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62e8166940 con 0x7f62e806de50 2024-05-08T04:43:52.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.947+0000 7f62dfe00640 1 -- 172.21.15.164:0/285672293 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62d8010040 con 0x7f62e806de50 2024-05-08T04:43:52.950 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 msgr2=0x7f62c806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 0x7f62c806a1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cce60 tx=0x7f62d4009040 comp rx=0 tx=0).stop 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 msgr2=0x7f62e814fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009ec0 tx=0x7f62d800c490 comp rx=0 tx=0).stop 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 shutdown_connections 2024-05-08T04:43:52.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f62c8067ce0 0x7f62c806a1a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62e815ba20 0x7f62e81f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e80aedf0 0x7f62e8150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 --2- 172.21.15.164:0/285672293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e806de50 0x7f62e814fa80 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 >> 172.21.15.164:0/285672293 conn(0x7f62e80cf770 msgr2=0x7f62e815e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 shutdown_connections 2024-05-08T04:43:52.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:52.951+0000 7f62ef000640 1 -- 172.21.15.164:0/285672293 wait complete. 2024-05-08T04:43:52.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T04:43:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.107+0000 7f8cc5600640 1 Processor -- start 2024-05-08T04:43:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.107+0000 7f8cc5600640 1 -- start start 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00cec40 0x7f8cc015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8cc00b1d40 0x7f8cc00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc5600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc015c900 con 0x7f8cc00b1960 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015cec0 con 0x7f8cc00b1d40 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc015cbe0 con 0x7f8cc00cec40 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cbfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8cc00b1960 0x7f8cc0071aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58870/0 (socket says 172.21.15.164:58870) 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cbfe00640 1 -- 172.21.15.164:0/283327237 learned_addr learned my addr 172.21.15.164:0/283327237 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:53.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/283327237 >> v2:172.21.15.172:3300/0 conn(0x7f8cc00b1d40 0x7f8cc00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cb7400640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00cec40 0x7f8cc015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cbf400640 1 -- 172.21.15.164:0/283327237 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719973656 0 0) 0x7f8cc015c900 con 0x7f8cc00b1960 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cbf400640 1 -- 172.21.15.164:0/283327237 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca4003530 con 0x7f8cc00b1960 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc4c00640 1 -- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00cec40 msgr2=0x7f8cc015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00cec40 0x7f8cc015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc4c00640 1 -- 172.21.15.164:0/283327237 >> v1:172.21.15.164:6789/0 conn(0x7f8cc00b1960 legacy=0x7f8cc0071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.111+0000 7f8cc4c00640 1 -- 172.21.15.164:0/283327237 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00712a0 con 0x7f8cc00b1d40 2024-05-08T04:43:53.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ccca0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb801d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be4b8c31aa9d147 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cbf400640 1 -- 172.21.15.164:0/283327237 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80b2070 con 0x7f8cc00b1d40 2024-05-08T04:43:53.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cbf400640 1 -- 172.21.15.164:0/283327237 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb80a5930 con 0x7f8cc00b1d40 2024-05-08T04:43:53.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cbf400640 1 -- 172.21.15.164:0/283327237 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80a5d00 con 0x7f8cc00b1d40 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 msgr2=0x7f8cc00ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ccca0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009d90 tx=0x7f8cb801d2b0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- 172.21.15.164:0/283327237 shutdown_connections 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ccca0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 --2- 172.21.15.164:0/283327237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00cec40 0x7f8cc015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- 172.21.15.164:0/283327237 >> 172.21.15.164:0/283327237 conn(0x7f8cc006ac90 msgr2=0x7f8cc006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- 172.21.15.164:0/283327237 shutdown_connections 2024-05-08T04:43:53.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- 172.21.15.164:0/283327237 wait complete. 2024-05-08T04:43:53.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 Processor -- start 2024-05-08T04:43:53.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.115+0000 7f8cc5600640 1 -- start start 2024-05-08T04:43:53.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 0x7f8cc014b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b3d70 con 0x7f8cc00cec40 2024-05-08T04:43:53.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b3bf0 con 0x7f8cc00b1d40 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b3ef0 con 0x7f8cc00b1960 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42194/0 (socket says 172.21.15.164:42194) 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 -- 172.21.15.164:0/3599776916 learned_addr learned my addr 172.21.15.164:0/3599776916 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbfe00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 0x7f8cc014b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cb7400640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 -- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 msgr2=0x7f8cc014b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 0x7f8cc014b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 -- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 msgr2=0x7f8cc00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 -- 172.21.15.164:0/3599776916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc01508d0 con 0x7f8cc00cec40 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbfe00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 0x7f8cc014b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc4c00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8cb801a480 tx=0x7f8cb80195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80bc030 con 0x7f8cc00cec40 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb80ad440 con 0x7f8cc00cec40 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0150bb0 con 0x7f8cc00cec40 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80b2040 con 0x7f8cc00cec40 2024-05-08T04:43:53.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc015e9c0 con 0x7f8cc00cec40 2024-05-08T04:43:53.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc004ef30 con 0x7f8cc00cec40 2024-05-08T04:43:53.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.119+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f8cb80bf070 con 0x7f8cc00cec40 2024-05-08T04:43:53.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.123+0000 7f8cbcc00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 0x7f8c9c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.123+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8cb811e3f0 con 0x7f8cc00cec40 2024-05-08T04:43:53.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.123+0000 7f8cbfe00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 0x7f8c9c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.123+0000 7f8cbfe00640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 0x7f8c9c06a1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005890 tx=0x7f8cb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.127+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cb80c6070 con 0x7f8cc00cec40 2024-05-08T04:43:53.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8cc0167700 con 0x7f8cc00cec40 2024-05-08T04:43:53.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cbcc00640 1 -- 172.21.15.164:0/3599776916 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cb8016560 con 0x7f8cc00cec40 2024-05-08T04:43:53.280 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-05-08T04:43:53.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 msgr2=0x7f8c9c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 0x7f8c9c06a1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005890 tx=0x7f8cb0009040 comp rx=0 tx=0).stop 2024-05-08T04:43:53.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 msgr2=0x7f8cc00addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8cb801a480 tx=0x7f8cb80195f0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 shutdown_connections 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f8c9c067ce0 0x7f8c9c06a1a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cc00cec40 0x7f8cc00addd0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cc00b1d40 0x7f8cc00ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 --2- 172.21.15.164:0/3599776916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8cc00b1960 0x7f8cc014b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 >> 172.21.15.164:0/3599776916 conn(0x7f8cc006ac90 msgr2=0x7f8cc00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 shutdown_connections 2024-05-08T04:43:53.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.279+0000 7f8cc5600640 1 -- 172.21.15.164:0/3599776916 wait complete. 2024-05-08T04:43:53.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.435+0000 7fc68b400640 1 Processor -- start 2024-05-08T04:43:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.435+0000 7fc68b400640 1 -- start start 2024-05-08T04:43:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc68415bc50 0x7fc6840b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc6840d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6840b4d50 0x7fc6840d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840b42a0 con 0x7fc6840b4d50 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840cd4e0 con 0x7fc68415bc50 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc68b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cd7c0 con 0x7fc6840ccfe0 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6840b4d50 0x7fc6840d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6840b4d50 0x7fc6840d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42206/0 (socket says 172.21.15.164:42206) 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 -- 172.21.15.164:0/3265901600 learned_addr learned my addr 172.21.15.164:0/3265901600 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc688a00640 1 --2- 172.21.15.164:0/3265901600 >> v2:172.21.15.172:3300/0 conn(0x7fc68415bc50 0x7fc6840b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc683e00640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc6840d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 -- 172.21.15.164:0/3265901600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 msgr2=0x7fc6840d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc6840d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 -- 172.21.15.164:0/3265901600 >> v2:172.21.15.172:3300/0 conn(0x7fc68415bc50 msgr2=0x7fc6840b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 --2- 172.21.15.164:0/3265901600 >> v2:172.21.15.172:3300/0 conn(0x7fc68415bc50 0x7fc6840b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.439+0000 7fc689400640 1 -- 172.21.15.164:0/3265901600 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840cf590 con 0x7fc6840b4d50 2024-05-08T04:43:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc689400640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc6840d05b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc684071a00 tx=0x7fc67c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264458c5892f6424 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc683400640 1 -- 172.21.15.164:0/3265901600 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0a57b0 con 0x7fc6840b4d50 2024-05-08T04:43:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc683400640 1 -- 172.21.15.164:0/3265901600 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c0a5950 con 0x7fc6840b4d50 2024-05-08T04:43:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc683400640 1 -- 172.21.15.164:0/3265901600 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0a5d20 con 0x7fc6840b4d50 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- 172.21.15.164:0/3265901600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 msgr2=0x7fc6840d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc6840d05b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc684071a00 tx=0x7fc67c01c930 comp rx=0 tx=0).stop 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- 172.21.15.164:0/3265901600 shutdown_connections 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc6840d05b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- 172.21.15.164:0/3265901600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc6840d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- 172.21.15.164:0/3265901600 >> v2:172.21.15.172:3300/0 conn(0x7fc68415bc50 0x7fc6840b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- 172.21.15.164:0/3265901600 >> 172.21.15.164:0/3265901600 conn(0x7fc6840b34f0 msgr2=0x7fc6840b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- 172.21.15.164:0/3265901600 shutdown_connections 2024-05-08T04:43:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- 172.21.15.164:0/3265901600 wait complete. 2024-05-08T04:43:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 Processor -- start 2024-05-08T04:43:53.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- start start 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc68415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68415bc50 0x7fc68415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cea90 con 0x7fc6840b4d50 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840ce910 con 0x7fc68415bc50 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc68b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cec10 con 0x7fc6840ccfe0 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc688a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc688a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42214/0 (socket says 172.21.15.164:42214) 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.443+0000 7fc688a00640 1 -- 172.21.15.164:0/936382925 learned_addr learned my addr 172.21.15.164:0/936382925 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:53.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc683e00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc68415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc689400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68415bc50 0x7fc68415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 -- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 msgr2=0x7fc68415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc68415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 -- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68415bc50 msgr2=0x7fc68415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68415bc50 0x7fc68415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 -- 172.21.15.164:0/936382925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840c9db0 con 0x7fc6840b4d50 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc688a00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc67400e870 tx=0x7fc67400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674017070 con 0x7fc6840b4d50 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67400c940 con 0x7fc6840b4d50 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67400ccf0 con 0x7fc6840b4d50 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68406cc20 con 0x7fc6840b4d50 2024-05-08T04:43:53.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68406d1e0 con 0x7fc6840b4d50 2024-05-08T04:43:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc684069d60 con 0x7fc6840b4d50 2024-05-08T04:43:53.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.447+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fc67401a070 con 0x7fc6840b4d50 2024-05-08T04:43:53.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.451+0000 7fc680c00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 0x7fc66006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.451+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc674083f30 con 0x7fc6840b4d50 2024-05-08T04:43:53.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.451+0000 7fc683e00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 0x7fc66006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.451+0000 7fc683e00640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 0x7fc66006a1a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b2be0 tx=0x7fc66c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.455+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc674056090 con 0x7fc6840b4d50 2024-05-08T04:43:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6840b7940 con 0x7fc6840b4d50 2024-05-08T04:43:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc680c00640 1 -- 172.21.15.164:0/936382925 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc67405a130 con 0x7fc6840b4d50 2024-05-08T04:43:53.644 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:53.645 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:53.645 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 83 2024-05-08T04:43:53.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 msgr2=0x7fc66006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 0x7fc66006a1a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b2be0 tx=0x7fc66c009190 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 msgr2=0x7fc684151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc67400e870 tx=0x7fc67400ec70 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 shutdown_connections 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc660067ce0 0x7fc66006a1a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6840ccfe0 0x7fc68415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68415bc50 0x7fc68415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 --2- 172.21.15.164:0/936382925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6840b4d50 0x7fc684151570 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 >> 172.21.15.164:0/936382925 conn(0x7fc6840b34f0 msgr2=0x7fc6840b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 shutdown_connections 2024-05-08T04:43:53.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.643+0000 7fc68b400640 1 -- 172.21.15.164:0/936382925 wait complete. 2024-05-08T04:43:53.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:53.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.799+0000 7f0e44c00640 1 Processor -- start 2024-05-08T04:43:53.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.799+0000 7f0e44c00640 1 -- start start 2024-05-08T04:43:53.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e44c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e400aee30 0x7f0e400cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e44c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0e400cef80 0x7f0e4006c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e44c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4006e0e0 con 0x7f0e4015b390 2024-05-08T04:43:53.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e44c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e400cf3d0 con 0x7f0e400cef80 2024-05-08T04:43:53.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e44c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4015e8f0 con 0x7f0e400aee30 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0e400cef80 0x7f0e4006c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3de00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0e4015b390 0x7f0e4006d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58882/0 (socket says 172.21.15.164:58882) 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3de00640 1 -- 172.21.15.164:0/2230675521 learned_addr learned my addr 172.21.15.164:0/2230675521 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e35400640 1 --2- 172.21.15.164:0/2230675521 >> v2:172.21.15.172:3300/0 conn(0x7f0e400aee30 0x7f0e400cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3e800640 1 -- 172.21.15.164:0/2230675521 >> v2:172.21.15.172:3300/0 conn(0x7f0e400aee30 msgr2=0x7f0e400cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3e800640 1 --2- 172.21.15.164:0/2230675521 >> v2:172.21.15.172:3300/0 conn(0x7f0e400aee30 0x7f0e400cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3e800640 1 -- 172.21.15.164:0/2230675521 >> v1:172.21.15.164:6789/0 conn(0x7f0e4015b390 legacy=0x7f0e4006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3e800640 1 -- 172.21.15.164:0/2230675521 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4015ed80 con 0x7f0e400cef80 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.803+0000 7f0e3d400640 1 -- 172.21.15.164:0/2230675521 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051790000 0 0) 0x7f0e4006e0e0 con 0x7f0e4015b390 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e3e800640 1 --2- 172.21.15.164:0/2230675521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400cef80 0x7f0e4006c3d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0e400d0550 tx=0x7f0e3801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df9af48aee0cf91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e3d400640 1 -- 172.21.15.164:0/2230675521 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380b2070 con 0x7f0e400cef80 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e3d400640 1 -- 172.21.15.164:0/2230675521 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e380a5880 con 0x7f0e400cef80 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e3d400640 1 -- 172.21.15.164:0/2230675521 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380a5c50 con 0x7f0e400cef80 2024-05-08T04:43:53.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- 172.21.15.164:0/2230675521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400cef80 msgr2=0x7f0e4006c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 --2- 172.21.15.164:0/2230675521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400cef80 0x7f0e4006c3d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0e400d0550 tx=0x7f0e3801c930 comp rx=0 tx=0).stop 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- 172.21.15.164:0/2230675521 shutdown_connections 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 --2- 172.21.15.164:0/2230675521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400cef80 0x7f0e4006c3d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 --2- 172.21.15.164:0/2230675521 >> v2:172.21.15.172:3300/0 conn(0x7f0e400aee30 0x7f0e400cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- 172.21.15.164:0/2230675521 >> 172.21.15.164:0/2230675521 conn(0x7f0e4006ac90 msgr2=0x7f0e4006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- 172.21.15.164:0/2230675521 shutdown_connections 2024-05-08T04:43:53.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- 172.21.15.164:0/2230675521 wait complete. 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 Processor -- start 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.807+0000 7f0e44c00640 1 -- start start 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e400cef80 0x7f0e400b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4015b390 0x7f0e40159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4015b770 con 0x7f0e400aee30 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4015a8c0 con 0x7f0e4015b390 2024-05-08T04:43:53.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4006c910 con 0x7f0e400cef80 2024-05-08T04:43:53.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42222/0 (socket says 172.21.15.164:42222) 2024-05-08T04:43:53.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 -- 172.21.15.164:0/3057369852 learned_addr learned my addr 172.21.15.164:0/3057369852 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 -- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e400cef80 msgr2=0x7f0e400b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e400cef80 0x7f0e400b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 -- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4015b390 msgr2=0x7f0e40159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4015b390 0x7f0e40159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 -- 172.21.15.164:0/3057369852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400b38e0 con 0x7f0e400aee30 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e3de00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c520 tx=0x7f0e3000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30018070 con 0x7f0e400aee30 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300074e0 con 0x7f0e400aee30 2024-05-08T04:43:53.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300049f0 con 0x7f0e400aee30 2024-05-08T04:43:53.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4014e1c0 con 0x7f0e400aee30 2024-05-08T04:43:53.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4014e780 con 0x7f0e400aee30 2024-05-08T04:43:53.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e40069d60 con 0x7f0e400aee30 2024-05-08T04:43:53.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.811+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f0e30004b90 con 0x7f0e400aee30 2024-05-08T04:43:53.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.815+0000 7f0e36a00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 0x7f0e1c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:53.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.815+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f0e30083010 con 0x7f0e400aee30 2024-05-08T04:43:53.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.815+0000 7f0e35400640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 0x7f0e1c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:53.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.815+0000 7f0e35400640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 0x7f0e1c06a1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e401f9cd0 tx=0x7f0e28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:53.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.819+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e30055170 con 0x7f0e400aee30 2024-05-08T04:43:53.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.971+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e40071f30 con 0x7f0e400aee30 2024-05-08T04:43:53.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.971+0000 7f0e36a00640 1 -- 172.21.15.164:0/3057369852 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f0e30059210 con 0x7f0e400aee30 2024-05-08T04:43:53.973 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:53.973 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 msgr2=0x7f0e1c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 0x7f0e1c06a1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e401f9cd0 tx=0x7f0e28008040 comp rx=0 tx=0).stop 2024-05-08T04:43:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 msgr2=0x7f0e40149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c520 tx=0x7f0e3000c9e0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 shutdown_connections 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f0e1c067ce0 0x7f0e1c06a1a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e400cef80 0x7f0e400b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4015b390 0x7f0e40159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 --2- 172.21.15.164:0/3057369852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e400aee30 0x7f0e40149150 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 >> 172.21.15.164:0/3057369852 conn(0x7f0e4006ac90 msgr2=0x7f0e400cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 shutdown_connections 2024-05-08T04:43:53.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:53.975+0000 7f0e44c00640 1 -- 172.21.15.164:0/3057369852 wait complete. 2024-05-08T04:43:53.995 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:43:53.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:54.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.127+0000 7f2a7fc00640 1 Processor -- start 2024-05-08T04:43:54.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.127+0000 7f2a7fc00640 1 -- start start 2024-05-08T04:43:54.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7fc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 0x7f2a780cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7fc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7806c770 con 0x7f2a7815ab60 2024-05-08T04:43:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7fc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a780d0060 con 0x7f2a780b26c0 2024-05-08T04:43:54.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7806ca50 con 0x7f2a780cfbd0 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a77e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 0x7f2a780cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7d200640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2a7815ab60 0x7f2a78072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:59208/0 (socket says 172.21.15.164:59208) 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:45912/0 (socket says 172.21.15.164:45912) 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7d200640 1 -- 172.21.15.164:0/2156287827 learned_addr learned my addr 172.21.15.164:0/2156287827 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 -- 172.21.15.164:0/2156287827 >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 msgr2=0x7f2a780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 --2- 172.21.15.164:0/2156287827 >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 0x7f2a780cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 -- 172.21.15.164:0/2156287827 >> v1:172.21.15.172:6789/0 conn(0x7f2a7815ab60 legacy=0x7f2a78072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a7dc00640 1 -- 172.21.15.164:0/2156287827 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7806cd30 con 0x7f2a780cfbd0 2024-05-08T04:43:54.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.131+0000 7f2a77e00640 1 --2- 172.21.15.164:0/2156287827 >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 0x7f2a780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:54.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7dc00640 1 --2- 172.21.15.164:0/2156287827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67e16ffc63b74fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a77400640 1 -- 172.21.15.164:0/2156287827 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b0070 con 0x7f2a780cfbd0 2024-05-08T04:43:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a77400640 1 -- 172.21.15.164:0/2156287827 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a700ab600 con 0x7f2a780cfbd0 2024-05-08T04:43:54.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a77400640 1 -- 172.21.15.164:0/2156287827 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700ab9d0 con 0x7f2a780cfbd0 2024-05-08T04:43:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 -- 172.21.15.164:0/2156287827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 msgr2=0x7f2a780cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/2156287827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 -- 172.21.15.164:0/2156287827 shutdown_connections 2024-05-08T04:43:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/2156287827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a780cd910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/2156287827 >> v2:172.21.15.172:3300/0 conn(0x7f2a780b26c0 0x7f2a780cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 -- 172.21.15.164:0/2156287827 >> 172.21.15.164:0/2156287827 conn(0x7f2a7806ac90 msgr2=0x7f2a7806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 -- 172.21.15.164:0/2156287827 shutdown_connections 2024-05-08T04:43:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 -- 172.21.15.164:0/2156287827 wait complete. 2024-05-08T04:43:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.135+0000 7f2a7fc00640 1 Processor -- start 2024-05-08T04:43:54.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- start start 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a780b26c0 0x7f2a78154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a7815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780aeeb0 con 0x7f2a7815ab60 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780aed30 con 0x7f2a780b26c0 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78167970 con 0x7f2a780cfbd0 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42228/0 (socket says 172.21.15.164:42228) 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 -- 172.21.15.164:0/1475511451 learned_addr learned my addr 172.21.15.164:0/1475511451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:54.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7d200640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a780b26c0 0x7f2a78154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a77e00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a7815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 -- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 msgr2=0x7f2a7815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a7815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 -- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a780b26c0 msgr2=0x7f2a78154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a780b26c0 0x7f2a78154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 -- 172.21.15.164:0/1475511451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78158b70 con 0x7f2a7815ab60 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a77e00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a7815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7dc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f2a700ae040 tx=0x7f2a70019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b0040 con 0x7f2a7815ab60 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a700ba070 con 0x7f2a7815ab60 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a78158e30 con 0x7f2a7815ab60 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70006e20 con 0x7f2a7815ab60 2024-05-08T04:43:54.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a781593f0 con 0x7f2a7815ab60 2024-05-08T04:43:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.139+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a78069d60 con 0x7f2a7815ab60 2024-05-08T04:43:54.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.143+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f2a700b4630 con 0x7f2a7815ab60 2024-05-08T04:43:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.143+0000 7f2a74c00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 0x7f2a5806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.143+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2a7011da80 con 0x7f2a7815ab60 2024-05-08T04:43:54.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.143+0000 7f2a7d200640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 0x7f2a5806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.143+0000 7f2a7d200640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 0x7f2a5806a1a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a680012f0 tx=0x7f2a680019a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.147+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a70016090 con 0x7f2a7815ab60 2024-05-08T04:43:54.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.331+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a780ad890 con 0x7f2a7815ab60 2024-05-08T04:43:54.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.331+0000 7f2a74c00640 1 -- 172.21.15.164:0/1475511451 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f2a700f0170 con 0x7f2a7815ab60 2024-05-08T04:43:54.334 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:54.334 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:54.334 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 83 2024-05-08T04:43:54.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 msgr2=0x7f2a5806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 0x7f2a5806a1a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a680012f0 tx=0x7f2a680019a0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 msgr2=0x7f2a7813e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f2a700ae040 tx=0x7f2a70019240 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 shutdown_connections 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f2a58067ce0 0x7f2a5806a1a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a780cfbd0 0x7f2a7815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a7815ab60 0x7f2a7813e850 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 --2- 172.21.15.164:0/1475511451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a780b26c0 0x7f2a78154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 >> 172.21.15.164:0/1475511451 conn(0x7f2a7806ac90 msgr2=0x7f2a780b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 shutdown_connections 2024-05-08T04:43:54.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.335+0000 7f2a7fc00640 1 -- 172.21.15.164:0/1475511451 wait complete. 2024-05-08T04:43:54.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:43:54.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.491+0000 7f3970c00640 1 Processor -- start 2024-05-08T04:43:54.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.491+0000 7f3970c00640 1 -- start start 2024-05-08T04:43:54.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3970c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f396c15ba70 0x7f396c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3970c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f396c0cfda0 0x7f396c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3970c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c0cd9f0 con 0x7f396c15be50 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3970c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396c15fe70 con 0x7f396c0cfda0 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3970c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396c0cdcd0 con 0x7f396c15ba70 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f396c0cfda0 0x7f396c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f396c0cfda0 0x7f396c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42232/0 (socket says 172.21.15.164:42232) 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 -- 172.21.15.164:0/3357381005 learned_addr learned my addr 172.21.15.164:0/3357381005 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:54.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3969e00640 1 --2- 172.21.15.164:0/3357381005 >> v2:172.21.15.172:3300/0 conn(0x7f396c15ba70 0x7f396c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 -- 172.21.15.164:0/3357381005 >> v2:172.21.15.172:3300/0 conn(0x7f396c15ba70 msgr2=0x7f396c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:54.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 --2- 172.21.15.164:0/3357381005 >> v2:172.21.15.172:3300/0 conn(0x7f396c15ba70 0x7f396c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 -- 172.21.15.164:0/3357381005 >> v1:172.21.15.172:6789/0 conn(0x7f396c15be50 legacy=0x7f396c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f396a800640 1 -- 172.21.15.164:0/3357381005 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c15f820 con 0x7f396c0cfda0 2024-05-08T04:43:54.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.495+0000 7f3968a00640 1 -- 172.21.15.164:0/3357381005 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283792620 0 0) 0x7f396c0cd9f0 con 0x7f396c15be50 2024-05-08T04:43:54.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f396a800640 1 --2- 172.21.15.164:0/3357381005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c0cfda0 0x7f396c0b31a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f396c06bc70 tx=0x7f396401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f41567f7826a111 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3968a00640 1 -- 172.21.15.164:0/3357381005 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640a5790 con 0x7f396c0cfda0 2024-05-08T04:43:54.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3968a00640 1 -- 172.21.15.164:0/3357381005 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39640a5930 con 0x7f396c0cfda0 2024-05-08T04:43:54.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3968a00640 1 -- 172.21.15.164:0/3357381005 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640a5d00 con 0x7f396c0cfda0 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- 172.21.15.164:0/3357381005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c0cfda0 msgr2=0x7f396c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- 172.21.15.164:0/3357381005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c0cfda0 0x7f396c0b31a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f396c06bc70 tx=0x7f396401c9c0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- 172.21.15.164:0/3357381005 shutdown_connections 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- 172.21.15.164:0/3357381005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c0cfda0 0x7f396c0b31a0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- 172.21.15.164:0/3357381005 >> v2:172.21.15.172:3300/0 conn(0x7f396c15ba70 0x7f396c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- 172.21.15.164:0/3357381005 >> 172.21.15.164:0/3357381005 conn(0x7f396c06ac90 msgr2=0x7f396c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- 172.21.15.164:0/3357381005 shutdown_connections 2024-05-08T04:43:54.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- 172.21.15.164:0/3357381005 wait complete. 2024-05-08T04:43:54.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 Processor -- start 2024-05-08T04:43:54.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- start start 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f396c0cfda0 0x7f396c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f396c15ba70 0x7f396c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c168d40 con 0x7f396c15be50 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c168bc0 con 0x7f396c15ba70 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3970c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c168ec0 con 0x7f396c0cfda0 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f396a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f396a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42236/0 (socket says 172.21.15.164:42236) 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f396a800640 1 -- 172.21.15.164:0/2109760388 learned_addr learned my addr 172.21.15.164:0/2109760388 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3969400640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f396c15ba70 0x7f396c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.499+0000 7f3969e00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f396c0cfda0 0x7f396c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 -- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f396c0cfda0 msgr2=0x7f396c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f396c0cfda0 0x7f396c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 -- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f396c15ba70 msgr2=0x7f396c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f396c15ba70 0x7f396c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 -- 172.21.15.164:0/2109760388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c15df30 con 0x7f396c15be50 2024-05-08T04:43:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f396a800640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3964009e20 tx=0x7f3964019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640b8670 con 0x7f396c15be50 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39640be070 con 0x7f396c15be50 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c15e210 con 0x7f396c15be50 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640b8a50 con 0x7f396c15be50 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c15e7d0 con 0x7f396c15be50 2024-05-08T04:43:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c069d60 con 0x7f396c15be50 2024-05-08T04:43:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f39640b8bf0 con 0x7f396c15be50 2024-05-08T04:43:54.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.503+0000 7f3962000640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 0x7f394c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.507+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f396411f1a0 con 0x7f396c15be50 2024-05-08T04:43:54.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.507+0000 7f3969e00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 0x7f394c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.507+0000 7f3969e00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 0x7f394c06a1a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f395c009b60 tx=0x7f395c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.511+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39640165e0 con 0x7f396c15be50 2024-05-08T04:43:54.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.699+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f396c06e210 con 0x7f396c15be50 2024-05-08T04:43:54.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.699+0000 7f3962000640 1 -- 172.21.15.164:0/2109760388 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f39640f13a0 con 0x7f396c15be50 2024-05-08T04:43:54.702 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:54.702 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T04:43:30.835958+0000","modified":"2024-05-08T04:43:33.847549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6837/333884670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":333884670},{"type":"v1","addr":"172.21.15.164:6837","nonce":333884670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-08T04:43:54.702 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 83 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 msgr2=0x7f394c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 0x7f394c06a1a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f395c009b60 tx=0x7f395c009190 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 msgr2=0x7f396c14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3964009e20 tx=0x7f3964019760 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 shutdown_connections 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f394c067ce0 0x7f394c06a1a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396c15be50 0x7f396c14f850 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f396c15ba70 0x7f396c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 --2- 172.21.15.164:0/2109760388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f396c0cfda0 0x7f396c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 >> 172.21.15.164:0/2109760388 conn(0x7f396c06ac90 msgr2=0x7f396c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:54.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 shutdown_connections 2024-05-08T04:43:54.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.703+0000 7f3970c00640 1 -- 172.21.15.164:0/2109760388 wait complete. 2024-05-08T04:43:54.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:43:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.855+0000 7f4721e00640 1 Processor -- start 2024-05-08T04:43:54.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 -- start start 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f471c15b9f0 0x7f471c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f471c0cfa20 0x7f471c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c15f400 con 0x7f471c06ef60 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c15f6e0 con 0x7f471c15b9f0 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c15f9c0 con 0x7f471c0cfa20 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f471c15b9f0 0x7f471c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f471c15b9f0 0x7f471c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42242/0 (socket says 172.21.15.164:42242) 2024-05-08T04:43:54.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 -- 172.21.15.164:0/1783073538 learned_addr learned my addr 172.21.15.164:0/1783073538 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:54.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4721400640 1 --2- 172.21.15.164:0/1783073538 >> v2:172.21.15.172:3300/0 conn(0x7f471c0cfa20 0x7f471c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 -- 172.21.15.164:0/1783073538 >> v2:172.21.15.172:3300/0 conn(0x7f471c0cfa20 msgr2=0x7f471c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:54.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 --2- 172.21.15.164:0/1783073538 >> v2:172.21.15.172:3300/0 conn(0x7f471c0cfa20 0x7f471c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 -- 172.21.15.164:0/1783073538 >> v1:172.21.15.172:6789/0 conn(0x7f471c06ef60 legacy=0x7f471c06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.859+0000 7f4720a00640 1 -- 172.21.15.164:0/1783073538 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c15fca0 con 0x7f471c15b9f0 2024-05-08T04:43:54.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4720a00640 1 --2- 172.21.15.164:0/1783073538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c06ac90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f471c06bb90 tx=0x7f470c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4f21a27719126 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f471be00640 1 -- 172.21.15.164:0/1783073538 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0a6040 con 0x7f471c15b9f0 2024-05-08T04:43:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f471be00640 1 -- 172.21.15.164:0/1783073538 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c0af070 con 0x7f471c15b9f0 2024-05-08T04:43:54.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f471be00640 1 -- 172.21.15.164:0/1783073538 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0aa860 con 0x7f471c15b9f0 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- 172.21.15.164:0/1783073538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 msgr2=0x7f471c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 --2- 172.21.15.164:0/1783073538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c06ac90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f471c06bb90 tx=0x7f470c01c4e0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- 172.21.15.164:0/1783073538 shutdown_connections 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 --2- 172.21.15.164:0/1783073538 >> v2:172.21.15.172:3300/0 conn(0x7f471c0cfa20 0x7f471c06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 --2- 172.21.15.164:0/1783073538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c06ac90 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- 172.21.15.164:0/1783073538 >> 172.21.15.164:0/1783073538 conn(0x7f471c066fa0 msgr2=0x7f471c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- 172.21.15.164:0/1783073538 shutdown_connections 2024-05-08T04:43:54.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- 172.21.15.164:0/1783073538 wait complete. 2024-05-08T04:43:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 Processor -- start 2024-05-08T04:43:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.863+0000 7f4721e00640 1 -- start start 2024-05-08T04:43:54.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfa20 0x7f471c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c168b30 con 0x7f471c15b9f0 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1689b0 con 0x7f471c06ef60 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c168cb0 con 0x7f471c0cfa20 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.164:35344/0 (socket says 172.21.15.164:35344) 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 -- 172.21.15.164:0/3959633409 learned_addr learned my addr 172.21.15.164:0/3959633409 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4713e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfa20 0x7f471c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 -- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfa20 msgr2=0x7f471c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfa20 0x7f471c0a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 -- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 msgr2=0x7f471c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:54.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 -- 172.21.15.164:0/3959633409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c1fa390 con 0x7f471c06ef60 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721400640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4720a00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f470c01a110 tx=0x7f470c01ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0bd3f0 con 0x7f471c06ef60 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c0bd590 con 0x7f471c06ef60 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c1fa670 con 0x7f471c06ef60 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0bd940 con 0x7f471c06ef60 2024-05-08T04:43:54.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c1fac30 con 0x7f471c06ef60 2024-05-08T04:43:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.867+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c04ef30 con 0x7f471c06ef60 2024-05-08T04:43:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.871+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f470c0bdae0 con 0x7f471c06ef60 2024-05-08T04:43:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.871+0000 7f4719600640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 0x7f46f806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.871+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f470c11d550 con 0x7f471c06ef60 2024-05-08T04:43:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.871+0000 7f4713e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 0x7f46f806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.871+0000 7f4713e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 0x7f46f806a1d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f471c15f0b0 tx=0x7f4704007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:54.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:54.875+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f470c0160d0 con 0x7f471c06ef60 2024-05-08T04:43:55.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.027+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f471c0af330 con 0x7f471c06ef60 2024-05-08T04:43:55.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4719600640 1 -- 172.21.15.164:0/3959633409 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f470c0efae0 con 0x7f471c06ef60 2024-05-08T04:43:55.032 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:43:55.032 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":126,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:32.452656+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-08T04:43:29.191800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-08T04:43:29.875561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 msgr2=0x7f46f806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 0x7f46f806a1d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f471c15f0b0 tx=0x7f4704007e40 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 msgr2=0x7f471c14fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f470c01a110 tx=0x7f470c01ad20 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 shutdown_connections 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f46f8067d10 0x7f46f806a1d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfa20 0x7f471c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f471c15b9f0 0x7f471c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 --2- 172.21.15.164:0/3959633409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f471c06ef60 0x7f471c14fd10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 >> 172.21.15.164:0/3959633409 conn(0x7f471c066fa0 msgr2=0x7f471c159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 shutdown_connections 2024-05-08T04:43:55.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.031+0000 7f4721e00640 1 -- 172.21.15.164:0/3959633409 wait complete. 2024-05-08T04:43:55.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T04:43:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.187+0000 7f7adf000640 1 Processor -- start 2024-05-08T04:43:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.187+0000 7f7adf000640 1 -- start start 2024-05-08T04:43:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7adf000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ad815ba70 0x7f7ad8160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7adf000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cd9f0 con 0x7f7ad815be50 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7adf000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cdcd0 con 0x7f7ad80cfda0 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7adf000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad815fe70 con 0x7f7ad815ba70 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ad815ba70 0x7f7ad8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7ad815be50 0x7f7ad80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58898/0 (socket says 172.21.15.164:58898) 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7400640 1 -- 172.21.15.164:0/3746104577 learned_addr learned my addr 172.21.15.164:0/3746104577 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7e00640 1 -- 172.21.15.164:0/3746104577 >> v1:172.21.15.172:6789/0 conn(0x7f7ad80cfda0 legacy=0x7f7ad80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7e00640 1 -- 172.21.15.164:0/3746104577 >> v1:172.21.15.164:6789/0 conn(0x7f7ad815be50 legacy=0x7f7ad80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad7e00640 1 -- 172.21.15.164:0/3746104577 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad815f820 con 0x7f7ad815ba70 2024-05-08T04:43:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.191+0000 7f7ad6a00640 1 -- 172.21.15.164:0/3746104577 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92742709 0 0) 0x7f7ad80cd9f0 con 0x7f7ad815be50 2024-05-08T04:43:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7ad7e00640 1 --2- 172.21.15.164:0/3746104577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad815ba70 0x7f7ad8160300 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009d90 tx=0x7f7ac801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0dc620185de6f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7ad6a00640 1 -- 172.21.15.164:0/3746104577 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b2070 con 0x7f7ad815ba70 2024-05-08T04:43:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7ad6a00640 1 -- 172.21.15.164:0/3746104577 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80a58d0 con 0x7f7ad815ba70 2024-05-08T04:43:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7ad6a00640 1 -- 172.21.15.164:0/3746104577 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80a5ca0 con 0x7f7ad815ba70 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- 172.21.15.164:0/3746104577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad815ba70 msgr2=0x7f7ad8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 --2- 172.21.15.164:0/3746104577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad815ba70 0x7f7ad8160300 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009d90 tx=0x7f7ac801c900 comp rx=0 tx=0).stop 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- 172.21.15.164:0/3746104577 shutdown_connections 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 --2- 172.21.15.164:0/3746104577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad815ba70 0x7f7ad8160300 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- 172.21.15.164:0/3746104577 >> 172.21.15.164:0/3746104577 conn(0x7f7ad806ac90 msgr2=0x7f7ad806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- 172.21.15.164:0/3746104577 shutdown_connections 2024-05-08T04:43:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- 172.21.15.164:0/3746104577 wait complete. 2024-05-08T04:43:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 Processor -- start 2024-05-08T04:43:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- start start 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad815ba70 0x7f7ad80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad815be50 0x7f7ad80addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad8169f60 con 0x7f7ad80cfda0 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad8169de0 con 0x7f7ad815be50 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.195+0000 7f7adf000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad816a0e0 con 0x7f7ad815ba70 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42264/0 (socket says 172.21.15.164:42264) 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 -- 172.21.15.164:0/326773801 learned_addr learned my addr 172.21.15.164:0/326773801 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7add000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad815be50 0x7f7ad80addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7400640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad815ba70 0x7f7ad80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 -- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad815ba70 msgr2=0x7f7ad80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad815ba70 0x7f7ad80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 -- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad815be50 msgr2=0x7f7ad80addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad815be50 0x7f7ad80addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 -- 172.21.15.164:0/326773801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad81fb060 con 0x7f7ad80cfda0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7ad7e00640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8019510 tx=0x7f7ac80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80bd020 con 0x7f7ad80cfda0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80ad740 con 0x7f7ad80cfda0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b2040 con 0x7f7ad80cfda0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad81fb340 con 0x7f7ad80cfda0 2024-05-08T04:43:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad81fb900 con 0x7f7ad80cfda0 2024-05-08T04:43:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad8069d60 con 0x7f7ad80cfda0 2024-05-08T04:43:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f7ac80c0020 con 0x7f7ad80cfda0 2024-05-08T04:43:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.199+0000 7f7acfe00640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 0x7f7ab806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.203+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7ac811edd0 con 0x7f7ad80cfda0 2024-05-08T04:43:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.203+0000 7f7ad7400640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 0x7f7ab806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.203+0000 7f7ad7400640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 0x7f7ab806a1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80b0d60 tx=0x7f7ac4007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.207+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ac80c7030 con 0x7f7ad80cfda0 2024-05-08T04:43:55.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.391+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ad806cd10 con 0x7f7ad80cfda0 2024-05-08T04:43:55.394 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-05-08T04:43:55.391+0000 7fd327e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T04:43:55.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.395+0000 7f7acfe00640 1 -- 172.21.15.164:0/326773801 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7ac8016520 con 0x7f7ad80cfda0 2024-05-08T04:43:55.396 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T04:43:55.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 msgr2=0x7f7ab806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 0x7f7ab806a1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80b0d60 tx=0x7f7ac4007df0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 msgr2=0x7f7ad814a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8019510 tx=0x7f7ac80196a0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 shutdown_connections 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f7ab8067ce0 0x7f7ab806a1a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ad815ba70 0x7f7ad80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad815be50 0x7f7ad80addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 --2- 172.21.15.164:0/326773801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ad80cfda0 0x7f7ad814a160 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 >> 172.21.15.164:0/326773801 conn(0x7f7ad806ac90 msgr2=0x7f7ad806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 shutdown_connections 2024-05-08T04:43:55.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.399+0000 7f7adf000640 1 -- 172.21.15.164:0/326773801 wait complete. 2024-05-08T04:43:55.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T04:43:55.465 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2024-05-08T04:43:55.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 Processor -- start 2024-05-08T04:43:55.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 -- start start 2024-05-08T04:43:55.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2c815ab10 0x7fe2c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c815e870 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c815eb50 con 0x7fe2c80cee90 2024-05-08T04:43:55.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cfa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c806bb80 con 0x7fe2c815ab10 2024-05-08T04:43:55.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cd000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2c815ab10 0x7fe2c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2bfe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe2c80ae6e0 0x7fe2c80ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58904/0 (socket says 172.21.15.164:58904) 2024-05-08T04:43:55.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2bfe00640 1 -- 172.21.15.164:0/30866979 learned_addr learned my addr 172.21.15.164:0/30866979 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:55.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cd000640 1 -- 172.21.15.164:0/30866979 >> v1:172.21.15.172:6789/0 conn(0x7fe2c80cee90 legacy=0x7fe2c80ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cd000640 1 -- 172.21.15.164:0/30866979 >> v1:172.21.15.164:6789/0 conn(0x7fe2c80ae6e0 legacy=0x7fe2c80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.567+0000 7fe2cd000640 1 -- 172.21.15.164:0/30866979 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c815ee30 con 0x7fe2c815ab10 2024-05-08T04:43:55.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cd000640 1 --2- 172.21.15.164:0/30866979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c815ab10 0x7fe2c806cce0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80097c0 tx=0x7fe2b801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb6fe97ef53434 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2c7e00640 1 -- 172.21.15.164:0/30866979 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80b1070 con 0x7fe2c815ab10 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2c7e00640 1 -- 172.21.15.164:0/30866979 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b80a8050 con 0x7fe2c815ab10 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2c7e00640 1 -- 172.21.15.164:0/30866979 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80ac860 con 0x7fe2c815ab10 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 -- 172.21.15.164:0/30866979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c815ab10 msgr2=0x7fe2c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/30866979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c815ab10 0x7fe2c806cce0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80097c0 tx=0x7fe2b801c3e0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 -- 172.21.15.164:0/30866979 shutdown_connections 2024-05-08T04:43:55.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/30866979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c815ab10 0x7fe2c806cce0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 -- 172.21.15.164:0/30866979 >> 172.21.15.164:0/30866979 conn(0x7fe2c80cf770 msgr2=0x7fe2c80cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:55.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 -- 172.21.15.164:0/30866979 shutdown_connections 2024-05-08T04:43:55.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.571+0000 7fe2cfa00640 1 -- 172.21.15.164:0/30866979 wait complete. 2024-05-08T04:43:55.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 Processor -- start 2024-05-08T04:43:55.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- start start 2024-05-08T04:43:55.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2c80cee90 0x7fe2c815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2c815ab10 0x7fe2c8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80cd670 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815aef0 con 0x7fe2c80cee90 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80cd7f0 con 0x7fe2c815ab10 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42294/0 (socket says 172.21.15.164:42294) 2024-05-08T04:43:55.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 -- 172.21.15.164:0/2316813575 learned_addr learned my addr 172.21.15.164:0/2316813575 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2bfe00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2c80cee90 0x7fe2c815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cda00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2c815ab10 0x7fe2c8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 -- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2c815ab10 msgr2=0x7fe2c8157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2c815ab10 0x7fe2c8157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 -- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2c80cee90 msgr2=0x7fe2c815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:55.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2c80cee90 0x7fe2c815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 -- 172.21.15.164:0/2316813575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8157a70 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cd000640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8018ff0 tx=0x7fe2b801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80b1040 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b80bd070 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80b7660 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c8158d90 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c8157fd0 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.575+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c8072d30 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.579+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fe2b80b7800 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.579+0000 7fe2c5600640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 0x7fe2a406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:55.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.579+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5879+0+0 (secure 0 0 0) 0x7fe2b811e170 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.579+0000 7fe2bfe00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 0x7fe2a406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:55.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.579+0000 7fe2bfe00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 0x7fe2a406a1a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe2c815a240 tx=0x7fe2b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:55.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.583+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2b80c7020 con 0x7fe2c80ae6e0 2024-05-08T04:43:55.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:55.771+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2c804f010 con 0x7fe2c80ae6e0 2024-05-08T04:43:56.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.403+0000 7fe2c5600640 1 -- 172.21.15.164:0/2316813575 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe2c8157a70 con 0x7fe2c80ae6e0 2024-05-08T04:43:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 msgr2=0x7fe2a406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 0x7fe2a406a1a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe2c815a240 tx=0x7fe2b0009040 comp rx=0 tx=0).stop 2024-05-08T04:43:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 msgr2=0x7fe2c81522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8018ff0 tx=0x7fe2b801ac20 comp rx=0 tx=0).stop 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 shutdown_connections 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe2a4067ce0 0x7fe2a406a1a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe2c815ab10 0x7fe2c8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2c80cee90 0x7fe2c815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 --2- 172.21.15.164:0/2316813575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2c80ae6e0 0x7fe2c81522c0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 >> 172.21.15.164:0/2316813575 conn(0x7fe2c80cf770 msgr2=0x7fe2c80af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 shutdown_connections 2024-05-08T04:43:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.407+0000 7fe2cfa00640 1 -- 172.21.15.164:0/2316813575 wait complete. 2024-05-08T04:43:56.428 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T04:43:56.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 Processor -- start 2024-05-08T04:43:56.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 -- start start 2024-05-08T04:43:56.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f413015f2b0 0x7f41300d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4130070280 con 0x7f41300b3fe0 2024-05-08T04:43:56.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413015f690 con 0x7f413015f2b0 2024-05-08T04:43:56.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f4134c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cff80 con 0x7f41300cd560 2024-05-08T04:43:56.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:45992/0 (socket says 172.21.15.164:45992) 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 -- 172.21.15.164:0/3280667675 learned_addr learned my addr 172.21.15.164:0/3280667675 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412de00640 1 --2- 172.21.15.164:0/3280667675 >> v2:172.21.15.172:3300/0 conn(0x7f413015f2b0 0x7f41300d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 -- 172.21.15.164:0/3280667675 >> v2:172.21.15.172:3300/0 conn(0x7f413015f2b0 msgr2=0x7f41300d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 --2- 172.21.15.164:0/3280667675 >> v2:172.21.15.172:3300/0 conn(0x7f413015f2b0 0x7f41300d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 -- 172.21.15.164:0/3280667675 >> v1:172.21.15.172:6789/0 conn(0x7f41300b3fe0 legacy=0x7f41300cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.611+0000 7f412e800640 1 -- 172.21.15.164:0/3280667675 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130070560 con 0x7f41300cd560 2024-05-08T04:43:56.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f412e800640 1 --2- 172.21.15.164:0/3280667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa1d2a38390cf086 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:56.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f412ca00640 1 -- 172.21.15.164:0/3280667675 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280a95f0 con 0x7f41300cd560 2024-05-08T04:43:56.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f412ca00640 1 -- 172.21.15.164:0/3280667675 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41280a9790 con 0x7f41300cd560 2024-05-08T04:43:56.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f412ca00640 1 -- 172.21.15.164:0/3280667675 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280a9b60 con 0x7f41300cd560 2024-05-08T04:43:56.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 -- 172.21.15.164:0/3280667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 msgr2=0x7f413015f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 --2- 172.21.15.164:0/3280667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412801c530 comp rx=0 tx=0).stop 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 -- 172.21.15.164:0/3280667675 shutdown_connections 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 --2- 172.21.15.164:0/3280667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41300cd560 0x7f413015f8b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 --2- 172.21.15.164:0/3280667675 >> v2:172.21.15.172:3300/0 conn(0x7f413015f2b0 0x7f41300d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 -- 172.21.15.164:0/3280667675 >> 172.21.15.164:0/3280667675 conn(0x7f41300b34f0 msgr2=0x7f41300b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 -- 172.21.15.164:0/3280667675 shutdown_connections 2024-05-08T04:43:56.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.615+0000 7f4134c00640 1 -- 172.21.15.164:0/3280667675 wait complete. 2024-05-08T04:43:56.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 Processor -- start 2024-05-08T04:43:56.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 -- start start 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41300cd560 0x7f41300c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f413015f2b0 0x7f41300c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cdb70 con 0x7f41300b3fe0 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cd9f0 con 0x7f41300cd560 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f4134c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4130168bf0 con 0x7f413015f2b0 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42306/0 (socket says 172.21.15.164:42306) 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 -- 172.21.15.164:0/3334781016 learned_addr learned my addr 172.21.15.164:0/3334781016 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412d400640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41300cd560 0x7f41300c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412e800640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f413015f2b0 0x7f41300c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 -- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f413015f2b0 msgr2=0x7f41300c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f413015f2b0 0x7f41300c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 -- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41300cd560 msgr2=0x7f41300c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41300cd560 0x7f41300c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 -- 172.21.15.164:0/3334781016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413015d460 con 0x7f41300b3fe0 2024-05-08T04:43:56.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.619+0000 7f412de00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4120009d10 tx=0x7f412000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120018020 con 0x7f41300b3fe0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120014480 con 0x7f41300b3fe0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413015d740 con 0x7f41300b3fe0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120014830 con 0x7f41300b3fe0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413015dd00 con 0x7f41300b3fe0 2024-05-08T04:43:56.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413009fae0 con 0x7f41300b3fe0 2024-05-08T04:43:56.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f412001b020 con 0x7f41300b3fe0 2024-05-08T04:43:56.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 0x7f411006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:56.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5879+0+0 (secure 0 0 0) 0x7f41200847e0 con 0x7f41300b3fe0 2024-05-08T04:43:56.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.623+0000 7f412d400640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 0x7f411006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:56.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.627+0000 7f412d400640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 0x7f411006a1a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f411c001650 tx=0x7f411c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:56.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.627+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4120022070 con 0x7f41300b3fe0 2024-05-08T04:43:56.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:56.783+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f413006bda0 con 0x7f41300b3fe0 2024-05-08T04:43:57.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.627+0000 7f4126000640 1 -- 172.21.15.164:0/3334781016 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4120010040 con 0x7f41300b3fe0 2024-05-08T04:43:57.628 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-05-08T04:43:57.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 msgr2=0x7f411006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 0x7f411006a1a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f411c001650 tx=0x7f411c009040 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 msgr2=0x7f4130155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4120009d10 tx=0x7f412000c490 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 shutdown_connections 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4110067ce0 0x7f411006a1a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f413015f2b0 0x7f41300c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41300cd560 0x7f41300c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 --2- 172.21.15.164:0/3334781016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41300b3fe0 0x7f4130155080 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 >> 172.21.15.164:0/3334781016 conn(0x7f41300b34f0 msgr2=0x7f41300b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 shutdown_connections 2024-05-08T04:43:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.631+0000 7f4134c00640 1 -- 172.21.15.164:0/3334781016 wait complete. 2024-05-08T04:43:57.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T04:43:57.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 Processor -- start 2024-05-08T04:43:57.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 -- start start 2024-05-08T04:43:57.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd880ce930 0x7fcd88168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:57.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd880cfe60 con 0x7fcd880cdb10 2024-05-08T04:43:57.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd880d0140 con 0x7fcd880d0c20 2024-05-08T04:43:57.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd8ca00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd880d0790 con 0x7fcd880ce930 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd85e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcd880cdb10 0x7fcd8806efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58906/0 (socket says 172.21.15.164:58906) 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd85e00640 1 -- 172.21.15.164:0/262668126 learned_addr learned my addr 172.21.15.164:0/262668126 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd86800640 1 --2- 172.21.15.164:0/262668126 >> v2:172.21.15.164:3300/0 conn(0x7fcd880ce930 0x7fcd88168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd84a00640 1 -- 172.21.15.164:0/262668126 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747400928 0 0) 0x7fcd880cfe60 con 0x7fcd880cdb10 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd84a00640 1 -- 172.21.15.164:0/262668126 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd6c003530 con 0x7fcd880cdb10 2024-05-08T04:43:57.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd86800640 1 -- 172.21.15.164:0/262668126 >> v1:172.21.15.172:6789/0 conn(0x7fcd880d0c20 legacy=0x7fcd880ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd86800640 1 -- 172.21.15.164:0/262668126 >> v1:172.21.15.164:6789/0 conn(0x7fcd880cdb10 legacy=0x7fcd8806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.799+0000 7fcd86800640 1 -- 172.21.15.164:0/262668126 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd880d0420 con 0x7fcd880ce930 2024-05-08T04:43:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd86800640 1 --2- 172.21.15.164:0/262668126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd88168ec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009d90 tx=0x7fcd8001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf93b83e17fe6155 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd84a00640 1 -- 172.21.15.164:0/262668126 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd800b2070 con 0x7fcd880ce930 2024-05-08T04:43:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd84a00640 1 -- 172.21.15.164:0/262668126 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd800a58d0 con 0x7fcd880ce930 2024-05-08T04:43:57.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd84a00640 1 -- 172.21.15.164:0/262668126 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd800a5ca0 con 0x7fcd880ce930 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 -- 172.21.15.164:0/262668126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 msgr2=0x7fcd88168ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/262668126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd88168ec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009d90 tx=0x7fcd8001c8e0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 -- 172.21.15.164:0/262668126 shutdown_connections 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/262668126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd88168ec0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 -- 172.21.15.164:0/262668126 >> 172.21.15.164:0/262668126 conn(0x7fcd880b34f0 msgr2=0x7fcd880b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 -- 172.21.15.164:0/262668126 shutdown_connections 2024-05-08T04:43:57.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.803+0000 7fcd8ca00640 1 -- 172.21.15.164:0/262668126 wait complete. 2024-05-08T04:43:57.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 Processor -- start 2024-05-08T04:43:57.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- start start 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd880cdb10 0x7fcd880c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd880d0c20 0x7fcd8809f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8816bda0 con 0x7fcd880ce930 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8816bc20 con 0x7fcd880d0c20 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8816bf20 con 0x7fcd880cdb10 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42316/0 (socket says 172.21.15.164:42316) 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 -- 172.21.15.164:0/2977508114 learned_addr learned my addr 172.21.15.164:0/2977508114 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd86800640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd880d0c20 0x7fcd8809f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:57.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85e00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd880cdb10 0x7fcd880c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 -- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd880cdb10 msgr2=0x7fcd880c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd880cdb10 0x7fcd880c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 -- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd880d0c20 msgr2=0x7fcd8809f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd880d0c20 0x7fcd8809f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 -- 172.21.15.164:0/2977508114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88155f00 con 0x7fcd880ce930 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd85400640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c340 tx=0x7fcd7400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740052d0 con 0x7fcd880ce930 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74005470 con 0x7fcd880ce930 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74007260 con 0x7fcd880ce930 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8809ba80 con 0x7fcd880ce930 2024-05-08T04:43:57.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.807+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8809c320 con 0x7fcd880ce930 2024-05-08T04:43:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8809fae0 con 0x7fcd880ce930 2024-05-08T04:43:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fcd74007400 con 0x7fcd880ce930 2024-05-08T04:43:57.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd7e000640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 0x7fcd6806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fcd740831a0 con 0x7fcd880ce930 2024-05-08T04:43:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd85e00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 0x7fcd6806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:57.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.811+0000 7fcd85e00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 0x7fcd6806a1a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcd780011f0 tx=0x7fcd78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:57.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.815+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd7408c080 con 0x7fcd880ce930 2024-05-08T04:43:57.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:57.971+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd8804f010 con 0x7fcd880ce930 2024-05-08T04:43:58.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.631+0000 7fcd7e000640 1 -- 172.21.15.164:0/2977508114 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcd74055950 con 0x7fcd880ce930 2024-05-08T04:43:58.635 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-05-08T04:43:58.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 msgr2=0x7fcd6806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 0x7fcd6806a1a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcd780011f0 tx=0x7fcd78009040 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 msgr2=0x7fcd8809ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c340 tx=0x7fcd7400c800 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 shutdown_connections 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fcd68067ce0 0x7fcd6806a1a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd880d0c20 0x7fcd8809f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd880ce930 0x7fcd8809ef00 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 --2- 172.21.15.164:0/2977508114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd880cdb10 0x7fcd880c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.635+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 >> 172.21.15.164:0/2977508114 conn(0x7fcd880b34f0 msgr2=0x7fcd881611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.639+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 shutdown_connections 2024-05-08T04:43:58.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.639+0000 7fcd8ca00640 1 -- 172.21.15.164:0/2977508114 wait complete. 2024-05-08T04:43:58.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T04:43:58.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.795+0000 7f5c5ba00640 1 Processor -- start 2024-05-08T04:43:58.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.795+0000 7f5c5ba00640 1 -- start start 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c5ba00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c5406cce0 0x7f5c540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c5ba00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5415e870 con 0x7f5c540cecc0 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c5ba00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5415eb50 con 0x7f5c5415ab10 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c5ba00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5406bb80 con 0x7f5c5406cce0 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5c540cecc0 0x7f5c540ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58916/0 (socket says 172.21.15.164:58916) 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59a00640 1 -- 172.21.15.164:0/1383104765 learned_addr learned my addr 172.21.15.164:0/1383104765 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 --2- 172.21.15.164:0/1383104765 >> v2:172.21.15.172:3300/0 conn(0x7f5c5406cce0 0x7f5c540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247826798 0 0) 0x7f5c5415e870 con 0x7f5c540cecc0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c38005740 con 0x7f5c540cecc0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704996586 0 0) 0x7f5c38005740 con 0x7f5c540cecc0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 >> v2:172.21.15.172:3300/0 conn(0x7f5c5406cce0 msgr2=0x7f5c540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 --2- 172.21.15.164:0/1383104765 >> v2:172.21.15.172:3300/0 conn(0x7f5c5406cce0 0x7f5c540ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 >> v1:172.21.15.172:6789/0 conn(0x7f5c5415ab10 legacy=0x7f5c540ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5415ee30 con 0x7f5c540cecc0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7f5c4c001a00 con 0x7f5c540cecc0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 >> v1:172.21.15.164:6789/0 conn(0x7f5c540cecc0 legacy=0x7f5c540ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c38005bd0 0x7f5c38005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 0x7f5c38006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c38007340 0x7f5c3800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5406bb80 con 0x7f5c380065d0 2024-05-08T04:43:58.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800e280 con 0x7f5c38005bd0 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800e480 con 0x7f5c38007340 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 0x7f5c38006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59000640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c38005bd0 0x7f5c38005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59a00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c38007340 0x7f5c3800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 -- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c38007340 msgr2=0x7f5c3800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c38007340 0x7f5c3800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 -- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c38005bd0 msgr2=0x7f5c38005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c38005bd0 0x7f5c38005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c4be00640 1 -- 172.21.15.164:0/1383104765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38005740 con 0x7f5c380065d0 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59000640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c38005bd0 0x7f5c38005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:58.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.799+0000 7f5c59a00640 1 -- 172.21.15.164:0/1383104765 reap_dead start 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c4be00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 0x7f5c38006aa0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ae0 tx=0x7f5c3c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2583b0e93b990821 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0b2070 con 0x7f5c380065d0 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c0a5660 con 0x7f5c380065d0 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c53e00640 1 -- 172.21.15.164:0/1383104765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0096b0 con 0x7f5c380065d0 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 msgr2=0x7f5c38006aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 0x7f5c38006aa0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ae0 tx=0x7f5c3c01ca70 comp rx=0 tx=0).stop 2024-05-08T04:43:58.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- 172.21.15.164:0/1383104765 shutdown_connections 2024-05-08T04:43:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/1383104765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380065d0 0x7f5c38006aa0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- 172.21.15.164:0/1383104765 >> 172.21.15.164:0/1383104765 conn(0x7f5c540cf770 msgr2=0x7f5c540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- 172.21.15.164:0/1383104765 shutdown_connections 2024-05-08T04:43:58.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- 172.21.15.164:0/1383104765 wait complete. 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 Processor -- start 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.803+0000 7f5c5ba00640 1 -- start start 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 0x7f5c54147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c540ad890 0x7f5c540add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5415f3f0 con 0x7f5c54071c30 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540cf0f0 con 0x7f5c380065d0 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5415aef0 con 0x7f5c540ad890 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42334/0 (socket says 172.21.15.164:42334) 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 -- 172.21.15.164:0/640107887 learned_addr learned my addr 172.21.15.164:0/640107887 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c59000640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 0x7f5c54147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c59a00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c540ad890 0x7f5c540add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 -- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c540ad890 msgr2=0x7f5c540add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c540ad890 0x7f5c540add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 -- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 msgr2=0x7f5c54147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 0x7f5c54147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 -- 172.21.15.164:0/640107887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c54158970 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c59000640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 0x7f5c54147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c4be00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0b0040 tx=0x7f5c3c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0b2040 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00f000 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c54072130 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c004260 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54158600 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.807+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c54158110 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.811+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f5c3c003620 con 0x7f5c54071c30 2024-05-08T04:43:58.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.811+0000 7f5c51600640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 0x7f5c3006a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.811+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f5c3c11d160 con 0x7f5c54071c30 2024-05-08T04:43:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.811+0000 7f5c59000640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 0x7f5c3006a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:58.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.811+0000 7f5c59000640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 0x7f5c3006a1d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5c44007ad0 tx=0x7f5c4400a940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:58.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.815+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c3c016780 con 0x7f5c54071c30 2024-05-08T04:43:58.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.967+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5c54072d30 con 0x7f5c54071c30 2024-05-08T04:43:58.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.971+0000 7f5c51600640 1 -- 172.21.15.164:0/640107887 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5c3c0efbc0 con 0x7f5c54071c30 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 msgr2=0x7f5c3006a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 0x7f5c3006a1d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5c44007ad0 tx=0x7f5c4400a940 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 msgr2=0x7f5c54070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0b0040 tx=0x7f5c3c019810 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 shutdown_connections 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f5c30067d10 0x7f5c3006a1d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c540ad890 0x7f5c540add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c54071c30 0x7f5c54070d60 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 --2- 172.21.15.164:0/640107887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c380065d0 0x7f5c54147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 >> 172.21.15.164:0/640107887 conn(0x7f5c540cf770 msgr2=0x7f5c54155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 shutdown_connections 2024-05-08T04:43:58.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:58.975+0000 7f5c5ba00640 1 -- 172.21.15.164:0/640107887 wait complete. 2024-05-08T04:43:58.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume' 2024-05-08T04:43:59.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 Processor -- start 2024-05-08T04:43:59.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 -- start start 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4e4815ba70 0x7f4e48160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e4815be50 0x7f4e480cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480cd9f0 con 0x7f4e480aeda0 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4815fe70 con 0x7f4e4815ba70 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e4d200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480cdcd0 con 0x7f4e4815be50 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4e4815ba70 0x7f4e48160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e46800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4e480aeda0 0x7f4e480b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58926/0 (socket says 172.21.15.164:58926) 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e46800640 1 -- 172.21.15.164:0/3872313571 learned_addr learned my addr 172.21.15.164:0/3872313571 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:43:59.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45400640 1 --2- 172.21.15.164:0/3872313571 >> v2:172.21.15.172:3300/0 conn(0x7f4e4815be50 0x7f4e480cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45e00640 1 -- 172.21.15.164:0/3872313571 >> v2:172.21.15.172:3300/0 conn(0x7f4e4815be50 msgr2=0x7f4e480cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:43:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45e00640 1 --2- 172.21.15.164:0/3872313571 >> v2:172.21.15.172:3300/0 conn(0x7f4e4815be50 0x7f4e480cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45e00640 1 -- 172.21.15.164:0/3872313571 >> v1:172.21.15.164:6789/0 conn(0x7f4e480aeda0 legacy=0x7f4e480b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e44a00640 1 -- 172.21.15.164:0/3872313571 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384783090 0 0) 0x7f4e480cd9f0 con 0x7f4e480aeda0 2024-05-08T04:43:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.131+0000 7f4e45e00640 1 -- 172.21.15.164:0/3872313571 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4815f820 con 0x7f4e4815ba70 2024-05-08T04:43:59.136 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:43:59.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e45e00640 1 --2- 172.21.15.164:0/3872313571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e4815ba70 0x7f4e48160300 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4e4815f780 tx=0x7f4e3801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32943f32fb3214ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:59.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e44a00640 1 -- 172.21.15.164:0/3872313571 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380b2070 con 0x7f4e4815ba70 2024-05-08T04:43:59.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e44a00640 1 -- 172.21.15.164:0/3872313571 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380a9070 con 0x7f4e4815ba70 2024-05-08T04:43:59.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e44a00640 1 -- 172.21.15.164:0/3872313571 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380ad880 con 0x7f4e4815ba70 2024-05-08T04:43:59.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 -- 172.21.15.164:0/3872313571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e4815ba70 msgr2=0x7f4e48160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 --2- 172.21.15.164:0/3872313571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e4815ba70 0x7f4e48160300 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4e4815f780 tx=0x7f4e3801c450 comp rx=0 tx=0).stop 2024-05-08T04:43:59.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 -- 172.21.15.164:0/3872313571 shutdown_connections 2024-05-08T04:43:59.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 --2- 172.21.15.164:0/3872313571 >> v2:172.21.15.172:3300/0 conn(0x7f4e4815be50 0x7f4e480cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 --2- 172.21.15.164:0/3872313571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e4815ba70 0x7f4e48160300 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 -- 172.21.15.164:0/3872313571 >> 172.21.15.164:0/3872313571 conn(0x7f4e4806ac90 msgr2=0x7f4e4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:59.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 -- 172.21.15.164:0/3872313571 shutdown_connections 2024-05-08T04:43:59.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.135+0000 7f4e4d200640 1 -- 172.21.15.164:0/3872313571 wait complete. 2024-05-08T04:43:59.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 Processor -- start 2024-05-08T04:43:59.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:43:59.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- start start 2024-05-08T04:43:59.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4815ba70 0x7f4e4814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 0x7f4e4814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48168d40 con 0x7f4e480aeda0 2024-05-08T04:43:59.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48168bc0 con 0x7f4e4815ba70 2024-05-08T04:43:59.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48168ec0 con 0x7f4e4815be50 2024-05-08T04:43:59.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42356/0 (socket says 172.21.15.164:42356) 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 -- 172.21.15.164:0/357378700 learned_addr learned my addr 172.21.15.164:0/357378700 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e46800640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 0x7f4e4814bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45400640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4815ba70 0x7f4e4814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 -- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 msgr2=0x7f4e4814bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 0x7f4e4814bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 -- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4815ba70 msgr2=0x7f4e4814ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4815ba70 0x7f4e4814ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 -- 172.21.15.164:0/357378700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48138e20 con 0x7f4e480aeda0 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e46800640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 0x7f4e4814bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e45e00640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4e380b0040 tx=0x7f4e3801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:59.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380b2050 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380be070 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e48149d10 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380b9690 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4814a5c0 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.139+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48069d60 con 0x7f4e480aeda0 2024-05-08T04:43:59.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.143+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f4e380ab070 con 0x7f4e480aeda0 2024-05-08T04:43:59.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.143+0000 7f4e3e000640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 0x7f4e2806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:43:59.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.143+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f4e3811e4a0 con 0x7f4e480aeda0 2024-05-08T04:43:59.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.143+0000 7f4e45400640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 0x7f4e2806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:43:59.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.143+0000 7f4e45400640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 0x7f4e2806a1a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4e4814c930 tx=0x7f4e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:43:59.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.147+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e38016070 con 0x7f4e480aeda0 2024-05-08T04:43:59.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.359+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume"]} v 0) v1 -- 0x7f4e48166930 con 0x7f4e480aeda0 2024-05-08T04:43:59.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.635+0000 7f4e3e000640 1 -- 172.21.15.164:0/357378700 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_retain_snapshot_recreate_subvolume"]}]=0 v341) v1 ==== 177+0+0 (secure 0 0 0) 0x7f4e380f0d70 con 0x7f4e480aeda0 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.635+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 msgr2=0x7f4e2806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 0x7f4e2806a1a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4e4814c930 tx=0x7f4e34008040 comp rx=0 tx=0).stop 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 msgr2=0x7f4e481388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4e380b0040 tx=0x7f4e3801ac90 comp rx=0 tx=0).stop 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 shutdown_connections 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e28067ce0 0x7f4e2806a1a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e4815be50 0x7f4e4814bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4815ba70 0x7f4e4814ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 --2- 172.21.15.164:0/357378700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e480aeda0 0x7f4e481388e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:43:59.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 >> 172.21.15.164:0/357378700 conn(0x7f4e4806ac90 msgr2=0x7f4e4806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:43:59.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 shutdown_connections 2024-05-08T04:43:59.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:43:59.639+0000 7f4e4d200640 1 -- 172.21.15.164:0/357378700 wait complete. 2024-05-08T04:43:59.662 INFO:tasks.cephfs_test_runner:ensure a retained subvolume can be recreated and further snapshotted ... ERROR 2024-05-08T04:43:59.662 INFO:tasks.cephfs_test_runner: 2024-05-08T04:43:59.662 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner:ensure a retained subvolume can be recreated and further snapshotted 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 4768, in test_subvolume_retain_snapshot_recreate_subvolume 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: snap_info = json.loads(self._get_subvolume_snapshot_info(self.volname, subvolume, snapshot1)) 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 237, in _get_subvolume_snapshot_info 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: snap_md = self._fs_cmd(*args) 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-05-08T04:43:59.663 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_manager.py", line 1621, in run_cluster_cmd 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol45980 subvol_snap74790' 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner:Ran 13 tests in 302.319s 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner: 2024-05-08T04:43:59.664 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner:ensure a retained subvolume can be recreated and further snapshotted 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 4768, in test_subvolume_retain_snapshot_recreate_subvolume 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: snap_info = json.loads(self._get_subvolume_snapshot_info(self.volname, subvolume, snapshot1)) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 237, in _get_subvolume_snapshot_info 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: snap_md = self._fs_cmd(*args) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-05-08T04:43:59.665 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/ceph_manager.py", line 1621, in run_cluster_cmd 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol45980 subvol_snap74790' 2024-05-08T04:43:59.666 INFO:tasks.cephfs_test_runner: 2024-05-08T04:43:59.666 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:43:59.869 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=52b651016cc543f9b62d99cb4e844676 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-05-08T04:43:59.871 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T04:43:59.880 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T04:43:59.923 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T04:43:59.923 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:59.923 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:59.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:59.938 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-08T04:43:59.938 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:43:59.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1868 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T04:43:59.991 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:43:59.991 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:44:00.001 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:44:00.001 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-08T04:44:00.001 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T04:44:00.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2852 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T04:44:00.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.054 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:44:00.063 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T04:44:00.064 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T04:44:00.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T04:44:00.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.127 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T04:44:00.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-05-08T04:44:00.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.381 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-05-08T04:44:00.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-05-08T04:44:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-05-08T04:44:00.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:44:00.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.458 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:44:00.468 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T04:44:00.468 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T04:44:00.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.469 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T04:44:00.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.537 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T04:44:00.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.607 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-05-08T04:44:00.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.792 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-05-08T04:44:00.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T04:44:00.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-05-08T04:44:00.866 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-05-08T04:44:00.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:44:00.868 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T04:44:00.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T04:44:00.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:44:01.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.011+0000 7fab82800640 1 Processor -- start 2024-05-08T04:44:01.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.011+0000 7fab82800640 1 -- start start 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab82800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab7c15ba70 0x7fab7c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab82800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0cd9f0 con 0x7fab7c15be50 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab82800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0cdcd0 con 0x7fab7c0cfda0 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab82800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c15fe70 con 0x7fab7c15ba70 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7b400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab7c15ba70 0x7fab7c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7aa00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fab7c15be50 0x7fab7c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50458/0 (socket says 172.21.15.164:50458) 2024-05-08T04:44:01.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7aa00640 1 -- 172.21.15.164:0/2198778128 learned_addr learned my addr 172.21.15.164:0/2198778128 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:01.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7b400640 1 -- 172.21.15.164:0/2198778128 >> v1:172.21.15.172:6789/0 conn(0x7fab7c0cfda0 legacy=0x7fab7c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7b400640 1 -- 172.21.15.164:0/2198778128 >> v1:172.21.15.164:6789/0 conn(0x7fab7c15be50 legacy=0x7fab7c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.015+0000 7fab7b400640 1 -- 172.21.15.164:0/2198778128 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c15f820 con 0x7fab7c15ba70 2024-05-08T04:44:01.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab7b400640 1 --2- 172.21.15.164:0/2198778128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c160300 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eeaf1a3c9ca66d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab7a000640 1 -- 172.21.15.164:0/2198778128 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2070 con 0x7fab7c15ba70 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab7a000640 1 -- 172.21.15.164:0/2198778128 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c0a5910 con 0x7fab7c15ba70 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab7a000640 1 -- 172.21.15.164:0/2198778128 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a5ce0 con 0x7fab7c15ba70 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- 172.21.15.164:0/2198778128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 msgr2=0x7fab7c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 --2- 172.21.15.164:0/2198778128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c160300 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c940 comp rx=0 tx=0).stop 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- 172.21.15.164:0/2198778128 shutdown_connections 2024-05-08T04:44:01.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 --2- 172.21.15.164:0/2198778128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c160300 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- 172.21.15.164:0/2198778128 >> 172.21.15.164:0/2198778128 conn(0x7fab7c06ac90 msgr2=0x7fab7c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- 172.21.15.164:0/2198778128 shutdown_connections 2024-05-08T04:44:01.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- 172.21.15.164:0/2198778128 wait complete. 2024-05-08T04:44:01.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 Processor -- start 2024-05-08T04:44:01.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- start start 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 0x7fab7c139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 0x7fab7c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c169f60 con 0x7fab7c15ba70 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c169de0 con 0x7fab7c0cfda0 2024-05-08T04:44:01.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.019+0000 7fab82800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c16a0e0 con 0x7fab7c15be50 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35222/0 (socket says 172.21.15.164:35222) 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 -- 172.21.15.164:0/1236985043 learned_addr learned my addr 172.21.15.164:0/1236985043 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7b400640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 0x7fab7c139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 -- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 msgr2=0x7fab7c150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7be00640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 0x7fab7c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 0x7fab7c150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 -- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 msgr2=0x7fab7c139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 0x7fab7c139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 -- 172.21.15.164:0/1236985043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0b0840 con 0x7fab7c15ba70 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7b400640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 0x7fab7c139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7be00640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 0x7fab7c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab7aa00640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c500 tx=0x7fab6800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68017070 con 0x7fab7c15ba70 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68005cf0 con 0x7fab7c15ba70 2024-05-08T04:44:01.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c13a490 con 0x7fab7c15ba70 2024-05-08T04:44:01.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680072f0 con 0x7fab7c15ba70 2024-05-08T04:44:01.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c0affa0 con 0x7fab7c15ba70 2024-05-08T04:44:01.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c069d60 con 0x7fab7c15ba70 2024-05-08T04:44:01.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fab6801a070 con 0x7fab7c15ba70 2024-05-08T04:44:01.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.023+0000 7fab73400640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 0x7fab5c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.027+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fab68082fd0 con 0x7fab7c15ba70 2024-05-08T04:44:01.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.027+0000 7fab7b400640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 0x7fab5c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.027+0000 7fab7b400640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 0x7fab5c06a1a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0af000 tx=0x7fab6c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.031+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab68055130 con 0x7fab7c15ba70 2024-05-08T04:44:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab7c06e210 con 0x7fab5c067ce0 2024-05-08T04:44:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab73400640 1 -- 172.21.15.164:0/1236985043 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13474 (secure 0 0 0) 0x7fab7c06e210 con 0x7fab5c067ce0 2024-05-08T04:44:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:44:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 msgr2=0x7fab5c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 0x7fab5c06a1a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0af000 tx=0x7fab6c0a7070 comp rx=0 tx=0).stop 2024-05-08T04:44:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 msgr2=0x7fab7c150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c500 tx=0x7fab6800c9c0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 shutdown_connections 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fab5c067ce0 0x7fab5c06a1a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab7c15be50 0x7fab7c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab7c15ba70 0x7fab7c150540 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 --2- 172.21.15.164:0/1236985043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab7c0cfda0 0x7fab7c139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 >> 172.21.15.164:0/1236985043 conn(0x7fab7c06ac90 msgr2=0x7fab7c153670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 shutdown_connections 2024-05-08T04:44:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.179+0000 7fab82800640 1 -- 172.21.15.164:0/1236985043 wait complete. 2024-05-08T04:44:01.201 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-08T04:43:59.496966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263320,"kb_used_data":13880,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711400,"statfs":{"total":773094113280,"available":772824473600,"internally_reserved":0,"allocated":14213120,"data_stored":7011654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129647,"num_objects":-26,"num_object_clones":-1,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":-1,"num_read":-4,"num_read_kb":-4,"num_write":-98,"num_write_kb":-166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001673"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T04:43:55.962833+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:43:55.962833+0000","last_peered":"2024-05-08T04:43:55.962833+0000","last_clean":"2024-05-08T04:43:55.962833+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:43:55.962833+0000","last_undegraded":"2024-05-08T04:43:55.962833+0000","last_fullsized":"2024-05-08T04:43:55.962833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1415636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":825042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":696453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2179072,"data_stored":1286821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:44:01.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:44:01.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 Processor -- start 2024-05-08T04:44:01.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 -- start start 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce3b0 0x7efd040b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 0x7efd040b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efd04160f20 0x7efd041613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0406ca40 con 0x7efd04160f20 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd041618f0 con 0x7efd040ce790 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040b3090 con 0x7efd040ce3b0 2024-05-08T04:44:01.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efd04160f20 0x7efd041613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efd04160f20 0x7efd041613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35232/0 (socket says 172.21.15.164:35232) 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 -- 172.21.15.164:0/38995657 learned_addr learned my addr 172.21.15.164:0/38995657 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd08e00640 1 --2- 172.21.15.164:0/38995657 >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 0x7efd040b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd09800640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce3b0 0x7efd040b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 -- 172.21.15.164:0/38995657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce3b0 msgr2=0x7efd040b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce3b0 0x7efd040b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 -- 172.21.15.164:0/38995657 >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 msgr2=0x7efd040b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:01.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 --2- 172.21.15.164:0/38995657 >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 0x7efd040b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd0a200640 1 -- 172.21.15.164:0/38995657 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040d22a0 con 0x7efd04160f20 2024-05-08T04:44:01.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.339+0000 7efd08e00640 1 --2- 172.21.15.164:0/38995657 >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 0x7efd040b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:01.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0a200640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd04160f20 0x7efd041613b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7efd0409b420 tx=0x7efcfc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafcdf90d3835471 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd03e00640 1 -- 172.21.15.164:0/38995657 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0a57b0 con 0x7efd04160f20 2024-05-08T04:44:01.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd03e00640 1 -- 172.21.15.164:0/38995657 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc0a5950 con 0x7efd04160f20 2024-05-08T04:44:01.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd03e00640 1 -- 172.21.15.164:0/38995657 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0a5d20 con 0x7efd04160f20 2024-05-08T04:44:01.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- 172.21.15.164:0/38995657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd04160f20 msgr2=0x7efd041613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd04160f20 0x7efd041613b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7efd0409b420 tx=0x7efcfc01c930 comp rx=0 tx=0).stop 2024-05-08T04:44:01.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- 172.21.15.164:0/38995657 shutdown_connections 2024-05-08T04:44:01.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd04160f20 0x7efd041613b0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 --2- 172.21.15.164:0/38995657 >> v2:172.21.15.172:3300/0 conn(0x7efd040ce790 0x7efd040b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 --2- 172.21.15.164:0/38995657 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce3b0 0x7efd040b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- 172.21.15.164:0/38995657 >> 172.21.15.164:0/38995657 conn(0x7efd0409aa20 msgr2=0x7efd040d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- 172.21.15.164:0/38995657 shutdown_connections 2024-05-08T04:44:01.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- 172.21.15.164:0/38995657 wait complete. 2024-05-08T04:44:01.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 Processor -- start 2024-05-08T04:44:01.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.343+0000 7efd0ac00640 1 -- start start 2024-05-08T04:44:01.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce790 0x7efd04156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd04160f20 0x7efd04157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040d1ec0 con 0x7efd040ce3b0 2024-05-08T04:44:01.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040b13e0 con 0x7efd04160f20 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040d2040 con 0x7efd040ce790 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35248/0 (socket says 172.21.15.164:35248) 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 -- 172.21.15.164:0/3136359844 learned_addr learned my addr 172.21.15.164:0/3136359844 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0a200640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd04160f20 0x7efd04157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd08e00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce790 0x7efd04156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 -- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce790 msgr2=0x7efd04156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce790 0x7efd04156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 -- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd04160f20 msgr2=0x7efd04157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd04160f20 0x7efd04157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 -- 172.21.15.164:0/3136359844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040c7f90 con 0x7efd040ce3b0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd09800640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efcf800d800 tx=0x7efcf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8018030 con 0x7efd040ce3b0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf80096a0 con 0x7efd040ce3b0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8009870 con 0x7efd040ce3b0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0415ce40 con 0x7efd040ce3b0 2024-05-08T04:44:01.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0415d400 con 0x7efd040ce3b0 2024-05-08T04:44:01.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.347+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0404ef30 con 0x7efd040ce3b0 2024-05-08T04:44:01.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.351+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7efcf801b070 con 0x7efd040ce3b0 2024-05-08T04:44:01.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.351+0000 7efd01600640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 0x7efce406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.351+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7efcf8083f00 con 0x7efd040ce3b0 2024-05-08T04:44:01.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.351+0000 7efd08e00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 0x7efce406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.351+0000 7efd08e00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 0x7efce406a1a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efcf00045f0 tx=0x7efcf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.355+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcf8056060 con 0x7efd040ce3b0 2024-05-08T04:44:01.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd040cf000 con 0x7efce4067ce0 2024-05-08T04:44:01.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd01600640 1 -- 172.21.15.164:0/3136359844 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13474 (secure 0 0 0) 0x7efd040cf000 con 0x7efce4067ce0 2024-05-08T04:44:01.508 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:01.508 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 msgr2=0x7efce406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 0x7efce406a1a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efcf00045f0 tx=0x7efcf0008040 comp rx=0 tx=0).stop 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 msgr2=0x7efd040c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efcf800d800 tx=0x7efcf800dcc0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 shutdown_connections 2024-05-08T04:44:01.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7efce4067ce0 0x7efce406a1a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd040ce790 0x7efd04156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd04160f20 0x7efd04157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 --2- 172.21.15.164:0/3136359844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd040ce3b0 0x7efd040c7960 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 >> 172.21.15.164:0/3136359844 conn(0x7efd0409aa20 msgr2=0x7efd040b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 shutdown_connections 2024-05-08T04:44:01.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.507+0000 7efd0ac00640 1 -- 172.21.15.164:0/3136359844 wait complete. 2024-05-08T04:44:01.528 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T04:44:01.497240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263320,"kb_used_data":13880,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711400,"statfs":{"total":773094113280,"available":772824473600,"internally_reserved":0,"allocated":14213120,"data_stored":7011654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129647,"num_objects":-26,"num_object_clones":-1,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":-1,"num_read":-4,"num_read_kb":-4,"num_write":-98,"num_write_kb":-166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.098253"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T04:43:55.962833+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:43:55.962833+0000","last_peered":"2024-05-08T04:43:55.962833+0000","last_clean":"2024-05-08T04:43:55.962833+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:43:55.962833+0000","last_undegraded":"2024-05-08T04:43:55.962833+0000","last_fullsized":"2024-05-08T04:43:55.962833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1415636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":825042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":696453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2179072,"data_stored":1286821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:44:01.529 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T04:44:01.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:44:01.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 Processor -- start 2024-05-08T04:44:01.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 -- start start 2024-05-08T04:44:01.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3100aedb0 0x7fa3100cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa3100cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3100ceac0 con 0x7fa310163540 2024-05-08T04:44:01.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3100ce860 con 0x7fa3100aedb0 2024-05-08T04:44:01.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa317c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100cc1e0 con 0x7fa3100cf6c0 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3100aedb0 0x7fa3100cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3100aedb0 0x7fa3100cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35252/0 (socket says 172.21.15.164:35252) 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 -- 172.21.15.164:0/276327880 learned_addr learned my addr 172.21.15.164:0/276327880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315c00640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa3100cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 -- 172.21.15.164:0/276327880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 msgr2=0x7fa3100cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa3100cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 -- 172.21.15.164:0/276327880 >> v1:172.21.15.172:6789/0 conn(0x7fa310163540 legacy=0x7fa3100ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315200640 1 -- 172.21.15.164:0/276327880 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100cc4c0 con 0x7fa3100aedb0 2024-05-08T04:44:01.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.667+0000 7fa315c00640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa3100cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:01.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa315200640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3100aedb0 0x7fa3100cccc0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa30001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea53ae648f2d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa30fe00640 1 -- 172.21.15.164:0/276327880 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000b1070 con 0x7fa3100aedb0 2024-05-08T04:44:01.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa30fe00640 1 -- 172.21.15.164:0/276327880 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3000a8050 con 0x7fa3100aedb0 2024-05-08T04:44:01.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa30fe00640 1 -- 172.21.15.164:0/276327880 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000ac860 con 0x7fa3100aedb0 2024-05-08T04:44:01.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 -- 172.21.15.164:0/276327880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3100aedb0 msgr2=0x7fa3100cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3100aedb0 0x7fa3100cccc0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa30001c4d0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 -- 172.21.15.164:0/276327880 shutdown_connections 2024-05-08T04:44:01.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa3100cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 --2- 172.21.15.164:0/276327880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3100aedb0 0x7fa3100cccc0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 -- 172.21.15.164:0/276327880 >> 172.21.15.164:0/276327880 conn(0x7fa31006ac90 msgr2=0x7fa31006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 -- 172.21.15.164:0/276327880 shutdown_connections 2024-05-08T04:44:01.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.671+0000 7fa317c00640 1 -- 172.21.15.164:0/276327880 wait complete. 2024-05-08T04:44:01.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 Processor -- start 2024-05-08T04:44:01.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- start start 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100aedb0 0x7fa3101576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa310156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31006dbc0 con 0x7fa310163540 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31006da40 con 0x7fa3100aedb0 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31006dd40 con 0x7fa3100cf6c0 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100aedb0 0x7fa3101576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa307e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa310156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35262/0 (socket says 172.21.15.164:35262) 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 -- 172.21.15.164:0/2838087170 learned_addr learned my addr 172.21.15.164:0/2838087170 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 -- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 msgr2=0x7fa310156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa310156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 -- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100aedb0 msgr2=0x7fa3101576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100aedb0 0x7fa3101576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 -- 172.21.15.164:0/2838087170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310157e60 con 0x7fa310163540 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa315c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa30800cef0 tx=0x7fa30800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30801a070 con 0x7fa310163540 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30800d860 con 0x7fa310163540 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800dc50 con 0x7fa310163540 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa310158140 con 0x7fa310163540 2024-05-08T04:44:01.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.675+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310158700 con 0x7fa310163540 2024-05-08T04:44:01.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa310069d60 con 0x7fa310163540 2024-05-08T04:44:01.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fa30801d070 con 0x7fa310163540 2024-05-08T04:44:01.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa30d600640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 0x7fa2ec06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fa308084510 con 0x7fa310163540 2024-05-08T04:44:01.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa315200640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 0x7fa2ec06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.679+0000 7fa315200640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 0x7fa2ec06a1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ae000 tx=0x7fa3000a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:01.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.683+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa308056730 con 0x7fa310163540 2024-05-08T04:44:01.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.835+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3100ce640 con 0x7fa2ec067ce0 2024-05-08T04:44:01.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.835+0000 7fa30d600640 1 -- 172.21.15.164:0/2838087170 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13474 (secure 0 0 0) 0x7fa3100ce640 con 0x7fa2ec067ce0 2024-05-08T04:44:01.838 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:01.838 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 msgr2=0x7fa2ec06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 0x7fa2ec06a1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ae000 tx=0x7fa3000a7000 comp rx=0 tx=0).stop 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 msgr2=0x7fa310164d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa30800cef0 tx=0x7fa30800a710 comp rx=0 tx=0).stop 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 shutdown_connections 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fa2ec067ce0 0x7fa2ec06a1a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3100cf6c0 0x7fa310156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa310163540 0x7fa310164d60 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 --2- 172.21.15.164:0/2838087170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100aedb0 0x7fa3101576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 >> 172.21.15.164:0/2838087170 conn(0x7fa31006ac90 msgr2=0x7fa3100717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 shutdown_connections 2024-05-08T04:44:01.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.839+0000 7fa317c00640 1 -- 172.21.15.164:0/2838087170 wait complete. 2024-05-08T04:44:01.858 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T04:44:01.497240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263320,"kb_used_data":13880,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711400,"statfs":{"total":773094113280,"available":772824473600,"internally_reserved":0,"allocated":14213120,"data_stored":7011654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129647,"num_objects":-26,"num_object_clones":-1,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":-1,"num_read":-4,"num_read_kb":-4,"num_write":-98,"num_write_kb":-166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.098253"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T04:43:55.962833+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:43:55.962833+0000","last_peered":"2024-05-08T04:43:55.962833+0000","last_clean":"2024-05-08T04:43:55.962833+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:43:55.962833+0000","last_undegraded":"2024-05-08T04:43:55.962833+0000","last_fullsized":"2024-05-08T04:43:55.962833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:38.348677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1415636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":825042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":696653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":697329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":696453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2179072,"data_stored":1286821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:44:01.859 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:44:01.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 Processor -- start 2024-05-08T04:44:01.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 -- start start 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98f006c7b0 0x7f98f01620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f98f01625f0 0x7f98f00b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f006cc70 con 0x7f98f015dd70 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f006cf50 con 0x7f98f006c7b0 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f6c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f00b1570 con 0x7f98f01625f0 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98efe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f98f015dd70 0x7f98f01619a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50466/0 (socket says 172.21.15.164:50466) 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98efe00640 1 -- 172.21.15.164:0/978774636 learned_addr learned my addr 172.21.15.164:0/978774636 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:01.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98e7400640 1 --2- 172.21.15.164:0/978774636 >> v2:172.21.15.164:3300/0 conn(0x7f98f006c7b0 0x7f98f01620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98f4c00640 1 --2- 172.21.15.164:0/978774636 >> v2:172.21.15.172:3300/0 conn(0x7f98f01625f0 0x7f98f00b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:01.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98e7400640 1 -- 172.21.15.164:0/978774636 >> v2:172.21.15.172:3300/0 conn(0x7f98f01625f0 msgr2=0x7f98f00b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:01.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98e7400640 1 --2- 172.21.15.164:0/978774636 >> v2:172.21.15.172:3300/0 conn(0x7f98f01625f0 0x7f98f00b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:01.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98e7400640 1 -- 172.21.15.164:0/978774636 >> v1:172.21.15.164:6789/0 conn(0x7f98f015dd70 legacy=0x7f98f01619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:01.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.995+0000 7f98e7400640 1 -- 172.21.15.164:0/978774636 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00b1850 con 0x7f98f006c7b0 2024-05-08T04:44:02.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98e7400640 1 --2- 172.21.15.164:0/978774636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f01620b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c8a20 tx=0x7f98d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cda7f82c68d7d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:02.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98ef400640 1 -- 172.21.15.164:0/978774636 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b0070 con 0x7f98f006c7b0 2024-05-08T04:44:02.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98ef400640 1 -- 172.21.15.164:0/978774636 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80ab600 con 0x7f98f006c7b0 2024-05-08T04:44:02.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98ef400640 1 -- 172.21.15.164:0/978774636 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ab9d0 con 0x7f98f006c7b0 2024-05-08T04:44:02.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 -- 172.21.15.164:0/978774636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 msgr2=0x7f98f01620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 --2- 172.21.15.164:0/978774636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f01620b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c8a20 tx=0x7f98d801c480 comp rx=0 tx=0).stop 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 -- 172.21.15.164:0/978774636 shutdown_connections 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 --2- 172.21.15.164:0/978774636 >> v2:172.21.15.172:3300/0 conn(0x7f98f01625f0 0x7f98f00b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 --2- 172.21.15.164:0/978774636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f01620b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 -- 172.21.15.164:0/978774636 >> 172.21.15.164:0/978774636 conn(0x7f98f00b34f0 msgr2=0x7f98f00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:02.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 -- 172.21.15.164:0/978774636 shutdown_connections 2024-05-08T04:44:02.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:01.999+0000 7f98f6c00640 1 -- 172.21.15.164:0/978774636 wait complete. 2024-05-08T04:44:02.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 Processor -- start 2024-05-08T04:44:02.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- start start 2024-05-08T04:44:02.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:02.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98f015dd70 0x7f98f0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98f01625f0 0x7f98f00b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f00b2e50 con 0x7f98f006c7b0 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f00b2cd0 con 0x7f98f015dd70 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f00b2fd0 con 0x7f98f01625f0 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35270/0 (socket says 172.21.15.164:35270) 2024-05-08T04:44:02.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 -- 172.21.15.164:0/2873449674 learned_addr learned my addr 172.21.15.164:0/2873449674 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98e7400640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98f015dd70 0x7f98f0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f4c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98f01625f0 0x7f98f00b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 -- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98f01625f0 msgr2=0x7f98f00b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98f01625f0 0x7f98f00b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 -- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98f015dd70 msgr2=0x7f98f0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98f015dd70 0x7f98f0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 -- 172.21.15.164:0/2873449674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0153c80 con 0x7f98f006c7b0 2024-05-08T04:44:02.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98efe00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3a0 tx=0x7f98e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:02.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98f006c7b0 2024-05-08T04:44:02.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e0005cf0 con 0x7f98f006c7b0 2024-05-08T04:44:02.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f00b6960 con 0x7f98f006c7b0 2024-05-08T04:44:02.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e00072f0 con 0x7f98f006c7b0 2024-05-08T04:44:02.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f00c5670 con 0x7f98f006c7b0 2024-05-08T04:44:02.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.003+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f0069d60 con 0x7f98f006c7b0 2024-05-08T04:44:02.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.007+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f98e001a070 con 0x7f98f006c7b0 2024-05-08T04:44:02.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.007+0000 7f98ecc00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 0x7f98cc06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:02.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.007+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f98e0083040 con 0x7f98f006c7b0 2024-05-08T04:44:02.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.007+0000 7f98e7400640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 0x7f98cc06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:02.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.007+0000 7f98e7400640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 0x7f98cc06a1a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f98d80ad000 tx=0x7f98d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:02.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.011+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98e00551a0 con 0x7f98f006c7b0 2024-05-08T04:44:02.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.167+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98f00b4010 con 0x7f98f006c7b0 2024-05-08T04:44:02.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.167+0000 7f98ecc00640 1 -- 172.21.15.164:0/2873449674 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+13224 (secure 0 0 0) 0x7f98e0059240 con 0x7f98f006c7b0 2024-05-08T04:44:02.170 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:02.170 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":129,"fsid":"6150cf8f-12a4-433b-b39d-bc24e302fd2a","created":"2024-05-08T04:38:04.373719+0000","modified":"2024-05-08T04:43:58.629250+0000","last_up_change":"2024-05-08T04:38:16.653225+0000","last_in_change":"2024-05-08T04:38:06.913892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:38:17.494575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"74ce1b3b-c77b-4e2c-97d8-24b6d18fff3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6801","nonce":2287928810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6803","nonce":2287928810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6807","nonce":2287928810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2287928810},{"type":"v1","addr":"172.21.15.164:6805","nonce":2287928810}]},"public_addr":"172.21.15.164:6801/2287928810","cluster_addr":"172.21.15.164:6803/2287928810","heartbeat_back_addr":"172.21.15.164:6807/2287928810","heartbeat_front_addr":"172.21.15.164:6805/2287928810","state":["exists","up"]},{"osd":1,"uuid":"342a2bc7-f643-4df7-baf3-b04d228500ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6810","nonce":2349768482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6814","nonce":2349768482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6822","nonce":2349768482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2349768482},{"type":"v1","addr":"172.21.15.164:6818","nonce":2349768482}]},"public_addr":"172.21.15.164:6810/2349768482","cluster_addr":"172.21.15.164:6814/2349768482","heartbeat_back_addr":"172.21.15.164:6822/2349768482","heartbeat_front_addr":"172.21.15.164:6818/2349768482","state":["exists","up"]},{"osd":2,"uuid":"454494bb-a031-44bb-806e-9922df32773d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6825","nonce":4253785060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6827","nonce":4253785060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6831","nonce":4253785060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4253785060},{"type":"v1","addr":"172.21.15.164:6829","nonce":4253785060}]},"public_addr":"172.21.15.164:6825/4253785060","cluster_addr":"172.21.15.164:6827/4253785060","heartbeat_back_addr":"172.21.15.164:6831/4253785060","heartbeat_front_addr":"172.21.15.164:6829/4253785060","state":["exists","up"]},{"osd":3,"uuid":"d4386cf2-89cd-4873-8d2f-d1831eea8277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6811","nonce":2335523075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6815","nonce":2335523075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6823","nonce":2335523075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2335523075},{"type":"v1","addr":"172.21.15.164:6819","nonce":2335523075}]},"public_addr":"172.21.15.164:6811/2335523075","cluster_addr":"172.21.15.164:6815/2335523075","heartbeat_back_addr":"172.21.15.164:6823/2335523075","heartbeat_front_addr":"172.21.15.164:6819/2335523075","state":["exists","up"]},{"osd":4,"uuid":"d500c818-d468-4b1e-ac4c-274147bc4bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6801","nonce":3510673411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6803","nonce":3510673411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6807","nonce":3510673411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3510673411},{"type":"v1","addr":"172.21.15.172:6805","nonce":3510673411}]},"public_addr":"172.21.15.172:6801/3510673411","cluster_addr":"172.21.15.172:6803/3510673411","heartbeat_back_addr":"172.21.15.172:6807/3510673411","heartbeat_front_addr":"172.21.15.172:6805/3510673411","state":["exists","up"]},{"osd":5,"uuid":"e28992ff-444b-42c9-b859-87d309757426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6815","nonce":3686545583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6818","nonce":3686545583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6823","nonce":3686545583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3686545583},{"type":"v1","addr":"172.21.15.172:6821","nonce":3686545583}]},"public_addr":"172.21.15.172:6815/3686545583","cluster_addr":"172.21.15.172:6818/3686545583","heartbeat_back_addr":"172.21.15.172:6823/3686545583","heartbeat_front_addr":"172.21.15.172:6821/3686545583","state":["exists","up"]},{"osd":6,"uuid":"eda60f02-6743-495e-b97a-0fb6ff0d9bc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6825","nonce":2968599637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6827","nonce":2968599637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6831","nonce":2968599637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2968599637},{"type":"v1","addr":"172.21.15.172:6829","nonce":2968599637}]},"public_addr":"172.21.15.172:6825/2968599637","cluster_addr":"172.21.15.172:6827/2968599637","heartbeat_back_addr":"172.21.15.172:6831/2968599637","heartbeat_front_addr":"172.21.15.172:6829/2968599637","state":["exists","up"]},{"osd":7,"uuid":"67d26eaa-8ce3-4724-8384-b144879d1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6809","nonce":1836955626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6811","nonce":1836955626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6819","nonce":1836955626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1836955626},{"type":"v1","addr":"172.21.15.172:6813","nonce":1836955626}]},"public_addr":"172.21.15.172:6809/1836955626","cluster_addr":"172.21.15.172:6811/1836955626","heartbeat_back_addr":"172.21.15.172:6819/1836955626","heartbeat_front_addr":"172.21.15.172:6813/1836955626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.565717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.647566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.731657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.721842+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.889335+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.660497+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.968845+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:38:12.577743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6837/333884670":"2024-05-09T04:43:55.395009+0000","172.21.15.164:6835/333884670":"2024-05-09T04:43:55.395009+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":10}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:44:02.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 msgr2=0x7f98cc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:02.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 0x7f98cc06a1a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f98d80ad000 tx=0x7f98d8008040 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 msgr2=0x7f98f00c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3a0 tx=0x7f98e000c860 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 shutdown_connections 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f98cc067ce0 0x7f98cc06a1a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98f01625f0 0x7f98f00b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98f015dd70 0x7f98f0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 --2- 172.21.15.164:0/2873449674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f006c7b0 0x7f98f00c5e70 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 >> 172.21.15.164:0/2873449674 conn(0x7f98f00b34f0 msgr2=0x7f98f00cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:02.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 shutdown_connections 2024-05-08T04:44:02.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:02.171+0000 7f98f6c00640 1 -- 172.21.15.164:0/2873449674 wait complete. 2024-05-08T04:44:03.196 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T04:44:03.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:03.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.335+0000 7f1a43000640 1 Processor -- start 2024-05-08T04:44:03.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.335+0000 7f1a43000640 1 -- start start 2024-05-08T04:44:03.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.335+0000 7f1a43000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1a3c15ba20 0x7f1a3c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a43000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a43000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c0c6600 con 0x7f1a3c06de50 2024-05-08T04:44:03.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a43000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c15fbe0 con 0x7f1a3c15ba20 2024-05-08T04:44:03.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a43000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c6040 con 0x7f1a3c0aedf0 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a41000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1a3c15ba20 0x7f1a3c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a41000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.164:58110/0 (socket says 172.21.15.164:58110) 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a41000640 1 -- 172.21.15.164:0/543352790 learned_addr learned my addr 172.21.15.164:0/543352790 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3b400640 1 --1- 172.21.15.164:0/543352790 >> v1:172.21.15.172:6789/0 conn(0x7f1a3c06de50 0x7f1a3c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:53156/0 (socket says 172.21.15.164:53156) 2024-05-08T04:44:03.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3be00640 1 -- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 msgr2=0x7f1a3c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:03.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3be00640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3be00640 1 -- 172.21.15.164:0/543352790 >> v1:172.21.15.172:6789/0 conn(0x7f1a3c06de50 legacy=0x7f1a3c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a3be00640 1 -- 172.21.15.164:0/543352790 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0c6320 con 0x7f1a3c15ba20 2024-05-08T04:44:03.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.339+0000 7f1a41000640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:44:03.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a3be00640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c160070 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009d90 tx=0x7f1a2c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e80771c8e38c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a3aa00640 1 -- 172.21.15.164:0/543352790 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0b2070 con 0x7f1a3c15ba20 2024-05-08T04:44:03.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a3aa00640 1 -- 172.21.15.164:0/543352790 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c0a59a0 con 0x7f1a3c15ba20 2024-05-08T04:44:03.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a3aa00640 1 -- 172.21.15.164:0/543352790 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0a5d70 con 0x7f1a3c15ba20 2024-05-08T04:44:03.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 msgr2=0x7f1a3c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c160070 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009d90 tx=0x7f1a2c01c940 comp rx=0 tx=0).stop 2024-05-08T04:44:03.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- 172.21.15.164:0/543352790 shutdown_connections 2024-05-08T04:44:03.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 --2- 172.21.15.164:0/543352790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c160070 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- 172.21.15.164:0/543352790 >> 172.21.15.164:0/543352790 conn(0x7f1a3c0cf770 msgr2=0x7f1a3c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- 172.21.15.164:0/543352790 shutdown_connections 2024-05-08T04:44:03.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- 172.21.15.164:0/543352790 wait complete. 2024-05-08T04:44:03.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 Processor -- start 2024-05-08T04:44:03.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 -- start start 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 0x7f1a3c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.343+0000 7f1a43000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c14e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ce000 con 0x7f1a3c06de50 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0cde80 con 0x7f1a3c15ba20 2024-05-08T04:44:03.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c168b70 con 0x7f1a3c0aedf0 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 0x7f1a3c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 0x7f1a3c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35282/0 (socket says 172.21.15.164:35282) 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 -- 172.21.15.164:0/425019025 learned_addr learned my addr 172.21.15.164:0/425019025 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3b400640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a41000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 -- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 msgr2=0x7f1a3c14e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a3c0aedf0 0x7f1a3c14e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 -- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 msgr2=0x7f1a3c0ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c0ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0cad10 con 0x7f1a3c06de50 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a41000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a3c15ba20 0x7f1a3c0ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a3be00640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 0x7f1a3c0c6840 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b0040 tx=0x7f1a2c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0b2050 con 0x7f1a3c06de50 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c0be070 con 0x7f1a3c06de50 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0ada50 con 0x7f1a3c06de50 2024-05-08T04:44:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c14ed80 con 0x7f1a3c06de50 2024-05-08T04:44:03.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c14f340 con 0x7f1a3c06de50 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a3c072d30 con 0x7f1a3c06de50 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.347+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f1a2c0adbf0 con 0x7f1a3c06de50 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a1c067ce0 0x7f1a1c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f1a2c11f8f0 con 0x7f1a3c06de50 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a3b400640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a1c067ce0 0x7f1a1c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1a1c06dc00 0x7f1a1c070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a2c0091a0 con 0x7f1a1c06dc00 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a2c122200 con 0x7f1a3c06de50 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a41000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1a1c06dc00 0x7f1a1c070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a3b400640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a1c067ce0 0x7f1a1c06a1a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15a9c0 tx=0x7f1a28007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a41000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1a1c06dc00 0x7f1a1c070060 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.351+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== osd.0 v2:172.21.15.164:6800/2287928810 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1a2c0091a0 con 0x7f1a1c06dc00 2024-05-08T04:44:03.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 --> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a3c160390 con 0x7f1a1c06dc00 2024-05-08T04:44:03.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a33e00640 1 -- 172.21.15.164:0/425019025 <== osd.0 v2:172.21.15.164:6800/2287928810 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1a3c160390 con 0x7f1a1c06dc00 2024-05-08T04:44:03.366 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:03.366 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:03.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1a1c06dc00 msgr2=0x7f1a1c070060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6800/2287928810,v1:172.21.15.164:6801/2287928810] conn(0x7f1a1c06dc00 0x7f1a1c070060 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a1c067ce0 msgr2=0x7f1a1c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f1a1c067ce0 0x7f1a1c06a1a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15a9c0 tx=0x7f1a28007cf0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 msgr2=0x7f1a3c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 --2- 172.21.15.164:0/425019025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a3c06de50 0x7f1a3c0c6840 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b0040 tx=0x7f1a2c0196e0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a41000640 1 -- 172.21.15.164:0/425019025 reap_dead start 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 shutdown_connections 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 >> 172.21.15.164:0/425019025 conn(0x7f1a3c0cf770 msgr2=0x7f1a3c15e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 shutdown_connections 2024-05-08T04:44:03.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.363+0000 7f1a43000640 1 -- 172.21.15.164:0/425019025 wait complete. 2024-05-08T04:44:03.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T04:44:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 Processor -- start 2024-05-08T04:44:03.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 -- start start 2024-05-08T04:44:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 0x7fc6000cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6000c5530 0x7fc6000c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000ccc40 con 0x7fc6000cbed0 2024-05-08T04:44:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60015ae90 con 0x7fc6000c5530 2024-05-08T04:44:03.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc606000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000c5ed0 con 0x7fc60015aab0 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc605600640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc6000cbed0 0x7fc6000cc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50472/0 (socket says 172.21.15.164:50472) 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc605600640 1 -- 172.21.15.164:0/2859700538 learned_addr learned my addr 172.21.15.164:0/2859700538 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc5ffe00640 1 --2- 172.21.15.164:0/2859700538 >> v2:172.21.15.172:3300/0 conn(0x7fc6000c5530 0x7fc6000c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc604c00640 1 --2- 172.21.15.164:0/2859700538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 0x7fc6000cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc5ff400640 1 -- 172.21.15.164:0/2859700538 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486410688 0 0) 0x7fc6000ccc40 con 0x7fc6000cbed0 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc5ff400640 1 -- 172.21.15.164:0/2859700538 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e4003530 con 0x7fc6000cbed0 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc604c00640 1 -- 172.21.15.164:0/2859700538 >> v2:172.21.15.172:3300/0 conn(0x7fc6000c5530 msgr2=0x7fc6000c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:03.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc604c00640 1 --2- 172.21.15.164:0/2859700538 >> v2:172.21.15.172:3300/0 conn(0x7fc6000c5530 0x7fc6000c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc604c00640 1 -- 172.21.15.164:0/2859700538 >> v1:172.21.15.164:6789/0 conn(0x7fc6000cbed0 legacy=0x7fc6000cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.523+0000 7fc604c00640 1 -- 172.21.15.164:0/2859700538 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000ce170 con 0x7fc60015aab0 2024-05-08T04:44:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc604c00640 1 --2- 172.21.15.164:0/2859700538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 0x7fc6000cec50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097c0 tx=0x7fc5f001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e33474e3850cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc5ff400640 1 -- 172.21.15.164:0/2859700538 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00b0070 con 0x7fc60015aab0 2024-05-08T04:44:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc5ff400640 1 -- 172.21.15.164:0/2859700538 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f00ab600 con 0x7fc60015aab0 2024-05-08T04:44:03.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc5ff400640 1 -- 172.21.15.164:0/2859700538 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00ab9d0 con 0x7fc60015aab0 2024-05-08T04:44:03.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 -- 172.21.15.164:0/2859700538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 msgr2=0x7fc6000cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 --2- 172.21.15.164:0/2859700538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 0x7fc6000cec50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097c0 tx=0x7fc5f001c4a0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 -- 172.21.15.164:0/2859700538 shutdown_connections 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 --2- 172.21.15.164:0/2859700538 >> v2:172.21.15.172:3300/0 conn(0x7fc6000c5530 0x7fc6000c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 --2- 172.21.15.164:0/2859700538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc60015aab0 0x7fc6000cec50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 -- 172.21.15.164:0/2859700538 >> 172.21.15.164:0/2859700538 conn(0x7fc60006b670 msgr2=0x7fc6000b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 -- 172.21.15.164:0/2859700538 shutdown_connections 2024-05-08T04:44:03.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.527+0000 7fc606000640 1 -- 172.21.15.164:0/2859700538 wait complete. 2024-05-08T04:44:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 Processor -- start 2024-05-08T04:44:03.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- start start 2024-05-08T04:44:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6000c5530 0x7fc600155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6000cbed0 0x7fc600156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000724b0 con 0x7fc60015aab0 2024-05-08T04:44:03.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000713f0 con 0x7fc6000c5530 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000ce7c0 con 0x7fc6000cbed0 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35296/0 (socket says 172.21.15.164:35296) 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 -- 172.21.15.164:0/4049272010 learned_addr learned my addr 172.21.15.164:0/4049272010 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc604c00640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6000c5530 0x7fc600155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc5ffe00640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6000cbed0 0x7fc600156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 -- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6000cbed0 msgr2=0x7fc600156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6000cbed0 0x7fc600156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 -- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6000c5530 msgr2=0x7fc600155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6000c5530 0x7fc600155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 -- 172.21.15.164:0/4049272010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600157850 con 0x7fc60015aab0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc605600640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800cb40 tx=0x7fc5f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8017070 con 0x7fc60015aab0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800aca0 con 0x7fc60015aab0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8012620 con 0x7fc60015aab0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc600157b30 con 0x7fc60015aab0 2024-05-08T04:44:03.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.531+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6001580f0 con 0x7fc60015aab0 2024-05-08T04:44:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60006c330 con 0x7fc60015aab0 2024-05-08T04:44:03.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fc5f801a070 con 0x7fc60015aab0 2024-05-08T04:44:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc5fcc00640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 0x7fc5e006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc5f8083ab0 con 0x7fc60015aab0 2024-05-08T04:44:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc604c00640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 0x7fc5e006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.535+0000 7fc604c00640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 0x7fc5e006a1a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ad000 tx=0x7fc5f00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.539+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc60006c330 con 0x7fc60015aab0 2024-05-08T04:44:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc600068170 con 0x7fc5e0067ce0 2024-05-08T04:44:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc5fcc00640 1 -- 172.21.15.164:0/4049272010 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc600068170 con 0x7fc5e0067ce0 2024-05-08T04:44:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 msgr2=0x7fc5e006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 0x7fc5e006a1a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ad000 tx=0x7fc5f00a6000 comp rx=0 tx=0).stop 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 msgr2=0x7fc6000ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800cb40 tx=0x7fc5f800a060 comp rx=0 tx=0).stop 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 shutdown_connections 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fc5e0067ce0 0x7fc5e006a1a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6000cbed0 0x7fc600156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc60015aab0 0x7fc6000ad890 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 --2- 172.21.15.164:0/4049272010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6000c5530 0x7fc600155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 >> 172.21.15.164:0/4049272010 conn(0x7fc60006b670 msgr2=0x7fc6000cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 shutdown_connections 2024-05-08T04:44:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.695+0000 7fc606000640 1 -- 172.21.15.164:0/4049272010 wait complete. 2024-05-08T04:44:03.716 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T04:44:03.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 Processor -- start 2024-05-08T04:44:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 -- start start 2024-05-08T04:44:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f968015ab10 0x7f968006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800cee90 0x7f96800aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968015e870 con 0x7f96800ae6e0 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968006bb80 con 0x7f968015ab10 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f9684c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968015eb50 con 0x7f96800cee90 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f968015ab10 0x7f968006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f968015ab10 0x7f968006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35304/0 (socket says 172.21.15.164:35304) 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 -- 172.21.15.164:0/1168649242 learned_addr learned my addr 172.21.15.164:0/1168649242 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 -- 172.21.15.164:0/1168649242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800cee90 msgr2=0x7f96800aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 --2- 172.21.15.164:0/1168649242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800cee90 0x7f96800aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 -- 172.21.15.164:0/1168649242 >> v1:172.21.15.172:6789/0 conn(0x7f96800ae6e0 legacy=0x7f96800ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.855+0000 7f967de00640 1 -- 172.21.15.164:0/1168649242 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968015ee30 con 0x7f968015ab10 2024-05-08T04:44:03.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f967de00640 1 --2- 172.21.15.164:0/1168649242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968006cce0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9670009870 tx=0x7f967001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313060cef9869e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f967d400640 1 -- 172.21.15.164:0/1168649242 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700ab600 con 0x7f968015ab10 2024-05-08T04:44:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f967d400640 1 -- 172.21.15.164:0/1168649242 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96700b0070 con 0x7f968015ab10 2024-05-08T04:44:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f967d400640 1 -- 172.21.15.164:0/1168649242 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700ab9c0 con 0x7f968015ab10 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- 172.21.15.164:0/1168649242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 msgr2=0x7f968006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 --2- 172.21.15.164:0/1168649242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968006cce0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9670009870 tx=0x7f967001c480 comp rx=0 tx=0).stop 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- 172.21.15.164:0/1168649242 shutdown_connections 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 --2- 172.21.15.164:0/1168649242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800cee90 0x7f96800aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 --2- 172.21.15.164:0/1168649242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968006cce0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- 172.21.15.164:0/1168649242 >> 172.21.15.164:0/1168649242 conn(0x7f96800cf770 msgr2=0x7f96800cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- 172.21.15.164:0/1168649242 shutdown_connections 2024-05-08T04:44:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- 172.21.15.164:0/1168649242 wait complete. 2024-05-08T04:44:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 Processor -- start 2024-05-08T04:44:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.859+0000 7f9684c00640 1 -- start start 2024-05-08T04:44:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800ae6e0 0x7f968015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800cee90 0x7f968014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800cb070 con 0x7f968015ab10 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800caef0 con 0x7f96800cee90 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800cb1f0 con 0x7f96800ae6e0 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35314/0 (socket says 172.21.15.164:35314) 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 -- 172.21.15.164:0/550539101 learned_addr learned my addr 172.21.15.164:0/550539101 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9675400640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800cee90 0x7f968014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967de00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800ae6e0 0x7f968015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 -- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800ae6e0 msgr2=0x7f968015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800ae6e0 0x7f968015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 -- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800cee90 msgr2=0x7f968014d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800cee90 0x7f968014d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968015e030 con 0x7f968015ab10 2024-05-08T04:44:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f967e800640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968014d630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f967800ba10 tx=0x7f967800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678019070 con 0x7f968015ab10 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967800cb20 con 0x7f968015ab10 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968015d190 con 0x7f968015ab10 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678014440 con 0x7f968015ab10 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968015b1e0 con 0x7f968015ab10 2024-05-08T04:44:03.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.863+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9680072d30 con 0x7f968015ab10 2024-05-08T04:44:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f967801c070 con 0x7f968015ab10 2024-05-08T04:44:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f965c067ce0 0x7f965c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f9678084090 con 0x7f968015ab10 2024-05-08T04:44:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f967de00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f965c067ce0 0x7f965c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f965c06dba0 0x7f965c070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96780091c0 con 0x7f965c06dba0 2024-05-08T04:44:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9678086db0 con 0x7f968015ab10 2024-05-08T04:44:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9675400640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f965c06dba0 0x7f965c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f967de00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f965c067ce0 0x7f965c06a1a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96700af000 tx=0x7f96700a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9675400640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f965c06dba0 0x7f965c070000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:03.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.867+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== osd.1 v2:172.21.15.164:6808/2349768482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f96780091c0 con 0x7f965c06dba0 2024-05-08T04:44:03.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.879+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 --> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f968004f010 con 0x7f965c06dba0 2024-05-08T04:44:03.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9676a00640 1 -- 172.21.15.164:0/550539101 <== osd.1 v2:172.21.15.164:6808/2349768482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f968004f010 con 0x7f965c06dba0 2024-05-08T04:44:03.883 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:03.883 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:03.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f965c06dba0 msgr2=0x7f965c070000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6808/2349768482,v1:172.21.15.164:6810/2349768482] conn(0x7f965c06dba0 0x7f965c070000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f965c067ce0 msgr2=0x7f965c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f965c067ce0 0x7f965c06a1a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96700af000 tx=0x7f96700a7000 comp rx=0 tx=0).stop 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 msgr2=0x7f968014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 --2- 172.21.15.164:0/550539101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968015ab10 0x7f968014d630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f967800ba10 tx=0x7f967800bed0 comp rx=0 tx=0).stop 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f967e800640 1 -- 172.21.15.164:0/550539101 reap_dead start 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 shutdown_connections 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 >> 172.21.15.164:0/550539101 conn(0x7f96800cf770 msgr2=0x7f968006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 shutdown_connections 2024-05-08T04:44:03.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:03.883+0000 7f9684c00640 1 -- 172.21.15.164:0/550539101 wait complete. 2024-05-08T04:44:03.901 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T04:44:04.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.035+0000 7f52cb600640 1 Processor -- start 2024-05-08T04:44:04.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.035+0000 7f52cb600640 1 -- start start 2024-05-08T04:44:04.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52cb600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52c415bc50 0x7f52c40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52cb600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40b42a0 con 0x7f52c40d0780 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52cb600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40cd4e0 con 0x7f52c40ccfe0 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52cb600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40cd7c0 con 0x7f52c415bc50 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52c415bc50 0x7f52c40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52c415bc50 0x7f52c40b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35318/0 (socket says 172.21.15.164:35318) 2024-05-08T04:44:04.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 -- 172.21.15.164:0/395411618 learned_addr learned my addr 172.21.15.164:0/395411618 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 -- 172.21.15.164:0/395411618 >> v1:172.21.15.172:6789/0 conn(0x7f52c40ccfe0 legacy=0x7f52c40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 -- 172.21.15.164:0/395411618 >> v1:172.21.15.164:6789/0 conn(0x7f52c40d0780 legacy=0x7f52c415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 -- 172.21.15.164:0/395411618 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40cf590 con 0x7f52c415bc50 2024-05-08T04:44:04.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c3400640 1 -- 172.21.15.164:0/395411618 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326525644 0 0) 0x7f52c40b42a0 con 0x7f52c40d0780 2024-05-08T04:44:04.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.039+0000 7f52c8c00640 1 --2- 172.21.15.164:0/395411618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c415bc50 0x7f52c40b4810 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69127f28566bed95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3400640 1 -- 172.21.15.164:0/395411618 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40a5830 con 0x7f52c415bc50 2024-05-08T04:44:04.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3400640 1 -- 172.21.15.164:0/395411618 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b40a59d0 con 0x7f52c415bc50 2024-05-08T04:44:04.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3400640 1 -- 172.21.15.164:0/395411618 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40a5da0 con 0x7f52c415bc50 2024-05-08T04:44:04.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- 172.21.15.164:0/395411618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c415bc50 msgr2=0x7f52c40b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 --2- 172.21.15.164:0/395411618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c415bc50 0x7f52c40b4810 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b401c970 comp rx=0 tx=0).stop 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- 172.21.15.164:0/395411618 shutdown_connections 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 --2- 172.21.15.164:0/395411618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c415bc50 0x7f52c40b4810 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- 172.21.15.164:0/395411618 >> 172.21.15.164:0/395411618 conn(0x7f52c40b34f0 msgr2=0x7f52c40b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- 172.21.15.164:0/395411618 shutdown_connections 2024-05-08T04:44:04.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- 172.21.15.164:0/395411618 wait complete. 2024-05-08T04:44:04.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 Processor -- start 2024-05-08T04:44:04.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- start start 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52c40ccfe0 0x7f52c4156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c415bc50 0x7f52c4156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4169c70 con 0x7f52c40d0780 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4169af0 con 0x7f52c415bc50 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52cb600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4169df0 con 0x7f52c40ccfe0 2024-05-08T04:44:04.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35322/0 (socket says 172.21.15.164:35322) 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c3e00640 1 -- 172.21.15.164:0/2567703712 learned_addr learned my addr 172.21.15.164:0/2567703712 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c8c00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52c40ccfe0 0x7f52c4156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.043+0000 7f52c9600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c415bc50 0x7f52c4156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 -- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52c40ccfe0 msgr2=0x7f52c4156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52c40ccfe0 0x7f52c4156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 -- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c415bc50 msgr2=0x7f52c4156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c415bc50 0x7f52c4156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 -- 172.21.15.164:0/2567703712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40c96e0 con 0x7f52c40d0780 2024-05-08T04:44:04.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c3e00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c3c0 tx=0x7f52b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0017070 con 0x7f52c40d0780 2024-05-08T04:44:04.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b00052d0 con 0x7f52c40d0780 2024-05-08T04:44:04.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003840 con 0x7f52c40d0780 2024-05-08T04:44:04.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c415c4f0 con 0x7f52c40d0780 2024-05-08T04:44:04.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c415cab0 con 0x7f52c40d0780 2024-05-08T04:44:04.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4069d60 con 0x7f52c40d0780 2024-05-08T04:44:04.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f52b001a070 con 0x7f52c40d0780 2024-05-08T04:44:04.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 0x7f52a406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.047+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f52b0082fb0 con 0x7f52c40d0780 2024-05-08T04:44:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.051+0000 7f52c8c00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 0x7f52a406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.051+0000 7f52c8c00640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 0x7f52a406a1a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f52b40ae000 tx=0x7f52b40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.051+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52c40c96e0 con 0x7f52c40d0780 2024-05-08T04:44:04.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.203+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f52c415c2a0 con 0x7f52a4067ce0 2024-05-08T04:44:04.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52c0c00640 1 -- 172.21.15.164:0/2567703712 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f52c415c2a0 con 0x7f52a4067ce0 2024-05-08T04:44:04.207 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 msgr2=0x7f52a406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 0x7f52a406a1a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f52b40ae000 tx=0x7f52b40a7070 comp rx=0 tx=0).stop 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 msgr2=0x7f52c40c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c3c0 tx=0x7f52b000c880 comp rx=0 tx=0).stop 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 shutdown_connections 2024-05-08T04:44:04.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f52a4067ce0 0x7f52a406a1a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c415bc50 0x7f52c4156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c40d0780 0x7f52c40c91a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 --2- 172.21.15.164:0/2567703712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52c40ccfe0 0x7f52c4156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 >> 172.21.15.164:0/2567703712 conn(0x7f52c40b34f0 msgr2=0x7f52c415af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 shutdown_connections 2024-05-08T04:44:04.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.207+0000 7f52cb600640 1 -- 172.21.15.164:0/2567703712 wait complete. 2024-05-08T04:44:04.228 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T04:44:04.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:04.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 Processor -- start 2024-05-08T04:44:04.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 -- start start 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8dc15ba20 0x7fe8dc160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe8dc06de50 0x7fe8dc06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0c6600 con 0x7fe8dc0ae6e0 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc15fbe0 con 0x7fe8dc15ba20 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e2e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c6040 con 0x7fe8dc06de50 2024-05-08T04:44:04.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8dbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8dc15ba20 0x7fe8dc160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e0e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe8dc0ae6e0 0x7fe8dc0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50486/0 (socket says 172.21.15.164:50486) 2024-05-08T04:44:04.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8e0e00640 1 -- 172.21.15.164:0/2945078841 learned_addr learned my addr 172.21.15.164:0/2945078841 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:04.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8db400640 1 --2- 172.21.15.164:0/2945078841 >> v2:172.21.15.172:3300/0 conn(0x7fe8dc06de50 0x7fe8dc06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8dbe00640 1 -- 172.21.15.164:0/2945078841 >> v2:172.21.15.172:3300/0 conn(0x7fe8dc06de50 msgr2=0x7fe8dc06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:04.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8dbe00640 1 --2- 172.21.15.164:0/2945078841 >> v2:172.21.15.172:3300/0 conn(0x7fe8dc06de50 0x7fe8dc06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8dbe00640 1 -- 172.21.15.164:0/2945078841 >> v1:172.21.15.164:6789/0 conn(0x7fe8dc0ae6e0 legacy=0x7fe8dc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.367+0000 7fe8dbe00640 1 -- 172.21.15.164:0/2945078841 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0c6320 con 0x7fe8dc15ba20 2024-05-08T04:44:04.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8dbe00640 1 --2- 172.21.15.164:0/2945078841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc160070 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0b5980 tx=0x7fe8cc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8e092479ca289 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8daa00640 1 -- 172.21.15.164:0/2945078841 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0b2070 con 0x7fe8dc15ba20 2024-05-08T04:44:04.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8daa00640 1 -- 172.21.15.164:0/2945078841 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0a5960 con 0x7fe8dc15ba20 2024-05-08T04:44:04.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8daa00640 1 -- 172.21.15.164:0/2945078841 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0a5d30 con 0x7fe8dc15ba20 2024-05-08T04:44:04.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 -- 172.21.15.164:0/2945078841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc15ba20 msgr2=0x7fe8dc160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/2945078841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc160070 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0b5980 tx=0x7fe8cc01c900 comp rx=0 tx=0).stop 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 -- 172.21.15.164:0/2945078841 shutdown_connections 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/2945078841 >> v2:172.21.15.172:3300/0 conn(0x7fe8dc06de50 0x7fe8dc06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/2945078841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc160070 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 -- 172.21.15.164:0/2945078841 >> 172.21.15.164:0/2945078841 conn(0x7fe8dc0cf770 msgr2=0x7fe8dc0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 -- 172.21.15.164:0/2945078841 shutdown_connections 2024-05-08T04:44:04.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.371+0000 7fe8e2e00640 1 -- 172.21.15.164:0/2945078841 wait complete. 2024-05-08T04:44:04.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 Processor -- start 2024-05-08T04:44:04.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- start start 2024-05-08T04:44:04.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 0x7fe8dc150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8dc0ae6e0 0x7fe8dc0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0aec40 con 0x7fe8dc06de50 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0aeac0 con 0x7fe8dc15ba20 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0aedc0 con 0x7fe8dc0ae6e0 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 0x7fe8dc150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 0x7fe8dc150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35334/0 (socket says 172.21.15.164:35334) 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 -- 172.21.15.164:0/594346308 learned_addr learned my addr 172.21.15.164:0/594346308 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e0e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8db400640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8dc0ae6e0 0x7fe8dc0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 -- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8dc0ae6e0 msgr2=0x7fe8dc0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8dc0ae6e0 0x7fe8dc0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 -- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8dc15ba20 msgr2=0x7fe8dc0b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc0b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc146d70 con 0x7fe8dc06de50 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e0e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8dc15ba20 0x7fe8dc0b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:04.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8db400640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8dc0ae6e0 0x7fe8dc0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8dbe00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 0x7fe8dc150390 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc019510 tx=0x7fe8cc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0b2050 con 0x7fe8dc06de50 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0be070 con 0x7fe8dc06de50 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0ada70 con 0x7fe8dc06de50 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0c83d0 con 0x7fe8dc06de50 2024-05-08T04:44:04.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0afca0 con 0x7fe8dc06de50 2024-05-08T04:44:04.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.375+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8dc0c88f0 con 0x7fe8dc06de50 2024-05-08T04:44:04.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fe8cc0adc10 con 0x7fe8dc06de50 2024-05-08T04:44:04.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8bc067ce0 0x7fe8bc06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fe8cc11fb90 con 0x7fe8dc06de50 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8db400640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8bc067ce0 0x7fe8bc06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7fe8bc06dba0 0x7fe8bc070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8cc0c5e70 con 0x7fe8bc06dba0 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8cc122470 con 0x7fe8dc06de50 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8e0e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7fe8bc06dba0 0x7fe8bc070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8db400640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8bc067ce0 0x7fe8bc06a1a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0b5660 tx=0x7fe8c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8e0e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7fe8bc06dba0 0x7fe8bc070000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.379+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== osd.2 v2:172.21.15.164:6824/4253785060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe8cc0c5e70 con 0x7fe8bc06dba0 2024-05-08T04:44:04.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 --> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe8dc072d30 con 0x7fe8bc06dba0 2024-05-08T04:44:04.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8d3e00640 1 -- 172.21.15.164:0/594346308 <== osd.2 v2:172.21.15.164:6824/4253785060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe8dc072d30 con 0x7fe8bc06dba0 2024-05-08T04:44:04.395 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:04.395 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:04.395 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7fe8bc06dba0 msgr2=0x7fe8bc070000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6824/4253785060,v1:172.21.15.164:6825/4253785060] conn(0x7fe8bc06dba0 0x7fe8bc070000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8bc067ce0 msgr2=0x7fe8bc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe8bc067ce0 0x7fe8bc06a1a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0b5660 tx=0x7fe8c8008040 comp rx=0 tx=0).stop 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 msgr2=0x7fe8dc150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 --2- 172.21.15.164:0/594346308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8dc06de50 0x7fe8dc150390 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc019510 tx=0x7fe8cc0196a0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e0e00640 1 -- 172.21.15.164:0/594346308 reap_dead start 2024-05-08T04:44:04.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 shutdown_connections 2024-05-08T04:44:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 >> 172.21.15.164:0/594346308 conn(0x7fe8dc0cf770 msgr2=0x7fe8dc15d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 shutdown_connections 2024-05-08T04:44:04.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.395+0000 7fe8e2e00640 1 -- 172.21.15.164:0/594346308 wait complete. 2024-05-08T04:44:04.414 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T04:44:04.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 Processor -- start 2024-05-08T04:44:04.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 -- start start 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9bc15be50 0x7fb9bc160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0cd9f0 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0cdcd0 con 0x7fb9bc15ba70 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9c0a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc15fe70 con 0x7fb9bc15be50 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9bc15be50 0x7fb9bc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9bc15be50 0x7fb9bc160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35350/0 (socket says 172.21.15.164:35350) 2024-05-08T04:44:04.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 -- 172.21.15.164:0/1232048166 learned_addr learned my addr 172.21.15.164:0/1232048166 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9b9400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb9bc0cfda0 0x7fb9bc0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50492/0 (socket says 172.21.15.164:50492) 2024-05-08T04:44:04.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 -- 172.21.15.164:0/1232048166 >> v1:172.21.15.172:6789/0 conn(0x7fb9bc15ba70 legacy=0x7fb9bc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 -- 172.21.15.164:0/1232048166 >> v1:172.21.15.164:6789/0 conn(0x7fb9bc0cfda0 legacy=0x7fb9bc0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9ba800640 1 -- 172.21.15.164:0/1232048166 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc15f820 con 0x7fb9bc15be50 2024-05-08T04:44:04.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.551+0000 7fb9b8a00640 1 -- 172.21.15.164:0/1232048166 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911392870 0 0) 0x7fb9bc0cd9f0 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9ba800640 1 --2- 172.21.15.164:0/1232048166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc15be50 0x7fb9bc160300 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da59b2a90469df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9b8a00640 1 -- 172.21.15.164:0/1232048166 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40ab640 con 0x7fb9bc15be50 2024-05-08T04:44:04.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9b8a00640 1 -- 172.21.15.164:0/1232048166 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b40b0070 con 0x7fb9bc15be50 2024-05-08T04:44:04.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9b8a00640 1 -- 172.21.15.164:0/1232048166 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40aba00 con 0x7fb9bc15be50 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1232048166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc15be50 msgr2=0x7fb9bc160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1232048166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc15be50 0x7fb9bc160300 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b401c530 comp rx=0 tx=0).stop 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1232048166 shutdown_connections 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1232048166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc15be50 0x7fb9bc160300 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1232048166 >> 172.21.15.164:0/1232048166 conn(0x7fb9bc06ac90 msgr2=0x7fb9bc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1232048166 shutdown_connections 2024-05-08T04:44:04.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.555+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1232048166 wait complete. 2024-05-08T04:44:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 Processor -- start 2024-05-08T04:44:04.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- start start 2024-05-08T04:44:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9bc15ba70 0x7fb9bc14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9bc15be50 0x7fb9bc1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc169f60 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc169de0 con 0x7fb9bc15be50 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc16a0e0 con 0x7fb9bc15ba70 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35358/0 (socket says 172.21.15.164:35358) 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 -- 172.21.15.164:0/1206770383 learned_addr learned my addr 172.21.15.164:0/1206770383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9ba800640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9bc15be50 0x7fb9bc1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9400640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9bc15ba70 0x7fb9bc14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 -- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9bc15ba70 msgr2=0x7fb9bc14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9bc15ba70 0x7fb9bc14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 -- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9bc15be50 msgr2=0x7fb9bc1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9bc15be50 0x7fb9bc1431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 -- 172.21.15.164:0/1206770383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc14f570 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b9e00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00c820 tx=0x7fb9ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac003b80 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac003d20 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00aa90 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc14ea50 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.559+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc14efb0 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc069d60 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fb9ac00ac30 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9b2000640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 0x7fb99c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb9ac083e20 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9b9400640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 0x7fb99c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.563+0000 7fb9b9400640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 0x7fb99c06a1a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.567+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9bc14ea50 con 0x7fb9bc0cfda0 2024-05-08T04:44:04.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9bc06e210 con 0x7fb99c067ce0 2024-05-08T04:44:04.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9b2000640 1 -- 172.21.15.164:0/1206770383 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb9bc06e210 con 0x7fb99c067ce0 2024-05-08T04:44:04.728 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T04:44:04.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 msgr2=0x7fb99c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 0x7fb99c06a1a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8007ba0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 msgr2=0x7fb9bc14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00c820 tx=0x7fb9ac00cce0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 shutdown_connections 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb99c067ce0 0x7fb99c06a1a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9bc15ba70 0x7fb9bc14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9bc15be50 0x7fb9bc1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 --2- 172.21.15.164:0/1206770383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc0cfda0 0x7fb9bc14aee0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 >> 172.21.15.164:0/1206770383 conn(0x7fb9bc06ac90 msgr2=0x7fb9bc0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 shutdown_connections 2024-05-08T04:44:04.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.727+0000 7fb9c0a00640 1 -- 172.21.15.164:0/1206770383 wait complete. 2024-05-08T04:44:04.749 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T04:44:04.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:04.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.887+0000 7f6e4c800640 1 Processor -- start 2024-05-08T04:44:04.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.887+0000 7f6e4c800640 1 -- start start 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.887+0000 7f6e4c800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e4415ce10 0x7f6e4406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e4c800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440c6600 con 0x7f6e4406e390 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e4c800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440c6040 con 0x7f6e4415ba20 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e4c800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4415fbe0 con 0x7f6e4415ce10 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e4a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6e4406e390 0x7f6e440ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50496/0 (socket says 172.21.15.164:50496) 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e4a800640 1 -- 172.21.15.164:0/373038571 learned_addr learned my addr 172.21.15.164:0/373038571 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:04.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e49400640 1 --2- 172.21.15.164:0/373038571 >> v2:172.21.15.172:3300/0 conn(0x7f6e4415ce10 0x7f6e4406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e48a00640 1 -- 172.21.15.164:0/373038571 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134856070 0 0) 0x7f6e440c6600 con 0x7f6e4406e390 2024-05-08T04:44:04.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e48a00640 1 -- 172.21.15.164:0/373038571 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c003530 con 0x7f6e4406e390 2024-05-08T04:44:04.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e49400640 1 -- 172.21.15.164:0/373038571 >> v1:172.21.15.172:6789/0 conn(0x7f6e4415ba20 legacy=0x7f6e44160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e49400640 1 -- 172.21.15.164:0/373038571 >> v1:172.21.15.164:6789/0 conn(0x7f6e4406e390 legacy=0x7f6e440ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e49400640 1 -- 172.21.15.164:0/373038571 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440c6320 con 0x7f6e4415ce10 2024-05-08T04:44:04.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.891+0000 7f6e49400640 1 --2- 172.21.15.164:0/373038571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ce10 0x7f6e4406de50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e3401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36949388ce727be3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e48a00640 1 -- 172.21.15.164:0/373038571 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340b0070 con 0x7f6e4415ce10 2024-05-08T04:44:04.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e48a00640 1 -- 172.21.15.164:0/373038571 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e340ab600 con 0x7f6e4415ce10 2024-05-08T04:44:04.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e48a00640 1 -- 172.21.15.164:0/373038571 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340ab9d0 con 0x7f6e4415ce10 2024-05-08T04:44:04.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- 172.21.15.164:0/373038571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ce10 msgr2=0x7f6e4406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 --2- 172.21.15.164:0/373038571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ce10 0x7f6e4406de50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e3401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- 172.21.15.164:0/373038571 shutdown_connections 2024-05-08T04:44:04.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 --2- 172.21.15.164:0/373038571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ce10 0x7f6e4406de50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- 172.21.15.164:0/373038571 >> 172.21.15.164:0/373038571 conn(0x7f6e440cf770 msgr2=0x7f6e440cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- 172.21.15.164:0/373038571 shutdown_connections 2024-05-08T04:44:04.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- 172.21.15.164:0/373038571 wait complete. 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 Processor -- start 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- start start 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 0x7f6e440b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ba20 0x7f6e440b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4415ce10 0x7f6e440b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440ce000 con 0x7f6e4406e390 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cde80 con 0x7f6e4415ba20 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e4c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44169b80 con 0x7f6e4415ce10 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 0x7f6e440b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e49e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 0x7f6e440b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35370/0 (socket says 172.21.15.164:35370) 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.895+0000 7f6e49e00640 1 -- 172.21.15.164:0/114150078 learned_addr learned my addr 172.21.15.164:0/114150078 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49400640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ba20 0x7f6e440b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e4a800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4415ce10 0x7f6e440b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 -- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4415ce10 msgr2=0x7f6e440b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4415ce10 0x7f6e440b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 -- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ba20 msgr2=0x7f6e440b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4415ba20 0x7f6e440b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440b1050 con 0x7f6e4406e390 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49e00640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 0x7f6e440b44e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c500 tx=0x7f6e3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38016020 con 0x7f6e4406e390 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38005220 con 0x7f6e4406e390 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440b4b50 con 0x7f6e4406e390 2024-05-08T04:44:04.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38007030 con 0x7f6e4406e390 2024-05-08T04:44:04.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440b5110 con 0x7f6e4406e390 2024-05-08T04:44:04.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e44072d30 con 0x7f6e4406e390 2024-05-08T04:44:04.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f6e380071f0 con 0x7f6e4406e390 2024-05-08T04:44:04.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e28067ce0 0x7f6e2806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f6e38083640 con 0x7f6e4406e390 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7f6e2806dba0 0x7f6e28070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e49400640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e28067ce0 0x7f6e2806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.899+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e38020e50 con 0x7f6e2806dba0 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.903+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e38088600 con 0x7f6e4406e390 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.903+0000 7f6e4a800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7f6e2806dba0 0x7f6e28070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.903+0000 7f6e4a800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7f6e2806dba0 0x7f6e28070000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.903+0000 7f6e49400640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e28067ce0 0x7f6e2806a1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6e340ad000 tx=0x7f6e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:04.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.903+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== osd.3 v2:172.21.15.164:6809/2335523075 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6e38020e50 con 0x7f6e2806dba0 2024-05-08T04:44:04.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 --> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6e4404f010 con 0x7f6e2806dba0 2024-05-08T04:44:04.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e3e000640 1 -- 172.21.15.164:0/114150078 <== osd.3 v2:172.21.15.164:6809/2335523075 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6e4404f010 con 0x7f6e2806dba0 2024-05-08T04:44:04.918 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:04.918 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:04.918 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:04.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7f6e2806dba0 msgr2=0x7f6e28070000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6809/2335523075,v1:172.21.15.164:6811/2335523075] conn(0x7f6e2806dba0 0x7f6e28070000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e28067ce0 msgr2=0x7f6e2806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f6e28067ce0 0x7f6e2806a1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6e340ad000 tx=0x7f6e34008040 comp rx=0 tx=0).stop 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 msgr2=0x7f6e440b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 --2- 172.21.15.164:0/114150078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e4406e390 0x7f6e440b44e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c500 tx=0x7f6e3800c9c0 comp rx=0 tx=0).stop 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4a800640 1 -- 172.21.15.164:0/114150078 reap_dead start 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 shutdown_connections 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 >> 172.21.15.164:0/114150078 conn(0x7f6e440cf770 msgr2=0x7f6e4415e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 shutdown_connections 2024-05-08T04:44:04.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:04.915+0000 7f6e4c800640 1 -- 172.21.15.164:0/114150078 wait complete. 2024-05-08T04:44:04.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T04:44:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 Processor -- start 2024-05-08T04:44:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 -- start start 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe62c163540 0x7fe62c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62c0ceac0 con 0x7fe62c0cf6c0 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62c0cc1e0 con 0x7fe62c0aedb0 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe632200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62c0ce860 con 0x7fe62c163540 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe62c163540 0x7fe62c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe62c163540 0x7fe62c0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35386/0 (socket says 172.21.15.164:35386) 2024-05-08T04:44:05.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 -- 172.21.15.164:0/124763522 learned_addr learned my addr 172.21.15.164:0/124763522 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62be00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe62c0cf6c0 0x7fe62c06cfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50506/0 (socket says 172.21.15.164:50506) 2024-05-08T04:44:05.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 -- 172.21.15.164:0/124763522 >> v1:172.21.15.172:6789/0 conn(0x7fe62c0aedb0 legacy=0x7fe62c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 -- 172.21.15.164:0/124763522 >> v1:172.21.15.164:6789/0 conn(0x7fe62c0cf6c0 legacy=0x7fe62c06cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.071+0000 7fe62aa00640 1 -- 172.21.15.164:0/124763522 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0cc4c0 con 0x7fe62c163540 2024-05-08T04:44:05.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe62aa00640 1 --2- 172.21.15.164:0/124763522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c163540 0x7fe62c0cccc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe618009870 tx=0x7fe62c06df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457a4b32a9d5e1f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe62a000640 1 -- 172.21.15.164:0/124763522 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6180aa620 con 0x7fe62c163540 2024-05-08T04:44:05.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe62a000640 1 -- 172.21.15.164:0/124763522 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6180af070 con 0x7fe62c163540 2024-05-08T04:44:05.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe62a000640 1 -- 172.21.15.164:0/124763522 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6180aa9e0 con 0x7fe62c163540 2024-05-08T04:44:05.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 -- 172.21.15.164:0/124763522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c163540 msgr2=0x7fe62c0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 --2- 172.21.15.164:0/124763522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c163540 0x7fe62c0cccc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe618009870 tx=0x7fe62c06df00 comp rx=0 tx=0).stop 2024-05-08T04:44:05.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 -- 172.21.15.164:0/124763522 shutdown_connections 2024-05-08T04:44:05.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 --2- 172.21.15.164:0/124763522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c163540 0x7fe62c0cccc0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 -- 172.21.15.164:0/124763522 >> 172.21.15.164:0/124763522 conn(0x7fe62c06ac90 msgr2=0x7fe62c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 -- 172.21.15.164:0/124763522 shutdown_connections 2024-05-08T04:44:05.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.075+0000 7fe632200640 1 -- 172.21.15.164:0/124763522 wait complete. 2024-05-08T04:44:05.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 Processor -- start 2024-05-08T04:44:05.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- start start 2024-05-08T04:44:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62c0cf6c0 0x7fe62c166460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62c163540 0x7fe62c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c170350 con 0x7fe62c0aedb0 2024-05-08T04:44:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1701d0 con 0x7fe62c163540 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1704d0 con 0x7fe62c0cf6c0 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35390/0 (socket says 172.21.15.164:35390) 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 -- 172.21.15.164:0/941777274 learned_addr learned my addr 172.21.15.164:0/941777274 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62be00640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62c163540 0x7fe62c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62aa00640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62c0cf6c0 0x7fe62c166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 -- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62c0cf6c0 msgr2=0x7fe62c166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62c0cf6c0 0x7fe62c166460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 -- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62c163540 msgr2=0x7fe62c160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62c163540 0x7fe62c160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 -- 172.21.15.164:0/941777274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c1416c0 con 0x7fe62c0aedb0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe62b400640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00c790 tx=0x7fe61c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c016020 con 0x7fe62c0aedb0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c006d60 con 0x7fe62c0aedb0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c003bc0 con 0x7fe62c0aedb0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c144000 con 0x7fe62c0aedb0 2024-05-08T04:44:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.079+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c1445c0 con 0x7fe62c0aedb0 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c069d60 con 0x7fe62c0aedb0 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fe61c00a850 con 0x7fe62c0aedb0 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe623400640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 0x7fe60c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fe61c083ac0 con 0x7fe62c0aedb0 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe62aa00640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 0x7fe60c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.083+0000 7fe62aa00640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 0x7fe60c06a1a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe6180ae000 tx=0x7fe618004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.087+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe61c020020 con 0x7fe62c0aedb0 2024-05-08T04:44:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe62c0ae660 con 0x7fe60c067ce0 2024-05-08T04:44:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe623400640 1 -- 172.21.15.164:0/941777274 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe62c0ae660 con 0x7fe60c067ce0 2024-05-08T04:44:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T04:44:05.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 msgr2=0x7fe60c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 0x7fe60c06a1a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe6180ae000 tx=0x7fe618004740 comp rx=0 tx=0).stop 2024-05-08T04:44:05.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 msgr2=0x7fe62c1ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00c790 tx=0x7fe61c00cc50 comp rx=0 tx=0).stop 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 shutdown_connections 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fe60c067ce0 0x7fe60c06a1a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe62c0cf6c0 0x7fe62c166460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62c163540 0x7fe62c160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 --2- 172.21.15.164:0/941777274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62c0aedb0 0x7fe62c1ffa20 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 >> 172.21.15.164:0/941777274 conn(0x7fe62c06ac90 msgr2=0x7fe62c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 shutdown_connections 2024-05-08T04:44:05.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.243+0000 7fe632200640 1 -- 172.21.15.164:0/941777274 wait complete. 2024-05-08T04:44:05.265 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T04:44:05.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:05.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 Processor -- start 2024-05-08T04:44:05.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 -- start start 2024-05-08T04:44:05.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4fd415ba20 0x7f4fd4160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 0x7f4fd406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40c6600 con 0x7f4fd40ae6e0 2024-05-08T04:44:05.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd415fbe0 con 0x7f4fd415ba20 2024-05-08T04:44:05.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.403+0000 7f4fd9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40c6040 con 0x7f4fd406de50 2024-05-08T04:44:05.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd3400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4fd40ae6e0 0x7f4fd40cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.164:53222/0 (socket says 172.21.15.164:53222) 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd3400640 1 -- 172.21.15.164:0/1561399191 learned_addr learned my addr 172.21.15.164:0/1561399191 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 --2- 172.21.15.164:0/1561399191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 0x7f4fd406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2a00640 1 --2- 172.21.15.164:0/1561399191 >> v2:172.21.15.172:3300/0 conn(0x7f4fd415ba20 0x7f4fd4160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2000640 1 -- 172.21.15.164:0/1561399191 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417933266 0 0) 0x7f4fd40c6600 con 0x7f4fd40ae6e0 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2000640 1 -- 172.21.15.164:0/1561399191 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8003530 con 0x7f4fd40ae6e0 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 -- 172.21.15.164:0/1561399191 >> v2:172.21.15.172:3300/0 conn(0x7f4fd415ba20 msgr2=0x7f4fd4160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 --2- 172.21.15.164:0/1561399191 >> v2:172.21.15.172:3300/0 conn(0x7f4fd415ba20 0x7f4fd4160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 -- 172.21.15.164:0/1561399191 >> v1:172.21.15.172:6789/0 conn(0x7f4fd40ae6e0 legacy=0x7f4fd40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 -- 172.21.15.164:0/1561399191 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40c6320 con 0x7f4fd406de50 2024-05-08T04:44:05.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fca000640 1 --2- 172.21.15.164:0/1561399191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 0x7f4fd406e290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4fd415fd60 tx=0x7f4fbc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e7063baafa459b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2000640 1 -- 172.21.15.164:0/1561399191 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0ab610 con 0x7f4fd406de50 2024-05-08T04:44:05.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2000640 1 -- 172.21.15.164:0/1561399191 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc0b0070 con 0x7f4fd406de50 2024-05-08T04:44:05.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.407+0000 7f4fd2000640 1 -- 172.21.15.164:0/1561399191 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0ab9d0 con 0x7f4fd406de50 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- 172.21.15.164:0/1561399191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 msgr2=0x7f4fd406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/1561399191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 0x7f4fd406e290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4fd415fd60 tx=0x7f4fbc01c470 comp rx=0 tx=0).stop 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- 172.21.15.164:0/1561399191 shutdown_connections 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/1561399191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd406de50 0x7f4fd406e290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/1561399191 >> v2:172.21.15.172:3300/0 conn(0x7f4fd415ba20 0x7f4fd4160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- 172.21.15.164:0/1561399191 >> 172.21.15.164:0/1561399191 conn(0x7f4fd40cf770 msgr2=0x7f4fd40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- 172.21.15.164:0/1561399191 shutdown_connections 2024-05-08T04:44:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- 172.21.15.164:0/1561399191 wait complete. 2024-05-08T04:44:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 Processor -- start 2024-05-08T04:44:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- start start 2024-05-08T04:44:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 0x7f4fd414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ae6e0 0x7f4fd4152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd415ba20 0x7f4fd4152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40aec40 con 0x7f4fd406de50 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40aeac0 con 0x7f4fd415ba20 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40aedc0 con 0x7f4fd40ae6e0 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 0x7f4fd414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 0x7f4fd414fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35398/0 (socket says 172.21.15.164:35398) 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd2a00640 1 -- 172.21.15.164:0/3224193238 learned_addr learned my addr 172.21.15.164:0/3224193238 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fca000640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ae6e0 0x7f4fd4152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.411+0000 7f4fd3400640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd415ba20 0x7f4fd4152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 -- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ae6e0 msgr2=0x7f4fd4152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fd40ae6e0 0x7f4fd4152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 -- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd415ba20 msgr2=0x7f4fd4152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd415ba20 0x7f4fd4152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd41551f0 con 0x7f4fd406de50 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd2a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 0x7f4fd414fc10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400e960 tx=0x7f4fc400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4017070 con 0x7f4fd406de50 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400ca00 con 0x7f4fd406de50 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd4150150 con 0x7f4fd406de50 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400cdb0 con 0x7f4fd406de50 2024-05-08T04:44:05.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd4155710 con 0x7f4fd406de50 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fd4072d30 con 0x7f4fd406de50 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f4fc401a070 con 0x7f4fd406de50 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb0067ce0 0x7f4fb006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f4fc4083ce0 con 0x7f4fd406de50 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fca000640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb0067ce0 0x7f4fb006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f4fb006dba0 0x7f4fb0070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fb00706d0 con 0x7f4fb006dba0 2024-05-08T04:44:05.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.415+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc408d050 con 0x7f4fd406de50 2024-05-08T04:44:05.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.419+0000 7f4fd3400640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f4fb006dba0 0x7f4fb0070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.419+0000 7f4fca000640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb0067ce0 0x7f4fb006a1a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0af000 tx=0x7f4fbc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.419+0000 7f4fd3400640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f4fb006dba0 0x7f4fb0070000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.419+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== osd.4 v2:172.21.15.172:6800/3510673411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4fb00706d0 con 0x7f4fb006dba0 2024-05-08T04:44:05.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 --> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4fd4160330 con 0x7f4fb006dba0 2024-05-08T04:44:05.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fcb400640 1 -- 172.21.15.164:0/3224193238 <== osd.4 v2:172.21.15.172:6800/3510673411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4fd4160330 con 0x7f4fb006dba0 2024-05-08T04:44:05.434 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:05.434 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:05.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f4fb006dba0 msgr2=0x7f4fb0070000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.172:6800/3510673411,v1:172.21.15.172:6801/3510673411] conn(0x7f4fb006dba0 0x7f4fb0070000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb0067ce0 msgr2=0x7f4fb006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4fb0067ce0 0x7f4fb006a1a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0af000 tx=0x7f4fbc0a7000 comp rx=0 tx=0).stop 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 msgr2=0x7f4fd414fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 --2- 172.21.15.164:0/3224193238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd406de50 0x7f4fd414fc10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400e960 tx=0x7f4fc400ee20 comp rx=0 tx=0).stop 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd3400640 1 -- 172.21.15.164:0/3224193238 reap_dead start 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 shutdown_connections 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 >> 172.21.15.164:0/3224193238 conn(0x7f4fd40cf770 msgr2=0x7f4fd40cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 shutdown_connections 2024-05-08T04:44:05.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.431+0000 7f4fd9a00640 1 -- 172.21.15.164:0/3224193238 wait complete. 2024-05-08T04:44:05.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T04:44:05.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 Processor -- start 2024-05-08T04:44:05.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 -- start start 2024-05-08T04:44:05.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb41015ba70 0x7fb410160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100cd9f0 con 0x7fb4100cfda0 2024-05-08T04:44:05.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100cdcd0 con 0x7fb41015be50 2024-05-08T04:44:05.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb417c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41015fe70 con 0x7fb41015ba70 2024-05-08T04:44:05.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb41015ba70 0x7fb410160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb4100cfda0 0x7fb4100b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50508/0 (socket says 172.21.15.164:50508) 2024-05-08T04:44:05.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415c00640 1 -- 172.21.15.164:0/2002234868 learned_addr learned my addr 172.21.15.164:0/2002234868 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:05.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb41015ba70 0x7fb410160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35402/0 (socket says 172.21.15.164:35402) 2024-05-08T04:44:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415200640 1 -- 172.21.15.164:0/2002234868 >> v1:172.21.15.172:6789/0 conn(0x7fb41015be50 legacy=0x7fb4100cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415200640 1 -- 172.21.15.164:0/2002234868 >> v1:172.21.15.164:6789/0 conn(0x7fb4100cfda0 legacy=0x7fb4100b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb415200640 1 -- 172.21.15.164:0/2002234868 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41015f820 con 0x7fb41015ba70 2024-05-08T04:44:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.591+0000 7fb40f400640 1 -- 172.21.15.164:0/2002234868 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821116842 0 0) 0x7fb4100cd9f0 con 0x7fb4100cfda0 2024-05-08T04:44:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb415200640 1 --2- 172.21.15.164:0/2002234868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb410160300 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb40001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346160fa78c6f0f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb40f400640 1 -- 172.21.15.164:0/2002234868 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000b0070 con 0x7fb41015ba70 2024-05-08T04:44:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb40f400640 1 -- 172.21.15.164:0/2002234868 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4000ab600 con 0x7fb41015ba70 2024-05-08T04:44:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb40f400640 1 -- 172.21.15.164:0/2002234868 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000ab9d0 con 0x7fb41015ba70 2024-05-08T04:44:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- 172.21.15.164:0/2002234868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 msgr2=0x7fb410160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 --2- 172.21.15.164:0/2002234868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb410160300 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb40001c430 comp rx=0 tx=0).stop 2024-05-08T04:44:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- 172.21.15.164:0/2002234868 shutdown_connections 2024-05-08T04:44:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 --2- 172.21.15.164:0/2002234868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb410160300 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- 172.21.15.164:0/2002234868 >> 172.21.15.164:0/2002234868 conn(0x7fb41006ac90 msgr2=0x7fb41006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- 172.21.15.164:0/2002234868 shutdown_connections 2024-05-08T04:44:05.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- 172.21.15.164:0/2002234868 wait complete. 2024-05-08T04:44:05.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 Processor -- start 2024-05-08T04:44:05.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.595+0000 7fb417c00640 1 -- start start 2024-05-08T04:44:05.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4100cfda0 0x7fb410151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41015be50 0x7fb410146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410169f60 con 0x7fb41015ba70 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410169de0 con 0x7fb41015be50 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41016a0e0 con 0x7fb4100cfda0 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35412/0 (socket says 172.21.15.164:35412) 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 -- 172.21.15.164:0/4271810127 learned_addr learned my addr 172.21.15.164:0/4271810127 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb415c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41015be50 0x7fb410146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb415200640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4100cfda0 0x7fb410151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 -- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4100cfda0 msgr2=0x7fb410151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4100cfda0 0x7fb410151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 -- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41015be50 msgr2=0x7fb410146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41015be50 0x7fb410146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 -- 172.21.15.164:0/4271810127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41014cf70 con 0x7fb41015ba70 2024-05-08T04:44:05.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40fe00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b410 tx=0x7fb3fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc015030 con 0x7fb41015ba70 2024-05-08T04:44:05.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc004020 con 0x7fb41015ba70 2024-05-08T04:44:05.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410151cb0 con 0x7fb41015ba70 2024-05-08T04:44:05.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc004450 con 0x7fb41015ba70 2024-05-08T04:44:05.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410152270 con 0x7fb41015ba70 2024-05-08T04:44:05.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410069d60 con 0x7fb41015ba70 2024-05-08T04:44:05.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.599+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fb3fc018070 con 0x7fb41015ba70 2024-05-08T04:44:05.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.603+0000 7fb40cc00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 0x7fb3f006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.603+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb3fc083850 con 0x7fb41015ba70 2024-05-08T04:44:05.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.603+0000 7fb415200640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 0x7fb3f006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.603+0000 7fb415200640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 0x7fb3f006a1a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb41015fde0 tx=0x7fb4000a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.607+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb41014cf70 con 0x7fb41015ba70 2024-05-08T04:44:05.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb41006c580 con 0x7fb3f0067ce0 2024-05-08T04:44:05.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb40cc00640 1 -- 172.21.15.164:0/4271810127 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb41006c580 con 0x7fb3f0067ce0 2024-05-08T04:44:05.764 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T04:44:05.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 msgr2=0x7fb3f006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 0x7fb3f006a1a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb41015fde0 tx=0x7fb4000a6000 comp rx=0 tx=0).stop 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 msgr2=0x7fb41014ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b410 tx=0x7fb3fc00b8d0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 shutdown_connections 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3f0067ce0 0x7fb3f006a1a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41015be50 0x7fb410146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41015ba70 0x7fb41014ca30 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 --2- 172.21.15.164:0/4271810127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4100cfda0 0x7fb410151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 >> 172.21.15.164:0/4271810127 conn(0x7fb41006ac90 msgr2=0x7fb41006d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 shutdown_connections 2024-05-08T04:44:05.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.763+0000 7fb417c00640 1 -- 172.21.15.164:0/4271810127 wait complete. 2024-05-08T04:44:05.784 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T04:44:05.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:05.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.919+0000 7f86ff000640 1 Processor -- start 2024-05-08T04:44:05.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.919+0000 7f86ff000640 1 -- start start 2024-05-08T04:44:05.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 0x7f86f806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86f80d0c20 0x7f86f80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f806f520 0x7f86f80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f80d0790 con 0x7f86f80d0c20 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f80cfe60 con 0x7f86f80cdb10 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86ff000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80d0140 con 0x7f86f806f520 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86f80d0c20 0x7f86f80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86f80d0c20 0x7f86f80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35418/0 (socket says 172.21.15.164:35418) 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 -- 172.21.15.164:0/1514103921 learned_addr learned my addr 172.21.15.164:0/1514103921 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86fd000640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f806f520 0x7f86f80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7e00640 1 --2- 172.21.15.164:0/1514103921 >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 0x7f86f806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 -- 172.21.15.164:0/1514103921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f806f520 msgr2=0x7f86f80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:05.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f806f520 0x7f86f80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 -- 172.21.15.164:0/1514103921 >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 msgr2=0x7f86f806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:05.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 --2- 172.21.15.164:0/1514103921 >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 0x7f86f806efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7400640 1 -- 172.21.15.164:0/1514103921 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80d0420 con 0x7f86f80d0c20 2024-05-08T04:44:05.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.923+0000 7f86f7e00640 1 --2- 172.21.15.164:0/1514103921 >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 0x7f86f806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:05.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86f7400640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f80d0c20 0x7f86f80ce220 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f86f8071940 tx=0x7f86e401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b8c762697163f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86f6a00640 1 -- 172.21.15.164:0/1514103921 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40a7070 con 0x7f86f80d0c20 2024-05-08T04:44:05.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86f6a00640 1 -- 172.21.15.164:0/1514103921 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e40b0070 con 0x7f86f80d0c20 2024-05-08T04:44:05.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86f6a00640 1 -- 172.21.15.164:0/1514103921 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40ab890 con 0x7f86f80d0c20 2024-05-08T04:44:05.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- 172.21.15.164:0/1514103921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f80d0c20 msgr2=0x7f86f80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f80d0c20 0x7f86f80ce220 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f86f8071940 tx=0x7f86e401c490 comp rx=0 tx=0).stop 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- 172.21.15.164:0/1514103921 shutdown_connections 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f806f520 0x7f86f80ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 --2- 172.21.15.164:0/1514103921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f80d0c20 0x7f86f80ce220 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 --2- 172.21.15.164:0/1514103921 >> v2:172.21.15.172:3300/0 conn(0x7f86f80cdb10 0x7f86f806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- 172.21.15.164:0/1514103921 >> 172.21.15.164:0/1514103921 conn(0x7f86f80b34f0 msgr2=0x7f86f80b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- 172.21.15.164:0/1514103921 shutdown_connections 2024-05-08T04:44:05.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- 172.21.15.164:0/1514103921 wait complete. 2024-05-08T04:44:05.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 Processor -- start 2024-05-08T04:44:05.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.927+0000 7f86ff000640 1 -- start start 2024-05-08T04:44:05.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 0x7f86f80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f80cdb10 0x7f86f80b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f80d0c20 0x7f86f8161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8162730 con 0x7f86f806f520 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81625b0 con 0x7f86f80cdb10 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81628b0 con 0x7f86f80d0c20 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 0x7f86f80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 0x7f86f80b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35422/0 (socket says 172.21.15.164:35422) 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 -- 172.21.15.164:0/3190073280 learned_addr learned my addr 172.21.15.164:0/3190073280 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:05.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7400640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f80cdb10 0x7f86f80b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86fd000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f80d0c20 0x7f86f8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 -- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f80d0c20 msgr2=0x7f86f8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f80d0c20 0x7f86f8161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 -- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f80cdb10 msgr2=0x7f86f80b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f80cdb10 0x7f86f80b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80b5f20 con 0x7f86f806f520 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86fd000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f80d0c20 0x7f86f8161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:05.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86f7e00640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 0x7f86f80b4ff0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d830 tx=0x7f86e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8018020 con 0x7f86f806f520 2024-05-08T04:44:05.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e80096c0 con 0x7f86f806f520 2024-05-08T04:44:05.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8009ab0 con 0x7f86f806f520 2024-05-08T04:44:05.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f80b5680 con 0x7f86f806f520 2024-05-08T04:44:05.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f80b5c40 con 0x7f86f806f520 2024-05-08T04:44:05.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86f809fae0 con 0x7f86f806f520 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.931+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f86e801b020 con 0x7f86f806f520 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86d8067ce0 0x7f86d806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f86e8084ae0 con 0x7f86f806f520 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86f7400640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86d8067ce0 0x7f86d806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f86d806dba0 0x7f86d8070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86d80706d0 con 0x7f86d806dba0 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e8087440 con 0x7f86f806f520 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86fd000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f86d806dba0 0x7f86d8070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:05.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86f7400640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86d8067ce0 0x7f86d806a1a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86e40af000 tx=0x7f86e40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86fd000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f86d806dba0 0x7f86d8070000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:05.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.935+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== osd.5 v2:172.21.15.172:6814/3686545583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f86d80706d0 con 0x7f86d806dba0 2024-05-08T04:44:05.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 --> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86f804f010 con 0x7f86d806dba0 2024-05-08T04:44:05.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86efe00640 1 -- 172.21.15.164:0/3190073280 <== osd.5 v2:172.21.15.172:6814/3686545583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f86f804f010 con 0x7f86d806dba0 2024-05-08T04:44:05.951 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:05.952 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:05.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f86d806dba0 msgr2=0x7f86d8070000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.172:6814/3686545583,v1:172.21.15.172:6815/3686545583] conn(0x7f86d806dba0 0x7f86d8070000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86d8067ce0 msgr2=0x7f86d806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f86d8067ce0 0x7f86d806a1a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86e40af000 tx=0x7f86e40a7000 comp rx=0 tx=0).stop 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 msgr2=0x7f86f80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 --2- 172.21.15.164:0/3190073280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f806f520 0x7f86f80b4ff0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d830 tx=0x7f86e800dcf0 comp rx=0 tx=0).stop 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86fd000640 1 -- 172.21.15.164:0/3190073280 reap_dead start 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 shutdown_connections 2024-05-08T04:44:05.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 >> 172.21.15.164:0/3190073280 conn(0x7f86f80b34f0 msgr2=0x7f86f806c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:05.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 shutdown_connections 2024-05-08T04:44:05.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:05.951+0000 7f86ff000640 1 -- 172.21.15.164:0/3190073280 wait complete. 2024-05-08T04:44:05.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T04:44:06.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 Processor -- start 2024-05-08T04:44:06.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 -- start start 2024-05-08T04:44:06.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f650415be50 0x7f6504160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cd9f0 con 0x7f650415ba70 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cdcd0 con 0x7f65040cfda0 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.103+0000 7f650aa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650415fe70 con 0x7f650415be50 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f650415be50 0x7f6504160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f650415be50 0x7f6504160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35430/0 (socket says 172.21.15.164:35430) 2024-05-08T04:44:06.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 -- 172.21.15.164:0/113042421 learned_addr learned my addr 172.21.15.164:0/113042421 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 -- 172.21.15.164:0/113042421 >> v1:172.21.15.172:6789/0 conn(0x7f65040cfda0 legacy=0x7f65040b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 -- 172.21.15.164:0/113042421 >> v1:172.21.15.164:6789/0 conn(0x7f650415ba70 legacy=0x7f65040cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 -- 172.21.15.164:0/113042421 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650415f820 con 0x7f650415be50 2024-05-08T04:44:06.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6502a00640 1 -- 172.21.15.164:0/113042421 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959587453 0 0) 0x7f65040cd9f0 con 0x7f650415ba70 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6508a00640 1 --2- 172.21.15.164:0/113042421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650415be50 0x7f6504160300 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b62214a216929f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6502a00640 1 -- 172.21.15.164:0/113042421 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0ab640 con 0x7f650415be50 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6502a00640 1 -- 172.21.15.164:0/113042421 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc0b0070 con 0x7f650415be50 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.107+0000 7f6502a00640 1 -- 172.21.15.164:0/113042421 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0aba00 con 0x7f650415be50 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- 172.21.15.164:0/113042421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650415be50 msgr2=0x7f6504160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 --2- 172.21.15.164:0/113042421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650415be50 0x7f6504160300 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc01c530 comp rx=0 tx=0).stop 2024-05-08T04:44:06.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- 172.21.15.164:0/113042421 shutdown_connections 2024-05-08T04:44:06.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 --2- 172.21.15.164:0/113042421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f650415be50 0x7f6504160300 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- 172.21.15.164:0/113042421 >> 172.21.15.164:0/113042421 conn(0x7f650406ac90 msgr2=0x7f650406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- 172.21.15.164:0/113042421 shutdown_connections 2024-05-08T04:44:06.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- 172.21.15.164:0/113042421 wait complete. 2024-05-08T04:44:06.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 Processor -- start 2024-05-08T04:44:06.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- start start 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f650415ba70 0x7f6504153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f650415be50 0x7f65040715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504169f60 con 0x7f65040cfda0 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504169de0 con 0x7f650415be50 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f650aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650416a0e0 con 0x7f650415ba70 2024-05-08T04:44:06.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35442/0 (socket says 172.21.15.164:35442) 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 -- 172.21.15.164:0/827550804 learned_addr learned my addr 172.21.15.164:0/827550804 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503400640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f650415ba70 0x7f6504153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6508a00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f650415be50 0x7f65040715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 -- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f650415ba70 msgr2=0x7f6504153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f650415ba70 0x7f6504153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 -- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f650415be50 msgr2=0x7f65040715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f650415be50 0x7f65040715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 -- 172.21.15.164:0/827550804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65041554d0 con 0x7f65040cfda0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.111+0000 7f6503e00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c870 tx=0x7f64f400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003db0 con 0x7f65040cfda0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4005070 con 0x7f65040cfda0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4005460 con 0x7f65040cfda0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504154430 con 0x7f65040cfda0 2024-05-08T04:44:06.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65041549f0 con 0x7f65040cfda0 2024-05-08T04:44:06.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6504067820 con 0x7f65040cfda0 2024-05-08T04:44:06.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f64f4005600 con 0x7f65040cfda0 2024-05-08T04:44:06.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 0x7f64e406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f64f4083bf0 con 0x7f65040cfda0 2024-05-08T04:44:06.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f6503400640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 0x7f64e406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.115+0000 7f6503400640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 0x7f64e406a1a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f650414f9b0 tx=0x7f64f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.119+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64f40563a0 con 0x7f65040cfda0 2024-05-08T04:44:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6504069d60 con 0x7f64e4067ce0 2024-05-08T04:44:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f64fbe00640 1 -- 172.21.15.164:0/827550804 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6504069d60 con 0x7f64e4067ce0 2024-05-08T04:44:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T04:44:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 msgr2=0x7f64e406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 0x7f64e406a1a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f650414f9b0 tx=0x7f64f0009040 comp rx=0 tx=0).stop 2024-05-08T04:44:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 msgr2=0x7f6504070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c870 tx=0x7f64f400cd30 comp rx=0 tx=0).stop 2024-05-08T04:44:06.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 shutdown_connections 2024-05-08T04:44:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f64e4067ce0 0x7f64e406a1a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f650415ba70 0x7f6504153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f650415be50 0x7f65040715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 --2- 172.21.15.164:0/827550804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65040cfda0 0x7f6504070370 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 >> 172.21.15.164:0/827550804 conn(0x7f650406ac90 msgr2=0x7f65041586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 shutdown_connections 2024-05-08T04:44:06.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.275+0000 7f650aa00640 1 -- 172.21.15.164:0/827550804 wait complete. 2024-05-08T04:44:06.295 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T04:44:06.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:06.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 Processor -- start 2024-05-08T04:44:06.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 -- start start 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e7415ab10 0x7f4e7406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e740aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7415e870 con 0x7f4e740ae6e0 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7406bb80 con 0x7f4e7415ab10 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e79000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7415eb50 con 0x7f4e740cee90 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e71400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4e740ae6e0 0x7f4e740ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50526/0 (socket says 172.21.15.164:50526) 2024-05-08T04:44:06.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e71400640 1 -- 172.21.15.164:0/2565509538 learned_addr learned my addr 172.21.15.164:0/2565509538 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e71e00640 1 --2- 172.21.15.164:0/2565509538 >> v2:172.21.15.172:3300/0 conn(0x7f4e7415ab10 0x7f4e7406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e72800640 1 --2- 172.21.15.164:0/2565509538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e740aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e70a00640 1 -- 172.21.15.164:0/2565509538 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220189151 0 0) 0x7f4e7415e870 con 0x7f4e740ae6e0 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e70a00640 1 -- 172.21.15.164:0/2565509538 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e58003530 con 0x7f4e740ae6e0 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e72800640 1 -- 172.21.15.164:0/2565509538 >> v2:172.21.15.172:3300/0 conn(0x7f4e7415ab10 msgr2=0x7f4e7406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e72800640 1 --2- 172.21.15.164:0/2565509538 >> v2:172.21.15.172:3300/0 conn(0x7f4e7415ab10 0x7f4e7406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e72800640 1 -- 172.21.15.164:0/2565509538 >> v1:172.21.15.164:6789/0 conn(0x7f4e740ae6e0 legacy=0x7f4e740ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.435+0000 7f4e72800640 1 -- 172.21.15.164:0/2565509538 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7415ee30 con 0x7f4e740cee90 2024-05-08T04:44:06.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e72800640 1 --2- 172.21.15.164:0/2565509538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e740aeb70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009d90 tx=0x7f4e6c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be00a48040601ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e70a00640 1 -- 172.21.15.164:0/2565509538 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0b2070 con 0x7f4e740cee90 2024-05-08T04:44:06.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e70a00640 1 -- 172.21.15.164:0/2565509538 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c0a5930 con 0x7f4e740cee90 2024-05-08T04:44:06.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e70a00640 1 -- 172.21.15.164:0/2565509538 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0a5d00 con 0x7f4e740cee90 2024-05-08T04:44:06.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 -- 172.21.15.164:0/2565509538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 msgr2=0x7f4e740aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 --2- 172.21.15.164:0/2565509538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e740aeb70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009d90 tx=0x7f4e6c01c8b0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 -- 172.21.15.164:0/2565509538 shutdown_connections 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 --2- 172.21.15.164:0/2565509538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e740aeb70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 --2- 172.21.15.164:0/2565509538 >> v2:172.21.15.172:3300/0 conn(0x7f4e7415ab10 0x7f4e7406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 -- 172.21.15.164:0/2565509538 >> 172.21.15.164:0/2565509538 conn(0x7f4e740cf770 msgr2=0x7f4e740cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 -- 172.21.15.164:0/2565509538 shutdown_connections 2024-05-08T04:44:06.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.439+0000 7f4e79000640 1 -- 172.21.15.164:0/2565509538 wait complete. 2024-05-08T04:44:06.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 Processor -- start 2024-05-08T04:44:06.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- start start 2024-05-08T04:44:06.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e740ae6e0 0x7f4e74070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e7414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 0x7f4e740ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740cb070 con 0x7f4e7415ab10 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740caef0 con 0x7f4e740ae6e0 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740cb1f0 con 0x7f4e740cee90 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 0x7f4e740ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 0x7f4e740ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35456/0 (socket says 172.21.15.164:35456) 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 -- 172.21.15.164:0/2975252506 learned_addr learned my addr 172.21.15.164:0/2975252506 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e71e00640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e740ae6e0 0x7f4e74070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e71400640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e7414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 -- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 msgr2=0x7f4e7414e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e7414e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 -- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e740ae6e0 msgr2=0x7f4e74070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e740ae6e0 0x7f4e74070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7414f3f0 con 0x7f4e7415ab10 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e71400640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e740cee90 0x7f4e7414e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e72800640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 0x7f4e740ad890 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c01d150 tx=0x7f4e6c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0bb020 con 0x7f4e7415ab10 2024-05-08T04:44:06.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c0ad640 con 0x7f4e7415ab10 2024-05-08T04:44:06.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0b2040 con 0x7f4e7415ab10 2024-05-08T04:44:06.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e74071260 con 0x7f4e7415ab10 2024-05-08T04:44:06.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.443+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74071820 con 0x7f4e7415ab10 2024-05-08T04:44:06.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e74072d30 con 0x7f4e7415ab10 2024-05-08T04:44:06.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f4e6c0be020 con 0x7f4e7415ab10 2024-05-08T04:44:06.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e54067d10 0x7f4e5406a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f4e6c11e800 con 0x7f4e7415ab10 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e71e00640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e54067d10 0x7f4e5406a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7f4e5406dbd0 0x7f4e54070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e6c0b7e50 con 0x7f4e5406dbd0 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6c126340 con 0x7f4e7415ab10 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e71400640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7f4e5406dbd0 0x7f4e54070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e71e00640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e54067d10 0x7f4e5406a1d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009a50 tx=0x7f4e64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.447+0000 7f4e71400640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7f4e5406dbd0 0x7f4e54070030 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.451+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== osd.6 v2:172.21.15.172:6824/2968599637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4e6c0b7e50 con 0x7f4e5406dbd0 2024-05-08T04:44:06.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 --> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4e740c7750 con 0x7f4e5406dbd0 2024-05-08T04:44:06.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e6a000640 1 -- 172.21.15.164:0/2975252506 <== osd.6 v2:172.21.15.172:6824/2968599637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4e740c7750 con 0x7f4e5406dbd0 2024-05-08T04:44:06.465 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:06.465 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:06.465 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:06.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7f4e5406dbd0 msgr2=0x7f4e54070030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.172:6824/2968599637,v1:172.21.15.172:6825/2968599637] conn(0x7f4e5406dbd0 0x7f4e54070030 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e54067d10 msgr2=0x7f4e5406a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f4e54067d10 0x7f4e5406a1d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009a50 tx=0x7f4e64008040 comp rx=0 tx=0).stop 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 msgr2=0x7f4e740ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 --2- 172.21.15.164:0/2975252506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e7415ab10 0x7f4e740ad890 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c01d150 tx=0x7f4e6c019650 comp rx=0 tx=0).stop 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e72800640 1 -- 172.21.15.164:0/2975252506 reap_dead start 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 shutdown_connections 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 >> 172.21.15.164:0/2975252506 conn(0x7f4e740cf770 msgr2=0x7f4e740cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 shutdown_connections 2024-05-08T04:44:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.463+0000 7f4e79000640 1 -- 172.21.15.164:0/2975252506 wait complete. 2024-05-08T04:44:06.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T04:44:06.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.619+0000 7f373bc00640 1 Processor -- start 2024-05-08T04:44:06.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.619+0000 7f373bc00640 1 -- start start 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f373bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415ba70 0x7f3734160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f373bc00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37340cfda0 0x7f37340b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f373bc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37340cd9f0 con 0x7f373415be50 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f373bc00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373415fe70 con 0x7f37340cfda0 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f373bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37340cdcd0 con 0x7f373415ba70 2024-05-08T04:44:06.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37340cfda0 0x7f37340b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37340cfda0 0x7f37340b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35464/0 (socket says 172.21.15.164:35464) 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 -- 172.21.15.164:0/4126359395 learned_addr learned my addr 172.21.15.164:0/4126359395 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739200640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415ba70 0x7f3734160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 -- 172.21.15.164:0/4126359395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415ba70 msgr2=0x7f3734160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415ba70 0x7f3734160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 -- 172.21.15.164:0/4126359395 >> v1:172.21.15.172:6789/0 conn(0x7f373415be50 legacy=0x7f37340cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.623+0000 7f3739c00640 1 -- 172.21.15.164:0/4126359395 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373415f820 con 0x7f37340cfda0 2024-05-08T04:44:06.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f3739c00640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37340cfda0 0x7f37340b31a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f372c009e50 tx=0x7f372c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7481ce0a1a2a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f3733e00640 1 -- 172.21.15.164:0/4126359395 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0a57e0 con 0x7f37340cfda0 2024-05-08T04:44:06.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f3733e00640 1 -- 172.21.15.164:0/4126359395 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c0a5980 con 0x7f37340cfda0 2024-05-08T04:44:06.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f3733e00640 1 -- 172.21.15.164:0/4126359395 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0a5d50 con 0x7f37340cfda0 2024-05-08T04:44:06.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- 172.21.15.164:0/4126359395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37340cfda0 msgr2=0x7f37340b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37340cfda0 0x7f37340b31a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f372c009e50 tx=0x7f372c01ca00 comp rx=0 tx=0).stop 2024-05-08T04:44:06.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- 172.21.15.164:0/4126359395 shutdown_connections 2024-05-08T04:44:06.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37340cfda0 0x7f37340b31a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- 172.21.15.164:0/4126359395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415ba70 0x7f3734160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- 172.21.15.164:0/4126359395 >> 172.21.15.164:0/4126359395 conn(0x7f373406ac90 msgr2=0x7f373406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- 172.21.15.164:0/4126359395 shutdown_connections 2024-05-08T04:44:06.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- 172.21.15.164:0/4126359395 wait complete. 2024-05-08T04:44:06.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 Processor -- start 2024-05-08T04:44:06.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- start start 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37340cfda0 0x7f3734147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415be50 0x7f3734071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734168d40 con 0x7f373415ba70 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734168bc0 con 0x7f37340cfda0 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.627+0000 7f373bc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3734168ec0 con 0x7f373415be50 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35466/0 (socket says 172.21.15.164:35466) 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 -- 172.21.15.164:0/1899795927 learned_addr learned my addr 172.21.15.164:0/1899795927 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3739200640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37340cfda0 0x7f3734147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3739c00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415be50 0x7f3734071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 -- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415be50 msgr2=0x7f3734071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415be50 0x7f3734071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 -- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37340cfda0 msgr2=0x7f3734147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37340cfda0 0x7f3734147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 -- 172.21.15.164:0/1899795927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340b1090 con 0x7f373415ba70 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f372be00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c340 tx=0x7f371c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c0052d0 con 0x7f373415ba70 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c005470 con 0x7f373415ba70 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373415a9a0 con 0x7f373415ba70 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c007260 con 0x7f373415ba70 2024-05-08T04:44:06.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373415a6c0 con 0x7f373415ba70 2024-05-08T04:44:06.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3734069d60 con 0x7f373415ba70 2024-05-08T04:44:06.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f371c007400 con 0x7f373415ba70 2024-05-08T04:44:06.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.631+0000 7f3731600640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 0x7f371006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.635+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f371c083120 con 0x7f373415ba70 2024-05-08T04:44:06.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.635+0000 7f3739200640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 0x7f371006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.635+0000 7f3739200640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 0x7f371006a1a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3724007a60 tx=0x7f3724008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.635+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37340b1090 con 0x7f373415ba70 2024-05-08T04:44:06.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.791+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f373406e210 con 0x7f3710067ce0 2024-05-08T04:44:06.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.791+0000 7f3731600640 1 -- 172.21.15.164:0/1899795927 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f373406e210 con 0x7f3710067ce0 2024-05-08T04:44:06.794 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T04:44:06.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 msgr2=0x7f371006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 0x7f371006a1a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3724007a60 tx=0x7f3724008040 comp rx=0 tx=0).stop 2024-05-08T04:44:06.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 msgr2=0x7f37340b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c340 tx=0x7f371c00c800 comp rx=0 tx=0).stop 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 shutdown_connections 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3710067ce0 0x7f371006a1a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f373415be50 0x7f3734071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373415ba70 0x7f37340b0b50 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 --2- 172.21.15.164:0/1899795927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37340cfda0 0x7f3734147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 >> 172.21.15.164:0/1899795927 conn(0x7f373406ac90 msgr2=0x7f37341553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 shutdown_connections 2024-05-08T04:44:06.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.795+0000 7f373bc00640 1 -- 172.21.15.164:0/1899795927 wait complete. 2024-05-08T04:44:06.815 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T04:44:06.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:44:06.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.951+0000 7fd321200640 1 Processor -- start 2024-05-08T04:44:06.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.951+0000 7fd321200640 1 -- start start 2024-05-08T04:44:06.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd321200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd31c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd321200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31c15e870 con 0x7fd31c0cee90 2024-05-08T04:44:06.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd321200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31c15eb50 con 0x7fd31c0ae6e0 2024-05-08T04:44:06.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd321200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c06bb80 con 0x7fd31c15ab10 2024-05-08T04:44:06.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd31a800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd31c0cee90 0x7fd31c0ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50536/0 (socket says 172.21.15.164:50536) 2024-05-08T04:44:06.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd31a800640 1 -- 172.21.15.164:0/1619866758 learned_addr learned my addr 172.21.15.164:0/1619866758 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:06.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd31c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540900354 0 0) 0x7fd31c15e870 con 0x7fd31c0cee90 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd300005740 con 0x7fd31c0cee90 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167954310 0 0) 0x7fd300005740 con 0x7fd31c0cee90 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 msgr2=0x7fd31c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd31c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 >> v1:172.21.15.172:6789/0 conn(0x7fd31c0ae6e0 legacy=0x7fd31c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c15ee30 con 0x7fd31c0cee90 2024-05-08T04:44:06.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fd314001680 con 0x7fd31c0cee90 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 >> v1:172.21.15.164:6789/0 conn(0x7fd31c0cee90 legacy=0x7fd31c0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300005c00 0x7fd300005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd300006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300006f60 0x7fd3000073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c06bb80 con 0x7fd300005c00 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30000dc90 con 0x7fd300006f60 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30000de90 con 0x7fd31c15ab10 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300005c00 0x7fd300005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd311400640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd300006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd31a800640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300006f60 0x7fd3000073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 -- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 msgr2=0x7fd300006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15ab10 0x7fd300006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 -- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300006f60 msgr2=0x7fd3000073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300006f60 0x7fd3000073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd319e00640 1 -- 172.21.15.164:0/1619866758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300005740 con 0x7fd300005c00 2024-05-08T04:44:06.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.955+0000 7fd31a800640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300006f60 0x7fd3000073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:06.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd319e00640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300005c00 0x7fd300005fe0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009fc0 tx=0x7fd30c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7ba85462e44192 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0b2070 con 0x7fd300005c00 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c0a5b50 con 0x7fd300005c00 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd319400640 1 -- 172.21.15.164:0/1619866758 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c009c10 con 0x7fd300005c00 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- 172.21.15.164:0/1619866758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300005c00 msgr2=0x7fd300005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 --2- 172.21.15.164:0/1619866758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300005c00 0x7fd300005fe0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009fc0 tx=0x7fd30c01cc50 comp rx=0 tx=0).stop 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd31a800640 1 -- 172.21.15.164:0/1619866758 reap_dead start 2024-05-08T04:44:06.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- 172.21.15.164:0/1619866758 shutdown_connections 2024-05-08T04:44:06.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- 172.21.15.164:0/1619866758 >> 172.21.15.164:0/1619866758 conn(0x7fd31c0cf770 msgr2=0x7fd31c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- 172.21.15.164:0/1619866758 shutdown_connections 2024-05-08T04:44:06.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- 172.21.15.164:0/1619866758 wait complete. 2024-05-08T04:44:06.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 Processor -- start 2024-05-08T04:44:06.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.959+0000 7fd321200640 1 -- start start 2024-05-08T04:44:06.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300005c00 0x7fd31c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 0x7fd31c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15db60 0x7fd31c1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0cd670 con 0x7fd31c15d6e0 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c15aef0 con 0x7fd300005c00 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0cd7f0 con 0x7fd31c15db60 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 0x7fd31c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 0x7fd31c153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35488/0 (socket says 172.21.15.164:35488) 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 -- 172.21.15.164:0/76366013 learned_addr learned my addr 172.21.15.164:0/76366013 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd31a800640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15db60 0x7fd31c1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd319e00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300005c00 0x7fd31c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 -- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15db60 msgr2=0x7fd31c1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15db60 0x7fd31c1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 -- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300005c00 msgr2=0x7fd31c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300005c00 0x7fd31c0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c15bd70 con 0x7fd31c15d6e0 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd31a800640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd31c15db60 0x7fd31c1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd319e00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd300005c00 0x7fd31c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd311400640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 0x7fd31c153db0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd30400e830 tx=0x7fd30400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3040050d0 con 0x7fd31c15d6e0 2024-05-08T04:44:06.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd304018070 con 0x7fd31c15d6e0 2024-05-08T04:44:06.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400c6c0 con 0x7fd31c15d6e0 2024-05-08T04:44:06.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31c15c050 con 0x7fd31c15d6e0 2024-05-08T04:44:06.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31c15c610 con 0x7fd31c15d6e0 2024-05-08T04:44:06.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.963+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd31c072d30 con 0x7fd31c15d6e0 2024-05-08T04:44:06.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fd30400ca70 con 0x7fd31c15d6e0 2024-05-08T04:44:06.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2f8067d10 0x7fd2f806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fd3040848d0 con 0x7fd31c15d6e0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd319e00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2f8067d10 0x7fd2f806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7fd2f806dbd0 0x7fd2f8070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd304028e50 con 0x7fd2f806dbd0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd304087540 con 0x7fd31c15d6e0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd319e00640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2f8067d10 0x7fd2f806a1d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0af000 tx=0x7fd30c0a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd31a800640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7fd2f806dbd0 0x7fd2f8070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:06.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd31a800640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7fd2f806dbd0 0x7fd2f8070030 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:06.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.967+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== osd.7 v2:172.21.15.172:6808/1836955626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd304028e50 con 0x7fd2f806dbd0 2024-05-08T04:44:06.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 --> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd31c0ce3e0 con 0x7fd2f806dbd0 2024-05-08T04:44:06.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd312a00640 1 -- 172.21.15.164:0/76366013 <== osd.7 v2:172.21.15.172:6808/1836955626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd31c0ce3e0 con 0x7fd2f806dbd0 2024-05-08T04:44:06.984 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-08T04:44:06.984 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:44:06.984 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-08T04:44:06.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7fd2f806dbd0 msgr2=0x7fd2f8070030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.172:6808/1836955626,v1:172.21.15.172:6809/1836955626] conn(0x7fd2f806dbd0 0x7fd2f8070030 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2f8067d10 msgr2=0x7fd2f806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fd2f8067d10 0x7fd2f806a1d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0af000 tx=0x7fd30c0a7040 comp rx=0 tx=0).stop 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 msgr2=0x7fd31c153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 --2- 172.21.15.164:0/76366013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd31c15d6e0 0x7fd31c153db0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd30400e830 tx=0x7fd30400ecf0 comp rx=0 tx=0).stop 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd31a800640 1 -- 172.21.15.164:0/76366013 reap_dead start 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 shutdown_connections 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 >> 172.21.15.164:0/76366013 conn(0x7fd31c0cf770 msgr2=0x7fd31c0c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:06.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 shutdown_connections 2024-05-08T04:44:06.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:06.983+0000 7fd321200640 1 -- 172.21.15.164:0/76366013 wait complete. 2024-05-08T04:44:07.002 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T04:44:07.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.135+0000 7ff65ec00640 1 Processor -- start 2024-05-08T04:44:07.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.135+0000 7ff65ec00640 1 -- start start 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff6580cdfa0 0x7ff6580cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 0x7ff65806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65ec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff658071980 con 0x7ff6580af480 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff658161810 con 0x7ff6580cdfa0 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65806eed0 con 0x7ff65806bca0 2024-05-08T04:44:07.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff6580cdfa0 0x7ff6580cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff6580cdfa0 0x7ff6580cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35498/0 (socket says 172.21.15.164:35498) 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff657e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff6580af480 0x7ff65806cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50548/0 (socket says 172.21.15.164:50548) 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 -- 172.21.15.164:0/2526819983 learned_addr learned my addr 172.21.15.164:0/2526819983 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65cc00640 1 --2- 172.21.15.164:0/2526819983 >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 0x7ff65806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 -- 172.21.15.164:0/2526819983 >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 msgr2=0x7ff65806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 --2- 172.21.15.164:0/2526819983 >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 0x7ff65806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 -- 172.21.15.164:0/2526819983 >> v1:172.21.15.164:6789/0 conn(0x7ff6580af480 legacy=0x7ff65806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff64f400640 1 -- 172.21.15.164:0/2526819983 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658071c00 con 0x7ff6580cdfa0 2024-05-08T04:44:07.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.139+0000 7ff65cc00640 1 --2- 172.21.15.164:0/2526819983 >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 0x7ff65806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:07.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff64f400640 1 --2- 172.21.15.164:0/2526819983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580cdfa0 0x7ff6580cf470 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff658066d20 tx=0x7ff64001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4956a33c88953bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff657400640 1 -- 172.21.15.164:0/2526819983 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400ab430 con 0x7ff6580cdfa0 2024-05-08T04:44:07.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff657400640 1 -- 172.21.15.164:0/2526819983 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400b0070 con 0x7ff6580cdfa0 2024-05-08T04:44:07.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff657400640 1 -- 172.21.15.164:0/2526819983 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400ab7f0 con 0x7ff6580cdfa0 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- 172.21.15.164:0/2526819983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580cdfa0 msgr2=0x7ff6580cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2526819983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580cdfa0 0x7ff6580cf470 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff658066d20 tx=0x7ff64001c520 comp rx=0 tx=0).stop 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- 172.21.15.164:0/2526819983 shutdown_connections 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2526819983 >> v2:172.21.15.172:3300/0 conn(0x7ff65806bca0 0x7ff65806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2526819983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580cdfa0 0x7ff6580cf470 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- 172.21.15.164:0/2526819983 >> 172.21.15.164:0/2526819983 conn(0x7ff6580ccab0 msgr2=0x7ff6580ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:07.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- 172.21.15.164:0/2526819983 shutdown_connections 2024-05-08T04:44:07.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- 172.21.15.164:0/2526819983 wait complete. 2024-05-08T04:44:07.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 Processor -- start 2024-05-08T04:44:07.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- start start 2024-05-08T04:44:07.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff65806bca0 0x7ff6580b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6580cdfa0 0x7ff65814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806c610 con 0x7ff6580af480 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658071f50 con 0x7ff65806bca0 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff65ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff658072bd0 con 0x7ff6580cdfa0 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff64f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff64f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35504/0 (socket says 172.21.15.164:35504) 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.143+0000 7ff64f400640 1 -- 172.21.15.164:0/2253748765 learned_addr learned my addr 172.21.15.164:0/2253748765 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:07.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff657e00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff65806bca0 0x7ff6580b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff65cc00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6580cdfa0 0x7ff65814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 -- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6580cdfa0 msgr2=0x7ff65814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6580cdfa0 0x7ff65814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 -- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff65806bca0 msgr2=0x7ff6580b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff65806bca0 0x7ff6580b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 -- 172.21.15.164:0/2253748765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580c4dc0 con 0x7ff6580af480 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff64f400640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff640019130 tx=0x7ff64001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400be5f0 con 0x7ff6580af480 2024-05-08T04:44:07.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400be790 con 0x7ff6580af480 2024-05-08T04:44:07.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400beb80 con 0x7ff6580af480 2024-05-08T04:44:07.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6580c50a0 con 0x7ff6580af480 2024-05-08T04:44:07.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65815e700 con 0x7ff6580af480 2024-05-08T04:44:07.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff658069d60 con 0x7ff6580af480 2024-05-08T04:44:07.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7ff6400b43f0 con 0x7ff6580af480 2024-05-08T04:44:07.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 0x7ff63406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7ff6400a8070 con 0x7ff6580af480 2024-05-08T04:44:07.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.147+0000 7ff657e00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 0x7ff63406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.151+0000 7ff657e00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 0x7ff63406a1a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff648000f70 tx=0x7ff648007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.151+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff640016110 con 0x7ff6580af480 2024-05-08T04:44:07.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff658066df0 con 0x7ff634067ce0 2024-05-08T04:44:07.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff654c00640 1 -- 172.21.15.164:0/2253748765 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff658066df0 con 0x7ff634067ce0 2024-05-08T04:44:07.308 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T04:44:07.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 msgr2=0x7ff63406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 0x7ff63406a1a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff648000f70 tx=0x7ff648007cd0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 msgr2=0x7ff65815d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff640019130 tx=0x7ff64001ad60 comp rx=0 tx=0).stop 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 shutdown_connections 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7ff634067ce0 0x7ff63406a1a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6580cdfa0 0x7ff65814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6580af480 0x7ff65815d940 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 --2- 172.21.15.164:0/2253748765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff65806bca0 0x7ff6580b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 >> 172.21.15.164:0/2253748765 conn(0x7ff6580ccab0 msgr2=0x7ff65809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 shutdown_connections 2024-05-08T04:44:07.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.307+0000 7ff65ec00640 1 -- 172.21.15.164:0/2253748765 wait complete. 2024-05-08T04:44:07.329 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:44:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.463+0000 7fb420e00640 1 Processor -- start 2024-05-08T04:44:07.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.463+0000 7fb420e00640 1 -- start start 2024-05-08T04:44:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb420e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb41c06fa90 0x7fb41c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb420e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb41c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb420e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0d0720 con 0x7fb41c0cda70 2024-05-08T04:44:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb420e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c070210 con 0x7fb41c06fa90 2024-05-08T04:44:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb420e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0705b0 con 0x7fb41c0cca00 2024-05-08T04:44:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb41c0cda70 0x7fb41c0cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50562/0 (socket says 172.21.15.164:50562) 2024-05-08T04:44:07.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419e00640 1 -- 172.21.15.164:0/600043736 learned_addr learned my addr 172.21.15.164:0/600043736 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160674966 0 0) 0x7fb41c0d0720 con 0x7fb41c0cda70 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb41c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb400003530 con 0x7fb41c0cda70 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419400640 1 --2- 172.21.15.164:0/600043736 >> v2:172.21.15.172:3300/0 conn(0x7fb41c06fa90 0x7fb41c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626626653 0 0) 0x7fb400003530 con 0x7fb41c0cda70 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 msgr2=0x7fb41c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:07.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb41c15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 >> v2:172.21.15.172:3300/0 conn(0x7fb41c06fa90 msgr2=0x7fb41c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 --2- 172.21.15.164:0/600043736 >> v2:172.21.15.172:3300/0 conn(0x7fb41c06fa90 0x7fb41c0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0d0160 con 0x7fb41c0cda70 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4035823827 0 0) 0x7fb40c001a00 con 0x7fb41c0cda70 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419400640 1 --2- 172.21.15.164:0/600043736 >> v2:172.21.15.172:3300/0 conn(0x7fb41c06fa90 0x7fb41c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 >> v1:172.21.15.164:6789/0 conn(0x7fb41c0cda70 legacy=0x7fb41c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb4000039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb400003f40 0x7fb400004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb400004d50 0x7fb4000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0705b0 con 0x7fb400004d50 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40000b9d0 con 0x7fb400003f40 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40000bba0 con 0x7fb41c0cca00 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb400004d50 0x7fb4000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419e00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb4000039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb419400640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb400003f40 0x7fb400004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 -- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 msgr2=0x7fb4000039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:07.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb41c0cca00 0x7fb4000039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 -- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb400003f40 msgr2=0x7fb400004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb400003f40 0x7fb400004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.467+0000 7fb41a800640 1 -- 172.21.15.164:0/600043736 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400003530 con 0x7fb400004d50 2024-05-08T04:44:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb41a800640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb400004d50 0x7fb4000051d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb414009ae0 tx=0x7fb41401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a729fda4cef6171 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140b2070 con 0x7fb400004d50 2024-05-08T04:44:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4140a5660 con 0x7fb400004d50 2024-05-08T04:44:07.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb418a00640 1 -- 172.21.15.164:0/600043736 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140096b0 con 0x7fb400004d50 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- 172.21.15.164:0/600043736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb400004d50 msgr2=0x7fb4000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 --2- 172.21.15.164:0/600043736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb400004d50 0x7fb4000051d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb414009ae0 tx=0x7fb41401ca70 comp rx=0 tx=0).stop 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb41a800640 1 -- 172.21.15.164:0/600043736 reap_dead start 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- 172.21.15.164:0/600043736 shutdown_connections 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- 172.21.15.164:0/600043736 >> 172.21.15.164:0/600043736 conn(0x7fb41c0b34f0 msgr2=0x7fb41c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- 172.21.15.164:0/600043736 shutdown_connections 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- 172.21.15.164:0/600043736 wait complete. 2024-05-08T04:44:07.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 Processor -- start 2024-05-08T04:44:07.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.471+0000 7fb420e00640 1 -- start start 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb400004d50 0x7fb41c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c0c8dc0 0x7fb41c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0cde50 con 0x7fb41c0b62b0 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0cce10 con 0x7fb41c0c8dc0 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0c9f40 con 0x7fb400004d50 2024-05-08T04:44:07.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35520/0 (socket says 172.21.15.164:35520) 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 -- 172.21.15.164:0/4209456259 learned_addr learned my addr 172.21.15.164:0/4209456259 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb419400640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c0c8dc0 0x7fb41c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb419e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb400004d50 0x7fb41c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 -- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb400004d50 msgr2=0x7fb41c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb400004d50 0x7fb41c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 -- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c0c8dc0 msgr2=0x7fb41c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c0c8dc0 0x7fb41c06d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 -- 172.21.15.164:0/4209456259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0c9330 con 0x7fb41c0b62b0 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb41a800640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b0040 tx=0x7fb400005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140b2040 con 0x7fb41c0b62b0 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41400f000 con 0x7fb41c0b62b0 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140096b0 con 0x7fb41c0b62b0 2024-05-08T04:44:07.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0c9610 con 0x7fb41c0b62b0 2024-05-08T04:44:07.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c0c9bd0 con 0x7fb41c0b62b0 2024-05-08T04:44:07.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c0b3900 con 0x7fb41c0b62b0 2024-05-08T04:44:07.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.475+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fb4140bd030 con 0x7fb41c0b62b0 2024-05-08T04:44:07.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.479+0000 7fb412000640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 0x7fb3fc06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:07.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.479+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb41411d470 con 0x7fb41c0b62b0 2024-05-08T04:44:07.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.479+0000 7fb419e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 0x7fb3fc06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:07.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.479+0000 7fb419e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 0x7fb3fc06a1d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000f90 tx=0x7fb40c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:07.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.483+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4140c4060 con 0x7fb41c0b62b0 2024-05-08T04:44:07.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb41c0cfe10 con 0x7fb3fc067d10 2024-05-08T04:44:07.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb412000640 1 -- 172.21.15.164:0/4209456259 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13344 (secure 0 0 0) 0x7fb41c0cfe10 con 0x7fb3fc067d10 2024-05-08T04:44:07.636 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:07.636 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:44:07.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 msgr2=0x7fb3fc06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 0x7fb3fc06a1d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000f90 tx=0x7fb40c006040 comp rx=0 tx=0).stop 2024-05-08T04:44:07.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 msgr2=0x7fb41c0b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:07.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb4140b0040 tx=0x7fb400005840 comp rx=0 tx=0).stop 2024-05-08T04:44:07.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 shutdown_connections 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb3fc067d10 0x7fb3fc06a1d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0b62b0 0x7fb41c0b6730 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c0c8dc0 0x7fb41c06d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 --2- 172.21.15.164:0/4209456259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb400004d50 0x7fb41c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 >> 172.21.15.164:0/4209456259 conn(0x7fb41c0b34f0 msgr2=0x7fb41c0d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 shutdown_connections 2024-05-08T04:44:07.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:07.635+0000 7fb420e00640 1 -- 172.21.15.164:0/4209456259 wait complete. 2024-05-08T04:44:07.657 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-05-08T04:44:07.498215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263256,"kb_used_data":13800,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711464,"statfs":{"total":773094113280,"available":772824539136,"internally_reserved":0,"allocated":14131200,"data_stored":6846248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-260,"num_objects":-3,"num_object_clones":-1,"num_object_copies":-6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":-1,"num_read":-1,"num_read_kb":-1,"num_write":-5,"num_write_kb":-3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.867958"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":372,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T04:44:00.034106+0000","last_change":"2024-05-08T04:38:19.598359+0000","last_active":"2024-05-08T04:44:00.034106+0000","last_peered":"2024-05-08T04:44:00.034106+0000","last_clean":"2024-05-08T04:44:00.034106+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:44:00.034106+0000","last_undegraded":"2024-05-08T04:44:00.034106+0000","last_fullsized":"2024-05-08T04:44:00.034106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:38:17.655552+0000","last_clean_scrub_stamp":"2024-05-08T04:38:17.655552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33340,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338500,"statfs":{"total":96636764160,"available":96602624000,"internally_reserved":0,"allocated":2211840,"data_stored":1298557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2211840,"data_stored":1298557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:44:07.659 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T04:38:17.655552+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=38, tm_sec=17, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=44, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:44:07.659 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T04:44:27.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:44:27.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.806+0000 7fb0b7800640 1 Processor -- start 2024-05-08T04:44:27.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.806+0000 7fb0b7800640 1 -- start start 2024-05-08T04:44:27.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b7800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 0x7fb0b006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b7800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0b00aee30 0x7fb0b00cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b7800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b006e0e0 con 0x7fb0b00cef80 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b7800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b015e8f0 con 0x7fb0b00aee30 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b7800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015a8c0 con 0x7fb0b015b390 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0b00aee30 0x7fb0b00cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0b00aee30 0x7fb0b00cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56282/0 (socket says 172.21.15.164:56282) 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 -- 172.21.15.164:0/2186314837 learned_addr learned my addr 172.21.15.164:0/2186314837 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b4e00640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 0x7fb0b006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 -- 172.21.15.164:0/2186314837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 msgr2=0x7fb0b006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 0x7fb0b006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 -- 172.21.15.164:0/2186314837 >> v1:172.21.15.172:6789/0 conn(0x7fb0b00cef80 legacy=0x7fb0b00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:27.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 -- 172.21.15.164:0/2186314837 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b015ed80 con 0x7fb0b00aee30 2024-05-08T04:44:27.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0b4e00640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 0x7fb0b006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:27.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.810+0000 7fb0afe00640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b00aee30 0x7fb0b00cea40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009870 tx=0x7fb0a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af844a20ee2f11c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:27.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0af400640 1 -- 172.21.15.164:0/2186314837 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00b0070 con 0x7fb0b00aee30 2024-05-08T04:44:27.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0af400640 1 -- 172.21.15.164:0/2186314837 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a00ab600 con 0x7fb0b00aee30 2024-05-08T04:44:27.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0af400640 1 -- 172.21.15.164:0/2186314837 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00ab9d0 con 0x7fb0b00aee30 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- 172.21.15.164:0/2186314837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b00aee30 msgr2=0x7fb0b00cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b00aee30 0x7fb0b00cea40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009870 tx=0x7fb0a001c520 comp rx=0 tx=0).stop 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- 172.21.15.164:0/2186314837 shutdown_connections 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b00aee30 0x7fb0b00cea40 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- 172.21.15.164:0/2186314837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b015b390 0x7fb0b006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- 172.21.15.164:0/2186314837 >> 172.21.15.164:0/2186314837 conn(0x7fb0b006ac90 msgr2=0x7fb0b006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- 172.21.15.164:0/2186314837 shutdown_connections 2024-05-08T04:44:27.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- 172.21.15.164:0/2186314837 wait complete. 2024-05-08T04:44:27.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 Processor -- start 2024-05-08T04:44:27.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- start start 2024-05-08T04:44:27.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b00aee30 0x7fb0b0169000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b00cef80 0x7fb0b0167360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015b770 con 0x7fb0b015b390 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b006ddf0 con 0x7fb0b00cef80 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b7800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b00cdd40 con 0x7fb0b00aee30 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56286/0 (socket says 172.21.15.164:56286) 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b5800640 1 -- 172.21.15.164:0/1342442772 learned_addr learned my addr 172.21.15.164:0/1342442772 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0afe00640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b00cef80 0x7fb0b0167360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.814+0000 7fb0b4e00640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b00aee30 0x7fb0b0169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 -- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b00aee30 msgr2=0x7fb0b0169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b00aee30 0x7fb0b0169000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 -- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b00cef80 msgr2=0x7fb0b0167360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b00cef80 0x7fb0b0167360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 -- 172.21.15.164:0/1342442772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b01678a0 con 0x7fb0b015b390 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b5800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c780 tx=0x7fb0a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004b70 con 0x7fb0b015b390 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8004d10 con 0x7fb0b015b390 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800aa90 con 0x7fb0b015b390 2024-05-08T04:44:27.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0167b40 con 0x7fb0b015b390 2024-05-08T04:44:27.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b0168100 con 0x7fb0b015b390 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b00aaeb0 con 0x7fb0b015b390 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.818+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7fb0a800ac30 con 0x7fb0b015b390 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.822+0000 7fb0acc00640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 0x7fb08c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.822+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb0a8083e90 con 0x7fb0b015b390 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.822+0000 7fb0b4e00640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 0x7fb08c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:27.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.822+0000 7fb0b4e00640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 0x7fb08c06a1a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00aeb30 tx=0x7fb098008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:27.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.826+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0a8020070 con 0x7fb0b015b390 2024-05-08T04:44:27.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 --> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0b0069d60 con 0x7fb08c067ce0 2024-05-08T04:44:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0acc00640 1 -- 172.21.15.164:0/1342442772 <== mgr.4101 v2:172.21.15.164:6832/572599534 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7fb0b0069d60 con 0x7fb08c067ce0 2024-05-08T04:44:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:27.980 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 msgr2=0x7fb08c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 0x7fb08c06a1a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00aeb30 tx=0x7fb098008040 comp rx=0 tx=0).stop 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 msgr2=0x7fb0b00b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c780 tx=0x7fb0a800cc40 comp rx=0 tx=0).stop 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 shutdown_connections 2024-05-08T04:44:27.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7fb08c067ce0 0x7fb08c06a1a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b015b390 0x7fb0b00b3730 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b00cef80 0x7fb0b0167360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 --2- 172.21.15.164:0/1342442772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0b00aee30 0x7fb0b0169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 >> 172.21.15.164:0/1342442772 conn(0x7fb0b006ac90 msgr2=0x7fb0b0153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 shutdown_connections 2024-05-08T04:44:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:27.978+0000 7fb0b7800640 1 -- 172.21.15.164:0/1342442772 wait complete. 2024-05-08T04:44:28.001 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-05-08T04:44:27.502369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263244,"kb_used_data":13800,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711476,"statfs":{"total":773094113280,"available":772824551424,"internally_reserved":0,"allocated":14131200,"data_stored":6846248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002676"},"pg_stats":[{"pgid":"1.0","version":"129'119","reported_seq":383,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T04:44:07.930725+0000","last_change":"2024-05-08T04:44:07.930725+0000","last_active":"2024-05-08T04:44:07.930725+0000","last_peered":"2024-05-08T04:44:07.930725+0000","last_clean":"2024-05-08T04:44:07.930725+0000","last_became_active":"2024-05-08T04:38:19.597939+0000","last_became_peered":"2024-05-08T04:38:19.597939+0000","last_unstale":"2024-05-08T04:44:07.930725+0000","last_undegraded":"2024-05-08T04:44:07.930725+0000","last_fullsized":"2024-05-08T04:44:07.930725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'119","last_scrub_stamp":"2024-05-08T04:44:07.930591+0000","last_deep_scrub":"129'119","last_deep_scrub_stamp":"2024-05-08T04:44:07.930591+0000","last_clean_scrub_stamp":"2024-05-08T04:44:07.930591+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:37:22.154799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2211840,"data_stored":1298557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":708189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2211840,"data_stored":1298557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:44:28.002 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T04:44:28.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.134+0000 7f3f7d800640 1 Processor -- start 2024-05-08T04:44:28.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.134+0000 7f3f7d800640 1 -- start start 2024-05-08T04:44:28.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f7d800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f7815ce10 0x7f3f7806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f7d800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f780c6600 con 0x7f3f7806e390 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f7d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f780c6040 con 0x7f3f7815ba20 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f7d800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7815fbe0 con 0x7f3f7815ce10 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f76000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f7815ce10 0x7f3f7806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f77400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3f7806e390 0x7f3f780ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:44900/0 (socket says 172.21.15.164:44900) 2024-05-08T04:44:28.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f77400640 1 -- 172.21.15.164:0/268589404 learned_addr learned my addr 172.21.15.164:0/268589404 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-05-08T04:44:28.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f76000640 1 -- 172.21.15.164:0/268589404 >> v1:172.21.15.172:6789/0 conn(0x7f3f7815ba20 legacy=0x7f3f78160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f76000640 1 -- 172.21.15.164:0/268589404 >> v1:172.21.15.164:6789/0 conn(0x7f3f7806e390 legacy=0x7f3f780ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f76000640 1 -- 172.21.15.164:0/268589404 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780c6320 con 0x7f3f7815ce10 2024-05-08T04:44:28.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f75600640 1 -- 172.21.15.164:0/268589404 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299117763 0 0) 0x7f3f780c6600 con 0x7f3f7806e390 2024-05-08T04:44:28.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f76000640 1 --2- 172.21.15.164:0/268589404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7815ce10 0x7f3f7806de50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3f640097c0 tx=0x7f3f6401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf9ea5e832ee9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f75600640 1 -- 172.21.15.164:0/268589404 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f640b1070 con 0x7f3f7815ce10 2024-05-08T04:44:28.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f75600640 1 -- 172.21.15.164:0/268589404 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f640ac410 con 0x7f3f7815ce10 2024-05-08T04:44:28.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.138+0000 7f3f75600640 1 -- 172.21.15.164:0/268589404 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f640ac7e0 con 0x7f3f7815ce10 2024-05-08T04:44:28.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- 172.21.15.164:0/268589404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7815ce10 msgr2=0x7f3f7806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 --2- 172.21.15.164:0/268589404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7815ce10 0x7f3f7806de50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3f640097c0 tx=0x7f3f6401c4a0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- 172.21.15.164:0/268589404 shutdown_connections 2024-05-08T04:44:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 --2- 172.21.15.164:0/268589404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7815ce10 0x7f3f7806de50 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- 172.21.15.164:0/268589404 >> 172.21.15.164:0/268589404 conn(0x7f3f780cf770 msgr2=0x7f3f780cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- 172.21.15.164:0/268589404 shutdown_connections 2024-05-08T04:44:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- 172.21.15.164:0/268589404 wait complete. 2024-05-08T04:44:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 Processor -- start 2024-05-08T04:44:28.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- start start 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f7815ba20 0x7f3f781476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 0x7f3f780afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780ce000 con 0x7f3f7806e390 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780cde80 con 0x7f3f7815ce10 2024-05-08T04:44:28.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f7d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78169b80 con 0x7f3f7815ba20 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56298/0 (socket says 172.21.15.164:56298) 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 -- 172.21.15.164:0/1318469095 learned_addr learned my addr 172.21.15.164:0/1318469095 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f77400640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 0x7f3f780afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76000640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f7815ba20 0x7f3f781476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 -- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f7815ba20 msgr2=0x7f3f781476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f7815ba20 0x7f3f781476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 -- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 msgr2=0x7f3f780afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 0x7f3f780afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f76a00640 1 -- 172.21.15.164:0/1318469095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780b0f50 con 0x7f3f7806e390 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.142+0000 7f3f77400640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 0x7f3f780afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f76a00640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800c500 tx=0x7f3f6800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68016020 con 0x7f3f7806e390 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f68005220 con 0x7f3f7806e390 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007030 con 0x7f3f7806e390 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78153240 con 0x7f3f7806e390 2024-05-08T04:44:28.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f78147c20 con 0x7f3f7806e390 2024-05-08T04:44:28.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f78072d30 con 0x7f3f7806e390 2024-05-08T04:44:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f3f680071f0 con 0x7f3f7806e390 2024-05-08T04:44:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 0x7f3f5806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f3f68083860 con 0x7f3f7806e390 2024-05-08T04:44:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.146+0000 7f3f76000640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 0x7f3f5806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.150+0000 7f3f76000640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 0x7f3f5806a1d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3f640ae000 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.150+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f68020020 con 0x7f3f7806e390 2024-05-08T04:44:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.306+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3f7815b1e0 con 0x7f3f7806e390 2024-05-08T04:44:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3f68055ed0 con 0x7f3f7806e390 2024-05-08T04:44:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f6ea00640 1 -- 172.21.15.164:0/1318469095 <== mon.0 v2:172.21.15.164:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3f680557d0 con 0x7f3f7806e390 2024-05-08T04:44:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 msgr2=0x7f3f5806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 0x7f3f5806a1d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3f640ae000 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 msgr2=0x7f3f7815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800c500 tx=0x7f3f6800c9c0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 shutdown_connections 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f3f58067d10 0x7f3f5806a1d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f7815ba20 0x7f3f781476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f7815ce10 0x7f3f780afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 --2- 172.21.15.164:0/1318469095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f7806e390 0x7f3f7815ab30 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.310+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 >> 172.21.15.164:0/1318469095 conn(0x7f3f780cf770 msgr2=0x7f3f7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.314+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 shutdown_connections 2024-05-08T04:44:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.314+0000 7f3f7d800640 1 -- 172.21.15.164:0/1318469095 wait complete. 2024-05-08T04:44:28.336 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T04:44:28.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T04:44:28.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.470+0000 7f4122e00640 1 Processor -- start 2024-05-08T04:44:28.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.470+0000 7f4122e00640 1 -- start start 2024-05-08T04:44:28.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4122e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 0x7f411c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4122e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f411c15be50 0x7f411c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4122e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0cd9f0 con 0x7f411c0aeda0 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4122e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c15fe70 con 0x7f411c15be50 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4122e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0cdcd0 con 0x7f411c15ba70 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f411c15be50 0x7f411c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f411c15be50 0x7f411c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56302/0 (socket says 172.21.15.164:56302) 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 -- 172.21.15.164:0/1665377404 learned_addr learned my addr 172.21.15.164:0/1665377404 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 -- 172.21.15.164:0/1665377404 >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 msgr2=0x7f411c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f411be00640 1 --2- 172.21.15.164:0/1665377404 >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 0x7f411c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 --2- 172.21.15.164:0/1665377404 >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 0x7f411c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 -- 172.21.15.164:0/1665377404 >> v1:172.21.15.164:6789/0 conn(0x7f411c0aeda0 legacy=0x7f411c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f4113e00640 1 -- 172.21.15.164:0/1665377404 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c15f820 con 0x7f411c15be50 2024-05-08T04:44:28.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f411b400640 1 -- 172.21.15.164:0/1665377404 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404038138 0 0) 0x7f411c0cd9f0 con 0x7f411c0aeda0 2024-05-08T04:44:28.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.474+0000 7f411be00640 1 --2- 172.21.15.164:0/1665377404 >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 0x7f411c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:44:28.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4113e00640 1 --2- 172.21.15.164:0/1665377404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c15be50 0x7f411c0cf690 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f411c06bc70 tx=0x7f410401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1326903f44f68627 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f411b400640 1 -- 172.21.15.164:0/1665377404 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040ab600 con 0x7f411c15be50 2024-05-08T04:44:28.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f411b400640 1 -- 172.21.15.164:0/1665377404 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f41040b0070 con 0x7f411c15be50 2024-05-08T04:44:28.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f411b400640 1 -- 172.21.15.164:0/1665377404 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040ab9e0 con 0x7f411c15be50 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- 172.21.15.164:0/1665377404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c15be50 msgr2=0x7f411c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 --2- 172.21.15.164:0/1665377404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c15be50 0x7f411c0cf690 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f411c06bc70 tx=0x7f410401c4f0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- 172.21.15.164:0/1665377404 shutdown_connections 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 --2- 172.21.15.164:0/1665377404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c15be50 0x7f411c0cf690 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 --2- 172.21.15.164:0/1665377404 >> v2:172.21.15.172:3300/0 conn(0x7f411c15ba70 0x7f411c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- 172.21.15.164:0/1665377404 >> 172.21.15.164:0/1665377404 conn(0x7f411c06ac90 msgr2=0x7f411c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:28.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- 172.21.15.164:0/1665377404 shutdown_connections 2024-05-08T04:44:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- 172.21.15.164:0/1665377404 wait complete. 2024-05-08T04:44:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 Processor -- start 2024-05-08T04:44:28.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.478+0000 7f4122e00640 1 -- start start 2024-05-08T04:44:28.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411c15ba70 0x7f411c14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f411c15be50 0x7f411c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c168d40 con 0x7f411c0aeda0 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c168bc0 con 0x7f411c15ba70 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c168ec0 con 0x7f411c15be50 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56318/0 (socket says 172.21.15.164:56318) 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 -- 172.21.15.164:0/2416563613 learned_addr learned my addr 172.21.15.164:0/2416563613 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4113e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411c15ba70 0x7f411c14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4120e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f411c15be50 0x7f411c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 -- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f411c15be50 msgr2=0x7f411c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f411c15be50 0x7f411c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 -- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411c15ba70 msgr2=0x7f411c14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411c15ba70 0x7f411c14f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 -- 172.21.15.164:0/2416563613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c14b700 con 0x7f411c0aeda0 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f411be00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c7c0 tx=0x7f410c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c007a30 con 0x7f411c0aeda0 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f410c007bd0 con 0x7f411c0aeda0 2024-05-08T04:44:28.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0113f0 con 0x7f411c0aeda0 2024-05-08T04:44:28.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c14b9e0 con 0x7f411c0aeda0 2024-05-08T04:44:28.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c14bfa0 con 0x7f411c0aeda0 2024-05-08T04:44:28.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.482+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c069d60 con 0x7f411c0aeda0 2024-05-08T04:44:28.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.486+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 87951+0+0 (secure 0 0 0) 0x7f410c0115b0 con 0x7f411c0aeda0 2024-05-08T04:44:28.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.486+0000 7f4118c00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 0x7f40f806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:44:28.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.486+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f410c083f60 con 0x7f411c0aeda0 2024-05-08T04:44:28.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.486+0000 7f4113e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 0x7f40f806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:44:28.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.486+0000 7f4113e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 0x7f40f806a1d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f41040af000 tx=0x7f41040a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:44:28.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.490+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f410c088e20 con 0x7f411c0aeda0 2024-05-08T04:44:28.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.674+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f411c15b0e0 con 0x7f411c0aeda0 2024-05-08T04:44:28.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.674+0000 7f4118c00640 1 -- 172.21.15.164:0/2416563613 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f410c015020 con 0x7f411c0aeda0 2024-05-08T04:44:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-08T04:44:28.678 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3466153510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3466153510},{"type":"v1","addr":"172.21.15.172:6835","nonce":3466153510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1771025621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1771025621},{"type":"v1","addr":"172.21.15.172:6834","nonce":1771025621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/3647200559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3647200559},{"type":"v1","addr":"172.21.15.164:6836","nonce":3647200559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":8793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/458768443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":458768443},{"type":"v1","addr":"172.21.15.164:6837","nonce":458768443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[]} 2024-05-08T04:44:28.678 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 85 2024-05-08T04:44:28.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 msgr2=0x7f40f806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 0x7f40f806a1d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f41040af000 tx=0x7f41040a7000 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 msgr2=0x7f411c13a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c7c0 tx=0x7f410c00cc80 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 shutdown_connections 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:6832/572599534,v1:172.21.15.164:6833/572599534] conn(0x7f40f8067d10 0x7f40f806a1d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f411c15be50 0x7f411c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411c15ba70 0x7f411c14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 --2- 172.21.15.164:0/2416563613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c0aeda0 0x7f411c13a160 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 >> 172.21.15.164:0/2416563613 conn(0x7f411c06ac90 msgr2=0x7f411c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 shutdown_connections 2024-05-08T04:44:28.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-08T04:44:28.678+0000 7f4122e00640 1 -- 172.21.15.164:0/2416563613 wait complete. 2024-05-08T04:44:28.698 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T04:44:28.698 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T04:44:28.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T04:44:28.699 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:28.711 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T04:44:28.711 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T04:44:28.711 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:28.767 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T04:44:28.767 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T04:44:28.767 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:28.781 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T04:44:28.781 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T04:44:28.781 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:28.842 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T04:44:28.842 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T04:44:28.843 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T04:44:28.843 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:29.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:29.212 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:31.556 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-08T04:44:34.945 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T04:44:34.946 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T04:44:34.946 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:35.007 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T04:44:35.008 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T04:44:35.008 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:37.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-08T04:44:37.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-08T04:44:41.110 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T04:44:41.111 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T04:44:41.111 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:41.180 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T04:44:41.181 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T04:44:41.181 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:41.279 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T04:44:41.279 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T04:44:41.280 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:43.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-08T04:44:43.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-08T04:44:47.382 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T04:44:47.382 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T04:44:47.383 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:47.466 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T04:44:47.466 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T04:44:47.467 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:48.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-08T04:44:48.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-08T04:44:53.568 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T04:44:53.569 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T04:44:53.570 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T04:44:53.570 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:53.632 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T04:44:53.633 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T04:44:53.633 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:53.687 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T04:44:53.687 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T04:44:53.688 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T04:44:53.688 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:53.708 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T04:44:53.708 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T04:44:53.708 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:53.725 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T04:44:53.725 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T04:44:53.725 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:44:53.785 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T04:44:53.785 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T04:44:53.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-08T04:44:53.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-08T04:44:53.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-08T04:44:53.819 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:44:53.819 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T04:44:54.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:44:54.052 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T04:44:54.249 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:44:54.249 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T04:44:54.422 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:44:54.422 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T04:44:54.590 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:44:54.590 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T04:44:54.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:44:54.791 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T04:44:54.979 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:44:54.980 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T04:44:55.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:44:55.187 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T04:44:55.363 INFO:tasks.ceph:Archiving mon data... 2024-05-08T04:44:55.364 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/data/mon.a.tgz 2024-05-08T04:44:55.365 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-05-08T04:44:55.373 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.9BPbl0gvzB 2024-05-08T04:44:55.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.9BPbl0gvzB 2024-05-08T04:44:55.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.9BPbl0gvzB 2024-05-08T04:44:55.649 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.9BPbl0gvzB is 817KB 2024-05-08T04:44:55.721 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.9BPbl0gvzB 2024-05-08T04:44:55.729 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/data/mon.b.tgz 2024-05-08T04:44:55.729 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-05-08T04:44:55.738 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.TZttW1ZzfK 2024-05-08T04:44:55.739 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TZttW1ZzfK 2024-05-08T04:44:55.940 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.TZttW1ZzfK 2024-05-08T04:44:56.015 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.TZttW1ZzfK is 817KB 2024-05-08T04:44:56.087 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.TZttW1ZzfK 2024-05-08T04:44:56.096 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/data/mon.c.tgz 2024-05-08T04:44:56.096 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-05-08T04:44:56.147 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.fAfscUMxmg 2024-05-08T04:44:56.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fAfscUMxmg 2024-05-08T04:44:56.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.fAfscUMxmg 2024-05-08T04:44:56.416 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.fAfscUMxmg is 817KB 2024-05-08T04:44:56.487 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.fAfscUMxmg 2024-05-08T04:44:56.495 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T04:44:56.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T04:44:56.498 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T04:44:56.570 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T04:44:56.603 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T04:44:56.604 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T04:44:56.605 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi164/crash 2024-05-08T04:44:56.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T04:44:56.637 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi172/crash 2024-05-08T04:44:56.638 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T04:44:56.661 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T04:44:56.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:56.684 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:56.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:56.772 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T04:44:56.814 INFO:tasks.ceph:Compressing logs... 2024-05-08T04:44:56.814 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T04:44:56.817 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T04:44:56.850 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36882.log 2024-05-08T04:44:56.851 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34285.log 2024-05-08T04:44:56.852 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39725.log 2024-05-08T04:44:56.852 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36882.log.gz 2024-05-08T04:44:56.853 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49339.log 2024-05-08T04:44:56.853 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34285.log.gz 2024-05-08T04:44:56.854 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-05-08T04:44:56.854 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39725.log.gz 2024-05-08T04:44:56.854 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27646.log 2024-05-08T04:44:56.855 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49339.log.gz 2024-05-08T04:44:56.855 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-05-08T04:44:56.856 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-05-08T04:44:56.856 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43473.log 2024-05-08T04:44:56.857 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27646.log.gz 2024-05-08T04:44:56.857 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43050.log 2024-05-08T04:44:56.857 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-05-08T04:44:56.859 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44979.log 2024-05-08T04:44:56.859 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43473.log.gz 2024-05-08T04:44:56.859 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2024-05-08T04:44:56.859 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43050.log.gz 2024-05-08T04:44:56.861 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-05-08T04:44:56.861 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44979.log.gz 2024-05-08T04:44:56.861 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-05-08T04:44:56.861 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2024-05-08T04:44:56.861 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36576.log 2024-05-08T04:44:56.862 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2024-05-08T04:44:56.862 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-05-08T04:44:56.862 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-05-08T04:44:56.863 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44303.log 2024-05-08T04:44:56.864 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36576.log.gz 2024-05-08T04:44:56.864 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42984.log 2024-05-08T04:44:56.864 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-05-08T04:44:56.864 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44953.log 2024-05-08T04:44:56.865 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44303.log.gz 2024-05-08T04:44:56.866 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.42984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42543.log 2024-05-08T04:44:56.866 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42984.log.gz 2024-05-08T04:44:56.866 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38434.log 2024-05-08T04:44:56.867 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44953.log.gz 2024-05-08T04:44:56.867 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51814.log 2024-05-08T04:44:56.867 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42543.log.gz 2024-05-08T04:44:56.868 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2024-05-08T04:44:56.868 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38434.log.gz 2024-05-08T04:44:56.869 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47278.log 2024-05-08T04:44:56.869 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51814.log.gz 2024-05-08T04:44:56.869 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50612.log 2024-05-08T04:44:56.870 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz 2024-05-08T04:44:56.870 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50288.log 2024-05-08T04:44:56.870 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47278.log.gz 2024-05-08T04:44:56.871 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44198.log 2024-05-08T04:44:56.872 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50612.log.gz 2024-05-08T04:44:56.872 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-05-08T04:44:56.872 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50288.log.gz 2024-05-08T04:44:56.873 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46788.log 2024-05-08T04:44:56.873 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz 2024-05-08T04:44:56.873 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-05-08T04:44:56.874 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.46788.log: gzip 91.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz -- 2024-05-08T04:44:56.874 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-client.admin.50235.log 2024-05-08T04:44:56.874 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46788.log.gz 2024-05-08T04:44:56.875 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T04:44:56.875 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-05-08T04:44:56.876 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51710.log 2024-05-08T04:44:56.876 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50235.log.gz 2024-05-08T04:44:56.876 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38920.log 2024-05-08T04:44:56.877 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2024-05-08T04:44:56.877 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51710.log.gz 2024-05-08T04:44:56.878 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-05-08T04:44:56.878 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38920.log.gz 2024-05-08T04:44:56.879 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-05-08T04:44:56.879 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2024-05-08T04:44:56.880 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48513.log 2024-05-08T04:44:56.880 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-05-08T04:44:56.881 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-05-08T04:44:56.881 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-05-08T04:44:56.882 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50488.log 2024-05-08T04:44:56.882 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48513.log.gz 2024-05-08T04:44:56.883 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-05-08T04:44:56.883 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-05-08T04:44:56.883 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34041.log 2024-05-08T04:44:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.50488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50488.log.gz 2024-05-08T04:44:56.885 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-05-08T04:44:56.885 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-05-08T04:44:56.885 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-05-08T04:44:56.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34673.log 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34673.log.gz 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36628.log 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-05-08T04:44:56.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46880.log 2024-05-08T04:44:56.895 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2024-05-08T04:44:56.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39349.log 2024-05-08T04:44:56.895 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36628.log.gz 2024-05-08T04:44:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34586.log 2024-05-08T04:44:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27856.log 2024-05-08T04:44:56.904 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34041.log.gz 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34586.log.gz 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34478.log 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27928.log 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27856.log.gz 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34478.log.gz 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27904.log 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33579.log 2024-05-08T04:44:56.905 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27928.log.gz 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33937.log 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27904.log.gz 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33579.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.1.33579.log.gz -5 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33937.log.gz 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33804.log 2024-05-08T04:44:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34473.log 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33804.log.gz 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33656.log 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33809.log 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34473.log.gz 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.33656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33656.log.gz 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27880.log 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33809.log.gz 2024-05-08T04:44:56.907 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27832.log 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27880.log.gz 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34261.log 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33561.log 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27832.log.gz 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34261.log.gz 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27784.log 2024-05-08T04:44:56.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34371.log 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33561.log.gz 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27784.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27784.log.gz 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34371.log.gz 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T04:44:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27808.log 2024-05-08T04:44:56.916 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34046.log 2024-05-08T04:44:56.916 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.27808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27952.log 2024-05-08T04:44:56.916 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27808.log.gz 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34046.log.gz 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr: 92.5%/var/log/ceph/ceph-client.admin.27952.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34581.log 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27952.log.gz 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T04:44:56.917 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33932.log 2024-05-08T04:44:56.918 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34581.log.gz 2024-05-08T04:44:56.918 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-08T04:44:56.918 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.33932.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-08T04:44:56.918 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33932.log.gz 2024-05-08T04:44:56.918 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34366.log 2024-05-08T04:44:56.919 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.1.34366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34366.log.gz 2024-05-08T04:44:56.919 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T04:44:56.920 INFO:teuthology.orchestra.run.smithi172.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T04:44:56.920 INFO:teuthology.orchestra.run.smithi172.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz/var/log/ceph/ceph-mon.b.log: 2024-05-08T04:44:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51684.log 2024-05-08T04:44:56.924 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46880.log.gz 2024-05-08T04:44:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33632.log 2024-05-08T04:44:56.924 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39349.log.gz 2024-05-08T04:44:56.924 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51684.log.gz 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34064.log 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33632.log.gz 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34064.log.gz 2024-05-08T04:44:56.925 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34072.log 2024-05-08T04:44:56.926 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-05-08T04:44:56.926 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-05-08T04:44:56.926 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz 2024-05-08T04:44:56.926 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2024-05-08T04:44:56.926 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34072.log.gz 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33313.log 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38751.log 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33313.log.gz 2024-05-08T04:44:56.930 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39991.log 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38751.log.gz 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43076.log 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39991.log.gz 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33392.log 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-05-08T04:44:56.931 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35382.log 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33392.log.gz 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35382.log.gz 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47767.log 2024-05-08T04:44:56.932 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35538.log 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47767.log.gz 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43632.log 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39252.log 2024-05-08T04:44:56.933 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35538.log.gz 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43632.log.gz 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.39252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39252.log.gz 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45233.log 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38226.log 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45233.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45233.log.gz 2024-05-08T04:44:56.934 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38226.log.gz 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-05-08T04:44:56.935 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34154.log 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33661.log 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34154.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.1.34149.log 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.1.34154.log.gz 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34149.log.gz 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.33661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33661.log.gz 2024-05-08T04:44:56.939 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34256.log 2024-05-08T04:44:56.940 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.34256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34256.log.gz 2024-05-08T04:44:56.948 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-05-08T04:44:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50816.log 2024-05-08T04:44:56.948 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz 2024-05-08T04:44:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36602.log 2024-05-08T04:44:56.948 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40781.log 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50816.log.gz 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36602.log.gz 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40781.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40781.log.gz 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42400.log 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41436.log 2024-05-08T04:44:56.949 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50764.log 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42400.log.gz 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45370.log 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41436.log.gz 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50764.log.gz 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39699.log 2024-05-08T04:44:56.950 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45370.log.gz 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39469.log 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gz 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47462.log 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39699.log.gz 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39469.log.gz 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38408.log 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47462.log.gz 2024-05-08T04:44:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37302.log 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41300.log 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38408.log.gz 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37302.log.gz 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41300.log.gz 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-05-08T04:44:56.952 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33815.log 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37094.log 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51167.log 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33815.log.gz 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36341.log 2024-05-08T04:44:56.953 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37094.log.gz 2024-05-08T04:44:56.954 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-05-08T04:44:56.954 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51167.log.gz 2024-05-08T04:44:56.954 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33866.log 2024-05-08T04:44:56.954 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36341.log.gz 2024-05-08T04:44:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51115.log 2024-05-08T04:44:56.958 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-05-08T04:44:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50261.log 2024-05-08T04:44:56.958 INFO:teuthology.orchestra.run.smithi164.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33866.log.gz 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51115.log: /var/log/ceph/ceph-client.admin.50261.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51115.log.gz 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43606.log 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50261.log.gz 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:gzip/var/log/ceph/ceph-client.admin.45017.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42322.log 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-05-08T04:44:56.959 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43606.log.gz 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42322.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42322.log.gz 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39415.log 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45396.log 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48539.log 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39415.log.gz 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45396.log.gz 2024-05-08T04:44:56.960 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50790.log 2024-05-08T04:44:56.961 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48539.log.gz 2024-05-08T04:44:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34278.log 2024-05-08T04:44:56.963 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-05-08T04:44:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-05-08T04:44:56.963 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50790.log.gz 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34278.log.gz 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51918.log 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-05-08T04:44:56.964 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-05-08T04:44:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51918.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22795.log 2024-05-08T04:44:56.966 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gz 2024-05-08T04:44:56.966 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40887.log: /var/log/ceph/ceph-client.admin.50676.log 2024-05-08T04:44:56.966 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-05-08T04:44:56.966 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.tmp-client.admin.22795.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22795.log.gz 2024-05-08T04:44:56.966 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42675.log 2024-05-08T04:44:56.967 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2024-05-08T04:44:56.968 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50676.log.gz 2024-05-08T04:44:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-05-08T04:44:56.968 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42675.log.gz 2024-05-08T04:44:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51245.log 2024-05-08T04:44:56.969 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2024-05-08T04:44:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45285.log 2024-05-08T04:44:56.970 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-05-08T04:44:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-05-08T04:44:56.970 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51245.log.gz 2024-05-08T04:44:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-05-08T04:44:56.971 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45285.log.gz 2024-05-08T04:44:56.972 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35434.log 2024-05-08T04:44:56.972 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-05-08T04:44:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47409.log 2024-05-08T04:44:56.973 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-05-08T04:44:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51271.log 2024-05-08T04:44:56.974 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35434.log.gz 2024-05-08T04:44:56.975 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34491.log 2024-05-08T04:44:56.975 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47409.log.gz 2024-05-08T04:44:56.976 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-05-08T04:44:56.976 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51271.log.gz 2024-05-08T04:44:56.977 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34491.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T04:44:56.977 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34491.log.gz 2024-05-08T04:44:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-05-08T04:44:56.978 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-05-08T04:44:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39808.log 2024-05-08T04:44:56.979 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35226.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-05-08T04:44:56.979 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-05-08T04:44:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36908.log 2024-05-08T04:44:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45097.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39808.log.gz 2024-05-08T04:44:56.981 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38525.log 2024-05-08T04:44:56.981 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2024-05-08T04:44:56.982 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48758.log 2024-05-08T04:44:56.982 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36908.log.gz 2024-05-08T04:44:56.983 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35304.log 2024-05-08T04:44:56.983 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38525.log.gz 2024-05-08T04:44:56.984 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-05-08T04:44:56.984 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48758.log.gz 2024-05-08T04:44:56.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-05-08T04:44:56.985 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35304.log.gz 2024-05-08T04:44:56.986 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-05-08T04:44:56.986 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-05-08T04:44:56.987 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-05-08T04:44:56.987 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-05-08T04:44:56.988 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-05-08T04:44:56.988 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-05-08T04:44:56.989 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-05-08T04:44:56.989 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44133.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-05-08T04:44:56.990 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39257.log 2024-05-08T04:44:56.990 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-05-08T04:44:56.991 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35886.log 2024-05-08T04:44:56.991 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-05-08T04:44:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41962.log 2024-05-08T04:44:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.39257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39257.log.gz 2024-05-08T04:44:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36180.log 2024-05-08T04:44:56.993 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35886.log.gz 2024-05-08T04:44:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-05-08T04:44:56.994 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41962.log.gz 2024-05-08T04:44:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-05-08T04:44:56.995 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36180.log.gz 2024-05-08T04:44:56.996 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36856.log 2024-05-08T04:44:56.996 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-05-08T04:44:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2024-05-08T04:44:56.997 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-05-08T04:44:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52048.log 2024-05-08T04:44:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36856.log: /var/log/ceph/ceph-client.admin.51970.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36856.log.gz 2024-05-08T04:44:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2024-05-08T04:44:57.000 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2024-05-08T04:44:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45071.log 2024-05-08T04:44:57.000 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52048.log.gz 2024-05-08T04:44:57.001 INFO:teuthology.orchestra.run.smithi172.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-08T04:44:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50534.log 2024-05-08T04:44:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49287.log: /var/log/ceph/ceph-client.admin.45071.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2024-05-08T04:44:57.003 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2024-05-08T04:44:57.003 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45071.log.gz 2024-05-08T04:44:57.003 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-05-08T04:44:57.004 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50534.log.gz 2024-05-08T04:44:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45915.log 2024-05-08T04:44:57.005 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2024-05-08T04:44:57.005 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44927.log 2024-05-08T04:44:57.006 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-05-08T04:44:57.006 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2024-05-08T04:44:57.007 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45915.log.gz 2024-05-08T04:44:57.007 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44927.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T04:44:57.008 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49775.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44927.log.gz 2024-05-08T04:44:57.008 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41702.log 2024-05-08T04:44:57.009 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz 2024-05-08T04:44:57.009 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36446.log 2024-05-08T04:44:57.010 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39323.log 2024-05-08T04:44:57.011 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41702.log.gz 2024-05-08T04:44:57.011 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-05-08T04:44:57.012 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36446.log.gz 2024-05-08T04:44:57.012 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35330.log 2024-05-08T04:44:57.013 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39323.log.gz 2024-05-08T04:44:57.014 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-05-08T04:44:57.014 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-05-08T04:44:57.015 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-05-08T04:44:57.015 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35330.log.gz 2024-05-08T04:44:57.016 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49021.log 2024-05-08T04:44:57.016 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-05-08T04:44:57.017 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2024-05-08T04:44:57.017 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-05-08T04:44:57.018 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48961.log 2024-05-08T04:44:57.018 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49021.log.gz 2024-05-08T04:44:57.019 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37483.log 2024-05-08T04:44:57.019 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2024-05-08T04:44:57.020 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51446.log 2024-05-08T04:44:57.021 INFO:teuthology.orchestra.run.smithi164.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48961.log.gz 2024-05-08T04:44:57.021 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.37483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49365.log 2024-05-08T04:44:57.021 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37483.log.gz 2024-05-08T04:44:57.022 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39297.log 2024-05-08T04:44:57.023 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51446.log.gz 2024-05-08T04:44:57.024 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47304.log 2024-05-08T04:44:57.024 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49365.log.gz 2024-05-08T04:44:57.024 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50738.log 2024-05-08T04:44:57.025 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39297.log.gz 2024-05-08T04:44:57.025 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-05-08T04:44:57.026 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47304.log.gz 2024-05-08T04:44:57.027 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39860.log 2024-05-08T04:44:57.027 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50738.log.gz 2024-05-08T04:44:57.028 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2024-05-08T04:44:57.028 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-05-08T04:44:57.029 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2024-05-08T04:44:57.029 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39860.log.gz 2024-05-08T04:44:57.030 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-05-08T04:44:57.031 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2024-05-08T04:44:57.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34465.log 2024-05-08T04:44:57.032 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2024-05-08T04:44:57.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43937.log 2024-05-08T04:44:57.033 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-05-08T04:44:57.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-05-08T04:44:57.034 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34465.log.gz 2024-05-08T04:44:57.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-05-08T04:44:57.035 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43937.log.gz 2024-05-08T04:44:57.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51762.log 2024-05-08T04:44:57.036 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz 2024-05-08T04:44:57.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2024-05-08T04:44:57.037 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-05-08T04:44:57.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35408.log 2024-05-08T04:44:57.038 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51762.log.gz 2024-05-08T04:44:57.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44856.log 2024-05-08T04:44:57.039 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2024-05-08T04:44:57.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-05-08T04:44:57.041 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35408.log.gz 2024-05-08T04:44:57.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.44856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34855.log 2024-05-08T04:44:57.042 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44856.log.gz 2024-05-08T04:44:57.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45967.log 2024-05-08T04:44:57.043 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-05-08T04:44:57.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42517.log 2024-05-08T04:44:57.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45967.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34855.log.gz 2024-05-08T04:44:57.045 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38174.log 2024-05-08T04:44:57.045 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45967.log.gz 2024-05-08T04:44:57.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-05-08T04:44:57.046 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42517.log.gz 2024-05-08T04:44:57.047 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34077.log 2024-05-08T04:44:57.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44422.log: /var/log/ceph/ceph-client.admin.38174.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-05-08T04:44:57.049 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38174.log.gz 2024-05-08T04:44:57.049 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2024-05-08T04:44:57.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34077.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34077.log.gz 2024-05-08T04:44:57.050 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51788.log 2024-05-08T04:44:57.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-05-08T04:44:57.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51788.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2024-05-08T04:44:57.052 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2024-05-08T04:44:57.052 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51788.log.gz 2024-05-08T04:44:57.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-05-08T04:44:57.053 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2024-05-08T04:44:57.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2024-05-08T04:44:57.054 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2024-05-08T04:44:57.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47904.log 2024-05-08T04:44:57.055 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-05-08T04:44:57.056 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38591.log 2024-05-08T04:44:57.057 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2024-05-08T04:44:57.057 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33148.log 2024-05-08T04:44:57.058 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47904.log.gz 2024-05-08T04:44:57.059 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-05-08T04:44:57.060 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38591.log.gz 2024-05-08T04:44:57.060 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-05-08T04:44:57.061 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33148.log.gz 2024-05-08T04:44:57.061 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33868.log 2024-05-08T04:44:57.061 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-05-08T04:44:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-05-08T04:44:57.062 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-05-08T04:44:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50984.log 2024-05-08T04:44:57.063 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33868.log.gz 2024-05-08T04:44:57.064 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38669.log 2024-05-08T04:44:57.065 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50984.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-05-08T04:44:57.065 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41806.log 2024-05-08T04:44:57.065 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50984.log.gz 2024-05-08T04:44:57.066 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2024-05-08T04:44:57.067 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38669.log.gz 2024-05-08T04:44:57.067 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-05-08T04:44:57.068 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41806.log.gz 2024-05-08T04:44:57.068 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40121.log 2024-05-08T04:44:57.069 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2024-05-08T04:44:57.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39104.log 2024-05-08T04:44:57.070 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-05-08T04:44:57.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39673.log 2024-05-08T04:44:57.071 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40121.log.gz 2024-05-08T04:44:57.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-05-08T04:44:57.072 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39104.log.gz 2024-05-08T04:44:57.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39000.log 2024-05-08T04:44:57.073 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39673.log.gz 2024-05-08T04:44:57.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-05-08T04:44:57.074 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-05-08T04:44:57.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34595.log 2024-05-08T04:44:57.076 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39000.log.gz 2024-05-08T04:44:57.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48162.log 2024-05-08T04:44:57.076 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz 2024-05-08T04:44:57.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52100.log 2024-05-08T04:44:57.077 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34595.log.gz 2024-05-08T04:44:57.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2024-05-08T04:44:57.079 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48162.log.gz 2024-05-08T04:44:57.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40611.log 2024-05-08T04:44:57.080 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52100.log.gz 2024-05-08T04:44:57.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-05-08T04:44:57.081 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-05-08T04:44:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 2024-05-08T04:44:57.082 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40611.log.gz 2024-05-08T04:44:57.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-05-08T04:44:57.083 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-05-08T04:44:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-05-08T04:44:57.084 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz 2024-05-08T04:44:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33762.log 2024-05-08T04:44:57.085 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-05-08T04:44:57.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-05-08T04:44:57.086 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-05-08T04:44:57.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-05-08T04:44:57.088 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33762.log.gz 2024-05-08T04:44:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-05-08T04:44:57.089 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-05-08T04:44:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-05-08T04:44:57.090 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-05-08T04:44:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-05-08T04:44:57.091 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-05-08T04:44:57.092 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47715.log 2024-05-08T04:44:57.092 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-05-08T04:44:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39751.log 2024-05-08T04:44:57.093 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-05-08T04:44:57.094 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50508.log 2024-05-08T04:44:57.094 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47715.log.gz/var/log/ceph/ceph-client.admin.39751.log: 2024-05-08T04:44:57.095 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-05-08T04:44:57.095 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39751.log.gz 2024-05-08T04:44:57.096 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-05-08T04:44:57.096 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-05-08T04:44:57.097 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42805.log 2024-05-08T04:44:57.097 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-05-08T04:44:57.099 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33858.log 2024-05-08T04:44:57.099 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-05-08T04:44:57.100 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34284.log 2024-05-08T04:44:57.100 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42805.log.gz 2024-05-08T04:44:57.101 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-05-08T04:44:57.101 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33858.log.gz 2024-05-08T04:44:57.102 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-05-08T04:44:57.102 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34284.log.gz 2024-05-08T04:44:57.103 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-05-08T04:44:57.103 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz 2024-05-08T04:44:57.104 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40147.log 2024-05-08T04:44:57.104 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-05-08T04:44:57.105 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47689.log 2024-05-08T04:44:57.105 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-05-08T04:44:57.106 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40965.log 2024-05-08T04:44:57.106 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40147.log.gz 2024-05-08T04:44:57.107 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-05-08T04:44:57.107 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47689.log.gz 2024-05-08T04:44:57.108 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2024-05-08T04:44:57.109 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40965.log.gz 2024-05-08T04:44:57.109 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34569.log 2024-05-08T04:44:57.110 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-05-08T04:44:57.111 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37861.log 2024-05-08T04:44:57.111 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz 2024-05-08T04:44:57.112 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-05-08T04:44:57.112 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34569.log.gz 2024-05-08T04:44:57.113 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48661.log 2024-05-08T04:44:57.113 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37861.log.gz 2024-05-08T04:44:57.114 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51736.log 2024-05-08T04:44:57.114 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-05-08T04:44:57.115 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.48661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48661.log.gz 2024-05-08T04:44:57.115 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T04:44:57.116 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2024-05-08T04:44:57.117 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51736.log.gz 2024-05-08T04:44:57.117 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-05-08T04:44:57.119 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50066.log 2024-05-08T04:44:57.119 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2024-05-08T04:44:57.120 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 2024-05-08T04:44:57.120 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-05-08T04:44:57.121 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50066.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50066.log.gz 2024-05-08T04:44:57.122 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-05-08T04:44:57.122 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2024-05-08T04:44:57.123 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2024-05-08T04:44:57.124 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44394.log 2024-05-08T04:44:57.124 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-05-08T04:44:57.125 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46253.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2024-05-08T04:44:57.126 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44394.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44394.log.gz 2024-05-08T04:44:57.127 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-05-08T04:44:57.128 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-05-08T04:44:57.129 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37224.log 2024-05-08T04:44:57.130 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-05-08T04:44:57.130 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33339.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-05-08T04:44:57.131 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37224.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37224.log.gz 2024-05-08T04:44:57.132 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42224.log 2024-05-08T04:44:57.133 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37328.log 2024-05-08T04:44:57.135 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-05-08T04:44:57.135 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42224.log.gz 2024-05-08T04:44:57.135 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37328.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37328.log.gz 2024-05-08T04:44:57.136 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47592.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz 2024-05-08T04:44:57.137 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46783.log 2024-05-08T04:44:57.138 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-05-08T04:44:57.139 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.46783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34283.log 2024-05-08T04:44:57.139 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46783.log.gz 2024-05-08T04:44:57.141 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51580.log 2024-05-08T04:44:57.141 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-05-08T04:44:57.141 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34283.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34283.log.gz 2024-05-08T04:44:57.142 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49047.log 2024-05-08T04:44:57.143 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-05-08T04:44:57.144 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51580.log.gz 2024-05-08T04:44:57.145 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40663.log 2024-05-08T04:44:57.145 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49047.log.gz 2024-05-08T04:44:57.146 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39965.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-05-08T04:44:57.146 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46423.log 2024-05-08T04:44:57.148 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40663.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40663.log.gz 2024-05-08T04:44:57.149 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5/var/log/ceph/ceph-client.admin.46423.log: --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2024-05-08T04:44:57.149 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46423.log.gz 2024-05-08T04:44:57.149 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48822.log 2024-05-08T04:44:57.151 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46162.log 2024-05-08T04:44:57.151 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2024-05-08T04:44:57.152 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48822.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48822.log.gz 2024-05-08T04:44:57.153 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46162.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46162.log.gz 2024-05-08T04:44:57.153 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34281.log 2024-05-08T04:44:57.155 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-05-08T04:44:57.156 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2024-05-08T04:44:57.156 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34281.log.gz 2024-05-08T04:44:57.157 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45766.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-05-08T04:44:57.158 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2024-05-08T04:44:57.159 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42165.log 2024-05-08T04:44:57.160 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44277.log 2024-05-08T04:44:57.160 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T04:44:57.161 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34543.log 2024-05-08T04:44:57.161 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42165.log.gz 2024-05-08T04:44:57.162 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44277.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44277.log.gz 2024-05-08T04:44:57.163 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34543.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34543.log.gz 2024-05-08T04:44:57.164 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36102.log 2024-05-08T04:44:57.165 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-05-08T04:44:57.166 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-05-08T04:44:57.166 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36102.log.gz 2024-05-08T04:44:57.167 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40400.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz 2024-05-08T04:44:57.168 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42348.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-05-08T04:44:57.169 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51193.log 2024-05-08T04:44:57.170 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47357.log 2024-05-08T04:44:57.171 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42933.log 2024-05-08T04:44:57.171 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51193.log.gz 2024-05-08T04:44:57.172 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47357.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47357.log.gz 2024-05-08T04:44:57.174 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42933.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50129.log 2024-05-08T04:44:57.174 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42933.log.gz 2024-05-08T04:44:57.175 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38840.log 2024-05-08T04:44:57.177 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-05-08T04:44:57.177 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50129.log.gz 2024-05-08T04:44:57.178 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38840.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38840.log.gz 2024-05-08T04:44:57.179 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49749.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-05-08T04:44:57.179 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2024-05-08T04:44:57.180 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-05-08T04:44:57.182 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47252.log 2024-05-08T04:44:57.183 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2024-05-08T04:44:57.183 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-05-08T04:44:57.183 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43552.log 2024-05-08T04:44:57.183 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-05-08T04:44:57.184 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41910.log 2024-05-08T04:44:57.184 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47252.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47252.log.gz 2024-05-08T04:44:57.185 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-05-08T04:44:57.185 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43552.log.gz 2024-05-08T04:44:57.185 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41132.log 2024-05-08T04:44:57.186 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41910.log.gz 2024-05-08T04:44:57.187 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37751.log 2024-05-08T04:44:57.187 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-05-08T04:44:57.188 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41113.log 2024-05-08T04:44:57.188 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41132.log.gz 2024-05-08T04:44:57.189 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35149.log 2024-05-08T04:44:57.189 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.41113.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37751.log.gz 2024-05-08T04:44:57.189 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41113.log.gz 2024-05-08T04:44:57.190 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2024-05-08T04:44:57.190 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.2.35149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35149.log.gz 2024-05-08T04:44:57.191 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33788.log 2024-05-08T04:44:57.192 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T04:44:57.192 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43338.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz 2024-05-08T04:44:57.193 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39647.log 2024-05-08T04:44:57.193 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33788.log.gz 2024-05-08T04:44:57.195 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.39647.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39647.log.gz 2024-05-08T04:44:57.196 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-05-08T04:44:57.197 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-05-08T04:44:57.198 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39052.log 2024-05-08T04:44:57.199 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gz 2024-05-08T04:44:57.199 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37497.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-05-08T04:44:57.201 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39052.log.gz 2024-05-08T04:44:57.201 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-05-08T04:44:57.202 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-05-08T04:44:57.204 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-05-08T04:44:57.204 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-05-08T04:44:57.205 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44250.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-05-08T04:44:57.206 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44224.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-05-08T04:44:57.206 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51658.log 2024-05-08T04:44:57.207 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40348.log 2024-05-08T04:44:57.209 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-05-08T04:44:57.209 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51658.log.gz 2024-05-08T04:44:57.210 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48277.log 2024-05-08T04:44:57.210 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40348.log.gz 2024-05-08T04:44:57.211 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41351.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-05-08T04:44:57.211 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-05-08T04:44:57.212 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48277.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48277.log.gz 2024-05-08T04:44:57.212 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48329.log 2024-05-08T04:44:57.214 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33865.log 2024-05-08T04:44:57.214 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-05-08T04:44:57.215 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T04:44:57.215 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48329.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48329.log.gz 2024-05-08T04:44:57.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33865.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33865.log.gz 2024-05-08T04:44:57.216 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-05-08T04:44:57.217 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49125.log 2024-05-08T04:44:57.219 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34055.log 2024-05-08T04:44:57.219 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-05-08T04:44:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39443.log 2024-05-08T04:44:57.220 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49125.log.gz 2024-05-08T04:44:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34055.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34055.log.gz 2024-05-08T04:44:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-05-08T04:44:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39443.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39443.log.gz 2024-05-08T04:44:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-05-08T04:44:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38786.log 2024-05-08T04:44:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-05-08T04:44:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40043.log 2024-05-08T04:44:57.227 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38786.log.gz 2024-05-08T04:44:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2024-05-08T04:44:57.228 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-05-08T04:44:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40043.log.gz 2024-05-08T04:44:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27765.log 2024-05-08T04:44:57.230 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2024-05-08T04:44:57.231 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42087.log 2024-05-08T04:44:57.232 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-05-08T04:44:57.232 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27765.log.gz 2024-05-08T04:44:57.234 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42087.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42087.log.gz 2024-05-08T04:44:57.234 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45870.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-05-08T04:44:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35912.log 2024-05-08T04:44:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-05-08T04:44:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-05-08T04:44:57.238 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35912.log.gz 2024-05-08T04:44:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-05-08T04:44:57.239 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-05-08T04:44:57.239 INFO:teuthology.orchestra.run.smithi172.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T04:44:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47057.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gz 2024-05-08T04:44:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36289.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-05-08T04:44:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41514.log 2024-05-08T04:44:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-05-08T04:44:57.244 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-05-08T04:44:57.244 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41514.log.gz 2024-05-08T04:44:57.245 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46227.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-05-08T04:44:57.246 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49983.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-05-08T04:44:57.247 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34725.log 2024-05-08T04:44:57.248 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45207.log 2024-05-08T04:44:57.249 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44861.log 2024-05-08T04:44:57.249 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34725.log.gz 2024-05-08T04:44:57.251 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45207.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45207.log.gz 2024-05-08T04:44:57.252 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.44861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44861.log.gz 2024-05-08T04:44:57.252 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45181.log 2024-05-08T04:44:57.253 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49521.log 2024-05-08T04:44:57.254 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-05-08T04:44:57.255 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45181.log.gz 2024-05-08T04:44:57.255 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49521.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49521.log.gz 2024-05-08T04:44:57.257 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47488.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-05-08T04:44:57.257 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-05-08T04:44:57.258 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51349.log 2024-05-08T04:44:57.260 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51632.log 2024-05-08T04:44:57.260 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-05-08T04:44:57.260 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51349.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51349.log.gz 2024-05-08T04:44:57.261 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51632.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51632.log.gz 2024-05-08T04:44:57.262 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43710.log 2024-05-08T04:44:57.263 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-05-08T04:44:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35512.log 2024-05-08T04:44:57.265 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43710.log.gz 2024-05-08T04:44:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36211.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-05-08T04:44:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35512.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35512.log.gz 2024-05-08T04:44:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2024-05-08T04:44:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39271.log 2024-05-08T04:44:57.269 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48435.log 2024-05-08T04:44:57.269 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2024-05-08T04:44:57.271 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39271.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39271.log.gz 2024-05-08T04:44:57.272 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48435.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48435.log.gz 2024-05-08T04:44:57.272 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39834.log 2024-05-08T04:44:57.273 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-05-08T04:44:57.275 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2024-05-08T04:44:57.275 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39834.log.gz 2024-05-08T04:44:57.276 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45714.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-05-08T04:44:57.277 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49723.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2024-05-08T04:44:57.277 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35460.log 2024-05-08T04:44:57.278 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34271.log 2024-05-08T04:44:57.280 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35590.log 2024-05-08T04:44:57.280 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35460.log.gz 2024-05-08T04:44:57.281 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34271.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34271.log.gz 2024-05-08T04:44:57.282 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35590.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35590.log.gz 2024-05-08T04:44:57.282 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-05-08T04:44:57.283 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33869.log 2024-05-08T04:44:57.285 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36524.log 2024-05-08T04:44:57.285 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz 2024-05-08T04:44:57.286 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33869.log.gz 2024-05-08T04:44:57.287 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36524.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36524.log.gz 2024-05-08T04:44:57.287 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51297.log 2024-05-08T04:44:57.289 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51472.log 2024-05-08T04:44:57.291 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-05-08T04:44:57.291 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51297.log.gz 2024-05-08T04:44:57.292 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51472.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51472.log.gz 2024-05-08T04:44:57.292 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39886.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-05-08T04:44:57.292 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-05-08T04:44:57.294 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-05-08T04:44:57.295 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-05-08T04:44:57.295 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-05-08T04:44:57.296 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41754.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-05-08T04:44:57.297 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-05-08T04:44:57.298 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40719.log 2024-05-08T04:44:57.299 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35356.log 2024-05-08T04:44:57.300 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-05-08T04:44:57.301 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40719.log.gz 2024-05-08T04:44:57.301 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35356.log.gz 2024-05-08T04:44:57.303 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48134.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz 2024-05-08T04:44:57.303 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2024-05-08T04:44:57.304 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43312.log 2024-05-08T04:44:57.305 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43578.log 2024-05-08T04:44:57.306 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz 2024-05-08T04:44:57.306 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-05-08T04:44:57.306 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43312.log.gz 2024-05-08T04:44:57.308 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43578.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43578.log.gz 2024-05-08T04:44:57.308 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2024-05-08T04:44:57.309 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-05-08T04:44:57.309 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37120.log 2024-05-08T04:44:57.311 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42979.log 2024-05-08T04:44:57.311 INFO:teuthology.orchestra.run.smithi164.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2024-05-08T04:44:57.312 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37120.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37120.log.gz 2024-05-08T04:44:57.312 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.42979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42979.log.gz 2024-05-08T04:44:57.313 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-05-08T04:44:57.314 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2024-05-08T04:44:57.316 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-05-08T04:44:57.316 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-05-08T04:44:57.316 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44875.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2024-05-08T04:44:57.317 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47644.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-05-08T04:44:57.318 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51996.log 2024-05-08T04:44:57.319 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51219.log 2024-05-08T04:44:57.320 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51996.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T04:44:57.321 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51996.log.gz 2024-05-08T04:44:57.321 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51219.log.gz 2024-05-08T04:44:57.322 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45993.log 2024-05-08T04:44:57.325 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.45993.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45993.log.gz 2024-05-08T04:44:57.325 INFO:teuthology.orchestra.run.smithi164.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T04:44:57.325 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-05-08T04:44:57.327 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38356.log 2024-05-08T04:44:57.328 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46332.log 2024-05-08T04:44:57.328 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-05-08T04:44:57.329 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38356.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38356.log.gz 2024-05-08T04:44:57.330 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46332.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2024-05-08T04:44:57.331 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52126.log 2024-05-08T04:44:57.332 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51606.log 2024-05-08T04:44:57.333 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34272.log 2024-05-08T04:44:57.334 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52126.log.gz 2024-05-08T04:44:57.334 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51606.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51606.log.gz 2024-05-08T04:44:57.335 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34272.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34272.log.gz 2024-05-08T04:44:57.336 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39026.log 2024-05-08T04:44:57.337 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50010.log 2024-05-08T04:44:57.339 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-05-08T04:44:57.339 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39026.log.gz 2024-05-08T04:44:57.340 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50010.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50010.log.gz 2024-05-08T04:44:57.341 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41017.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-05-08T04:44:57.342 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2024-05-08T04:44:57.343 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38695.log 2024-05-08T04:44:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2024-05-08T04:44:57.345 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2024-05-08T04:44:57.345 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38695.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38695.log.gz 2024-05-08T04:44:57.346 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40296.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2024-05-08T04:44:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41858.log 2024-05-08T04:44:57.348 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-05-08T04:44:57.350 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-05-08T04:44:57.350 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41858.log.gz 2024-05-08T04:44:57.351 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47169.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-05-08T04:44:57.352 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz 2024-05-08T04:44:57.352 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43658.log 2024-05-08T04:44:57.353 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33248.log 2024-05-08T04:44:57.355 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2024-05-08T04:44:57.355 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43658.log.gz 2024-05-08T04:44:57.356 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33248.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33248.log.gz 2024-05-08T04:44:57.357 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40017.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2024-05-08T04:44:57.357 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2024-05-08T04:44:57.358 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42256.log 2024-05-08T04:44:57.360 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42727.log 2024-05-08T04:44:57.360 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2024-05-08T04:44:57.361 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42256.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42256.log.gz 2024-05-08T04:44:57.362 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42727.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42727.log.gz 2024-05-08T04:44:57.362 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-05-08T04:44:57.363 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36472.log 2024-05-08T04:44:57.364 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T04:44:57.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-05-08T04:44:57.365 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-05-08T04:44:57.366 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36472.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36472.log.gz 2024-05-08T04:44:57.367 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38946.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-05-08T04:44:57.368 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43140.log 2024-05-08T04:44:57.369 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42831.log 2024-05-08T04:44:57.370 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-05-08T04:44:57.371 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43140.log.gz 2024-05-08T04:44:57.371 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42831.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42831.log.gz 2024-05-08T04:44:57.372 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34073.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-05-08T04:44:57.373 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-05-08T04:44:57.374 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-05-08T04:44:57.375 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43762.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-05-08T04:44:57.376 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-05-08T04:44:57.377 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35278.log 2024-05-08T04:44:57.378 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2024-05-08T04:44:57.379 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-05-08T04:44:57.380 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35278.log.gz 2024-05-08T04:44:57.380 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44526.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2024-05-08T04:44:57.381 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47871.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-05-08T04:44:57.382 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-05-08T04:44:57.383 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2024-05-08T04:44:57.385 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-05-08T04:44:57.385 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-05-08T04:44:57.386 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47793.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2024-05-08T04:44:57.387 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36021.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-05-08T04:44:57.387 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38252.log 2024-05-08T04:44:57.388 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34063.log 2024-05-08T04:44:57.390 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-05-08T04:44:57.390 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38252.log.gz 2024-05-08T04:44:57.391 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34063.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34063.log.gz 2024-05-08T04:44:57.392 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45636.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-05-08T04:44:57.392 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-05-08T04:44:57.393 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45844.log 2024-05-08T04:44:57.395 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35029.log 2024-05-08T04:44:57.395 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-05-08T04:44:57.396 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45844.log: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 88.6% 2024-05-08T04:44:57.396 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45844.log.gz 2024-05-08T04:44:57.397 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.35029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35029.log.gz 2024-05-08T04:44:57.397 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47330.log 2024-05-08T04:44:57.398 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-05-08T04:44:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-05-08T04:44:57.399 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47330.log.gz 2024-05-08T04:44:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48784.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-05-08T04:44:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33418.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-05-08T04:44:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-05-08T04:44:57.403 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-05-08T04:44:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-05-08T04:44:57.404 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-05-08T04:44:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34915.log 2024-05-08T04:44:57.406 INFO:teuthology.orchestra.run.smithi164.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-05-08T04:44:57.407 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35154.log 2024-05-08T04:44:57.407 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34915.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-05-08T04:44:57.408 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33862.log 2024-05-08T04:44:57.408 INFO:teuthology.orchestra.run.smithi164.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34915.log.gz 2024-05-08T04:44:57.409 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.2.35154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35154.log.gz 2024-05-08T04:44:57.409 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-05-08T04:44:57.409 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41118.log 2024-05-08T04:44:57.410 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33862.log.gz 2024-05-08T04:44:57.411 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-05-08T04:44:57.411 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-05-08T04:44:57.411 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.41118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41118.log.gz 2024-05-08T04:44:57.412 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36986.log 2024-05-08T04:44:57.413 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48223.log 2024-05-08T04:44:57.413 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz 2024-05-08T04:44:57.414 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47741.log 88.1% 2024-05-08T04:44:57.414 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36986.log.gz 2024-05-08T04:44:57.415 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37666.log 2024-05-08T04:44:57.415 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48223.log: /var/log/ceph/ceph-client.admin.47741.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48223.log.gz 2024-05-08T04:44:57.416 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39913.log 2024-05-08T04:44:57.416 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47741.log.gz 2024-05-08T04:44:57.416 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-05-08T04:44:57.417 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37666.log.gz 2024-05-08T04:44:57.418 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35564.log 2024-05-08T04:44:57.418 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39913.log.gz 2024-05-08T04:44:57.419 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36128.log 2024-05-08T04:44:57.419 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-05-08T04:44:57.420 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-05-08T04:44:57.420 INFO:teuthology.orchestra.run.smithi164.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35564.log.gz 2024-05-08T04:44:57.421 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-05-08T04:44:57.421 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36128.log.gz 2024-05-08T04:44:57.422 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44042.log 2024-05-08T04:44:57.422 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gz 2024-05-08T04:44:57.423 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2024-05-08T04:44:57.424 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-05-08T04:44:57.424 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T04:44:57.424 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42059.log 2024-05-08T04:44:57.424 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44042.log.gz 2024-05-08T04:44:57.425 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50493.log 2024-05-08T04:44:57.425 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-05-08T04:44:57.426 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-05-08T04:44:57.426 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42059.log.gz 2024-05-08T04:44:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.50493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50493.log.gz 2024-05-08T04:44:57.427 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38460.log 2024-05-08T04:44:57.428 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T04:44:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33854.log 2024-05-08T04:44:57.429 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-05-08T04:44:57.429 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39498.log 2024-05-08T04:44:57.429 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38460.log.gz 2024-05-08T04:44:57.430 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27881.log 2024-05-08T04:44:57.430 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33854.log.gz 2024-05-08T04:44:57.431 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43885.log 2024-05-08T04:44:57.431 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39498.log.gz 2024-05-08T04:44:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-05-08T04:44:57.432 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27881.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27881.log.gz 2024-05-08T04:44:57.433 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33606.log 2024-05-08T04:44:57.433 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43885.log.gz 2024-05-08T04:44:57.433 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T04:44:57.434 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40270.log 2024-05-08T04:44:57.435 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-05-08T04:44:57.435 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-05-08T04:44:57.437 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-05-08T04:44:57.437 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m0.573s 2024-05-08T04:44:57.437 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m2.333s 2024-05-08T04:44:57.437 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.285s 2024-05-08T04:44:57.438 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33606.log.gz 2024-05-08T04:44:57.438 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37432.log 2024-05-08T04:44:57.438 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40270.log.gz 2024-05-08T04:44:57.438 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-05-08T04:44:57.438 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35748.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-05-08T04:44:57.439 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2024-05-08T04:44:57.439 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37432.log.gz 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33684.log 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47005.log 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51498.log 2024-05-08T04:44:57.444 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33684.log.gz 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33736.log 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47005.log.gz 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35834.log 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51498.log.gz 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2024-05-08T04:44:57.445 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33736.log.gz 2024-05-08T04:44:57.447 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51036.log 2024-05-08T04:44:57.447 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35834.log.gz 2024-05-08T04:44:57.447 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-05-08T04:44:57.447 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2024-05-08T04:44:57.447 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-05-08T04:44:57.448 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51036.log.gz 2024-05-08T04:44:57.448 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37198.log 2024-05-08T04:44:57.448 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-05-08T04:44:57.449 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-05-08T04:44:57.449 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-05-08T04:44:57.449 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36498.log 2024-05-08T04:44:57.450 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37198.log.gz 2024-05-08T04:44:57.450 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41728.log 2024-05-08T04:44:57.451 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-05-08T04:44:57.451 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2024-05-08T04:44:57.451 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36498.log.gz 2024-05-08T04:44:57.452 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-05-08T04:44:57.452 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41728.log.gz 2024-05-08T04:44:57.453 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-05-08T04:44:57.453 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2024-05-08T04:44:57.454 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-05-08T04:44:57.454 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2024-05-08T04:44:57.454 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48848.log 2024-05-08T04:44:57.455 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-05-08T04:44:57.455 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T04:44:57.456 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-05-08T04:44:57.456 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43390.log 2024-05-08T04:44:57.456 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48848.log.gz 2024-05-08T04:44:57.457 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-05-08T04:44:57.458 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43390.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43390.log.gz 2024-05-08T04:44:57.458 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-05-08T04:44:57.459 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-08T04:44:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48407.log: /var/log/ceph/ceph-client.admin.47143.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-05-08T04:44:57.460 INFO:teuthology.orchestra.run.smithi164.stderr:gzip 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz -5 2024-05-08T04:44:57.461 INFO:teuthology.orchestra.run.smithi164.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35642.log 2024-05-08T04:44:57.461 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-05-08T04:44:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39939.log 2024-05-08T04:44:57.462 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42139.log 2024-05-08T04:44:57.462 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35642.log.gz 2024-05-08T04:44:57.463 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41884.log 2024-05-08T04:44:57.463 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39939.log.gz 2024-05-08T04:44:57.464 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-05-08T04:44:57.464 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42139.log.gz 2024-05-08T04:44:57.465 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41067.log 2024-05-08T04:44:57.465 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41884.log.gz 2024-05-08T04:44:57.466 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50094.log 2024-05-08T04:44:57.466 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-05-08T04:44:57.467 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37250.log 2024-05-08T04:44:57.467 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41067.log.gz 2024-05-08T04:44:57.468 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50094.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-08T04:44:57.468 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50094.log.gz 2024-05-08T04:44:57.469 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33500.log 2024-05-08T04:44:57.470 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37250.log.gz 2024-05-08T04:44:57.470 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35616.log 2024-05-08T04:44:57.471 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51088.log 2024-05-08T04:44:57.472 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33500.log.gz 2024-05-08T04:44:57.472 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35668.log 2024-05-08T04:44:57.472 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35616.log.gz 2024-05-08T04:44:57.473 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42649.log 2024-05-08T04:44:57.474 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-05-08T04:44:57.475 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40493.log 2024-05-08T04:44:57.475 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35668.log.gz 2024-05-08T04:44:57.476 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49234.log 2024-05-08T04:44:57.476 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42649.log.gz 2024-05-08T04:44:57.477 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38894.log 2024-05-08T04:44:57.477 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40493.log.gz 2024-05-08T04:44:57.478 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37699.log 2024-05-08T04:44:57.478 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49234.log.gz 2024-05-08T04:44:57.479 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40755.log 2024-05-08T04:44:57.479 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38894.log.gz 2024-05-08T04:44:57.480 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-05-08T04:44:57.481 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37699.log.gz 2024-05-08T04:44:57.482 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38382.log 2024-05-08T04:44:57.482 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40755.log.gz 2024-05-08T04:44:57.483 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37012.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gz 2024-05-08T04:44:57.484 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38382.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38382.log.gz 2024-05-08T04:44:57.484 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log 2024-05-08T04:44:57.485 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-05-08T04:44:57.486 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35047.log 2024-05-08T04:44:57.486 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2024-05-08T04:44:57.487 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36315.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-05-08T04:44:57.488 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.35047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35047.log.gz 2024-05-08T04:44:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42033.log 2024-05-08T04:44:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41545.log 2024-05-08T04:44:57.491 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-05-08T04:44:57.491 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42033.log.gz 2024-05-08T04:44:57.492 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41545.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41545.log.gz 2024-05-08T04:44:57.493 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46581.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-05-08T04:44:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-05-08T04:44:57.494 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-05-08T04:44:57.496 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38330.log 2024-05-08T04:44:57.496 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-05-08T04:44:57.497 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35995.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-05-08T04:44:57.498 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38330.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38330.log.gz 2024-05-08T04:44:57.498 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50586.log 2024-05-08T04:44:57.499 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34621.log 2024-05-08T04:44:57.500 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48303.log 2024-05-08T04:44:57.501 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50586.log.gz 2024-05-08T04:44:57.501 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34621.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34621.log.gz 2024-05-08T04:44:57.503 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48303.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48303.log.gz 2024-05-08T04:44:57.503 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34803.log 2024-05-08T04:44:57.504 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2024-05-08T04:44:57.505 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35808.log 2024-05-08T04:44:57.506 INFO:teuthology.orchestra.run.smithi164.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34803.log.gz 2024-05-08T04:44:57.506 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48615.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2024-05-08T04:44:57.508 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35808.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35808.log.gz 2024-05-08T04:44:57.508 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37913.log 2024-05-08T04:44:57.509 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-05-08T04:44:57.510 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46944.log 2024-05-08T04:44:57.511 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37913.log.gz 2024-05-08T04:44:57.511 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37939.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-05-08T04:44:57.513 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46944.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46944.log.gz 2024-05-08T04:44:57.513 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 2024-05-08T04:44:57.514 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-05-08T04:44:57.515 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-05-08T04:44:57.515 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-05-08T04:44:57.517 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38643.log 2024-05-08T04:44:57.517 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-05-08T04:44:57.518 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36550.log 2024-05-08T04:44:57.518 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-05-08T04:44:57.519 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-05-08T04:44:57.519 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38643.log.gz 2024-05-08T04:44:57.520 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46503.log 2024-05-08T04:44:57.520 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36550.log.gz 2024-05-08T04:44:57.521 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-05-08T04:44:57.521 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-05-08T04:44:57.522 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-05-08T04:44:57.522 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2024-05-08T04:44:57.523 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46306.log 2024-05-08T04:44:57.523 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-05-08T04:44:57.525 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47845.log 2024-05-08T04:44:57.525 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-05-08T04:44:57.526 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35252.log 2024-05-08T04:44:57.526 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46306.log.gz 2024-05-08T04:44:57.527 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-05-08T04:44:57.527 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47845.log.gz 2024-05-08T04:44:57.528 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-05-08T04:44:57.528 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35252.log.gz 2024-05-08T04:44:57.529 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34777.log 2024-05-08T04:44:57.529 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-05-08T04:44:57.530 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41832.log 2024-05-08T04:44:57.530 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-05-08T04:44:57.532 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48565.log 2024-05-08T04:44:57.532 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34777.log.gz 2024-05-08T04:44:57.533 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48935.log 2024-05-08T04:44:57.533 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41832.log.gz 2024-05-08T04:44:57.534 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48910.log 2024-05-08T04:44:57.535 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48935.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48565.log.gz 2024-05-08T04:44:57.535 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48935.log.gz 2024-05-08T04:44:57.536 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36680.log 2024-05-08T04:44:57.536 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48910.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-05-08T04:44:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-05-08T04:44:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-05-08T04:44:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36680.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36680.log.gz 2024-05-08T04:44:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43173.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-05-08T04:44:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51141.log 2024-05-08T04:44:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49417.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-05-08T04:44:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-05-08T04:44:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38278.log 2024-05-08T04:44:57.543 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51141.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51141.log.gz/var/log/ceph/ceph-client.admin.48106.log: 2024-05-08T04:44:57.543 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-05-08T04:44:57.544 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47540.log 2024-05-08T04:44:57.544 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38278.log.gz 2024-05-08T04:44:57.544 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42857.log 2024-05-08T04:44:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-05-08T04:44:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47540.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47540.log.gz 2024-05-08T04:44:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42857.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42857.log.gz 2024-05-08T04:44:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44552.log 2024-05-08T04:44:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42753.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-05-08T04:44:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48487.log 2024-05-08T04:44:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45344.log 2024-05-08T04:44:57.550 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44552.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44552.log.gz/var/log/ceph/ceph-client.admin.48487.log: 2024-05-08T04:44:57.550 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48487.log.gz 2024-05-08T04:44:57.551 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46045.log 2024-05-08T04:44:57.552 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45344.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45344.log.gz 2024-05-08T04:44:57.552 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39782.log 2024-05-08T04:44:57.553 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-05-08T04:44:57.554 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46045.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46045.log.gz 2024-05-08T04:44:57.554 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39782.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39782.log.gz 2024-05-08T04:44:57.555 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-05-08T04:44:57.555 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45688.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-05-08T04:44:57.556 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38148.log 2024-05-08T04:44:57.557 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38122.log 2024-05-08T04:44:57.557 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50181.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz/var/log/ceph/ceph-client.admin.38148.log: 2024-05-08T04:44:57.558 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38148.log.gz 2024-05-08T04:44:57.559 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40322.log 2024-05-08T04:44:57.559 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38122.log.gz 2024-05-08T04:44:57.559 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-05-08T04:44:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40559.log 2024-05-08T04:44:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40322.log: 89.0%/var/log/ceph/ceph-client.admin.41676.log: -- replaced with /var/log/ceph/ceph-client.admin.40322.log.gz 2024-05-08T04:44:57.562 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-05-08T04:44:57.563 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-05-08T04:44:57.563 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40559.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-05-08T04:44:57.563 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-05-08T04:44:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50340.log 2024-05-08T04:44:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42007.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz/var/log/ceph/ceph-client.admin.40835.log: 2024-05-08T04:44:57.566 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-05-08T04:44:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34699.log 2024-05-08T04:44:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50340.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2024-05-08T04:44:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-05-08T04:44:57.568 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50155.log 2024-05-08T04:44:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34699.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34699.log.gz/var/log/ceph/ceph-client.admin.37066.log: 2024-05-08T04:44:57.569 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-05-08T04:44:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41571.log 2024-05-08T04:44:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50155.log.gz 2024-05-08T04:44:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43814.log 2024-05-08T04:44:57.572 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39375.log 2024-05-08T04:44:57.573 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41571.log: 89.6%/var/log/ceph/ceph-client.admin.43814.log: -- replaced with /var/log/ceph/ceph-client.admin.41571.log.gz 2024-05-08T04:44:57.573 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43814.log.gz 2024-05-08T04:44:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49697.log 2024-05-08T04:44:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39375.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39375.log.gz 2024-05-08T04:44:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38972.log 2024-05-08T04:44:57.576 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2024-05-08T04:44:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49697.log.gz 2024-05-08T04:44:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38972.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38972.log.gz 2024-05-08T04:44:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-05-08T04:44:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49618.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2024-05-08T04:44:57.579 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37835.log 2024-05-08T04:44:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34829.log 2024-05-08T04:44:57.581 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49260.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz/var/log/ceph/ceph-client.admin.37835.log: 2024-05-08T04:44:57.581 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37835.log.gz 2024-05-08T04:44:57.582 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33710.log 2024-05-08T04:44:57.582 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34829.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34829.log.gz 2024-05-08T04:44:57.583 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34647.log 2024-05-08T04:44:57.584 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-05-08T04:44:57.584 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33710.log: /var/log/ceph/ceph-client.admin.34647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33710.log.gz 2024-05-08T04:44:57.585 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34647.log.gz 2024-05-08T04:44:57.586 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-05-08T04:44:57.586 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-05-08T04:44:57.586 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41623.log 2024-05-08T04:44:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-05-08T04:44:57.588 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39130.log: 89.2%/var/log/ceph/ceph-client.admin.41623.log: -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-05-08T04:44:57.589 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41623.log.gz 2024-05-08T04:44:57.589 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-05-08T04:44:57.590 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50953.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-05-08T04:44:57.591 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-05-08T04:44:57.591 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36934.log 2024-05-08T04:44:57.592 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42113.log: 89.4%/var/log/ceph/ceph-client.admin.42569.log: -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-05-08T04:44:57.593 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-05-08T04:44:57.593 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39206.log 2024-05-08T04:44:57.594 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36934.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36934.log.gz 2024-05-08T04:44:57.594 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40244.log 2024-05-08T04:44:57.595 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39206.log: /var/log/ceph/ceph-client.admin.44500.log 2024-05-08T04:44:57.596 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39206.log.gz 2024-05-08T04:44:57.596 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40244.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40244.log.gz 2024-05-08T04:44:57.597 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-05-08T04:44:57.598 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44500.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-05-08T04:44:57.598 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-05-08T04:44:57.599 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-05-08T04:44:57.600 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43525.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz/var/log/ceph/ceph-client.admin.33580.log: 2024-05-08T04:44:57.600 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-05-08T04:44:57.601 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36777.log 2024-05-08T04:44:57.601 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42998.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-05-08T04:44:57.602 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-05-08T04:44:57.603 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2024-05-08T04:44:57.604 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36777.log: 89.1%/var/log/ceph/ceph-client.admin.44682.log: -- replaced with /var/log/ceph/ceph-client.admin.36777.log.gz 2024-05-08T04:44:57.604 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-05-08T04:44:57.605 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48680.log 2024-05-08T04:44:57.605 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46529.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2024-05-08T04:44:57.606 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-05-08T04:44:57.607 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47618.log 2024-05-08T04:44:57.607 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48680.log.gz 2024-05-08T04:44:57.608 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48706.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-05-08T04:44:57.609 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37575.log 2024-05-08T04:44:57.609 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47618.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47618.log.gz 2024-05-08T04:44:57.610 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2024-05-08T04:44:57.611 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2024-05-08T04:44:57.611 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37575.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37575.log.gz 2024-05-08T04:44:57.612 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45453.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2024-05-08T04:44:57.612 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-05-08T04:44:57.613 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41236.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2024-05-08T04:44:57.613 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2024-05-08T04:44:57.614 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-05-08T04:44:57.615 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-05-08T04:44:57.616 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.39530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2024-05-08T04:44:57.616 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34257.log 2024-05-08T04:44:57.617 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37641.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-05-08T04:44:57.617 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40807.log 2024-05-08T04:44:57.618 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37478.log 2024-05-08T04:44:57.619 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34257.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34257.log.gz 2024-05-08T04:44:57.619 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40807.log.gz 2024-05-08T04:44:57.620 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-05-08T04:44:57.620 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.37478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37478.log.gz 2024-05-08T04:44:57.621 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49834.log 2024-05-08T04:44:57.622 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50560.log 2024-05-08T04:44:57.623 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42191.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz/var/log/ceph/ceph-client.admin.49834.log: 2024-05-08T04:44:57.623 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49834.log.gz 2024-05-08T04:44:57.624 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2024-05-08T04:44:57.624 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.50560.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50560.log.gz 2024-05-08T04:44:57.625 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-05-08T04:44:57.626 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2024-05-08T04:44:57.627 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51892.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz/var/log/ceph/ceph-client.admin.43447.log: 2024-05-08T04:44:57.627 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-05-08T04:44:57.628 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48249.log 2024-05-08T04:44:57.628 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46607.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2024-05-08T04:44:57.629 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-05-08T04:44:57.629 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-05-08T04:44:57.630 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48249.log: /var/log/ceph/ceph-client.admin.43102.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48249.log.gz 2024-05-08T04:44:57.631 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-05-08T04:44:57.632 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35067.log 2024-05-08T04:44:57.632 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48355.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-05-08T04:44:57.632 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38723.log 2024-05-08T04:44:57.633 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2024-05-08T04:44:57.634 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35067.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35067.log.gz/var/log/ceph/ceph-client.admin.38723.log: 2024-05-08T04:44:57.635 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38723.log.gz 2024-05-08T04:44:57.635 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2024-05-08T04:44:57.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48053.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz 2024-05-08T04:44:57.636 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51524.log 2024-05-08T04:44:57.637 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51944.log 2024-05-08T04:44:57.638 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37276.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2024-05-08T04:44:57.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51524.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51524.log.gz 2024-05-08T04:44:57.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.51944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 2024-05-08T04:44:57.640 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2024-05-08T04:44:57.640 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49671.log 2024-05-08T04:44:57.641 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-05-08T04:44:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44068.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz 2024-05-08T04:44:57.642 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49671.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49671.log.gz 2024-05-08T04:44:57.643 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42597.log 2024-05-08T04:44:57.644 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-05-08T04:44:57.644 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49931.log 2024-05-08T04:44:57.645 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-05-08T04:44:57.646 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42597.log: 88.8%/var/log/ceph/ceph-client.admin.49931.log: -- replaced with /var/log/ceph/ceph-client.admin.42597.log.gz 2024-05-08T04:44:57.646 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49931.log.gz 2024-05-08T04:44:57.647 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-05-08T04:44:57.647 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43684.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-05-08T04:44:57.648 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-05-08T04:44:57.649 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-05-08T04:44:57.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46019.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-05-08T04:44:57.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46659.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-05-08T04:44:57.651 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49645.log 2024-05-08T04:44:57.651 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49469.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-05-08T04:44:57.652 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-05-08T04:44:57.653 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43911.log 2024-05-08T04:44:57.654 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.49645.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49645.log.gz/var/log/ceph/ceph-client.admin.40585.log: 2024-05-08T04:44:57.654 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-05-08T04:44:57.655 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-05-08T04:44:57.655 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43911.log.gz 2024-05-08T04:44:57.655 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-05-08T04:44:57.656 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41326.log 2024-05-08T04:44:57.657 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47090.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz/var/log/ceph/ceph-client.admin.41936.log: 2024-05-08T04:44:57.658 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-05-08T04:44:57.658 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35486.log 2024-05-08T04:44:57.659 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41326.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.41326.log.gz 2024-05-08T04:44:57.659 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48197.log 2024-05-08T04:44:57.660 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45422.log 2024-05-08T04:44:57.662 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35486.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35486.log.gz/var/log/ceph/ceph-client.admin.48197.log: 2024-05-08T04:44:57.662 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48197.log.gz 2024-05-08T04:44:57.663 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2024-05-08T04:44:57.663 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45422.log.gz 2024-05-08T04:44:57.663 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49957.log 2024-05-08T04:44:57.664 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34751.log 2024-05-08T04:44:57.665 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.44101.log: 87.8%/var/log/ceph/ceph-client.admin.49957.log: -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2024-05-08T04:44:57.666 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49957.log.gz 2024-05-08T04:44:57.667 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 2024-05-08T04:44:57.667 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34751.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34751.log.gz 2024-05-08T04:44:57.667 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44578.log 2024-05-08T04:44:57.668 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-05-08T04:44:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35696.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz/var/log/ceph/ceph-client.admin.44578.log: 2024-05-08T04:44:57.670 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44578.log.gz 2024-05-08T04:44:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-05-08T04:44:57.671 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.42374.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-05-08T04:44:57.671 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40095.log 2024-05-08T04:44:57.672 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2024-05-08T04:44:57.673 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46977.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz/var/log/ceph/ceph-client.admin.40095.log: 2024-05-08T04:44:57.673 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40095.log.gz 2024-05-08T04:44:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40461.log 2024-05-08T04:44:57.674 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47514.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2024-05-08T04:44:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43364.log 2024-05-08T04:44:57.675 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-05-08T04:44:57.676 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.40461.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40461.log.gz/var/log/ceph/ceph-client.admin.43364.log: 2024-05-08T04:44:57.677 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43364.log.gz 2024-05-08T04:44:57.677 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-05-08T04:44:57.678 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36829.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-05-08T04:44:57.679 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36803.log 2024-05-08T04:44:57.679 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45126.log 2024-05-08T04:44:57.681 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.41488.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz/var/log/ceph/ceph-client.admin.36803.log: 2024-05-08T04:44:57.681 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36803.log.gz 2024-05-08T04:44:57.681 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-05-08T04:44:57.682 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45126.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45126.log.gz 2024-05-08T04:44:57.682 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36154.log 2024-05-08T04:44:57.683 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47566.log 2024-05-08T04:44:57.684 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37549.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz/var/log/ceph/ceph-client.admin.36154.log: 2024-05-08T04:44:57.684 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36154.log.gz 2024-05-08T04:44:57.685 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2024-05-08T04:44:57.686 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47566.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47566.log.gz 2024-05-08T04:44:57.686 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-05-08T04:44:57.687 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38617.log 2024-05-08T04:44:57.688 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46906.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-05-08T04:44:57.688 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36263.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-05-08T04:44:57.689 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-05-08T04:44:57.689 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38617.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38617.log.gz 2024-05-08T04:44:57.690 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43840.log 2024-05-08T04:44:57.691 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45584.log 2024-05-08T04:44:57.692 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.48080.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-05-08T04:44:57.692 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.43840.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43840.log.gz 2024-05-08T04:44:57.693 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46802.log 2024-05-08T04:44:57.694 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.45584.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45584.log.gz 2024-05-08T04:44:57.694 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2024-05-08T04:44:57.695 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.46802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48666.log 2024-05-08T04:44:57.696 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46802.log.gz 2024-05-08T04:44:57.696 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.47116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2024-05-08T04:44:57.697 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.48666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48666.log.gz 2024-05-08T04:44:57.699 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-08T04:44:57.701 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T04:44:57.707 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T04:44:57.740 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T04:44:57.949 INFO:teuthology.orchestra.run.smithi164.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-08T04:44:57.951 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-05-08T04:44:57.951 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m1.131s 2024-05-08T04:44:57.951 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m4.288s 2024-05-08T04:44:57.951 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m1.394s 2024-05-08T04:44:57.952 INFO:tasks.ceph:Archiving logs... 2024-05-08T04:44:57.952 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi164/log 2024-05-08T04:44:57.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T04:44:58.839 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-05-08T04:44:58.915 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi172/log 2024-05-08T04:44:58.916 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T04:44:59.218 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T04:44:59.228 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T04:44:59.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T04:44:59.232 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T04:44:59.312 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T04:44:59.322 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T04:44:59.322 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:44:59.326 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:44:59.337 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T04:44:59.337 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-05-08T04:44:59.337 INFO:teuthology.orchestra.run.smithi164.stdout:*hv01.front.sepi 67.205.162.81 3 u 62 64 377 0.082 -0.498 1.019 2024-05-08T04:44:59.338 INFO:teuthology.orchestra.run.smithi164.stdout:+hv02.front.sepi 63.231.80.2 3 u 59 64 377 0.065 -1.388 0.826 2024-05-08T04:44:59.338 INFO:teuthology.orchestra.run.smithi164.stdout:+hv03.front.sepi 74.6.168.72 3 u 65 64 377 0.090 -0.422 0.662 2024-05-08T04:44:59.338 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-05-08T04:44:59.341 INFO:teuthology.orchestra.run.smithi172.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T04:44:59.341 INFO:teuthology.orchestra.run.smithi172.stdout:============================================================================== 2024-05-08T04:44:59.341 INFO:teuthology.orchestra.run.smithi172.stdout:+hv01.front.sepi 67.205.162.81 3 u 3 64 377 0.101 -2.850 1.369 2024-05-08T04:44:59.342 INFO:teuthology.orchestra.run.smithi172.stdout:*hv02.front.sepi 63.231.80.2 3 u 59 64 377 0.099 -1.423 0.673 2024-05-08T04:44:59.342 INFO:teuthology.orchestra.run.smithi172.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.079 -2.344 1.357 2024-05-08T04:44:59.342 INFO:teuthology.orchestra.run.smithi172.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-05-08T04:44:59.343 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T04:44:59.395 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T04:44:59.395 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T04:44:59.443 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T04:44:59.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T04:44:59.552 INFO:teuthology.task.internal:Duration was 1189.055489 seconds 2024-05-08T04:44:59.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T04:44:59.603 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T04:44:59.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T04:44:59.607 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T04:44:59.702 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T04:44:59.702 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-05-08T04:44:59.702 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T04:44:59.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-05-08T04:44:59.723 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T04:44:59.745 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T04:44:59.746 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T04:44:59.769 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T04:44:59.827 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T04:44:59.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:44:59.853 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:44:59.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T04:44:59.950 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T04:44:59.950 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T04:44:59.970 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T04:44:59.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T04:45:00.004 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:45:00.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:45:00.035 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-05-08T04:45:00.053 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-05-08T04:45:00.071 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:45:00.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:45:00.104 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:45:00.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:45:00.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T04:45:00.129 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T04:45:00.130 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi164 2024-05-08T04:45:00.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T04:45:00.172 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289/remote/smithi172 2024-05-08T04:45:00.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T04:45:00.197 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T04:45:00.198 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T04:45:00.216 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T04:45:00.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T04:45:00.261 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T04:45:00.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T04:45:00.295 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T04:45:00.295 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T04:45:00.298 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T04:45:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: 655465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:45 /home/ubuntu/cephtest 2024-05-08T04:45:00.310 INFO:teuthology.orchestra.run.smithi172.stdout: 658123 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:45 /home/ubuntu/cephtest 2024-05-08T04:45:00.313 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T04:45:00.342 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T04:45:00.505 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T04:45:00.532 DEBUG:teuthology.nuke:shortname: smithi164 2024-05-08T04:45:00.533 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T04:45:00.557 DEBUG:teuthology.nuke:shortname: smithi172 2024-05-08T04:45:00.557 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T04:45:00.561 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 04:15:03.613620', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnS/3cYF83fvr2BaYE8Va8ASSrF+m2VisCVVn/fgNEYJNj1uesTjrUBynghqpdH1q2GwXAQmUum+3EExQ6XLI0='} 2024-05-08T04:45:00.582 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-08_04:04:15-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7697289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 04:15:03.612223', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRm1XvoHIXIVd8PIF/BL0esi4XwWpYetJ7hmo0GEjmxAW4xn7mAK0Hn60JgRhKulQR+VLqYNGDfdRp3h35qmbI='} 2024-05-08T04:45:00.612 INFO:teuthology.orchestra.console.smithi164:Power off 2024-05-08T04:45:00.613 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T04:45:00.635 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-05-08T04:45:00.636 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:00.654 INFO:teuthology.orchestra.console.smithi172:Power off 2024-05-08T04:45:00.655 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T04:45:00.665 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-05-08T04:45:00.677 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2024-05-08T04:45:00.678 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:00.701 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-05-08T04:45:04.666 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:04.702 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:04.776 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-05-08T04:45:04.812 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-05-08T04:45:08.778 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:08.813 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:45:08.892 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-05-08T04:45:08.923 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2024-05-08T04:45:08.993 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-05-08T04:45:09.025 INFO:teuthology.orchestra.console.smithi172:Power off completed 2024-05-08T04:45:09.130 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1189.0554893016815 failure_reason: 'Test failure: test_subvolume_retain_snapshot_recreate_subvolume (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=52b651016cc543f9b62d99cb4e844676 status: fail success: false 2024-05-08T04:45:09.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T04:45:09.212 INFO:teuthology.run:FAIL